TargetDB: A target information aggregation tool and tractability predictor

Stephane De Cesco; John B. Davis; Paul E. Brennan

doi:10.1371/journal.pone.0232644

Peer Review History

Original SubmissionApril 17, 2020
5 Jun 2020 Decision Letter - Joseph J Barchi, Editor PONE-D-20-11085 TargetDB: A target information aggregation tool and tractability predictor PLOS ONE Dear Dr. De Cesco, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. As you can see the 3 reviewers were quite positive overall, and all recommend publication. Please pay attention to the comments and respond accordingly---I would be happy to accept a revised version of this manuscript as soon as I receive and review your modified paper. Please submit your revised manuscript by Jul 20 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Joseph J Barchi Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please ensure that you refer to Figure 2 in your text as, if accepted, production will need this reference to link the reader to the figure. 3. Please upload a copy of Figure 7, to which you refer in your text. If the figure is no longer to be included as part of the submission please remove all reference to it within the text. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. Additional Editor Comments (if provided): [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: I Don't Know Reviewer #3: N/A ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes Reviewer #3: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have developed a handy tool to assist researchers with the challenging task of prioritising a list of potential targets. A large number of various data resources are integrated and final outputs in terms of excel sheets are reported. The ability to adjust weights for main categories is particular useful to customise ranking of targets. 1.) We installed the targetDB package via conda as described on the github page, on MacOS. An initial run failed due to missing xlsxwriter and xlrd packages. Perhaps those dependencies can be mentioned as part of the installation instructions. After installation of those packages the software ran successfully. 2.) Occasionally, an error is thrown, presumably if no protein expression levels are available. KeyError: "['Muscle tissues', 'Female tissues', 'Bone marrow & lymphoid tissues', 'Liver & gallbladder', 'Proximal digestive tract', 'Gastrointestinal tract', 'Kidney & urinary bladder', 'Male tissues', 'Adipose & soft tissue', 'Endocrine tissues'] not in index" For example, when entering SLC6A8 and SLC6A5 as the gene list in 'List mode'. These corner cases should be caught and dealt with instead of left just hanging. 3.) There is a spreadsheet called "Not in DB" in List mode. Is this supposed to provide the user with the subset of genes that were not found/matched in the DB? This is useful if dozens of genes were entered and the user can quickly check which genes were not found. However, the spreadsheet is empty at the moment if gene names were not found. 4.) The mode 'Spider plot only' does not give an option to save the produced figure, at least not on MacOS. 5.) A supplementary table should be included listing the 399 tractable targets, 400 intractable targets and their allocation into training and test sets. 6.) Table 1, it is not clear what 'SBDD' stands for. 7.) Figure 6, provide the reader with a short explanation of the 'dark proteome' 8.) What about other tools with very similar functionality, e.g. differences between TargetDB and the recently published TractaViewer should be discussed. 9.) It's somewhat unfortunate that the name "TargetDB" has been used previously for another DB (PMID: 15130928). The authors might want to consider renaming their software/DB, although it seems that this older DB and its original name is not in use anymore. Reviewer #2: This paper presents a useful tool that aggregates information on a given human gene and presents that information in an easy to read and manipulate format. I can see that this tool would be very useful for prioritising gene targets in the context of early stage drug discovery in industry and also academic pipelines. The ability to up/down-weight the various parameter spaces in terms of the scoring is a nice feature which makes this a generically useful tool. I was able to download and install the program successfully and run a number of targets through it. I only tested under Linux so cannot say if it would work well under Windows or Mac. Some comments and requests: • I note that the supplied database is for ChEMBL 24, which is now quite out of date. I would like to see some instructions on how a newer version of ChEMBL (i.e. 26) can be used, notwithstanding the fact that it’s likely some code changes will be required to accommodate any ChEMBL schema changes • I would like to see guidance on how the program can be extended in principle. For example, population information on common SNPs, or integration of data from gnomAD. • The spider plots are very useful visualisations. However, it’s a bit hard to get your head around the filled colour schemes in the single gene output. I would like to see a legend underneath to explain the fill colours to the reader. • I would like to see a more detailed discussion on the machine learning process and outcomes in the text. Whilst the jupyter notebook is reasonably understandable, more discussion is required on why the exact form of Random Forest was chosen, in the context of the other methods tested. This is important because it gives the community a better understanding of your experience of using a large and diverse parameter space, which others can take advantage of. • You state that “Safety is not considered in the MPO scoring at this time”. It is not clear to me whether you mean that the MPO weighting is not considered or just that in the example you set this to zero. Please clarify. • Open Targets is ‘misspelled’ in a number of places. Please use the correct version ‘Open Targets’ consistently. • Please provide a reference to Humanmine. • Please can you describe in more detail what the selectivity score is? You specific that it is (selectivity entropy – Shannon entropy) and reference the seminar 1948 information theory paper, but there is not enough information here to reproduce what you are trying to achieve. • It appears that there are no resolution cut-offs used for structures that go through fpocket analysis. Please can you explain if that is the case and why you feel this is appropriate, if so? An EM structure of >6 Angstroms is going to provide unreliable results in this context, for example. • What is the sustainability plan for targetDB? Since databases change all the time, how will you ensure that this platform is still usable even in 6 months’ time? • It is highly unusual to use ellipses in articles. Please remove these and use ‘etc.’ (for example) instead. • It would be good to have the article carefully proof read for English before it is finally accepted. Overall, a nice piece of work that I’m sure many will want to use. Reviewer #3: The authors construct a database application with a graphical interface that aggregates multiple disparate sources of evidence for novel drug target evaluation. They consider measures of druggability, structure, chemistry, biology, disease association, genetic association, general information, and safety when constructing their application. As output they produce different summaries of this evidence - either a summary for a single target, multiple targets, or a graphical summary (aka a spider plot). Furthermore, the authors provide an optimization approach (powered by a machine learning method) to allow users to weight multiple classes of evidence based on the users target validation needs and interests. Major comments - Overall the authors provide a potentially useful tool to aid structural biologists, chemists, and assay developers for target prioritization. That being said, I have the following comments: 1. How reproducible are the results generated by this - many of the databases that evidence is being pulled from may change over time. Is there some way to version the results, or to construct queries that refer to a specific version of a public database? 2. The user defined weighting is an interesting idea - I would like to see the authors discuss potential pitfalls of this - e.g. over optimizing for targets with interesting structural biology/chemistry as opposed to targets that will actually lead to therapies or deeper insights into the disease biology. Minor comments - 1. use of ellipses is distracting (Abstract, Introduction Paragraph 1). 2. “Percentage of threes predicting” - spelling/grammar throughout should be double checked. 3. SBDD acronym in Table 1 is not defined. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No Reviewer #3: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0232644.r001
Revision 1
22 Jul 2020 Author Response PREFACE: Some answers to reviewers contains figures or equations, therefore, all answers have been added to the cover letter as well Reviewers’ comments to the authors: Reviewer #1 The authors have developed a handy tool to assist researchers with the challenging task of prioritising a list of potential targets. A large number of various data resources are integrated and final outputs in terms of excel sheets are reported. The ability to adjust weights for main categories is particular useful to customise ranking of targets. Answer: We thank the reviewer for their kind words and are happy that he/she finds it useful. 1.) We installed the targetDB package via conda as described on the github page, on MacOS. An initial run failed due to missing xlsxwriter and xlrd packages. Perhaps those dependencies can be mentioned as part of the installation instructions. After installation of those packages the software ran successfully. Answer: Thanks for the comment, the conda package wasn’t yet created at the time of submission and it hasn’t been tested extensively. As it is another contributor that created the package I asked him to include these packages to the dependency list. 2.) Occasionally, an error is thrown, presumably if no protein expression levels are available. KeyError: "['Muscle tissues', 'Female tissues', 'Bone marrow & lymphoid tissues', 'Liver & gallbladder', 'Proximal digestive tract', 'Gastrointestinal tract', 'Kidney & urinary bladder', 'Male tissues', 'Adipose & soft tissue', 'Endocrine tissues'] not in index" For example, when entering SLC6A8 and SLC6A5 as the gene list in 'List mode'. These corner cases should be caught and dealt with instead of left just hanging. Answer: It has now been fixed in the code in version 1.3.1 of the python package 3.) There is a spreadsheet called "Not in DB" in List mode. Is this supposed to provide the user with the subset of genes that were not found/matched in the DB? This is useful if dozens of genes were entered and the user can quickly check which genes were not found. However, the spreadsheet is empty at the moment if gene names were not found. Answer: It has now been fixed in the code in version 1.3.1 of the python package 4.) The mode 'Spider plot only' does not give an option to save the produced figure, at least not on MacOS. Answer: We thank the reviewer for the suggestion, it has now been added to the version 1.3.1 of the python package. 5.) A supplementary table should be included listing the 399 tractable targets, 400 intractable targets and their allocation into training and test sets. Answer: An additional SI document titled “ML_TrainingTestingSplit_Targets.xlsx” has been added. The document contains two tabs with the testing and training set and each has 2 columns: Target Uniprot ID and Druggability class (1 = Druggable, 0=Intractable) 6.) Table 1, it is not clear what 'SBDD' stands for. Answer: This has now been clarified in the legend of the table. 7.) Figure 6, provide the reader with a short explanation of the 'dark proteome' Answer: We have added the following text under the figure: “The definition of dark proteome is taken from the Target Central Resource Database.” Including a reference to the definition on the TCRD platform website ( http://juniper.health.unm.edu/tcrd/) 8.) What about other tools with very similar functionality, e.g. differences between TargetDB and the recently published TractaViewer should be discussed. Answer: A sentence has been added to discuss the differences between the two tools: “More recently, a tool with similar features, TractaViewer, has been described in the literature [16]. While this tool allows the user to classify targets into different bins, it does not provide an area specific score or a general scoring system that can be used for target prioritization. The bin assignment combined with the scoring system of TargetDB, however, could provide valuable information to researchers seeking to assess target tractability 9.) It's somewhat unfortunate that the name "TargetDB" has been used previously for another DB (PMID: 15130928). The authors might want to consider renaming their software/DB, although it seems that this older DB and its original name is not in use anymore. Answer: While we certainly agree with the reviewer, we eventually decided not to change the name as, the reviewer pointed out, the older platform has not been in use for a long time. We believe the TargetDB name reflects best the purpose of this tool. Furthermore, a significant number of academic research teams in the UK and a few pharmaceutical companies are already familiar with the TargetDB name and are using our platform. Therefore, we feel it would be better to continue to use TargetDB Reviewer #2 This paper presents a useful tool that aggregates information on a given human gene and presents that information in an easy to read and manipulate format. I can see that this tool would be very useful for prioritising gene targets in the context of early stage drug discovery in industry and also academic pipelines. The ability to up/down-weight the various parameter spaces in terms of the scoring is a nice feature which makes this a generically useful tool. Answer: We thank the reviewer for their kind words and are happy that he/she finds it useful. I was able to download and install the program successfully and run a number of targets through it. I only tested under Linux so cannot say if it would work well under Windows or Mac. Some comments and requests: • I note that the supplied database is for ChEMBL 24, which is now quite out of date. I would like to see some instructions on how a newer version of ChEMBL (i.e. 26) can be used, notwithstanding the fact that it’s likely some code changes will be required to accommodate any ChEMBL schema changes Answer: The readme file on github has been updated to reflect the fact that the data in the present release has been used with ChEMBL 25. The next update of the database (scheduled for August) will be using ChEMBL 27 or the most recent release at the time of the update. • I would like to see guidance on how the program can be extended in principle. For example, population information on common SNPs, or integration of data from gnomAD. Answer: All the source code is available on GitHub and anyone with experience in python can branch the repository to add any valuable information to the output. This may be completed in a couple of steps: - Convert/parse the data into a pandas dataframe (example of this is in the druggability_DB.py file) and store it as an attribute of the Class Target (line 1290) - Add this dataframe to the sqlite database (and create a new table) using the function write_to_db() (line 1074 - druggability_DB.py) - To add it to the output these three files would need to be updated accordingly as well: targetDB\\druggability_report.py targetDB\\target_descriptors.py targetDB\\target_features.py We have added a sentence to convey this message of collaborative effort more clearly: “We strongly encourage anyone interested to download the code and participate in the project by adding new datasources and/or features.” • The spider plots are very useful visualisations. However, it’s a bit hard to get your head around the filled colour schemes in the single gene output. I would like to see a legend underneath to explain the fill colours to the reader. Answer: We agree that the spider plot combines a lot of data in different forms and a reading guide would be beneficial. We have added a key to the SI (S1_Fig), as well as to the documentation on the github page of the project, to help the user interpret the plot. We hope the reviewers find this useful. • I would like to see a more detailed discussion on the machine learning process and outcomes in the text. Whilst the jupyter notebook is reasonably understandable, more discussion is required on why the exact form of Random Forest was chosen, in the context of the other methods tested. This is important because it gives the community a better understanding of your experience of using a large and diverse parameter space, which others can take advantage of. Answer: While we certainly agree with the reviewer that others can take advantage of the process we went through, we believe that the technical details of the model training and evaluation are better suited for the supplemental information (S1 File), where we go into detail about picking/removing collinear features and selection/refinement of the best algorithm. We have added the following sentence to the main text to provide better context to why the random forest algorithm was picked: “This allowed us to narrow down to a set of features that were truly contributing to the performance of the model.” • You state that “Safety is not considered in the MPO scoring at this time”. It is not clear to me whether you mean that the MPO weighting is not considered or just that in the example you set this to zero. Please clarify. Answer: We are sorry this was not more clear. The original meaning was that the weight was set to zero in that specific instance. We have now reformulated the sentence to make it clearer. The sentence now reads: “ In this instance, the weight for the safety term was set to zero and was not, therefore, considered in the MPO scoring” • Open Targets is ‘misspelled’ in a number of places. Please use the correct version ‘Open Targets’ consistently. Answer: This has now been fixed in the revised manuscript. • Please provide a reference to Humanmine. Answer: We are sorry to have missed that, it has been added to the revised manuscript • Please can you describe in more detail what the selectivity score is? You specific that it is (selectivity entropy – Shannon entropy) and reference the seminar 1948 information theory paper, but there is not enough information here to reproduce what you are trying to achieve. Answer: A more detailed calculation has been added in the Supporting information S1 File The Shannon entropy was used as a measure of selectivity for the two following metrics. The global equation is the following: S_sel=-∑_i^T▒〖ρ_i log⁡〖ρ_i 〗〗 With Ssel = Selectivity Entropy Selectivity of compound T = Kd for different targets ρ_i = Probability for a Kd value ρ_((T) )=〖K_d〗_T /(∑_i▒〖K_d〗_i ) Where KdT = Binding association for target T Selectivity of tissue expression T = Expression in different tissue ρ_i = Probability of an expression value ρ_((T))=E_T /(∑_i▒〖E_T〗_i ) Where ET = Expression value in tissue T • It appears that there are no resolution cut-offs used for structures that go through fpocket analysis. Please can you explain if that is the case and why you feel this is appropriate, if so? An EM structure of >6 Angstroms is going to provide unreliable results in this context, for example. Answer: We thank the reviewer for his suggestion, as of now, it is true that no cut-offs are put in place but this will certainly be implemented in the next release of the database. In the meantime, the information on the resolution of the structure is captured on a different tab and can be used to manually remove any binding pocket result coming from such low resolution structures. (This has been captured as a enhancement request on the GitHub page: https://github.com/sdecesco/targetDB/issues/10) • What is the sustainability plan for targetDB? Since databases change all the time, how will you ensure that this platform is still usable even in 6 months’ time? Answer: As of now two avenues are open, we are committed to update the database bi-annually for the foreseeable future, and are also working on a separate version which would collect data live from various API of these datasources. The disadvantage of the live version would be that it would be slower and would more easily break down if any of the used datasource modify their outputs in a significant manner. Moreover, one of the main reason to make the entire source code available is to allow anyone with enough experience in python to modulate and/or fix TargetDB in the future. • It is highly unusual to use ellipses in articles. Please remove these and use ‘etc.’ (for example) instead. Answer: Ellipses have been removed in the revised manuscript • It would be good to have the article carefully proof read for English before it is finally accepted. Answer: We thank the reviewer for the advice and the manuscript has now been carefully proofread by native English speakers. Overall, a nice piece of work that I’m sure many will want to use. Answer: We thank the reviewer again for their kind words. Reviewer #3 The authors construct a database application with a graphical interface that aggregates multiple disparate sources of evidence for novel drug target evaluation. They consider measures of druggability, structure, chemistry, biology, disease association, genetic association, general information, and safety when constructing their application. As output they produce different summaries of this evidence - either a summary for a single target, multiple targets, or a graphical summary (aka a spider plot). Furthermore, the authors provide an optimization approach (powered by a machine learning method) to allow users to weight multiple classes of evidence based on the users target validation needs and interests. Major comments - Overall the authors provide a potentially useful tool to aid structural biologists, chemists, and assay developers for target prioritization. That being said, I have the following comments: 1. How reproducible are the results generated by this - many of the databases that evidence is being pulled from may change over time. Is there some way to version the results, or to construct queries that refer to a specific version of a public database? Answer: The current model is that the database behind TargetDB is generated at a certain date and time by pulling data from specific versions of databases (which are specified in the supporting information and/or fields in the database itself) or through the API of the sources at that same date (+/- few days as the creation of the TargetDB database takes a few days to complete) 2. The user defined weighting is an interesting idea - I would like to see the authors discuss potential pitfalls of this - e.g. over optimizing for targets with interesting structural biology/chemistry as opposed to targets that will actually lead to therapies or deeper insights into the disease biology. Answer: This is an interesting and important point raised by the reviewer. Providing this tool of course does not guarantee that it will always be used in a way that leads to therapies. We have added a few words of caution regarding the use of these weights in target prioritization: “As note of caution, while a user might decide to over-prioritize areas such as structural biology and chemistry it is by no means a guarantee that this will lead to a target with potential therapeutic applications. This facility also enables users to tailor their searches to highlight targets where there is an opportunity for their research expertise to make an impact; picking targets for which there is a lack structural information, lack of chemistry or even lack of biology, for example On the other hand, if a target is over-prioritized for strong disease and genetic links, it may be very difficult to develop safe and effective therapeutics due to low safety and structural druggability scores.” Minor comments - 1. use of ellipses is distracting (Abstract, Introduction Paragraph 1). Answer: These have now been removed from the revised manuscript (see reviewer #2 comments) 2. “Percentage of threes predicting” - spelling/grammar throughout should be double checked. Answer: Spelling and grammar has now been checked throughout the document 3. SBDD acronym in Table 1 is not defined. Answer: Definition of the acronym has now been added to the text https://doi.org/10.1371/journal.pone.0232644.r002
29 Jul 2020 Decision Letter - Joseph J Barchi, Editor TargetDB: A target information aggregation tool and tractability predictor PONE-D-20-11085R1 Dear Dr. De Cesco, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Joseph J Barchi Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0232644.r003
Formally Accepted
3 Aug 2020 Acceptance Letter - Joseph J Barchi, Editor PONE-D-20-11085R1 TargetDB: A target information aggregation tool and tractability predictor Dear Dr. De Cesco: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Joseph J Barchi Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0232644.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .