Peer Review History

Original SubmissionMay 26, 2019
Decision Letter - Jason A. Papin, Editor, Ferhat Ay, Editor

Dear Dr Ritz,

Thank you very much for submitting your manuscript, 'Connectivity measures for signaling pathway topologies', to PLOS Computational Biology. As with all papers submitted to the journal, yours was fully evaluated by the PLOS Computational Biology editorial team, and in this case, by independent peer reviewers including two from the GLBIO submission. The reviewers agreed that this revision is responsive to the major concerns raised but identified some aspects of the manuscript that should be improved.

We would therefore like to ask you to modify the manuscript according to the review recommendations before we can consider your manuscript for acceptance. Your revisions should address the specific points made by each reviewer and we encourage you to respond to particular issues. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.raised.

In addition, when you are ready to resubmit, please be prepared to provide the following:

(1) A detailed list of your responses to the review comments and the changes you have made in the manuscript. We require a file of this nature before your manuscript is passed back to the editors.

(2) A copy of your manuscript with the changes highlighted (encouraged). We encourage authors, if possible to show clearly where changes have been made to their manuscript e.g. by highlighting text.

(3) A striking still image to accompany your article (optional). If the image is judged to be suitable by the editors, it may be featured on our website and might be chosen as the issue image for that month. These square, high-quality images should be accompanied by a short caption. Please note as well that there should be no copyright restrictions on the use of the image, so that it can be published under the Open-Access license and be subject only to appropriate attribution.

Before you resubmit your manuscript, please consult our Submission Checklist to ensure your manuscript is formatted correctly for PLOS Computational Biology: http://www.ploscompbiol.org/static/checklist.action. Some key points to remember are:

- Figures uploaded separately as TIFF or EPS files (if you wish, your figures may remain in your main manuscript file in addition).

- Supporting Information uploaded as separate files, titled 'Dataset', 'Figure', 'Table', 'Text', 'Protocol', 'Audio', or 'Video'.

- Funding information in the 'Financial Disclosure' box in the online system.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com  PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we ask that you let us know the expected resubmission date by email at ploscompbiol@plos.org.

If you have any questions or concerns while you make these revisions, please let us know.

Sincerely,

Ferhat Ay, Ph.D

Associate Editor

PLOS Computational Biology

Jason Papin

Editor-in-Chief

PLOS Computational Biology

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact ploscompbiol@plos.org immediately:

[LINK]

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The authors have addressed my comments fully in the revised manuscript and response to reviewers. A very nice paper!

I have some minor typographical and cosmetic comments:

- The type size in the compiled manuscript in all figures except for 4 is too small. If possible, it would be nice to standardize the font size across the figures.

- I think that the cross-refs for Fig 3B and 3C might be flipped in the text.

- Figure 4A: Is it possible to stretch A so that one can look at the same pathway across both A and B?

- Figure 4B: What is the threshold for significance? Are the p-values corrected for multiple testing?

- In the Abstract: “pairs of pathways statistically significant influence” → “pairs of pathways with statistically significant influence”

- Page 3, line 43: k hasn’t actually been defined before this point

- Page 4, line 68: “built upon” – “encoded within”?

- Page 6, lines 145-162: I got a bit confused re-reading this section. The text describes three sets of nodes to remove: 2778 small molecules defined by Reactome, 3 highly connected gene products/complexed, and another 155 small molecules from PathwayCommons. Which of these are “blacklisted nodes” – the ubiquitins and NPC? Were both sets of small molecules combined into one? Do the stats on the numbers of affected nodes/hyperedges reflect the removal of all three sets of nodes?

- Page 10, line 242: “under any definition” -- definition of what?

- Page 11, line 307: “Reactome pathwa” → “Reactome pathway”

Reviewer #2: I was a reviewer for the conference version of the paper to GLBIO. The authors have done a great job in their response to reviewers. The manuscript is in great shape, and I don't have any major comments.

Minor comment: p.11 line 23: typo, pathwa --> pathway

Reviewer #3: As a new reviewer for this revised submission, these comments are intended to assess only the responsiveness of the revisions to the major concerns from the original reviewers. The major changes in the revision include several new analyses that better demonstrate how B-relaxation distance can be used in practice and how this distance measure relates to existing connectivity measures. The new figures showing reachable source-target paths as a function of distance satisfy prior concerns about how the parameterized B-relaxation measure relates to parameterized versions of other connectivity measures.

Several reviewers questioned how B-relaxation can be evaluated empirically with more comprehensive applications. The new evaluation of STRING edges by score and edge type is a strong response to these comments. The results depict how pairs of proteins that are B-connected or have a particular B-relaxation distance tend to have higher STRING edge scores, especially for the STRING combined scores. This emphasizes how B-relaxation provides a different way to evaluate node connectivity that is not trivially recapitulated by the existing connectivity measures or Reactome pathway membership. The reviewers noted several additional potential applications that the authors defer for future work. This is reasonable to keep outside the scope of this manuscript and shows the opportunities for future studies based on B-relaxation connectivity. In addition, the authors added new permutation-based statistical testing for their Reactome pathway influence analysis. This is an appropriate control because the authors are correct that simulating realistic pathway hypergraphs, a previously-suggested alternative way to test the influence measures, is non-trivial.

The minor comments regarding the precision and recall analysis for the pathway influence study would have been a helpful addition to the manuscript. However, this is a fairly minor request and would not substantially affect the conclusions about the merits of B-relaxation.

Overall, all major concerns have been directly or indirectly addressed. The GraphSpace network visualizations and software to reproduce the major analyses in the manuscript are added benefits that complement the strong manuscript. The specific comments below are all minor comments that could help improve the presentation.

Minor comments:

1) Reactome has a large number of pathways, but only 34 are used for the pathway influence analysis and 140 are used for the STRING edge score analysis. It is unclear why these subsets where selected and why different pathways were used for the influence and STRING analyses.

2) The STRING analysis excluded the "database" channel, which used Reactome as a source of evidence. However, if the "combined score" channel includes the "database" channel as an information source, there is still some degree of circularity in this analysis.

3) The figures for the STRING analysis show scores in the 0-1000 range. However, some of the text describes scores in the 0-1 range.

4) Using 2.2e-308, the smallest float in Python, in the STRING significance analysis may not be the best way to present these results. Wolfgang Huber discussed this issue recently in "Reporting p Values" (Cell Systems https://doi.org/10.1016/j.cels.2019.03.001).

5) In Figure 8 and other figures that visualize hypergraph properties, it is not immediately obvious that the blue regions overlap the red regions. Even if the figures are not changed, it would be helpful to explicitly note in the caption that node d is in both the blue and the red sets.

6) The Discussion notes that connected proteins that are not in the same pathway may be missed from gene-set-like enrichment approaches. For most applications of gene set enrichment, this seems like a desirable property.

7) The Python software is provided in https://github.com/Murali-group/halp and https://github.com/annaritz/pathway-connectivity and is well-organized in general. However:

- The GPL3 licenses are not stated directly in the manuscript.

- It is not clear which developmental branch of the halp repository was used for this study, which impedes reproducibility. Ideally this version would be merged to master and released.

- There are test cases, but the Travis CI testing fails.

- The pathway-connectivity repository contains data from third-party sources (e.g. STRING) that may require more direct attribution in the readme.

- An added bonus for both repositories would be to archive the software on third-party services like Zenodo, Figshare, or Software Heritage to ensure they are more permanently available to readers.

8) Typos and phrasing:

- Abstract: "pathways statistically significant" is missing a word

- P3: "molecules that that participate"

- P3: The end of the introduction notes relationships that "emerge for larger values of k" but the parameter k has not yet been introduced and is not intuitive before that introduction

- P9: "nodes that in"

- P11: "within the same Reactome pathwa"

- P16: "PaxTools java" could capitalize Java

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Computational Biology data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: No: The statistics underlying the figures are not provided as supplementary tables, but the authors have provided their input data and code to generate the results on GitHub.

Reviewer #2: Yes

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

Revision 1
Decision Letter - Jason A. Papin, Editor, Ferhat Ay, Editor

Dear Dr Ritz,

We are pleased to inform you that your manuscript 'Hypergraph-based connectivity measures for signaling pathway topologies' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

In the meantime, please log into Editorial Manager at https://www.editorialmanager.com/pcompbiol/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production and billing process.

One of the goals of PLOS is to make science accessible to educators and the public. PLOS staff issue occasional press releases and make early versions of PLOS Computational Biology articles available to science writers and journalists. PLOS staff also collaborate with Communication and Public Information Offices and would be happy to work with the relevant people at your institution or funding agency. If your institution or funding agency is interested in promoting your findings, please ask them to coordinate their releases with PLOS (contact ploscompbiol@plos.org).

Thank you again for supporting Open Access publishing. We look forward to publishing your paper in PLOS Computational Biology.

Sincerely,

Ferhat Ay, Ph.D

Associate Editor

PLOS Computational Biology

Jason Papin

Editor-in-Chief

PLOS Computational Biology

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #3: The authors have addressed all of my previous minor concerns with these revisions. I noted two trivial formatting issues in this version:

- "from" is repeated in line 46

- The "All Nodes" label cut off in Fig S6B

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Computational Biology data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Attachments
Attachment
Submitted filename: ResponseToReviewers.pdf
Formally Accepted
Acceptance Letter - Jason A. Papin, Editor, Ferhat Ay, Editor

PCOMPBIOL-D-19-00839R1

Hypergraph-based connectivity measures for signaling pathway topologies

Dear Dr Ritz,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Matt Lyles

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom ploscompbiol@plos.org | Phone +44 (0) 1223-442824 | ploscompbiol.org | @PLOSCompBiol

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .