Dear Dr Wallace,

Thank you very much for submitting your Research Article entitled 'A more accurate method for colocalisation analysis allowing for multiple causal variants' to PLOS Genetics.

The manuscript was fully evaluated at the editorial level and by independent peer reviewers. The reviewers made positive comments but also raised substantial concerns about the current manuscript. Based on the reviews, we will not be able to accept this version of the manuscript, but we would be willing to review a much-revised version. We cannot, of course, promise publication at that time.

Should you decide to revise the manuscript for further consideration here, your revisions should address the specific points made by each reviewer. We will also require a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

If you decide to revise the manuscript for further consideration at PLOS Genetics, please aim to resubmit within the next 60 days, unless it will take extra time to address the concerns of the reviewers, in which case we would appreciate an expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments are included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool.  PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, use the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

We are sorry that we cannot be more positive about your manuscript at this stage. Please do not hesitate to contact us if you have any concerns or questions.

Yours sincerely,

Heather J Cordell

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

Reviewer's Responses to Questions

Comments to the Authors:

Reviewer #1: Please see attached Word doc.

Reviewer #2: Review of Wallace

Summary

This paper introduces an extension of the "coloc" method for colocalization

to deal with multiple causal variants in a region. This extension exploits a

recently-introduced method for fine mapping (SuSiE). The extension is

attractive in its simplicity, and simulations show it to perform better than some

alternative approaches. The paper also suggests a way to speed up computations

by pre-filtering out "non-significant" SNPs.

The key idea of combining SuSiE and coloc is nice, and I think that with

some improvements to the presentation will make a nice publishable contribution.

The idea of speeding up SuSiE by pre-filtering SNPs is also attractive from

a practical point of view, but it has some potential downsides that I feel

are not sufficiently emphasized and explored (even though the manuscript does end

with a statement that trimming might be not beneficial in general final mapping).

Specifically trimming out non-significant SNPs

could increase the potential for false positive identifications,

and indeed such a result has been previously reported in

https://www.biorxiv.org/content/10.1101/631390v3

(their Figure S7). It's not clear to me how, if at all, this is reflected in the results

shown here. Maybe it is simply the case that, as the paper suggests in the discussion,

that "Coloc benefits from comparing posterior probabilities across... two traits".

But the overall way that the manuscript deals with false positive (or indeed

false negative) identifications

is not clear. (Maybe methods are applied with some

knowledge of the true number of causal effects? It isn't clear to me.)

Since there are also other potential ways to speed up computation (see comments below)

I am not really convinced that the pre-filtering approach is really the way to go,

and would like to see at least a stronger assessment of the potential downsides.

Main Comments

1. The presentation of the method requires more details, including more precise

equations showing how quantities computed by SuSiE are used/combined. For

example you could introduce $\\alpha_{lj}$ for the matrix of posterior probabilities output by susie

and then give explicit expressions for the Bayes Factors being computed

($BF_{lj}$) in terms of $\\alpha_{lj}$. I'm not sure what $P_0$ is (is it something output by SuSiE?)

Is $\\pi=1/p$ where p is the number of SNPs in the region, or something else? How

do you set the maximum number of effects in SuSiE (L in the SuSiE paper)? Do you get SuSiE to

estimate the number of effects by estimating the prior variance, or do fix the prior variance?

If $L_g$ is the number of effects identified by SuSiE in the GWAS and $L_e$ the

number identified by SuSiE in the eQTL study, do you end up running coloc $L_g * L_e$ times?

(as suggested by "for every pair of regressions across traits" on p3).

How do you combine/summarise the results from all these different runs of coloc?

2. Presentation of colocalization results also needs more details. Can you say explicitly

what is an "AA" or "BB" comparison and an "AB-like signal"? From the description on p3 I

thought the simulations would include settings where there were 2 causal variants in each trait,

but no sharing. But Fig 3 seems to suggest

only a small portion of potential configurations of up to 2 signals in each trait are actually

included - is that right? (why?) And in Fig 3, what happens if SuSiE finds a signal in one trait

and not in the other - what comparison do you make? (Or do you force SuSiE to find the right

number of effects in each trait by fixing L to the true value? If so, is that cheating?)

Is the smaller height of the AA bar for susie_0 compared with other methods -- and indeed

the slightly smaller height of all bars -- something to be

concerned about? Are all methods equally applicable if (as is always the case) you do not know

the true number of causal signals in each trait?

3. Figure 1 compares only the PIPs at causal variants. Since in practice we don't know the

causal variants, one should also care about PIPs at non-causal variants. Is there a tendency

for SuSiE to inflate PIPs at non-causal variants when trimming?

4. It seems there are many potential ways to improve computation than

filtering out non-significant SNPs, and many of them may ultimately be better choices

(although filtering is obviously very simple to implement!) I don't think the discussion

in the paper really adequately reflects the options available or the many

issues involved.

Although I did not see it explicitly said anywhere, I believe the

paper is using the susie_rss function for applying SuSiE to summary data.

The details of this function are not included in the original SuSiE publication, but at time of writing

this function works by performing an initial eigendecomposition of the reference LD matrix R, which

makes it possible to convert the summary data into "transformed data" to which

regular SuSiE can be applied. This approach is appealing from a software engineering

point of view, but not necessarily the most efficient, computationally. The eigendecomposition

of R is quite expensive, being O(p^3) where p is the number of SNPs.

The subsequent application of SuSiE

to the transformed data is O(p^2) per iteration.

Thus if p is sufficiently large the eigendecomposition step will likely

dominate the susie_rss computation (and Figure 2 does indeed suggest computation maybe

increase something like p^3?)

One way to reduce computational complexity would therefore be to avoid the eigendecomposition

step, and we are currently actively exploring these in our development of susie_rss.

However, note that computing R itself is already

an O(np^2) operation, where $n$ is the number of samples in the reference sample used to compute R. So

if n is big then this computation (which is basically considered free

in this paper since R is precomputed) could be the dominant computational cost. Alternatively

if n<<p, --="" avoid="" entirely="" forming="" one="" perhaps="" r="" should="" then="">in the case n<</p,>

SVD of the reference genotypes (O(n^2p)) which will cheaper than forming R (O(np^2)) when n<<p.

In the future it seems quite likely that pre-computed R and eigen(R) could be made

available for some large panels, avoiding the need for each user to compute them. Once

these pre-computations are done there may no longer be any need to filter SNPs.

Other comments/details

- p3 although the number of potential models increases exponentially, SuSiE computation

does not increase exponentially.

- p4: "We labelled each comparisons considered...." I did not understand this sentence.

- p4: "... having strongest posterior support for H_4" - this should be H_3?

- p8: " this does apply to single trait" - missing *not*?

- In the second row-set of Figure 3, is the figure on the LHS

wrong? (The methods suggest colocalization but the figure shows no shared variant...)

- on p7 the r2 threshold is 0.8 but on p4 it is 0.5. Are there referring to different thresholds?

This review is signed: Matthew Stephens</p.

Reviewer #3: This is an interesting paper. The method is solid and implements M Stephen group's SUSIE method in the coloc framework with some simulation based comparisons with other methods (and "trimming" rather than shrinkage to help compute time). Expanding coloc to multiple variants is a useful advance to the field, and that is what PLoS Genetics Methods section papers are supposed to do.

I only have minor comments.

The formatting of figure 3 - the scenarios - seems to have gone slightly awry and needs to be fixed.

I suggest the discussion could be extended slightly - it is rather brief (although sufficient).

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Matthew Stephens

Reviewer #3: No

Attachments

Attachment

Submitted filename: review wallace plos genetics.docx

Dear Dr Wallace,

Thank you very much for submitting your Research Article entitled 'A more accurate method for colocalisation analysis allowing for multiple causal variants' to PLOS Genetics.

The manuscript was fully evaluated at the editorial level and by independent peer reviewers. The reviewers appreciated the attention to an important topic but identified some concerns that we ask you address in a revised manuscript

We therefore ask you to modify the manuscript according to the review recommendations. Your revisions should address the specific points made by each reviewer.

In addition we ask that you:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Heather J Cordell

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

Reviewer's Responses to Questions

Comments to the Authors:

Reviewer #1: Please see attached Word doc.

Reviewer #2: Review of Wallace (revised)

Thank you for the responsiveness to issues raised in the previous review.

I have just two points to be addressed.

1. As noted, Susie is under active development, and

since v0.10.1 (March 16th 2021) the susie_rss function no longer performs

eigen-decomposition of R. This fact could be noted

in the discussion, and the version of susie

used to produce the results reported here should be reported.

2. I found Figure 1 hard to read. Most of the ink is not

very informative, and one has to read the actual numbers to

extract the information. Also change in total PIP is probably less relevant than

changes in individual PIPs (eg if all PIPs increase a very small amount,

the total change can be big, but it probably doesn't matter much.)

I think there should be better ways to convey the information.

Possibly a scatterplot of PIPs for each SNP, with vs without trimming,

might work - most of the points will presumably be near (0,0) but

any outliers should be immediately apparent?

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

Reviewer #2: None

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Matthew Stephens

Attachments

Attachment

Submitted filename: plos genetics review.docx

Dear Chris,

Thank you very much for submitting your Research Article entitled 'A more accurate method for colocalisation analysis allowing for multiple causal variants' to PLOS Genetics.

The manuscript was fully evaluated at the editorial level and by one independent peer reviewer. The reviewer now recommends acceptance, but identified some minor concerns that we ask you address in a further revised version before we can formally accept your manuscript.

We therefore ask you to modify the manuscript according to the review recommendations. Your revisions should address the specific points made by each reviewer.

In addition we ask that you:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Heather J Cordell

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

Reviewer's Responses to Questions

Reviewer #1: The manuscript is greatly improved since I last saw it. I have a few minor comments below, mostly to clarify some points that were not clear, but I trust the author will address them so I don’t need to see another revision of the manuscript before publication.

Minor comments:

I didn’t understand “ratio of effects” in this sentence: “Here, multiple causal variants are dealt with by requiring colocalisation across all causal variants in a region, and that the ratio of effects of each causal variant on the two traits is constant across variants.”

I didn’t follow this sentence, and how connected with the rest of the paragraph: “Thus, the user is presented with a list of tag SNPs per signal for each trait, and the matrix of pairwise posterior probabilities of H4 may be examined to infer which, if any, pairs of tags represent the same signal.”

“This situation is confusing, because the same signal in trait 1 appears to colocalise with different signals in trait 1.” Should this read “…signals in trait 2”?

I didn’t see where Fig. 2 is referred to in the text.

Why is there no green line in Fig. 3a? And why are there no green points show in Fig. 3a, d?

For completeness, I think Fig. 3 should also show trait 1, susie signal 2? (And similarly for Fig. S2.)

In the Fig. 3 caption you should also make clear what the truth is (I recognize that this is given in the text).

“S2 Fig shows an example where the stepwise approach is less able to correctly identify the separate signals.” By “stepwise approach” do you mean cond_abo?

The example in S2 Fig seems interesting and instructive—maybe it is worth putting in the main text? If I understand correctly, one difference is that susie iteratively improves the fit, whereas cond_abo does not iterate—it conditions B on A, then A on B, then stops. So perhaps one important improvement in susie is that it iteratively improves the fit until convergence? Perhaps this could explain why susie better identifies the signals in S2 Fig?

“However, when no credible sets can be detected with confidence by SuSiE, single-coloc may still be able to make some inference.” Do you know why susie fails in these cases? If there is an explanation, it would be helpful to add it here. I’m guessing that this failure occurs in cases where the support for association is not strong? On the surface the need for the “hybrid” method is a bit surprising, but there could very well be a good reason for it.

Under “Availability”, you might want to mention the susie vignette in the coloc package, which seems particularly helpful for those interested in applying the new susie-based coloc methods.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Dear Dr Wallace,

We are pleased to inform you that your manuscript entitled "A more accurate method for colocalisation analysis allowing for multiple causal variants" has been editorially accepted for publication in PLOS Genetics. Congratulations!

Before your submission can be formally accepted and sent to production you will need to complete our formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Please note: the accept date on your published article will reflect the date of this provisional acceptance, but your manuscript will not be scheduled for publication until the required changes have been made.

Once your paper is formally accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you’ve already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosgenetics@plos.org.

In the meantime, please log into Editorial Manager at https://www.editorialmanager.com/pgenetics/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production and billing process. Note that PLOS requires an ORCID iD for all corresponding authors. Therefore, please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field.  This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager.

If you have a press-related query, or would like to know about making your underlying data available (as you will be aware, this is required for publication), please see the end of this email. If your institution or institutions have a press office, please notify them about your upcoming article at this point, to enable them to help maximise its impact. Inform journal staff as soon as possible if you are preparing a press release for your article and need a publication date.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Genetics!

Yours sincerely,

Heather J Cordell

Associate Editor

PLOS Genetics

David Balding

Section Editor: Methods

PLOS Genetics

www.plosgenetics.org

Twitter: @PLOSGenetics

----------------------------------------------------

Data Deposition

If you have submitted a Research Article or Front Matter that has associated data that are not suitable for deposition in a subject-specific public repository (such as GenBank or ArrayExpress), one way to make that data available is to deposit it in the Dryad Digital Repository. As you may recall, we ask all authors to agree to make data available; this is one way to achieve that. A full list of recommended repositories can be found on our website.

The following link will take you to the Dryad record for your article, so you won't have to re‐enter its bibliographic information, and can upload your files directly: 

http://datadryad.org/submit?journalID=pgenetics&manu=PGENETICS-D-21-00266R3

More information about depositing data in Dryad is available at http://www.datadryad.org/depositing. If you experience any difficulties in submitting your data, please contact help@datadryad.org for support.

Additionally, please be aware that our data availability policy requires that all numerical data underlying display items are included with the submission, and you will need to provide this before we can formally accept your manuscript, if not already present.

----------------------------------------------------

Press Queries

If you or your institution will be preparing press materials for this manuscript, or if you need to know your paper's publication date for media purposes, please inform the journal staff as soon as possible so that your submission can be scheduled accordingly. Your manuscript will remain under a strict press embargo until the publication date and time. This means an early version of your manuscript will not be published ahead of your final version. PLOS Genetics may also choose to issue a press release for your article. If there's anything the journal should know or you'd like more information, please get in touch via plosgenetics@plos.org.