Dear Dr Steinrücken,

Thank you very much for submitting your Research Article entitled 'Polygenic score accuracy in ancient samples: quantifying the effects of allelic turnover' to PLOS Genetics.

The manuscript was fully evaluated at the editorial level and by independent peer reviewers. The reviewers appreciated the attention to an important topic but identified some concerns that we ask you address in a revised manuscript.

We therefore ask you to modify the manuscript according to the review recommendations. Your revisions should address the specific points made by each reviewer.

In addition we ask that you:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Kirk E Lohmueller

Guest Editor

PLOS Genetics

Bret Payseur

Section Editor: Evolution

PLOS Genetics

Thank you for submitting this work to PLOS Genetics. It has been evaluated by 3 reviewers. All generally liked the manuscript. The reviewers provide a number of comments to improve the manuscript. None stood out as being more critical than others. I think addressing these comments, even those that were presented as suggestions/not required by the reviewers, in your revision will improve the readability and applicability of your work.

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: This manuscript by Carlson and colleagues considers the effect of "allelic turnover" on polygenic scores fit to ancient samples. "Allelic turnover," as the authors define it, is relevant to the problem of polygenic score portability in part because causal variants that explain trait variation in an ancient sample may experience frequency changes that make them undetectable in modern GWAS. The authors use a diffusion approach to study several summaries of polygenic score performance. I have not checked all the equations, but the work appears to be well done. I have a few comments, which I list not as requirements for publication (which I leave up to the editor), but rather as friendly reactions and suggestions.

1) The model here assumes that the ancient sample is from a population directly ancestral to a contemporary population, when in fact many ancient samples may not fit this description straightforwardly. (See e.g. Schraiber 2018, PMID: 29167200). I think it is totally fair for the authors to model things the way they do, as it captures a lot of the intuition and allows them just to think about one forward diffusion. However, the paper might benefit from a clearer discussion of this potential caveat.

2) Along similar lines to (1), I was struck by how much of the framing is in terms of ancient samples when a lot of the intuitions developed are relevant to pairs of contemporary populations. The model is more straightforwardly related to ancient populations (though see (1)), but I think it'd be interesting for the authors to give more space to thinking about the relevance for contemporary populations. There is a paragraph on this in the discussion, but only a hint of a quantitative comparison is given, and as far as I can tell mainly for empirical results. It would be useful to include a comparison with the modeling approach of Wang, Guo et al (ref 16).

3) arXiv:1909.00892 has a nice discussion of the relevance of allelic turnover to polygenic score comparisons as proxies for phenotype comparisons. This is captured in the various summaries studied, but I would have liked more emphasis on distinguishing within-population accuracy vs. getting changes in the mean over time correct in the discussion.

4) A new preprint by Yair & Coop is relevant for this study and takes a complementary modeling approach https://doi.org/10.1101/2021.09.10.459833 . (I know what this is starting to look like, and I swear I'm not Graham Coop.)

5) I found the explanations of the results in the main text to be, in some places, a little wordy and hard to follow. I think it is worthwhile to communicate the parts of the intuition that have to do with allele frequencies changing into or out of the GWAS-detectable range, but I got lost in some of the verbal explanations. I also thought the summary in the intro was in too much depth for not having seen the model yet. The repeated structure of the results subsections, though helpful for comprehension, also get to feel redundant. I would urge the authors to cut down the main text, lean more on their figures, and even possibly make use of tables to summarize some of the main results (e.g. in the intro).

6) I appreciated that the authors brought in literature on genetic prediction outside humans.

7) The final paragraph is perhaps missing a mention of environmental differences and GxE.

Minor comments

a) line 139, I assume that N is a diploid population size because time units are scaled in 2N generations, but please specify.

b) line 167, is it meant that this quantity is often called a "genetic value" specifically in this paper? I have not seen it much in other places. If it's meant that it's a general term, please provide a citation.

c) p. 5, I might have missed it, but I don't see it explicitly stated here that eq. 2 assumes linkage equilibrium as well. (It shows up in the beginning of 2.3 but should probably be here first.)

Reviewer #2: In this manuscript, Carlson et al. provide a masterful assessment of polygenic score accuracy decreases due to allelic turnover, in the context of ancient populations that are ancestral to the GWAS panel on which a phenotype was measured. They quantify this decrease as a function of various parameters of interest, including the temporal distance between the ancient and present-day population, the power to detect significantly-associated loci, and trait heritability. They also produce simulations to determine how much weak selection on trait-associated variants biases the quantities of interest.

The manuscript is well written and it is clear that the authors have done a great deal of work to reach their conclusions, and evaluate various quantities of interest for geneticists to assess how reliable a score for an ancient individual would be. I only have very minimal suggestions for changes or additions to the manuscript, that I don't deem crucial for the manuscript to be accepted for publication:

To provide a clearer picture for researchers choosing to do (or not do) polygenic prediction on ancient samples, it would be nice to work with values from a concrete phenotype and GWAS study: given the size and heritability estimates from the latest UK Biobank GWAS or Biobank Japan on a particular phenotype (e.g. height), how much would one mis-estimate this phenotype in a population that is X years away from UKBB or BBJ, in the past, solely due to allelic turnover?

Rarely are polygenic scores in the ancient DNA literature computed for populations that are directly ancestral to the present-day GWAS population, as there has been substantial population turnover and admixture in human populations during the Holocene and Pleistocene. It might be worth emphasizing this more strong in the Discussion so that the reader is aware that a population sampled X years in the past (in the same location as a present-day population) does not necessarily correspond to your scenario of an ancestral population that is separated by X years from the present-day GWAS populations.

In this vein, the manuscript would benefit from moving some of the results from Supplementary Text S1.1 to the main text (at a minimum the figure and the final conclusion), as those would more generally applicable to the larger set of researchers not necessarily working with directly ancestral populations, or working with diverged populations from the GWAS one, outside the ancient genomic literature.

Following up on this, could the authors also express the turnover-caused decreased in accuracy in terms of Fst values, rather than the ancient sampling time in coalescent units? This would make it a lot easier for an empirical researcher to assess how much of a decrease in accuracy due to allelic turnover they would expect for their ancient (or out-of-space) polygenic predictions, given that Fst values between present-day and ancient populations are readily obtainable (assuming stationarity and that allelic turnover is the only contributor to the decrease in accuracy).

I would be curious to see how accuracy decreases with increasing the number of ancient individuals used in the computation of polygenic score, e.g. how much more / less problematic is it to compute the mean polygenic score for a population of ancient hunter-gatherers vs. computing a polygenic score individually for each ancient hunter-gatherer in the population. How is the variance for these different estimates affected by the sampling time?

Beyond recent directional selection, could the authors provide some intuition as to how persistent negative selection would affect their results? i.e. How would these be affected when the magnitude of the effect size estimates determines the probability that the individual variants will be removed from the population?

Reviewer #3: See attached

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Fernando Racimo

Reviewer #3: Yes: Luke J. O'Connor

Attachments

Attachment

Submitted filename: Maryn_PG_PRS_review.docx

Dear Dr Steinrücken,

Thank you very much for submitting your Research Article entitled 'Polygenic score accuracy in ancient samples: quantifying the effects of allelic turnover' to PLOS Genetics.

The manuscript was fully evaluated at the editorial level and by independent peer reviewers. The reviewers appreciated the attention to an important topic but identified some concerns that we ask you address in a revised manuscript.

We therefore ask you to modify the manuscript according to the review recommendations. Your revisions should address the specific points made by each reviewer.

In addition we ask that you:

1) Provide a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript.

2) Upload a Striking Image with a corresponding caption to accompany your manuscript if one is available (either a new image or an existing one from within your manuscript). If this image is judged to be suitable, it may be featured on our website. Images should ideally be high resolution, eye-catching, single panel square images. For examples, please browse our archive. If your image is from someone other than yourself, please ensure that the artist has read and agreed to the terms and conditions of the Creative Commons Attribution License. Note: we cannot publish copyrighted images.

We hope to receive your revised manuscript within the next 30 days. If you anticipate any delay in its return, we would ask you to let us know the expected resubmission date by email to plosgenetics@plos.org.

If present, accompanying reviewer attachments should be included with this email; please notify the journal office if any appear to be missing. They will also be available for download from the link below. You can use this link to log into the system when you are ready to submit a revised version, having first consulted our Submission Checklist.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Please be aware that our data availability policy requires that all numerical data underlying graphs or summary statistics are included with the submission, and you will need to provide this upon resubmission if not already present. In addition, we do not permit the inclusion of phrases such as "data not shown" or "unpublished results" in manuscripts. All points should be backed up by data provided with the submission.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

PLOS has incorporated Similarity Check, powered by iThenticate, into its journal-wide submission system in order to screen submitted content for originality before publication. Each PLOS journal undertakes screening on a proportion of submitted articles. You will be contacted if needed following the screening process.

To resubmit, you will need to go to the link below and 'Revise Submission' in the 'Submissions Needing Revision' folder.

[LINK]

Please let us know if you have any questions while making these revisions.

Yours sincerely,

Kirk E Lohmueller

Guest Editor

PLOS Genetics

Bret Payseur

Section Editor: Evolution

PLOS Genetics

Thank you for submitting your revised manuscript. Two of the three reviewers are satisfied with the changes. Reviewer 3 has some additional comments on the BLUP model in the Supplement. Please provide some qualitative description of what would change under a BLUP-type estimator with no threshold.

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The authors have addressed my first-round comments sufficiently, and I have no further comments. This was a nice paper even before the revision.

Reviewer #2: The authors have addressed all of my concerns. I recommend this paper for acceptance.

Reviewer #3: In their revised manuscript, Carlson et al. have addressed most of my comments from the initial submission, but I was not completely satisfied by their treatment of the BLUP PRS estimator, which I still think would produce qualitatively different results than the threshold-style PRS estimator they primarily analyze.

The approximation the authors use for the PRS is that there is some detection threshold based on the true effect size (and AF) of a variant, and we get a good estimate of the effect size for variants above the threshold, but a zero estimate for variants below it. This approximation is aligned with a threshold-style estimator, where you throw out variants using a threshold for their estimated (rather than true) effect size/significance.

It contrasts with the BLUP estimator, which does not throw out any variants. The new supplementary note seems to be arguing that the BLUP is approximately the same as the estimator under consideration because its weights are well approximated by the OLS effect size estimates. This is true (perhaps even obvious), but the OLS effect size estimates are only used in the threshold approximation *for variants passing the threshold*, and this threshold seems to be critical reason for the qualitative findings throughout the paper.

As I noted before, the difference between these estimators corresponds to the difference between a sparse vs. infinitesimal genetic architecture: under a sparse architecture, a method with some sort of feature selection is called for; under an infinitesimal architecture, when every SNP is causal, the BLUP estimator is optimal.

I do not at all think it would detract from the findings of the paper if they are only applicable to sparse architectures and threshold/variable selection style estimators. We know that real genetic architectures are not infinitesimal (especially for non-psychiatric traits), and in general, models with some thresholding or variable selection perform better than BLUP.

I don’t request new analyses to address this, but I am concerned that the new supplementary note misses the mark, and I recommend at least some qualitative description of what would change under a BLUP-type estimator with no threshold.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: None

Reviewer #2: Yes

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Fernando Racimo

Reviewer #3: Yes: Luke J O'Connor

Dear Dr Steinrücken,

We are pleased to inform you that your manuscript entitled "Polygenic score accuracy in ancient samples: quantifying the effects of allelic turnover" has been editorially accepted for publication in PLOS Genetics. Congratulations!

Before your submission can be formally accepted and sent to production you will need to complete our formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Please note: the accept date on your published article will reflect the date of this provisional acceptance, but your manuscript will not be scheduled for publication until the required changes have been made.

Once your paper is formally accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you’ve already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosgenetics@plos.org.

In the meantime, please log into Editorial Manager at https://www.editorialmanager.com/pgenetics/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production and billing process. Note that PLOS requires an ORCID iD for all corresponding authors. Therefore, please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field.  This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager.

If you have a press-related query, or would like to know about making your underlying data available (as you will be aware, this is required for publication), please see the end of this email. If your institution or institutions have a press office, please notify them about your upcoming article at this point, to enable them to help maximise its impact. Inform journal staff as soon as possible if you are preparing a press release for your article and need a publication date.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Genetics!

Yours sincerely,

Kirk E Lohmueller

Guest Editor

PLOS Genetics

Bret Payseur

Section Editor: Evolution

PLOS Genetics

www.plosgenetics.org

Twitter: @PLOSGenetics

----------------------------------------------------

Comments from the reviewers (if applicable):

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #3: The authors have addressed my comments.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Genetics data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #3: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Luke J O'Connor

----------------------------------------------------

Data Deposition

If you have submitted a Research Article or Front Matter that has associated data that are not suitable for deposition in a subject-specific public repository (such as GenBank or ArrayExpress), one way to make that data available is to deposit it in the Dryad Digital Repository. As you may recall, we ask all authors to agree to make data available; this is one way to achieve that. A full list of recommended repositories can be found on our website.

The following link will take you to the Dryad record for your article, so you won't have to re‐enter its bibliographic information, and can upload your files directly: 

http://datadryad.org/submit?journalID=pgenetics&manu=PGENETICS-D-21-01313R2

More information about depositing data in Dryad is available at http://www.datadryad.org/depositing. If you experience any difficulties in submitting your data, please contact help@datadryad.org for support.

Additionally, please be aware that our data availability policy requires that all numerical data underlying display items are included with the submission, and you will need to provide this before we can formally accept your manuscript, if not already present.

----------------------------------------------------

Press Queries

If you or your institution will be preparing press materials for this manuscript, or if you need to know your paper's publication date for media purposes, please inform the journal staff as soon as possible so that your submission can be scheduled accordingly. Your manuscript will remain under a strict press embargo until the publication date and time. This means an early version of your manuscript will not be published ahead of your final version. PLOS Genetics may also choose to issue a press release for your article. If there's anything the journal should know or you'd like more information, please get in touch via plosgenetics@plos.org.