Dear Dr Aubier,

Thank you for submitting your manuscript entitled "Transmissible cancers do not easily select for sexual reproduction" for consideration as a Research Article by PLOS Biology.

Your manuscript has now been evaluated by the PLOS Biology editorial staff as well as by an academic editor with relevant expertise and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by Apr 07 2020 11:59PM.

Login to Editorial Manager here: https://www.editorialmanager.com/pbiology

During resubmission, you will be invited to opt-in to posting your pre-review manuscript as a bioRxiv preprint. Visit http://journals.plos.org/plosbiology/s/preprints for full details. If you consent to posting your current manuscript as a preprint, please upload a single Preprint PDF when you re-submit.

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Given the disruptions resulting from the ongoing COVID-19 pandemic, please expect delays in the editorial process. We apologise in advance for any inconvenience caused and will do our best to minimize impact as far as possible.

Feel free to email us at plosbiology@plos.org if you have any queries relating to your submission.

Kind regards,

Di Jiang,

Associate Editor

PLOS Biology

Dear Dr Aubier,

Thank you very much for submitting your manuscript "Transmissible cancers do not easily select for sexual reproduction" for consideration as a Research Article at PLOS Biology. Your manuscript has been evaluated by the PLOS Biology editors, an Academic Editor with relevant expertise, and by three independent reviewers.

In light of the reviews (below), we will welcome re-submission of a revised version that takes into account the reviewers' comments. Our Academic Editor agrees with reviewer 1 that you should ideally consider an open-ended model where many loci co-evolve; however, as this could involve extensive additional work, we have decided to make it optional but not essential. Our Academic Editor suggests that the revision should consider vertical transmission (which should not be hard) and also at least discuss how facultative sex might evolve towards obligate sex. We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent for further evaluation by the reviewers.

We expect to receive your revised manuscript within 2 months.

Please email us (plosbiology@plos.org) if you have any questions or concerns, or would like to request an extension. At this stage, your manuscript remains formally under active consideration at our journal; please notify us by email if you do not intend to submit a revision so that we may end consideration of the manuscript at PLOS Biology.

**IMPORTANT - SUBMITTING YOUR REVISION**

Your revisions should address the specific points made by each reviewer. Please submit the following files along with your revised manuscript:

1. A 'Response to Reviewers' file - this should detail your responses to the editorial requests, present a point-by-point response to all of the reviewers' comments, and indicate the changes made to the manuscript.

*NOTE: In your point by point response to the reviewers, please provide the full context of each review. Do not selectively quote paragraphs or sentences to reply to. The entire set of reviewer comments should be present in full and each specific point should be responded to individually, point by point.

You should also cite any additional relevant literature that has been published since the original submission and mention any additional citations in your response.

2. In addition to a clean copy of the manuscript, please also upload a 'track-changes' version of your manuscript that specifies the edits made. This should be uploaded as a "Related" file type.

*Re-submission Checklist*

When you are ready to resubmit your revised manuscript, please refer to this re-submission checklist: https://plos.io/Biology_Checklist

To submit a revised version of your manuscript, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' where you will find your submission record.

Please make sure to read the following important policies and guidelines while preparing your revision:

*Published Peer Review*

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/

*PLOS Data Policy*

Please note that as a condition of publication PLOS' data policy (http://journals.plos.org/plosbiology/s/data-availability) requires that you make available all data used to draw the conclusions arrived at in your manuscript. If you have not already done so, you must include any data used in your manuscript either in appropriate repositories, within the body of the manuscript, or as supporting information (N.B. this includes any numerical values that were used to generate graphs, histograms etc.). For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5

*Blot and Gel Data Policy*

We require the original, uncropped and minimally adjusted images supporting all blot and gel results reported in an article's figures or Supporting Information files. We will require these files before a manuscript can be accepted so please prepare them now, if you have not already uploaded them. Please carefully read our guidelines for how to prepare and upload this data: https://journals.plos.org/plosbiology/s/figures#loc-blot-and-gel-reporting-requirements

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosbiology/s/submission-guidelines#loc-materials-and-methods

Thank you again for your submission to our journal. We hope that our editorial process has been constructive thus far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Di Jiang, PhD

Associate Editor

PLOS Biology

*****************************************************

REVIEWS:

Reviewer #1 (Joanna Masel, signed review): This manuscript is an extended rebuttal of the novel hypothesis recently proposed in an essay by Thomas et al. in PLoS Biology, that protection from transmissible cancer drove the evolution of obligate sexuality as animals evolved to become multicellular. I agree with the authors of the current manuscript that the proposal of Thomas et al. should have been accompanied by a mathematical demonstration of proof of principle. However, the current manuscript attempts to provide a disproof of principle, which is always a trickier affair, as changing any number of assumptions might change the outcome, and proof of principle requires only one plausible scenario to be found.

My first major concern is with a modeling assumption that is widespread within the host-parasite coevolution theory community, but utterly baffling to someone like myself outside that who comes in grounded in molecular evolution data. This assumption is that only a small number of loci (two in this manuscript) mediate the host-parasite interaction, that fluctuating selection dynamics occur within this extremely restricted genotype space, and that it is lags within these fluctuations that drive the advantage of host sex in evolving away from parasites. The alternative "arms race" dynamic (see eg doi:10.1111/j.1420-9101.2008.01598.x for discussion) envisages open-ended evolution in many more loci, without the return to previous genotypes that characterizes fluctuations in more restricted genotype spaces that have nowhere else to go. As an outsider, I do not understand why fluctuating selection dynamics have received so much more theoretical attention than arms races; molecular studies of host-parasite interactions show adaptation occurring across vast tracts of the host genome (see eg the work of David Enard). In the context of this manuscript, it seems to me that a much larger genotype space, such that sexual progeny are almost always different from their parents, would be closer to Fig. 1B of Thomas et al., and must be considered before their hypothesis is dismissed.

My second major concern is that my reading of Thomas et al. is that their hypothesis applies to the transition from facultatively sexual reproduction to obligately sexual reproduction. While Thomas et al. is in places sloppily written on this point (another reason that a formal model should have been required in that work to make it clearer), I nevertheless think this is still the only reasonable reading. But the current manuscript models the transition from zero sex to obligate sex, even while it makes critical remarks about sex being an ancestral trait. It would be more appropriate to model the transition from rare sex to obligate sex.

Because of these two major concerns, I do not find that the principle proposed by Thomas et al. has been satisfactorily disproved in this work. Major revisions to address these two concerns has the potential to change the conclusions of this work. Minor revisions to acknowledge without addressing the concerns would substantially reduce the impact of the work.

Signed,

Joanna Masel

Reviewer #2 (Frederic Thomas, signed review): Comments of the paper: "Transmissible cancers do not easily select for sexual reproduction", by Thomas G. Aubier, Matthias Galipaud, E. Yagmur Erten, Hanna Kokko

Few months ago, I proposed (Thomas et al. 2019) with few colleagues in a verbal paper a novel hypothesis to explain the evolution of sexual reproduction. Our hypothesis suggested that transmissible cancer cells could have played, and would still play, a major role for the evolution and maintenance of sexual reproduction in the living world. In this paper, Aubier et al. explored the conditions that are required for this hypothesis to be plausible. Their main conclusion is that the conditions for transmissible cancers to select for sex are most favorable if transmissible cancers exist in a situation with a low rate of neoplasia, a high transmission rate and an intermediate cancer-associated mortality rate in hosts with a low baseline mortality rate. Because these conditions are very particular, they conclude that transmissible cancers do not easily select for sexual reproduction.

I found this paper excellent, timely and well written. I am also glad to see that our hypothesis had stimulated other scientists, and yielded beautiful work like this one.

I suggested Minor Revisions, but in fact it is between minor and major, I think.

As authors acknowledge themselves at the end of their ms, they only considered horizontal transmission while we also suggested in Thomas et al. 2019 that vertical transmission could exert a major role, even a higher role than horizontal one. Thus, the correction I would ask, both to be fair and also to stimulate other complementary researches, is in their title. In my view, it should be something like : "Horizontal transmission of malignant cells does not easily select for sexual reproduction" (instead of Transmissible cancers do not easily select for sexual reproduction). I would REALLY appreciate if the authors could change their title accordingly.

In addition to the title problem, it is also relevant from a biological point of view. Because the the model only assumes horizontal transmission, it clearly removed a key difference between sexual and asexual reproducers in respect of transmissible cancers. I would expect rare vertical transmission when individuals reproduce sexually and almost invariable vertical transmission during asexual reproduction, which could potentially turn this whole model upside down. I also believe that implementing vertical transmission in the model is key and would not be difficult (maybe I am wrong?): an offspring would acquire the transmissible cancer of the parent(s) if it has the same genotype at the interaction loci as the parent with transmissible cancer (only in case some sort of parental care (e.g. gestation/post-embryonic care) is presumable, due to physical proximity (maybe not even applicable if we talk about early multicellular organisms)). On the other hand, asexual reproduction would result in vertical transmission, maybe with a given rate, in hydra for instance almost all buds of cancerous hydra develop cancer.

Also the model assumes no evolution of anti-cancer mechanisms, which is another key difference between sexual and asexual reproducers. It is clearly articulated in numerous articles, that sexual reproduction accelerates adaptive evolution (see e.g. Goddard 2016 Nature), so once a transmissible cancer is present in the population sexual reproduction should accelerate the development of immunity/suppression. Even in early stages of such adaptations, sexually reproducing individuals might live a little bit longer than asexuals after acquiring cancer, which might result in their slightly higher fitness. There are indications that even Devil Facial Tumor Disease can accelerate selection in immune- and tumor-related genes, which could arguably be much slower in asexually reproducing species, e.g. in the case of Devils and DFTD, selection of defence mechanisms against the cancer have rapidly evolved in response to the epidemic (4-6 generations). Surely this selection would be much slower or perhaps not existing in the case asexual reproduction. This component might be harder to implement in the model, but should be clearly indicated in the ms as a shortfall of the work I believe. Perhaps authors should acknowledge this and suggest that the results should be interpreted with caution. If the model assumes no evolution of anticancer mechanisms then is difficult to put the results in the context of our paper.

I was a bit confused with the epidemiological model and how they modelled betta. In the supp material they say that betta increases the prevalence of transmissible cancers at equilibrium but is this fully independent of mortality and birth rates? Also, the birth rate and K have no effect on the proportion of cancers or selection, but surely these parameters are in reality somehow connected? This is just a thought as I had trouble with following the model, which surpassed my knowledge.

Some other minor clarifications needed:

Line 330 -333, they claim three cases of transmissible cancers have been identified in dogs, devils and bivalves. There are more than three transmissible cancers, I think they should say transmissible cancers have been found on xx species or alter the text according to the number of transmissible cancers they are referring to.

Lines 363 - 366. More precisely, in Tasmanian devil, a new lineage of transmissible cancer has been shown to express the host's major histocompatibility complex class 1 (MHC1) molecules. … Simultaneously, however, an older lineage of transmissible cancers in Tasmanian devils avoids the immune response by downregulating the expression of the MHC1 molecules (Devil Facial Tumour 1;Caldwell and Siddle, 2017).

In my opinion, DFT2 should not be defined as a new 'lineage' of transmissible cancer as implies that it evolved from DFTD, which is not the case. The manuscript first defines the devil cancer as DFTD, but then it uses DFT1 which is the same cancer. I know this might sound pedantic, but it would be good promote consistency in the devil literature because more and more papers are mixing the cancers and using them incorrectly. Ideally, they should explain the terminology early in the manuscript and keep consistency.

Else congratulation for this nice piece of work!

Fred Thomas

Reviewer #3: In their manuscript the authors probe the theory that the evolution of sexual reproduction in multicellular organisms was driven by the emergence and spread of contagious cancers. Intuitively these cancers are similar to parasitic infection and as such the authors propose the hypothesis that the emergence and spread of these cancers initiates fluctuations in selection to drive the evolution of sexual reproduction, increasing genetic recombination. The authors initially test this hypothesis by building a population genetic model with an assumption that transmissibility is determined by a single locus defining self/non-self. This model predicts that fluctuations in selection are not significant enough to drive the emergence of sexual reproduction.

The population model is then supplemented with an epidemiological model to link the proportion of 'neocancers' and the selection coefficient. They are then able to vary a number of parameters to determine which may produce the outcome of sexual reproduction. Again, the emergence to sex is unlikely in most scenarios, with the exception of where the transmission rate is high and selection is high.

The authors propose that the genetic link between newly emerged contagious cancers (so called neocancers) and their host is tight, which reduces selection and the 'Red Queen' scenario which would favour sexual reproduction.

The paper articulates an interesting hypothesis and builds the models logically to test this hypothesis. The primary issue with the models is their simplicity, numerous assumptions are made to simplify the model, such as the presence of a single locus determining self/non-self and an assumption of no mutation in tumour and host in the population model. These assumptions are obviously fundamental to the outcomes of the tests.

Central to this is the assumption that a single genetic locus determines self/non-self. Thus, the likelihood that the cancers will transmit to 'matched' individuals is very high and explains the genetic link between the host and cancer. The authors should introduce additional loci to the genetic model and test the result. This could be done in a stepwise manner to assess the impact on selection.

Similarly, in figure 1 a three locus model and one locus model is tested (Figure 1A). There needs to be a clearer explanation of these two models and why one is favoured.

In the methods (line 96) the authors state that host and cancer cells form two large populations of constant size. This needs to be clarified, does this mean that the cancer emergence is at a rate that results in equal cell numbers between host and cancers. This would not seem likely.

Typo in line 150 - should read 'their sum is equal'.

Typo in line 170 - additional word - our

Dear Dr Aubier,

Thank you for submitting your revised Research Article entitled "Transmissible cancers and the evolution of sex under the Red Queen hypothesis" for publication in PLOS Biology. I have now obtained advice from the original reviewers and have discussed their comments with the Academic Editor.

Based on the reviews, we will probably accept this manuscript for publication, assuming that you will modify the manuscript to address the remaining points raised by the reviewers. Please also make sure to address the data and other policy-related requests noted at the end of this email.

IMPORTANT:

a) Please address the remaining requests from the reviewers, including the shared concern about your use of the Red Queen hypothesis.

b) After discussion with the Academic Editor, we think that this article would be best published as a Short Report. As your paper is already quite concise, no re-formatting is needed, but please select "Short Reports" as the article type when you re-submit. We can also do this for you.

c) We believe that all the graphs in your figures can be generated using the code which you intend to deposit in Dryad. Please can you provide a reviewer link or password so that we can check this, AND clearly cite the Dryad URL in the relevant main and supplementary Figure legends?

We expect to receive your revised manuscript within two weeks. Your revisions should address the specific points made by each reviewer. In addition to the remaining revisions and before we will be able to formally accept your manuscript and consider it "in press", we also need to ensure that your article conforms to our guidelines. A member of our team will be in touch shortly with a set of requests. As we can't proceed until these requirements are met, your swift response will help prevent delays to publication.

To submit your revision, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' to find your submission record. Your revised submission must include the following:

- a cover letter that should detail your responses to any editorial requests, if applicable

- a Response to Reviewers file that provides a detailed response to the reviewers' comments (if applicable)

- a track-changes file indicating any changes that you have made to the manuscript.

*Copyediting*

Upon acceptance of your article, your final files will be copyedited and typeset into the final PDF. While you will have an opportunity to review these files as proofs, PLOS will only permit corrections to spelling or significant scientific errors. Therefore, please take this final revision time to assess and make any remaining major changes to your manuscript.

NOTE: If Supporting Information files are included with your article, note that these are not copyedited and will be published as they are submitted. Please ensure that these files are legible and of high quality (at least 300 dpi) in an easily accessible file format. For this reason, please be aware that any references listed in an SI file will not be indexed. For more information, see our Supporting Information guidelines:

https://journals.plos.org/plosbiology/s/supporting-information

*Published Peer Review History*

Please note that you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:

https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/

*Early Version*

Please note that an uncorrected proof of your manuscript will be published online ahead of the final version, unless you opted out when submitting your manuscript. If, for any reason, you do not want an earlier version of your manuscript published online, uncheck the box. Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us as soon as possible if you or your institution is planning to press release the article.

*Protocols deposition*

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosbiology/s/submission-guidelines#loc-materials-and-methods

Please do not hesitate to contact me should you have any questions.

Sincerely,

Roli Roberts

Roland G Roberts, PhD,

Senior Editor,

rroberts@plos.org,

PLOS Biology

------------------------------------------------------------------------

REVIEWERS' COMMENTS:

Reviewer #1:

[identifies herself as Joanna Masel]

My concerns have been satisfactorily addressed. One small requested change re equating the Red Queen or parasite coevolution hypothesis with fluctuating selection dynamics is that Haldane 1949 should not be cited in support of FSD. The FSD hypothesis began I believe with the Hamilton paper cited, and I don't think it is appropriate to back-project it onto earlier papers about host-parasite coevolution more generally.

Optionally on a linguistic note, I know it is common practice to equate "Red Queen" with FSD, but I do not believe it is appropriate given the far broader definition of Red Queen in the original van Valen paleontology paper, concerning extinction rates, that I believe first introduced the term Red Queen to evolutionary biology. Unfortunately, I think the effect of this linguistic practice is to help crowd out alternative and more realistic theories of host-parasite coevolution. This is a pet peeve of mine that is not at all specific to the manuscript at hand, which has been satisfactorily revised.

Signed,

Joanna Masel

Reviewer #2:

[identifies himself as Frederic Thomas]

This version is really much improved, I love it. However, I still have two comments (the second one being really important in my opinion):

1- I think there is a relevant paper that should be mentioned in your discussion

Khosrotehrani, K. et al. (2005) Natural history of fetal cell microchimerism during and following murine pregnancy. Journal of Reproductive Immunology DOI: 10.1016/j.jri.2005.02.001

This paper in fact simulates our scenario using syngenic mice. Indeed, using syngenic mice these authors mimicked an asexual reproduction in mice, and they showed that when embryos are identical to their mother, microchimerism is significantly enhanced. They are not using cancerous cells, I agree, but this findings really supports our idea that vertical cell transmission is more difficult to control and/or to prevent when parent and offsprings are identical.

2- I still think that there is a problem in the title. You wrote:

"In this paper, we investigate whether antagonistic interactions between hosts and transmissible cancers can promote the evolution of sex under the Red Queen hypothesis. We analyze a population genetic model of fluctuating selection and complement it with an epidemiological model. The latter model builds an explicit epidemiological setting that we then use to examine the likely parameter values that the population genetic model takes. This combined use of two models allows us to evaluate how likely it is for the modelled system to find itself within a selection regime where Red Queen dynamics can favour sexual reproduction."

The basic of the assumption for our paper was that transmissible cheater/cancer cells evolved at the same time as multicellularity. Due to their clonal reproduction the multicellular host cells could not recognise the transmissible cheater/cancer cells. Consequently, sexual reproduction evolved as an adaptive trait to prevent horizontal and/or vertical transmission of cheater/cancer cells. This was a one off evolutionary adaptation and hence once it appeared no further adaptations occurred. Importantly, our scenario should therefore not be regarded as a red-queen process, which is a constant, and ongoing battle between host and pathogens.

Signed: Fred Thomas

Dear Dr Aubier,

On behalf of my colleagues and the Academic Editor, Nick H Barton, I am pleased to inform you that we will be delighted to publish your Short Reports in PLOS Biology.

The files will now enter our production system. You will receive a copyedited version of the manuscript, along with your figures for a final review. You will be given two business days to review and approve the copyedit. Then, within a week, you will receive a PDF proof of your typeset article. You will have two days to review the PDF and make any final corrections. If there is a chance that you'll be unavailable during the copy editing/proof review period, please provide us with contact details of one of the other authors whom you nominate to handle these stages on your behalf. This will ensure that any requested corrections reach the production department in time for publication.

Early Version

The version of your manuscript submitted at the copyedit stage will be posted online ahead of the final proof version, unless you have already opted out of the process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with biologypress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

Thank you again for submitting your manuscript to PLOS Biology and for your support of Open Access publishing. Please do not hesitate to contact me if I can provide any assistance during the production process.

Kind regards,

Alice Musson

Publishing Editor,

PLOS Biology

on behalf of

Roland Roberts,

Senior Editor

PLOS Biology