Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner’s Dilemma

Vincent Knight; Marc Harper; Nikoleta E. Glynatsi; Jonathan Gillard

doi:10.1371/journal.pone.0304641

Peer Review History

Original SubmissionDecember 8, 2023
29 Jan 2024 Decision Letter - The Anh Han, Editor PONE-D-23-41274Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner’s DilemmaPLOS ONE Dear Dr. Knight, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== ACADEMIC EDITOR: The two reviewers have provided the constructive comments. They both agreed that the paper has good merit, and made several suggestions for further improvement. Please take them carefully into account when revising the paper. ============================== Please submit your revised manuscript by Mar 14 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, The Anh Han, Ph.D. Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex. 3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: The two reviewers have provided the constructive comments. They both agreed that the paper has good merit, and made several suggestions for further improvement. Please take them carefully into account when revising the paper. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: I reviewed this manuscript in another journal before and finally gave "Accept" in their revision 1. However, the handling editor gave "Reject" because he/she thought that the presentation of the paper is problematic. I personally think this manuscript is worth publishing in PLOS ONE. However, at the same time, I think the points suggested by the handling editor at that time were also very reasonable. I just want to know if those points by the handling editor are clearly addressed in this submitted manuscript in PLOS ONE. If yes, please provide a detailed list of your revisions so I can check it. Other points I'd like to mention. It is not natural to divide the abstract into two paragraphs. It should be one paragraph. Also, there are too many paragraphs in one section on average. Could you make it lesser overall? Introduction: R, T, S, and P in the PD parameters should not be itemized. It is better to write them as a part of text. Reviewer #2: In this work, the authors address the issue of establishing and maintaining mutual cooperation in in agent-to-agent interactions thanks to the direct reciprocity. Specifically, they consider the Iterated Prisoner's Dilemma and focus on the mechanism of extorsion, which represents a specific type of zero-determinant strategies. The idea behind the extortionate strategies is that agents cooperate at a small cost with the expectation of future benefits. Most works on zero-determinant strategies evaluate their effectiveness by placing them in specific scenarios and assessing their performance either numerically or theoretically. Moreover, extortionate strategies are defined as algebraically rigid and memory-one, requiring complete knowledge of a strategy's cooperation probabilities. Thus, the main contribution consists of proposing a method able to detect extortionate behavior from the play history of an arbitrary strategy. This approach differs from previous studies that observed the effectiveness of theoretically extortionate strategies. Rather, the study empirically quantifies the extortionateness of the largest known collection of strategies. The proposed approach is validated empirically through a large tournament involving 204 strategies, showing that sophisticated strategies can recognize extortionate behavior and adapt to opponents. The study emphasizes the importance of adaptability for achieving evolutionary stability in the context of evolutionary dynamics. Authors show how extortionate strategies achieve a high number of wins, but they do generally not achieve a high score and fail to be evolutionarily stable. Nevertheless, more sophisticated and complex strategies are able to adapt to a variety of opponents and act extortionately only against weaker strategies while cooperating with like-minded strategies that are not susceptible to extortion. This adaptability can be crucial to maintaining sustained cooperation, as some of these strategies emerged naturally from evolutionary processes trained to maximize payoff in IPD tournaments and fixation in population dynamics. Overall, the work is well-structured and well-written and provides several interesting theoretical insights related to extortionate strategies. For instance, one behaviour observed from the experiments is that certain strategies can selectively behave extortionately toward some opponents while cooperating with others. Thus, while resistance to extortionate behavior is crucial for the evolution of cooperation, selectively extorting weaker opponents is possible and can be evolutionary advantageous. In addition, the authors provide both the GitHub link with the code and all the data used are open-sourced and freely accessible. In terms of presentation, both methods and results are rigorously presented and discussed. Please find below minor suggested corrections/improvement of the work: 1. Missing the reference to the section at the beginning of page 3: “In Section , the reverse problem is considered” 2. Equation (15) should be aligned to equations (13) and (14). 3. The second part of equation (24) can be a distinct equation, e.g. (25) 4. Although it is indicated in the title, Figs. 1 and 3 do not have an explicit label for the y-axis. I suggest including SSE as label of the y-axis. 5. A dot is missing in the first sentence of the subsection 3.3.2 before Figure 7. “In [24] a large data set of pairwise fixation probabilities in the Moran process is made available at [22]” 6. The quote reported at the end of section 3 could be in italics. 7. A comma is missing in the following sentence after “Rather”: “Rather more sophisticated strategies are able to adapt to a variety of opponents and act extortionately only against weaker strategies while cooperating with like-minded strategies that are not susceptible to extortion.” ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0304641.r001
Revision 1
20 Mar 2024 Author Response # Response to reviewers We would like to thank both reviewers for their comments on the manscript. We attach a point by point response to each of their points, and in the case of reviewer 1 a point by point response to each of the points raised by the editor for a previous review. We also include a marked up document showing the changes since our original submission to PLOS One. Although we not that this was done prior to changing the style of the manuscript to match the style of PLOS One: this is to simplify how the marked up document looks. Note that this marked up document does not include changes to any of the mathematical equations. ## Reviewer 1 > I reviewed this manuscript in another journal before and finally gave "Accept" in their revision 1. > However, the handling editor gave "Reject" because he/she thought that the presentation of the paper is problematic. > I personally think this manuscript is worth publishing in PLOS ONE. > However, at the same time, I think the points suggested by the handling editor at that time were also very reasonable. > I just want to know if those points by the handling editor are clearly addressed in this submitted manuscript in PLOS ONE. > If yes, please provide a detailed list of your revisions so I can check it. We agree that the comments by the handling editor at the time were reasonable. We took on their comments and made all the required changes. The comments from the area editor were: - It is not clear from Figure 1 whether sorting is made in an ascending or descending order from left to right. Readers have to guess it. This has been clarified in the caption. - Figure 1 and thereafter: the definition of "win" is not provided, and readers have to guess what it means. A definition of a win has been added to the caption. - The definition of P(DD) is not provided, so readers have difficulty in understanding Figure 2 and Table 3. A definition of P(DD) has been added to the caption. - Figure 4 presents a stationary distribution, but it is not clear how the initial condition of the replicator equation was chosen; in fact, because the replicator equation is 203=204-1 dimensional, there should be lots of equilibria in the system and the uniqueness of equilibrium is not at all guaranteed. In that case, the stationary distribution should be highly dependent on the initial condition, but the paper does not consider this problem and only cites "integration technique described in [29]". The initial distribution chosen was a uniform distribution of the strategies. This has been clarified in the text. - It is not clear how many and what sorts of explanatory variables were used for general linear models with recursive feature elimination. Moreover, Tables 4 and 5 seem like just a copy-and-paste from a statistical software, and there are no descriptions about how to read/understand them. Readers have to guess everything, which is apparently a bad sign for a scientific paper. I find this is very critical, as Tables 4 and 5 are one of the main results. Furthermore, there are lots of pieces of information that are not relevant to the discussion in the main text (for example, "Date" and "Time" are obviously not necessary. Most readers could not understand item such as "Omnibus", "Durbin-Watson", "Jarque-Bera", "Cond. No." and so on, and they are not relevant to or even cited in the main text.) The date and time have been removed and a few more details have been added to the caption. We purposely included all other summary measures of the linear regression process as good practice in research communication. If the reviewer feels strongly to remove it we will do so but would actually actively encourage all researchers to include as many measures from processes like this as possible to ensure availability and reproducibility of research. - The fixation probability kappa_1 is discussed in Section 3.3.2 (btw, x_1 in figure panels and in the legend remain uncorrected). However, there is no information about what type of Moran model was employed (e.g. how to transform payoffs to fitness, including the magnitude of selection strength), and only citations to [22] and [24] are given; readers have to look at these in order to figure out what assumptions are actually made. This is again problematic, because Figure 7 is one of the main results, based on which the authors claim that negative skew is a key to evolutionary success. Also, a justification is missing about why taking the average of (N kappa_1) over N=3 to 14 should be an appropriate way to measure evolutionary success. More explanation has been given here. - It is not clear why Eq.(23) is the condition for extortion. Eq.(23) means -beta < alpha, which means chi=(-beta)/alpha > 1 for alpha<0, but means chi=(-beta)/alpha < 1 for alpha>0. We have corrected Eq.(23). Another issue is that there are lots of grammatical unclarity throughout the manuscript, which prevents readers from smooth reading/understanding. To just list some from the first 3 pages; - Abstract, "... lack thereof of zero-determinant strategies is done by placing some zero-determinant strategy": What is the meaning of "the lack of ZD is done by placing ZD"? We have improved punctuation to hopefully clarify this sentence. - Abstract: "their extortionateness quantified" -> "their extortionateness is quantified" The wording has been fixed. - Section 1, 3rd paragraph: "opponents previous play" -> "opponent's previous play" - Section 1, 2nd last paragraph: "in to" -> "into" - Section 2, 2nd paragraph: "by elements of R^4 mapping" -> "by elements of R^4, mapping" - Below Eq.(6) "In Section, ": a section number is missing. These have all been addressed. - Below Eq.(15) "triangular (15) plane (13) in 3 dimensions (14)": This is not a reader-friendly way of citing equations. We have modified this sentence. - Below Eq.(15) "from [33] which is" -> "from [33], which is" - Section 2.2, 2nd paragraph: "Table 1 which shows" -> "Table 1, which shows" These have all been addressed. > Other points I'd like to mention. It is not natural to divide the abstract into two paragraphs. > It should be one paragraph. Also, there are too many paragraphs in one section on average. Could you make it lesser overall? This seemed like a strange request from the original editor. If the reviewer would like us to merge paragraphs we can do so. If not, we would prefer to keep the presentation as is as we feel it improves readability. ## Reviewer 2 > 1. Missing the reference to the section at the beginning of page 3: “In Section , the reverse problem is considered” This has been fixed. > 2. Equation (15) should be aligned to equations (13) and (14). The equality signs and strictly less than sign are now all aligned. > 3. The second part of equation (24) can be a distinct equation, e.g. (25) This has been done. > 4. Although it is indicated in the title, Figs. 1 and 3 do not have an explicit label for the y-axis. I suggest including SSE as label of the y-axis. These have been added. > 5. A dot is missing in the first sentence of the subsection 3.3.2 before Figure 7. > “In [24] a large data set of pairwise fixation probabilities in the Moran process is made available at [22]” This has been fixed. > 6. The quote reported at the end of section 3 could be in italics. This has been fixed. > 7. A comma is missing in the following sentence after “Rather”: “Rather more sophisticated strategies are > able to adapt to a variety of opponents and act extortionately only against weaker strategies while cooperating > with like-minded strategies that are not susceptible to extortion.” This has been fixed. Attachments Attachment Submitted filename: main.docx https://doi.org/10.1371/journal.pone.0304641.r002
16 May 2024 Decision Letter - The Anh Han, Editor Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner’s Dilemma PONE-D-23-41274R1 Dear Dr. Knight, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, The Anh Han, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) Reviewer #2: The authors have addressed all of my minor comments and have further enhanced the manuscript's readability and presentation. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** https://doi.org/10.1371/journal.pone.0304641.r003
Formally Accepted
20 May 2024 Acceptance Letter - The Anh Han, Editor PONE-D-23-41274R1 PLOS ONE Dear Dr. Knight, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. The Anh Han Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0304641.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .