Can cognitive training capitalise on near transfer effects? Limited evidence of transfer following online inhibition training in a randomised-controlled trial

David J. Harris; Mark R. Wilson; Kieran Chillingsworth; Gabriella Mitchell; Sarah Smith; Tom Arthur; Kirsty Brock; Samuel J. Vine

doi:10.1371/journal.pone.0293657

Peer Review History

Original SubmissionJanuary 5, 2023
29 Jun 2023 Decision Letter - Celia Andreu-Sánchez, Editor PONE-D-23-00368Can cognitive training capitalise on near transfer effects? Limited evidence of transfer following online inhibition training in a randomised-controlled trialPLOS ONE Dear Dr. Harris, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 13 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Celia Andreu-Sánchez Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following financial disclosure: “This work was funded by the Defence Science and Technology Laboratory via the Human Social Science Research Capability framework (HS1.030).” Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 3. We note that Figure 1 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright. We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission: a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license. We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text: “I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.” Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission. In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].” b.If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only. 4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ******** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The present study aimed to investigate whether an online inhibition training task could generate near and mid-transfer effects in the context of response inhibition tasks. Furthermore, the authors examine whether any benefits would persist over a 1-month interval. This study was pre-registered and used a randomized controlled trial design. Overall, n=73 participants were included in this study and allocated to either an inhibition training program (six training sessions of a visual search task with a singleton distractor) or a closely matched active control task (that omitted the distractor element). As a result, the authors report tentative evidence for near transfer and no proof of mid-transfer. Furthermore, there was no evidence that the magnitude of training improvement was related to transfer task performance. The study focuses on a scientific topic that interests readers of PLOS ONE. The English language is decent for being published. However, I am not a native speaker, so this should be checked elsewhere. The introduction has a straightforward structure and is well-written. The design of the study and the methods are described and conducted well and allow replication of the study. Also, the results are reported clearly and transparently. Finally, the discussion also is well-conducted. Accordingly, I only have some minor aspects that could be considered for a potential revision: - There was a relatively high drop-out rate. Did the authors consider a drop-out analysis to investigate whether any characteristics of the participants could result in the incompletion of the training? - On page 19, some parts of the manuscript are mixed up at the end of the page. - The authors did not report any limitations. Therefore, concededly, this study is well-conducted. However, were there any aspects that could be seen as a limitation and could help readers for future studies that should be mentioned? Reviewer #2: This is a well-written, rigorous study (in terms of design, in particular) of the kind that are much needed in the cognitive training field. I have a few comments that I hope will help improve the manuscript: 1. Given the variability in findings in healthy vs. clinical populations (e.g., far transfer effect results have been more promising in aging/mild cognitive impairment (Basak et al., 2020; Hill et al., 2017)), I think it would be useful from the start to clarify what your target population/target outcome is because it provides important context for your framing. Particularly given the fact that your participants are healthy younger adults; it seems to me that your goal is optimizing performance in healthy younger adults, but this only becomes clear in the last paragraph of the introduction. You should also caveat any interpretations of your finding that they may not generalize to clinical/older adult populations. 2. Related to the point above, it is not clear if this is a healthy sample. Given your very open inclusion/exclusion criteria, did you take any measure of potential neurological/psychiatric disorders? I think that including a population sample is fine, but it may be a limitation in terms of being able to understand the generalizability of your findings. 3. What is the ages of the participants? Again, this is important for interpretation and understanding the generalizability of findings, and is just generally always reported. If age can’t be reported or wasn’t collected, please state that clearly as a limitation. 4. Given that this was a (likely mostly) healthy younger population with a low-risk/non-invasive task, the retention rate seems very poor. Can you explain the poor retention rate, or is this more common for online trials? I think that performing the trial online has some benefits, but the limitations (e.g., retention rate, potentially lower participant engagement) should be clearly outlined. You should also mention that it is online at all opportunities including in the introduction. 5. Given the poor retention during training, how did you deal with missing data? I think it might be appropriate to perform an intention-to-treat analysis or something similar to see if drop-out characteristics affected results. 6. Have you considered performing an analysis such as GEE, MLM (Ma et al., 2012), or an analysis of change score controlling for baseline differences (Mattes & Roheger, 2020)? These may be more robust than an ANOVA, and the GEE and MLM can handle missing data as exists in your study. They also account for baseline differences which would make interpretation easier (may allow you to rule out catch up effects. Also, please confirm that the ANOVA you performed was repeated measures? It isn’t clear in the paper. I also think these more complex models may allow you to model the training gains analysis in a more rigorous way compared to just correlating change scores: I also think that this analysis only needs to be completed for effects that were significant in the main analysis, this might help power given the correction for multiple comparisons. 7. Given you found a grouptime interaction on a transfer variable (flanker task) in a pre-registered study with very minimal differences between the conditions in a relatively small sample, your general interpretation is slightly confusing: it seems like you are trying to downplay the result. A grouptime interaction is stronger evidence than a post-test t-test, which I would not include. Ideally, the analyses I have suggested will allow for clearer results, but in general I think this is a very promising findings that doesn’t match the tone in which it is described. References 8. Basak, C., Qin, S., & O'Connell, M. A. (2020). Differential effects of cognitive training modules in healthy aging and mild cognitive impairment: A comprehensive meta-analysis of randomized controlled trials. Psychology and aging, 35(2), 220. 9. Hill, N. T., Mowszowski, L., Naismith, S. L., Chadwick, V. L., Valenzuela, M., & Lampit, A. (2017). Computerized cognitive training in older adults with mild cognitive impairment or dementia: a systematic review and meta-analysis. American Journal of Psychiatry, 174(4), 329-340. 10. Ma, Y., Mazumdar, M., & Memtsoudis, S. G. (2012). Beyond repeated-measures analysis of variance: advanced statistical methods for the analysis of longitudinal data in anesthesia research. Regional Anesthesia & Pain Medicine, 37(1), 99-105. 11. Mattes, A., & Roheger, M. (2020). Nothing wrong about change: the adequate choice of the dependent variable and design in prediction of cognitive training success. BMC Medical Research Methodology, 20(1), 1-15. ******** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: PD Dr. Jan Christopher Cwik Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0293657.r001
Revision 1
24 Jul 2023 Author Response We wanted to thank the two expert reviewers for taking the time to appraise our work and provide helpful comments on the manuscript. We have made changes to the paper and provide point-by-point responses to each comment below. We think the changes have improved the manuscript and hopefully address any of the reviewers’ concerns. Journal requirements: 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. Response: Manuscript reformatted. 2. Thank you for stating the following financial disclosure: “This work was funded by the Defence Science and Technology Laboratory via the Human Social Science Research Capability framework (HS1.030).” Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." Response: Amended, thanks. 3. We note that Figure 1 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright. We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission: Response: We have removed the copyrighted images and replaced them with icons to avoid any reproduction issues. 4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Response: Checked, thanks. Reviewers comments Reviewer #1: The present study aimed to investigate whether an online inhibition training task could generate near and mid-transfer effects in the context of response inhibition tasks. Furthermore, the authors examine whether any benefits would persist over a 1-month interval. This study was pre-registered and used a randomized controlled trial design. Overall, n=73 participants were included in this study and allocated to either an inhibition training program (six training sessions of a visual search task with a singleton distractor) or a closely matched active control task (that omitted the distractor element). As a result, the authors report tentative evidence for near transfer and no proof of mid-transfer. Furthermore, there was no evidence that the magnitude of training improvement was related to transfer task performance. The study focuses on a scientific topic that interests readers of PLOS ONE. The English language is decent for being published. However, I am not a native speaker, so this should be checked elsewhere. The introduction has a straightforward structure and is well-written. The design of the study and the methods are described and conducted well and allow replication of the study. Also, the results are reported clearly and transparently. Finally, the discussion also is well-conducted. Accordingly, I only have some minor aspects that could be considered for a potential revision: Response: Thank you for the positive comments about the work. - There was a relatively high drop-out rate. Did the authors consider a drop-out analysis to investigate whether any characteristics of the participants could result in the incompletion of the training? Response: Thanks, we hadn’t considered this and it would be an interesting analysis to add. However, we don’t really have that much information on participant characteristics on which to base this analysis. We collected minimal demographic information (only gender) which was probably an error. We will definitely record more data on this in the future to enable this sort of analysis. - On page 19, some parts of the manuscript are mixed up at the end of the page. Response: The formatting might have gone awry when it was changed to a PDF, but it seems to just be the continuation of the footnote from page 18. It looks fine on the resubmitted version. - The authors did not report any limitations. Therefore, concededly, this study is well-conducted. However, were there any aspects that could be seen as a limitation and could help readers for future studies that should be mentioned? Response: Thanks, yes of course any study has limitations. We have added some to the discussion as suggested (lines 409-418): “The results of the current study should also be considered in the context of its relative strengths and weaknesses. The study was well-powered and pre-registered, providing greater rigour than some previous studies. As an online study we could not, however, ensure that participants performed the training and assessment sessions in a quiet distraction-free environment. Variation in their motivation, arousal levels, or environment could also have added noise to the data, reducing any potential training effects. Lastly, it is important to note that our findings are more applicable to the use of cognitive training for performance enhancement in healthy individuals than for addressing cognitive deficits in clinical or elderly populations. A limitation is that we did not explicitly screen for neurologic/psychiatric disorders, or collect detailed demographic information, but as the sample consisted of university students this was a predominantly young and healthy sample.” Reviewer #2: This is a well-written, rigorous study (in terms of design, in particular) of the kind that are much needed in the cognitive training field. I have a few comments that I hope will help improve the manuscript: 1. Given the variability in findings in healthy vs. clinical populations (e.g., far transfer effect results have been more promising in aging/mild cognitive impairment (Basak et al., 2020; Hill et al., 2017)), I think it would be useful from the start to clarify what your target population/target outcome is because it provides important context for your framing. Particularly given the fact that your participants are healthy younger adults; it seems to me that your goal is optimizing performance in healthy younger adults, but this only becomes clear in the last paragraph of the introduction. You should also caveat any interpretations of your finding that they may not generalize to clinical/older adult populations. Response: Thanks, we have added an earlier mention in the introduction that our focus here is on performance optimisation rather than addressing cognitive impairment (see line 7) as well as noting that the results don’t apply to clinical or older populations (line 414-416). 2. Related to the point above, it is not clear if this is a healthy sample. Given your very open inclusion/exclusion criteria, did you take any measure of potential neurological/psychiatric disorders? I think that including a population sample is fine, but it may be a limitation in terms of being able to understand the generalizability of your findings. Response: We did not ask for this information, but yes this would have been useful to check. Given that the population was university students we have a pretty good idea that this was a broadly young and healthy population. We have added a note on this (line 416-418). 3. What is the ages of the participants? Again, this is important for interpretation and understanding the generalizability of findings, and is just generally always reported. If age can’t be reported or wasn’t collected, please state that clearly as a limitation. Response: Added, thanks. 4. Given that this was a (likely mostly) healthy younger population with a low-risk/non-invasive task, the retention rate seems very poor. Can you explain the poor retention rate, or is this more common for online trials? I think that performing the trial online has some benefits, but the limitations (e.g., retention rate, potentially lower participant engagement) should be clearly outlined. You should also mention that it is online at all opportunities including in the introduction. Response: It is hard to pinpoint the reason for the drop out. Six sessions is quite a lot and the online nature means it is just very easy for people to decide that they don’t want to do it any more. Most of the drop out occurred after the first session – if people logged on for the second session they generally made it to the end. We have added some extra mentions of the online data collection as suggested. 5. Given the poor retention during training, how did you deal with missing data? I think it might be appropriate to perform an intention-to-treat analysis or something similar to see if drop-out characteristics affected results. Response: Thanks, we thought this was a nice idea and we looked into how to do it. However, the intention to treat analysis only works if the participants returned for the post test. But there was only a single participant that didn’t complete all the training but still did the post-test. Essentially, if people got bored and stopped the training, they didn’t return so there’s no data to look at. 6. Have you considered performing an analysis such as GEE, MLM (Ma et al., 2012), or an analysis of change score controlling for baseline differences (Mattes & Roheger, 2020)? These may be more robust than an ANOVA, and the GEE and MLM can handle missing data as exists in your study. They also account for baseline differences which would make interpretation easier (may allow you to rule out catch up effects. Also, please confirm that the ANOVA you performed was repeated measures? It isn’t clear in the paper. I also think these more complex models may allow you to model the training gains analysis in a more rigorous way compared to just correlating change scores: I also think that this analysis only needs to be completed for effects that were significant in the main analysis, this might help power given the correction for multiple comparisons. Response: We did initially consider running ANCOVAs for the main analyses with baseline scores as a covariate. We opted for repeated measures ANOVA in the pre-registration plan so we believe we should to stick to this analysis approach for the main paper. In response to this suggestion (and because there were some baseline differences between the groups) we have also run ANCOVA versions of the main analyses and placed them in a supplementary file. These analyses reinforce that there was no benefit of being in the training group. In fact the only group-level effects that were significant in this analysis indicated that the training group had more ‘false alarm’ responses and a more liberal response bias. 7. Given you found a grouptime interaction on a transfer variable (flanker task) in a pre-registered study with very minimal differences between the conditions in a relatively small sample, your general interpretation is slightly confusing: it seems like you are trying to downplay the result. A grouptime interaction is stronger evidence than a post-test t-test, which I would not include. Ideally, the analyses I have suggested will allow for clearer results, but in general I think this is a very promising findings that doesn’t match the tone in which it is described. Response: We have a slightly different interpretation of this result. Our initial interpretation was that the interaction effect could be due to the baseline differences in the flanker scores and therefore just an artefact. We were therefore wary to talking it up too much. The additional tests that you suggested have been useful in this regard, because when controlling for baseline differences there was no group effect. While we take the point that the interaction should not be ignored, there was no difference between the groups at post-training time point. In the context of testing the success of a training intervention this seems to be the stronger indicator – those in the training group were not better off than those in the control group. We have re-read the conclusion and have reworded in a couple places because we realise that we probably dismissed the interaction too easily, but we have stuck with our original interpretation that this interaction was probably a result of the baseline differences (see lines 360-377). Attachments Attachment Submitted filename: response to reviewers.docx https://doi.org/10.1371/journal.pone.0293657.r002
4 Sep 2023 Decision Letter - Celia Andreu-Sánchez, Editor PONE-D-23-00368R1Can cognitive training capitalise on near transfer effects? Limited evidence of transfer following online inhibition training in a randomised-controlled trialPLOS ONE Dear Dr. Harris, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please approach the comments made by reviewer 2. Please submit your revised manuscript by Oct 19 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Celia Andreu-Sánchez Academic Editor PLOS ONE [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: (No Response) ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: No ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ******** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors revised the manuscript very carefully and took all of my recommendations into consideration. From my point of view, the paper is of good quality and could be accepted for publication. Reviewer #2: I have a few responses to the revisions: Thank you for the changes to the introduction and discussion, I think they improve the paper significantly. 1. I think you should add to the limitations that it had a high drop-out rate and you need to find ways to mitigate that in future work if you want to really understand what is happening. 2. I agree with the first reviewer that at least a drop-out analysis should be performed. Even if you don’t have significant demographic info, you can still see if the participants that dropped out had significant differences on the variables you have measured (pre-test scores, gender, etc.). 3. I agree with the authors that basic intent to treat analyses would not help in this case, but there are versions that work with participants that drop out and do not complete post-test. See this paper for example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6022256/. I think it is fine if you don’t want to do this analysis, just wanted to provide some information. 4. I still disagree with the framing of the near transfer finding. Interpreting post-training differences is unhelpful because what you are really interested in is change from pre- to post-test. If you have differences at pre-test then post-test differences are even less useful. Sometimes, post-test t-tests are used because if randomization works correctly you can theoretically say that only post-test differences matter as pre-test differences are controlled by randomization, but as in your case randomization often doesn’t work perfectly (although your baseline differences are non-significant). In your study you have a grouptime interaction and significant improvement in the active but not control group. This is a positive finding. I agree with carefully interpreting it given the other analyses, but it’s still evidence for near transfer. This sentence: “Our initial interpretation was that the interaction effect could be due to the baseline differences in the flanker scores and therefore just an artefact” in particular suggests a misunderstanding of the goals of this sort of analysis. If you had baseline differences that were erased at post-test, this could either be regression to the mean (which you mention as an option) or it could be a real intervention effect that is being masked at post-test due to real baseline differences. I recommend against post-test t-tests as meaningful analysis in an intervention design and would remove them. Overall, I don’t think this requires a huge change in the framing (it is a very tentative positive finding given the other results), but I do find it strange that you hypothesized a positive result, got a positive result, but are framing it as a negative result (not a caveated positive result). 5. In the retention analysis, it is common to do pre-test vs. follow-up not post-test vs. follow-up. You want to know if there are differences at follow-up compared to baseline, not if differences emerged after post-test (which is unlikely). I would recommend repeating the retention analysis using pre-test. ******* 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Adam Turnbull ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0293657.r003
Revision 2
12 Oct 2023 Author Response Response: We would like to thank the reviewers for taking the time to re-evaluate the manuscript. The additional queries have been really useful and we have made some substantial changes/additions to the paper. Reviewer 2 Thank you for the changes to the introduction and discussion, I think they improve the paper significantly. 1. I think you should add to the limitations that it had a high drop-out rate and you need to find ways to mitigate that in future work if you want to really understand what is happening. Response: Yes, thanks for this suggestion. Added to the discussion (see lines 385-391). 2. I agree with the first reviewer that at least a drop-out analysis should be performed. Even if you don’t have significant demographic info, you can still see if the participants that dropped out had significant differences on the variables you have measured (pre-test scores, gender, etc.). Response: Thanks, we have added this analysis in a supplementary file and refer to it in the discussion where we discuss the dropout rate. We compared the flanker performance between those that withdrew and those that stayed in the study. There was no indication that baseline cognitive performance was different between those who dropped out and those who remained in the study. There was also no indication that one gender dropped out disproportionately. 3. I agree with the authors that basic intent to treat analyses would not help in this case, but there are versions that work with participants that drop out and do not complete post-test. See this paper for example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6022256/. I think it is fine if you don’t want to do this analysis, just wanted to provide some information. Response: Thanks this looks really useful. Hopefully the analysis we have done provides some indication that the withdrawals did not bias the sample and we will consider this method in future trials. 4. I still disagree with the framing of the near transfer finding. Interpreting post-training differences is unhelpful because what you are really interested in is change from pre- to post-test. If you have differences at pre-test then post-test differences are even less useful. Sometimes, post-test t-tests are used because if randomization works correctly you can theoretically say that only post-test differences matter as pre-test differences are controlled by randomization, but as in your case randomization often doesn’t work perfectly (although your baseline differences are non-significant). In your study you have a grouptime interaction and significant improvement in the active but not control group. This is a positive finding. I agree with carefully interpreting it given the other analyses, but it’s still evidence for near transfer. This sentence: “Our initial interpretation was that the interaction effect could be due to the baseline differences in the flanker scores and therefore just an artefact” in particular suggests a misunderstanding of the goals of this sort of analysis. If you had baseline differences that were erased at post-test, this could either be regression to the mean (which you mention as an option) or it could be a real intervention effect that is being masked at post-test due to real baseline differences. I recommend against post-test t-tests as meaningful analysis in an intervention design and would remove them. Overall, I don’t think this requires a huge change in the framing (it is a very tentative positive finding given the other results), but I do find it strange that you hypothesized a positive result, got a positive result, but are framing it as a negative result (not a caveated positive result). Response: Thanks, you make a good argument that we have been overly conservative! We have reworded some sections in the discussion (lines 330-342; 403) to frame this as a positive result with some caveats, rather than dismiss it as a negative result. We have made a small change to the abstract as well to reflect this. 5. In the retention analysis, it is common to do pre-test vs. follow-up not post-test vs. follow-up. You want to know if there are differences at follow-up compared to baseline, not if differences emerged after post-test (which is unlikely). I would recommend repeating the retention analysis using pre-test. Response: Thank you, yes this approach makes sense. We have repeated the retention analyses comparing to baseline instead (lines 283-308). We think the other analyses still have some value so we have added them as an online supplementary file. The new analyses don’t alter any of the conclusions. Attachments Attachment Submitted filename: Rebuttal.docx* https://doi.org/10.1371/journal.pone.0293657.r004
18 Oct 2023 Decision Letter - Celia Andreu-Sánchez, Editor Can cognitive training capitalise on near transfer effects? Limited evidence of transfer following online inhibition training in a randomised-controlled trial PONE-D-23-00368R2 Dear Dr. Harris, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Celia Andreu-Sánchez Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: Thank you for addressing my comments. I think that the manuscript is much improved and should be published. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: Yes: Adam Turnbull ******** https://doi.org/10.1371/journal.pone.0293657.r005
Formally Accepted
23 Oct 2023 Acceptance Letter - Celia Andreu-Sánchez, Editor PONE-D-23-00368R2 Can cognitive training capitalise on near transfer effects? Limited evidence of transfer following online inhibition training in a randomised-controlled trial Dear Dr. Harris: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Celia Andreu-Sánchez Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0293657.r006

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .