Online randomised trials with children: A scoping review

Simone Lepage; Aislinn Conway; Noah Goodson; Paul Wicks; Laura Flight; Declan Devane

doi:10.1371/journal.pone.0280965

Peer Review History

Original SubmissionJanuary 10, 2023
26 Apr 2023 Decision Letter - Cho Lee Wong, Editor PONE-D-23-00748Online randomised trials with children: A scoping reviewPLOS ONE Dear Dr. Lepage, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== ACADEMIC EDITOR: Thanks for the opportunity to review this paper. This scoping review aimed to identify methods and tools have been used to create and conduct entirely online decentralized trials with children and determine knowledge gaps. This review addresses important topic in the field and may interest to researchers who work in paediatric fields. Overall it is well written. Several points were provided for considerations when designing and conducting online trials with children. I agreed with other reviewers that more elaboration on facilitators and gaps in knowledge as well as implications of this review should be added. Please submit your revised manuscript by Jun 10 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Cho Lee Wong, PhD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Acknowledgments Section of your manuscript: "This work contributes to one of the authors’ (SL) doctoral projects. SL’s PhD is funded by the College of Medicine, Nursing and Health Sciences at the University of Galway and by the Health Research Board (HRB, Ireland) through a grant to the Health Research Board-Trials Methodology Research Network (HRB-TMRN). " We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "This research was supported by the Health Research Board [TMRN-2021-001] and by the College of Medicine, Nursing and Health Sciences at the University of Galway. Funding was received by DD from the Health Research Board of Ireland: https://www.hrb.ie/ The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 3. Thank you for stating the following in the Competing Interests section: "PW is an associate editor at the Journal of Medical Internet Research and is on the editorial advisory boards of The BMJ, BMC Medicine, The Patient, and Digital Biomarkers. PW is employed by Wicks Digital Health Ltd, which has received funding from Ada Health, AstraZeneca, Biogen, Bold Health, Camoni, Compass Pathways, Coronna, EIT, Endava, Happify, HealthUnlocked, Inbeeo, Kheiron Medical, Lindus Health, MedRhythms, PatientsLikeMe, Sano Genetics, Self Care Catalysts, The Learning Corp, The Wellcome Trust, THREAD Research, VeraSci, and Woebot. NG is an employee of THREAD, a decentralized platform provider. The remaining authors, AC, DD, LF, and SL have no conflicts of interest to report. The findings and conclusions in the document are those of the authors and not necessarily those of NICE." Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: Thanks for the opportunity to review this paper. This scoping review aimed to identify methods and tools have been used to create and conduct entirely online decentralized trials with children and determine knowledge gaps. This review addresses important topic in the field and may interest to researchers who work in paediatric fields. Overall it is well written. Several points were provided for considerations when designing and conducting online trials with children. I agreed with other reviewers that more elaboration on facilitators and gaps in knowledge as well as implications of this review should be added. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: N/A ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: No ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: As scoping reviews do not aim to produce a critically appraised and synthesised result/answer to a particular question, and rather aim to provide an overview or map of the evidence, an assessment of methodological limitations or risk of bias of the evidence included within a scoping review is generally not performed (unless there is a specific requirement due to the nature of the scoping review aim). In this paper, the aim was to evaluate fully online RCTs performed on children, so that the authors could identify the methods and tools used, and to determine the gaps in knowledge, in order to help them perform their own online trial in children. The methods used were thorough, necessary, and well documented. They looked at a large number of trials, and followed the protocol meticulously. They were left with 21 studies that met with all the criteria. Exclusion of other studies was well justified. What bothers me about online trials is bias in recruitment, accuracy and validity of the information gathered, and spurious results. Did the authors find ways in which the authors of the different studies accounted for this? Also, were there ways in which the authors of the studies gauged the comprehension of the participants of the tasks set out for them? For informed consent to be valid we need the recruits to understand the implications of their participation. In addition, it would be preferable to include all studies in languages other than English, so as to be more comprehensively cover the subject of study. I understand that lack of funds precluded the authors to include other languages studies, but that does not totally justify excluding them. I agree with the authors’ conclusion that more than one person should have screened all the studies, which would have provided more valid representation of the studies. I agree with the conclusions of the review. Reviewer #2: This study presented a research question in summarising the evidence in the online RCTs in children. However, many issues should be addressed first before the manuscript can be considered for publication, especially the research problem clarification. My suggestions are as below: Introduction: 1. Please reconsider the research question and make it more clearly. What specific online RCT did the authors want to evaluate. Which type of the children did the authors want to focus on. What specific problems of children did the authors want to address. After that, please also rewrite the title and introduction. 2. As only RCTs or quasi experimental studies were included, why a scoping review was still conducted, not a systematic review or meta-analysis? Methods: 1. After reclarifying the research problem, the authors should redefine the inclusion criteria of the studies 2. How about the quality appraisal process? Please add more details of ROB Results: 1. The authors seemed to write this review as a wrong direction. Actually, for the recruitment methods, consent, data collection, compensation, loss to follow-up, and public and patient involvement (PPI) in online trials within paediatric populations, these could be summarised as the quality appraisal results, but should not be the main focus of a systematic review. The main results of the review involving RCTs should summarised the components of the target intervention, such as intervention typer, mode, frequency, and the efficacy of the interventions. 2. What facilitators and gaps did the authors find out? Discussion: 1. The authors should reconsider the significance of the findings for international scholars. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. Attachments Attachment Submitted filename: Comments.docx https://doi.org/10.1371/journal.pone.0280965.r001
Revision 1
10 May 2023 Author Response Dear Dr Wong, We would like to extend our thanks and appreciation to all the reviewers for their commentary and feedback. This letter provides our response to the reviewers’ comments to our manuscript, “Online randomised trials with children: A scoping review”. Journal Requirements: 1. The manuscript file names comply with PLOS ONE style requirements. 2. Acknowledgment Section & Funding Statement The Acknowledgements Section has been amended and no longer contains any funding information. The Funding Statement should read: “This research was supported by the Health Research Board [TMRN-2021-001] and by the College of Medicine, Nursing and Health Sciences at the University of Galway. Funding was received by DD from the Health Research Board of Ireland: https://www.hrb.ie/. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.” We have amended our cover letter appropriately. 3. Competing Interests Section Our Competing Interests Section remains the same and reads as follows: “PW is an associate editor at the Journal of Medical Internet Research and is on the editorial advisory boards of the BMJ, BMC Medicine, The Patient, and Digital Biomarkers. PW is employed by Wicks Digital Health Ltd, which has received funding from Ada Health, AstraZeneca, Biogen, Bold Health, Camoni, Compass Pathways, Coronna, EIT, Endava, Happify, HealthUnlocked, Inbeeo, Kheiron Medical, Lindus Health, MedRhythms, PatientsLikeMe, Sano Genetics, Self Care Catalysts, The Learning Corp, The Wellcome Trust, THREAD Research, VeraSci, and Woebot. NG is an employee of THREAD, a decentralised platform provider. The remaining authors, AC, DD, LF, and SL have no conflicts of interest to report. The findings and conclusions in the document are those of the authors and not necessarily those of NICE.” However, we would like the following statement added: “This does not alter our adherence to PLOS ONE policies on sharing data and materials.” Points raised by the academic reviewer: 1. “More elaboration on the facilitators and gaps in knowledge and their implications should be added.” Our response: We have added more detail in the Discussion Section of the manuscript addressing the facilitators and gaps in the field and their implications. Points raised by Reviewer #1: 1. “As scoping reviews do not aim to produce a critically appraised and synthesised result/answer to a particular question, and rather aim to provide an overview or map of the evidence, an assessment of methodological limitations or risk of bias of the evidence included within a scoping review is generally not performed (unless there is a specific requirement due to the nature of the scoping review aim).” Our response: We acknowledge and appreciate that a Risk of Bias assessment is not customary in a scoping review, nor is it necessary; however, we believe its inclusion in our review enhances the overall value and utility of our findings. By evaluating the quality of the included studies, readers gain a clearer understanding of the evidence's reliability, which in turn, supports the development of more robust research methodologies in future studies. . 2. “What bothers me about online trials is bias in recruitment, accuracy and validity of the information gathered, and spurious results. Did the authors find ways in which the authors of the different studies accounted for this? Also, were there ways in which the authors of the studies gauged the comprehension of the participants of the tasks set out for them? For informed consent to be valid we need the recruits to understand the implications of their participation.” Our response: The concerns surrounding decentralised, online trials raised by Reviewer #1 are echoed in the literature and although we did address this in the Introduction and Discussion Sections, we have amended the manuscript to elaborate on these concerns. We hope our recommendations will guide researchers so that these challenges can be met in a more rigorous and transparent manner. 3. “In addition, it would be preferable to include all studies in languages other than English, so as to be more comprehensively cover the subject of study. I understand that lack of funds precluded the authors to include other languages studies, but that does not totally justify excluding them. “ Our response: The concern of including only English-language studies is a relevant one. This is a challenge that many researchers face when working within their budgets. We acknowledge that this is a limitation of our review and introduces bias to our findings. Unfortunately, we are unable to change our resource allocation at this stage of the process. Points raised by Reviewer #2: Comments to the Author Responses Question 3 posed to the reviewers: Have the authors made all data underlying the findings in their manuscript fully available? Reviewer #2 answered “No”. Our response: All underlying data are available either in the associated supporting documentation as well as on the Open Science Framework website: http://www.doi.org/10.17605/OSF.IO/WHJXY Introduction 1. “Please reconsider the research question and make it more clearly. What specific online RCT did the authors want to evaluate. Which type of the children did the authors want to focus on. What specific problems of children did the authors want to address. After that, please also rewrite the title and introduction. 2. “As only RCTs or quasi experimental studies were included, why a scoping review was still conducted, not a systematic review or meta-analysis?” Our response: We will address points 1 and 2 together. As discussed in our Introduction Section and echoed by both the Academic Reviewer and Reviewer #1, the purpose of conducting a scoping review was to determine what the gaps in the knowledge of this field are and identify what the methods and tools were used in entirely online, decentralised trials within paediatric populations. The aim of a systematic review is to guide a decision-making process or answer a focused research question, which was not our aim with this manuscript. Our reasoning for conducting a scoping review was to inform the planning and conduction of our own online, randomised controlled trial with children. Because of this, a scoping review was more appropriate. Our primary focus was not on the outcomes of the interventions; instead, we were more interested in examining the trial methodologies. This approach allowed us to gain valuable insights into the study designs and their implications on the research process. This is also the reason a meta-analysis was not conducted. We were less interested in the results of interventions, rather the methodologies of the trials were of interest. Methods 1. “After reclarifying the research problem, the authors should redefine the inclusion criteria of the studies.” Our response: As addressed above. The inclusion and exclusion criteria were established to identify studies that were either a quasi-randomised or a randomised trial, conducted in a paediatric population, primarily employing online methods for the majority of their procedures, and published in English. 2. “How about the quality appraisal process? Please add more details of ROB.” Our response: The full RoB assessments with justifications for decisions made are available in the supporting document S7 Appendix. Results 1. “The authors seemed to write this review as a wrong direction. Actually, for the recruitment methods, consent, data collection, compensation, loss to follow-up, and public and patient involvement (PPI) in online trials within paediatric populations, these could be summarised as the quality appraisal results, but should not be the main focus of a systematic review. The main results of the review involving RCTs should summarise the components of the target intervention, such as intervention type, mode, frequency, and the efficacy of the interventions. “ Our response: As addressed above under Introduction points 1 & 2 responses. 2. “What facilitators and gaps did the authors find out?” Our response: We have amended the Discussion section to clarify facilitators and gaps in the knowledge. Discussion 1. “The authors should reconsider the significance of the findings for international scholars.” Our response: Our aim in the recommendations made in the Discussion Section is that they are applicable to researchers in most countries and geographical areas. Again, we are thankful for the feedback provided by the reviewers of this manuscript. Attachments Attachment Submitted filename: Response to Reviewers.docx https://doi.org/10.1371/journal.pone.0280965.r002
11 May 2023 Decision Letter - Cho Lee Wong, Editor Online randomised trials with children: A scoping review PONE-D-23-00748R1 Dear Simone Lepage, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Cho Lee Wong, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0280965.r003
Formally Accepted
15 May 2023 Acceptance Letter - Cho Lee Wong, Editor PONE-D-23-00748R1 Online randomised trials with children: A scoping review Dear Dr. Lepage: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Cho Lee Wong Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0280965.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .