PONE-D-24-01120How do we learn about research ethics from published research? Developing editorial policy for emerging technologies using a scoping review and reference panelPLOS ONE

Dear Dr. Knight,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 25 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Primus Che Chi, Ph.D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Competing Interests section: 

"The research reported was instigated as a collaboration between the Journal of Learning Analytics (JLA), Australasian Journal of Educational Technology (AJET), and the British Journal of Educational Technology (BJET), by the lead author, who was a previous co-editor-in-chief of the JLA. The authors acknowledge BJET co-editor-in-chief Cathy Lewin, who joined the journal later in this process, and thus did not participate in authoring this piece.

The views expressed are those of the authors, and may not represent the views of the journals, scholarly societies, or other organisations to which they are affiliated. 

The work was begun over the course of the lead author’s sabbatical, which included periods co-located at UCL and KTH / the Swedish Digital Futures research centre, supporting direct interaction with editors based at those institutions."

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: ""This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. 

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

3. Please note that in order to use the direct billing option the corresponding author must be affiliated with the chosen institute. Please either amend your manuscript to change the affiliation or corresponding author, or email us at plosone@plos.org with a request to remove this option.

4. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

5. Please upload a copy of Figure 2, to which you refer in your text on page 38. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

6. Please ensure that you refer to Figure 2 and 3 in your text as, if accepted, production will need this reference to link the reader to the figure.

7. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 5 in your text; if accepted, production will need this reference to link the reader to the Table.

8. We notice that your supplementary [figures/tables] are included in the manuscript file. Please remove them and upload them with the file type 'Supporting Information'. Please ensure that each Supporting Information file has a legend listed in the manuscript after the references list.

9. We note that this manuscript is a systematic review or meta-analysis; our author guidelines therefore require that you use PRISMA guidance to help improve reporting quality of this type of study. Please upload copies of the completed PRISMA checklist as Supporting Information with a file name “PRISMA checklist”.

Additional Editor Comments:

Dear Authors,

We have received some detailed and excellent feedback from two reviewers. The reviewers have raised substantial issues with the manuscripts and how these issued can be addressed in the next revision of the manuscript. The most important of the issues raised are the role of AI technology within the manuscript and coherence/connectedness of the various sections of the write-up. I am confident that if these issues are carefully reviewed and addressed by the authors it would strengthen the paper and improve the chances of being recommended for publication.

Best wishes!

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review “How do we learn about research ethics from published research? Developing editorial policy for emerging technologies using a scoping review and reference panel”. The paper seeks to develop a new editorial policy for research ethics across social science. The paper reports original research, not published elsewhere, though requires additional detail to sufficiently describe the research and resulting conclusions.

My comments focus on areas in which to provide more information to better align the stated goals of the paper with the actual achievements of the paper, and to sufficiently describe the research so as to understand what is done:

• The title does not align with the paper. I was expecting to see some published research heuristics, like work published by Abel Brodeur and others, but that wasn’t there and so I think the paper could be simply retitled to better address the actual content.

• The paper gets at an idea which is an important one: we are often missing clear guidelines with respect to ethical behavior in new technologies. AI has scrambled the academic world from top to bottom and we are all struggling to catch up. Understanding a new set of guidelines is essential. Unfortunately, the AI perspective of this feels completely missing or tacked on later to the paper. AI seems to be an addition, rather than the focus. I wonder if the framing could be revised so that AI is a single element, rather than pitched as the focus. Or could it be removed and added as a dynamic considered, just as a mentioned topic. Just reading as is, things feel unbalanced and undelivered, and so more detail is needed, or other reframing is required to better align the stated contribution with the actual research. This comment is the most significant in requiring revision before the paper could be published.

• Something that I inherently do not understand, with respect to editorial polices and research is: why can an executive decision not be made, and changes implemented? If a journal sets a new policy that X action or Y requirement must be implemented for acceptance, perhaps with a date in the future for implementation, why is this not an option? Some of this article feels fussy around the edge, without letting us understand, as readers, why editors cannot just define new, better, and perhaps more strict ethical criteria. I have long assumed it is inertia, so it would be useful to motivate why things are not simply changed.

• The conversation about de facto versus de jure use of IRB was very interesting. I appreciate the perspective that IRBs and similar review boards have been the focus of journals, although we all generally acknowledge that they are insufficient. I would have liked to see a bit more of a discussion of whether these requirements have achieved anything or if we should move away from them.

• I want to express some pushback on the recommendation for international IRB – or the perception that there should be a lack of it (on pages 3 and 4). It strikes me as close to soft bigotry. Many international organizations across the globe have excellent IRBs and many countries now have national offices. This recommendation feels very out of line with the current state of IRBs.

o The two references cited 17 and 18 are from 2002 and 2015. Things have significantly changed with respect to IRBs outside of “Western” contexts. The authors should either engage with this with respect to more recent publications; better contextualize that this comment is a perception of editors; and/or remove it.

• One of the strategy foci presented in what I believe to be Table 5 on page 29 is training. Is this a realistic recommendation. Several of the foci in Table 5 gave me similar pause, but this recommendation stood out. Given the extreme constraints facing journals and editors, is training an actually realistic recommendation?

• It would be useful to understand what defines social science. At present a lot of it seems to simply be “education”. Which is fine but is not encompassing of social science. If this is the case (e.g., that the focus is on education not necessarily social science), the entire paper should be reframed.

• Ultimately, I read this paper twice through my process of reading and reviewing. On my second reading, I came away with the impression that it was several disjointed sections adhered together. This, of course, is not necessarily relevant to PLOS ONE, which wants to publish unique and well-done work – which I do think this paper is. But, if this is to be a useful piece for anyone but the authors, significant revisions are needed to make it coherent, comprehensible, and actionable.

Reviewer #2: At the beginning the article foregrounds AI. I couldn't tell at that point if AI was to be taken as an example of an emerging technology that challenges research ethics in ways of concern in publication or if, more than a passing reference, AI was to the focus of the article and your methods. AI is not mentioned again until the end. It doesn't appear that concerns about AI in particular shaped the methodology, unless as a non-methodologist I have missed something. Therefore as a reader I would have found it helpful to have more clarity about AI as one among other emerging technologies, perhaps the most pressing on at the moment, that your approach would hope could be addressed by all the parties responsible for ethical research practices in published work. This appears to be merely a framing problem that should be easily resolved.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Jonathan D Moreno

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Emerging Technologies and Research Ethics: Developing Editorial Policy Using a Scoping Review and Reference Panel

PONE-D-24-01120R1

Dear Dr. Knight,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Primus Che Chi, Ph.D

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Dear Authors,

Thank you for carefully addressing the comments raised from the initial submission. After another round of review the reviewers have recommended that the manuscript should be accepted for publication, a recommendation that I agree to. Congratulations to the entire team for the amazing work that has gone into developing the manuscript to this stage.

Best wishes,

Primus

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your thorough and thoughtful response. I appreciate you taking the time to address my remarks and to revise your manuscript.

Reviewer #2: I am satisfied with the responses to my concerns. The reframing of the manuscript meets the reservations I had about the original submission.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Jonathan D Moreno

**********