PONE-D-23-04580

Evidence from UK Research Ethics Committee members on what makes a good research ethics review, and what can be improved.

PLOS ONE

Dear Dr. Kolstoe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We have now received one review for your manuscript, but have been unable to secure a second.  So as not to delay the process any longer, as Academic Editor I have also undertaken a review. You will notice that I have focused on the introduction as I feel that the reviewer has clearly detailed the changes required to the other sections of the manuscript. 

Overall the manuscript needs significant revision to make it clear and understandable for a reader not an expert in the UK HRA and REC process. Related to this the discussion contains interpretation that appears to be based on expert personal reflections, rather than embedded within the relevant literature, and this needs addressing in the revisions. As highlighted by the reviewer the method section needs a major rewrite to include all the usual features of a methods section. 

Please submit your revised manuscript by May 11 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Charlotte Lennox

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure: 

    "The work described here was conducted as part of the normal duties of MS and CC as part of their employment with the Health Research Authority. SEK was seconded to the Health Research Authority to conduct this work, and was renumerated through the University of Portsmouth."

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in the Competing Interests section: 

    "MS and CC work for the UK Health Research Authority. SEK was seconded to the HRA to conduct this research, and is also chair of the Cambridgeshire and Hertfordshire Research Ethics Committee. SEK is also a trustee of the charity UKRIO."

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. 

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

5. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 

6. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1,4, 6 and 7in your text; if accepted, production will need this reference to link the reader to the Table.

Additional Editor Comments:

Introduction

Line 60 - RECS should be RECs

Line 67 - for a wider readership some more context is needed on 'Think Ethics', why was it established? What was the evidenced rationale for it? Is there a wider literature to support this, perhaps from other countries?

Line 70 - the four main findings need some additional detail as they lack clarity for example 'widespread support for alternative ethics review methods', how widespread, what alternative methods are being suggested - please add further details

Line 77 - If public contributors are not always familiar or experienced with/in the practicalities of conducting research ethics reviews, or indeed research itself - then the question is why was this public dialogue undertaken? It seems there is a level of assumed knowledge needed from the reader about the HRA.

Line 82 - ongoing projects seeking the opinion of REC members - such as? What's the rationale for focusing on this one in particular?

Line 83 -“REC members’ development days” - what are these?

Line 84 - These events were new to the HRA and their committees in 2022 - It is unclear which events are being referred to here. I am aware that training days for REC members existed even before the HRA existed, so would have assumed that a provision for REC members to come together existed under the HRA, prior to the pandemic - is this not the case? Perhaps some more detail of how the RECs operate and training etc is needed for the reader.

Line 91 - are the list of topics under discussion of relevance to the results? If not, you might want to remove this.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is an important topic area, and I am pleased to have been given the opportunity to review this manuscript.

I am in support of publication of this manuscript, subject to revision. At present the manuscript contains insufficient methodological information and the results section requires more synthesis and detail.

Please see my comments by section below.

Abstract

Line 39-37 It is unclear how ‘attending 37 annual training events’ is relevant here. Data were collected at 3 events.

Introduction

Line 49 State ‘Research Ethics Committees’ in full before using (REC).

Line 56 It would be helpful to define ‘research waste’.

Lines 60+ It would be useful here to provide an overview of the HRA review process. E.g., who are applicants, types of studies that require REC approval (as opposed to HRA only), panel members, role of the chair, use of ERF system etc. This will be common knowledge for most UK researchers who recruit via the NHS but not for those who don’t, or international readership, whose processes are likely to vary.

Material & Methods

Lines 100+ Most elements of a traditional methodology section are missing, namely headings and information related to: design, setting/participants, materials, procedure, analysis. Please reformat to ensure that all key elements of the methodology are present, clear, and replicable.

More information is required re. group facilitation and note taking. It is unclear why audio recording was not utilised so this should be considered as a limitation of the methodology.

The nature of the note taking needs for description. Exactly what was noted? Was this verbatim or summarised in the researchers’ words?

Line 125 A full description of the process of content analysis is required. I also note that this method of analysis is not supported by a reference. Which form of content analysis was used?

It is unclear how consensus themes were formed when analysis was conducted following two different procedures (NVivo/manual) – please provide more information. It is also unclear what manual means.

Results

Table 1. Best practice would be to collapse representation categories by region only, not by individual REC. For some RECs you have just 1 representative. Whilst it is unlikely that individuals’ can be identified, knowing that 1 member from a particular panel participants allows for process of elimination.

It is also unclear how these data were obtained as the methodology does not provide an overview of this data capture.

Line 134+ I note that much of the text here relates to respondents mentioning the procedures that underpin the ethical review process but not what your respondents perceived as facilitators/barriers to making ethical decisions. My assumption is that there is more context within the response data re. why the discussion is important, the importance of ‘consistency’, weighing up risk/benefit etc. I recommend revisiting the data and trying to provide a richer overview of the responses.

Line 136 Careful with use of terminology such as ‘theme’. We would expect this term to be referenced in a thematic analysis less so in a content analysis if this type.

Table 3. The theme names here provide little information without also containing a definition of the theme and/or a response example (this comment is relevant to all other theme tables.)

Line 157 ‘it was interesting’ – best practice is to avoid commentary/interpretation and state findings factually in the results section.

Line 162 The MHRA process will be unknown to most researchers. I think this highlights why it is important to provide an overview of the REC remit in the introduction, including its joint role with other agencies (e.g. MHRA, HMPPS etc.)

General comment:

As with my comment for Question 1, I imagine there is a lot more to unpack here other than respondents mentioning the same thing. What is it about these things that the respondents deemed important? E.g., ‘checking the clarity of the participant facing information was also a distinct theme’ – why? I also note that theme headings appear to overlap between Questions 1 and 2. I wonder if some further analysis is required here to move responses to the appropriate question? It is usual for respondents to provide comments to a question which are more relevant to a subsequent question.

It is accepted practice to re-group data where appropriate.

For example, I see ‘role of chair’ is a theme in Q2 but I imagine this response is more relevant to Q1.

Line 167-168 ‘a concern that committees need to stay focussed on fundamental issues, rather than being distracted by minor issues’ – this information is difficult to interpret if the reader does not understand the ethical review process (i.e. contentious ethical issues are discussed at panel and applicants receive comprehensive feedback on these and other minor issues in writing – this would be useful in the introduction).

Like 172 -174 This is presented as though it is a quotation, yet it is a researcher summary of responses, it should be clear that this is the case (this comment is relevant to all comment excerpts).

Lines 180-190 I see the ERF information is provided here which is great. I still recommend a simple description in the introduction, considering the ERF is mentioned in a previous question response.

201-202 What was/wasn’t seemed as useful about the headings?

Discussion

General comment:

There is much more detail provided re. the findings in the discussion than in the results section, where we should expect to see much of this summary (see comments about detail above). It would be useful to present the more of the discussion within the context of the literature base. My assumption is that the authors are very experienced REC members and much of the interpretation is personal reflection. It might be helpful to at least refer to REC terms of reference or policy documents when referring to process (assuming these are publicly accessible).

Line 329-333 Reference missing for these findings.

General comment:

Much of the discussion provided in the latter part of this section contains important information to aid HRA developments. However for a manuscript of this type I would expect to see application of these findings to policy for RECs in general. Are you able to apply these findings to other agencies E.g., university RECs, HMPPS, international agencies?

Line 374+ Please consider other limitations, e.g., the nature of the data collection.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-23-04580R1Evidence from UK Research Ethics Committee members on what makes a good research ethics review, and what can be improved.PLOS ONE

Dear Dr. Kolstoe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. We'd like to thank you for addressing the first round of comments in such detail.  The manuscript has been reviewed by the same reviewer(s) and they have identified some additional minor comments. As Academic Editor I do agree that these changes would benefit the manuscript, in terms of boosting the scientific rigor of the method and results.   

Please submit your revised manuscript by Jun 29 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Charlotte Lennox

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I am pleased to see that the authors have gone to great effort to address my comments and make changes to the manuscript.

I have a few further comments below which can be addressed via minor revision.

Materials and methods:

I can see great improvement here. Whilst I recognise that this data collection was not intended as research activity, there are still some details missing.

• Setting/participants – participants are not described. Who attended? How many?

• Design – the activity design remains unnamed. Perhaps ‘consultation exercise’ in appropriate.

• Procedure – more details are required re. note taking procedure.

• Analysis – I note you reference Braun and Clark (15) for content analysis however the paper referenced concerns thematic analysis. Please provide an appropriate reference.

• Analysis – more detail is required re. the content analysis procedure. Each step of grouping of content should be described.

• Analysis – this section, or the beginning of the results section should state how the findings are presented and explicitly state that the quotes included are note excerpts and not verbatim participant quotes.

Results:

Line 193-195: ‘although which aspects of risk and safety were most important was difficult to assess as notes often just stated “risk” or similar one-word answers’ – I recommend omitting this sentence and instead discussing the quality of note taking as a limitation in the discussion.

Discussion:

Line 384+:

‘It is unclear whether RECs themselves should be entirely representative of society due to the technical nature of documents under review, and also the specific skills needed by members to operate within the committee review structure’.

- I understand what you are getting at here, but I feel there is opportunity for misinterpretation. A less considered reader might surmise that you think groups with particular characteristics are less likely to have the skills required.

it might be pertinent here to reflect on professional roles of REC chair and members (clinical, academic etc.) and how these professionals are not representative of society, therefore it is perhaps unsurprising that the REC panel is not representative – then move onto praising the inclusion of lay and PPIE members.

Line 223 – a conclusion paragraph would be of great value here – with recommendations for future REC practice/HRA oversite/research into the area.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Evidence from UK Research Ethics Committee members on what makes a good research ethics review, and what can be improved.

PONE-D-23-04580R2

Dear Dr. Kolstoe,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Simon White

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: