PONE-D-25-11453Embracing Transparency: A Study of Open Science Practices Among Early Career HCI ResearchersPLOS ONE

Dear Dr. Chakravorti,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 07 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mu-Hsuan Huang

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please include additional information about your dataset and ensure that you have included a statement specifying whether the collection and analysis method complied with the terms and conditions for the source of the data.

3. In the ethics statement in the Methods, you have specified that verbal consent was obtained. Please provide additional details regarding how this consent was documented and witnessed, and state whether this was approved by the IRB.

4. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager.

5. Thank you for uploading your study's underlying data set. Unfortunately, the repository you have noted in your Data Availability statement does not qualify as an acceptable data repository according to PLOS's standards.

At this time, please upload the minimal data set necessary to replicate your study's findings to a stable, public repository (such as figshare or Dryad) and provide us with the relevant URLs, DOIs, or accession numbers that may be used to access these data. For a list of recommended repositories and additional information on PLOS standards for data deposition, please see https://journals.plos.org/plosone/s/recommended-repositories.

6. Please amend the manuscript submission data (via Edit Submission) to include author Sanjana Gautam.

7. Please amend your authorship list in your manuscript file to include author Sanjana state Gautam.

8. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 in your text; if accepted, production will need this reference to link the reader to the Table.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript provides a timely qualitative exploration of open science adoption barriers and perceptions within the HCI community. The study is well-motivated, methodologically sound, and offers actionable recommendations. However, some areas require clarification or improvement.

**Strengths:**

1. **Relevance**: Addresses a critical gap in understanding open science practices among early-career HCI researchers, aligning with broader reproducibility debates.

2. **Methodology**: Robust use of semi-structured interviews (n=18) and thematic analysis, with clear documentation (e.g., GitHub repository for protocols).

3. **Findings**: Identifies key barriers (e.g., lack of incentives, cultural resistance) and benefits (e.g., transparency, citations) with nuanced qualitative insights.

4. **Recommendations**: Practical suggestions (e.g., cultural shifts, conference mandates) are well-grounded in data.

**Weaknesses and Suggestions for Improvement:**

1. **Sample Limitations**:

- The study’s focus on U.S.-based researchers limits generalizability. Explicitly acknowledge this and recommend future cross-cultural studies.

- Consider adding demographic details (e.g., institutional types, subfields) to contextualize diversity within the sample.

2. **Self-Report Bias**:

- While noted in the abstract, emphasize how recall/social desirability biases might affect findings (e.g., overreporting openness). Suggest triangulation with behavioral data (e.g., artifact sharing rates).

3. **Thematic Analysis**:

- Clarify how inter-coder reliability was ensured during theme development. A brief description of the reconciliation process would strengthen rigor.

4. **Recommendations**:

- Expand on how to address intellectual property concerns (e.g., Creative Commons licensing examples).

- Discuss potential downsides of mandating practices (e.g., burden on qualitative researchers).

5. **Structural Notes**:

- The "Related Work" section could better integrate HCI-specific challenges (e.g., qualitative vs. quantitative transparency).

- Tables/visualizations (e.g., a summary of barriers/themes) would improve readability.

5. **Literature review**:

The manuscript should make greater use of other sources and studies that have addressed related topics. The following articles are among the relevant articles that need to be cited and used in this manuscript:

- Zarghani, Maryam; Nemati-Anaraki; Leila, Sedghi, Shahram; Noroozi Chakoli, Abdolreza; Rowhani-Farid, Anisa. (2023). "The Application of Open Science Potentials in Research Processes: A Comprehensive Literature Review". Libri. DOI: 10.1515/libri-2022-0007

- Zarghani, Maryam; Nemati-Anaraki, Leila; Sedghi, Shahram; Noroozi Chakoli, Abdolreza; Rowhani-Farid, Ansia. (2023). ”Iranian researchers’ perspective about concept and effect of open science on research publication”. BMC Health Services Research. 23, 437. DOI: 10.1186/s12913-023-09420-9

**Ethical and Data Considerations:**

- Ethics approval (IRB) and consent procedures are clearly documented.

- Data availability is commendable (GitHub), but ensure all links are functional at publication.

**Conclusion:**

This manuscript makes a valuable contribution to HCI and open science literature. With minor revisions to address limitations and clarify methods, it would be suitable for publication.

**Recommendation: **

The manuscript needs a major revision and re-peer review.

Reviewer #2: This article explores HCI researcher’s perceptions of open science practices through 18 semi-structured interviews with individuals skilled in qualitative and mixed methods. The authors state that five themes were identified: (1) knowledge of open science practices in the HCI community; (2) dynamics of peer review and open science practices; (3) potential benefits from open science; (4) barriers to adopt open science practices; and (5) incentives and recognition required to motivate. I agreed to review this article because I was excited by its focus and wanted to see whether common themes in past research were found to be similar, or contrasting, in the HCI field. Unfortunately, the paper left me feeling rather underwhelmed – the main issues are around the structure of the paper and its contents. At current, the paper does not feel that it is a finished, high-quality piece of writing – there are issues with the language, structure, and content. It also seems that it would be better placed in a HCI-specific journal rather than PLOS One which has a general readership. I now outline these in turn:

First, the Introduction is confusing with different sub-headings that do not differentiate from content that has gone before (e.g., “Related Work – Open Science Practices in HCI” follows on from a paragraph about OS practices in HCI”). It crucially excludes a lot of relevant literature about open science practices such as work that has explored barriers and challenges to O.S. through qualitative inquiry (e.g., Spitzer & Mueller (2023) and Sarafoglou et al. (2022) on barriers to preregistration). Furthermore, the authors outline the various OS practices, such as preregistration, open materials, data and code, but miss out the practice of Registered Reports (see Chambers & Tzavella, 2022) – surely this practice is applicable to HCI researchers, too? (although uptake may be lower at current, see the work of Norris et al. and Silverstein et al. 2024). Many of the references at current are from HCI/CHI conferences etc. and I must admit that the paper reads as though it was submitted to a HCI/CHI conference with PLOS One as an afterthought – this is due to the lack of integration of more general open science, seminal, papers!

Second, the Introduction needs to be better contextualise open science in the field of HCI – it appears from the themes that HCI researchers operate under different circumstances than other researchers. For example, they mention mainly presenting work at conferences, which will have different guidelines and incentives etc. than the pressures academics face in publishing their research or writing grants (I am not saying that these things don’t apply to HCI researchers, but it’s unclear from the Introduction what context and culture HCI researchers sit within). Furthermore, they mention developing prototypes which is unique to their field. Better contextualisation and discussion of these factors and potential barriers to O.S. would make for a lot stronger paper.

Third, the outline of themes in the Results section does not then follow the same order when each theme is presented. Further, sub-themes are presented in one theme that overlap with another (e.g., under the theme “Knowledge of Open Research Practices by Early Career HCI researchers” there is a sub-theme named “Experiences with Peer Review from a Reviewer’s Standpoint”, but then there is another major theme named “Dynamics of Peer Review…”). Better integration and separation of the themes is therefore required.

Finally, the Abstract and the last paragraph of the Introduction which outlines the results* paints an overall pessimistic overview of the themes and findings, focusing on the barriers mainly. However, many positive themes came out of this work which are not highlighted and explored. For example the Abstract states “Our findings highlight key barriers to the widespread adoption of data and materials sharing, and preregistration, namely: lack of clear incentives; cultural resistance…”, but what about the other themes of “knowledge of open science practices in the HCI community”, “dynamics of peer review and open science practices” and “potential benefits from open science” which paint a more optimistic picture of this field. The Abstract should be more balanced when discussing these themes, giving a broader overview. Furthermore, the overview of the findings should not just focus on these barriers, although they are important to circumvent.

*The overview of the findings should be removed from the Introduction and included as the opening paragraph of the Discussion.

Minor points:

� The terms ‘open science’ and ‘open research’ are used interchangeably throughout the article.

� Will a general authorship of PLOS One understand many of the abbreviations used throughout the article? For example, I recommend un-abbreviating HCI in the title (and also mentioning that this is U.S. researchers only), and un-abbreviating conferences so that a general readership knows what these are.

� It would be beneficial to outline in the Abstract that the 18 researchers used mixed methods and qualitative methodology. There are different challenges/barriers to O.S. between quantitative and qualitative researchers. Discussion of this could also be included in the Introduction.

� The last sentence of the Abstract (“Future studies will include…”) is confusing. Should it not be “should include”? What does “allowing more justifiable examples” mean?

� The paper talks about how to incentivise open science through, e.g. better training. A new paper by Skubera et al. (2025) would help to make this point. See recommended references below.

� The Introduction talks about ethics under open science, but this isn’t an open science practice – in other words, all researchers have to adhere to ethical practice/clearance and this has existed way before the advent of open science practices. Perhaps you can rephrase this as improving equality, diversity, and accessibility within scientific fields, which O.S. aims to do. The work of FORRT may help here: https://forrt.org/equityinos/ and https://forrt.org/os-developing-world/ etc.

� Structure – in the introduction you outline issues of replicability and reproducibility in the first paragraph, but also end this with showing increases in open data and open materials. This is confusing because these increases came about through the implementation of open science, but open science is mentioned in the next paragraph. It would be better to outline the issues of science and then the advent of open science and how this has led to positive improvements.

� “Post-doctorate” should be Post Doctoral Research Assistant, or similar, in the first instance. It doesn’t make sense as “Post-doctorate” alone.

� Did you use the seminal work of Braun and Clarke for thematic analysis? How does the work of Blanford differ, what recommendations do they make that you implement?

� One of the interpretations of themes suggests that participants suggest O.S. practices can reduce fraud (Page 10, Lines 380-385). However, the quote does not suggest this, and rather that assumption is made by the authors which goes beyond the data. I would revise this.

� “Potential benefits from open science” – this theme is not relayed in the Abstract.

� The “Enhancing Training and Education” section of the Discussion would benefit from explicit resources that researchers can use. For example, there is no discussion of how journals are implementing the JARS-QUANT and JARS-QUAL guidelines: e.g. https://apastyle.apa.org/jars/quantitative. Again, this review of the positive changes that have been implemented through O.S. would be helpful: https://royalsocietypublishing.org/doi/10.1098/rsos.241726

� Do you need to label the Discussion section as “Discussion” (rather than “Recommendation”).

� The Conclusion is very strong – this is written clearly and the same prose should be used throughout the article to ensure high-quality writing throughout.

Recommended references:

https://doi.org/10.1371/journal.pone.0281086

http://doi.org/10.1098/rsos.211997

https://doi.org/10.1111/bjop.12700

https://doi.org/10.31222/osf.io/w48yh

https://doi.org/10.1038/s41562-021-01193-7

https://doi.org/10.1098/rsos.241726

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Dr Charlotte R. Pennington

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Comments.docx

Open Science Practices Among Early-Career Human-Computer Interaction Researchers in the US

PONE-D-25-11453R1

Dear Dr. Chakravorti,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Mu-Hsuan Huang

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: All of the revisions have been conducted, and the manuscript is acceptable to me. My requested items about methodology, data analysis, conclusion, background and etc. have been meet in the revised manuscript.

Reviewer #3: In my opinion, the authors have adequately addressed the comments made in a previous round by the two previous reviewers.

I find it regrettable that the text does not consider the profound connections between Open Science, HCI, and libraries. It seems highly surprising that only subject P12 mentioned a relationship between his knowledge of Open Science and the library where he studied. In fact, I believe it would have been necessary to include a question on this subject in the interview. In this regard, for future studies, I would recommend that the authors consult the following paper:

"Amjid Khan, Abid Hussain, Muhammad Zareef; The use of human-computer interaction in libraries: a systematic literature review. Global Knowledge, Memory and Communication 6 March 2025; 74 (3-4): 864–882. https://doi.org/10.1108/GKMC-07-2022-0167."

Although the research only considers a sample taken from the US, I think it would be very interesting to conduct the same research in the future in other geographical areas in order to compare and discuss the results.

Beyond this, I would like to congratulate the authors because I believe that the research makes a valuable contribution to the scientific literature on open science and HCI, and I do not consider it necessary to make any further mandatory changes.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: Yes: Carlos Díaz-Redondo

**********