PONE-D-24-24617Exploring the Potential of Large language models for integration into an academic Statistical Consulting Service – The EXPOLS Study ProtocolPLOS ONE

Dear Dr. Fichtner,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 30 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Bekalu Tadesse Moges

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that funding information should not appear in any section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript.

3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. 

4. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 3 in your text; if accepted, production will need this reference to link the reader to the Table.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 

6. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Author(s), You need to consider improving the quality of the protocol, as suggested by the reviewers.

1. It is rationalized that training material to integrate LLMs into statistical results is needed. However, the second research question related to the development of such material is formulated. I think the material needs to be evaluated, based on the data collected, in terms of its quality and relevance for the advisee and consulting jobs. Thus, I suggest this very question needs revision.

2. if the project needs mixed-methods study, it is better to choose or develop a mixed methods design as appropraite, that fits into your study conceptualization. the mixed methods should allow genuine integration of both qualitative and quantitative data (Teddlie and Tashakorri, 2009, Creswell, 2020)

3. Why is the Mann-Whitney U test selected? it is a non parametric equivalent to the independent samples t-test, used when data doe not meet the assumptions of parametric test. please look into it. (Note that, If you argue otherwise to the comments, they should be convincing, and you are welcome)

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Major Points of Review

1. The title is interesting and ideal, make it summarize the specific

Part 1: Introduction

1. The introduction does not clearly articulate the problems of data analysis or the rationale for this specific study regarding the Potential of Large language models for integration into an academic Statistical Consulting Service. For example, it should explain the problem of data quality, the problem of statistical methods, and the gap of statistical software or another specific purpose. The novelty of large language models should be articulated, highlighting any advancements.

Part 2: The integration of an LLM into statistical consulting.

1. The integration of LLM into statistical consulting is not well explained for the pre-session preparation, data analysis guidance, and real-time assistance as well as assessing integration.

2. Some of the research questions are not measurable. For instance, how to measure, efficiency, utility, and user satisfaction?

Study Design

1. Why incorporate qualitative methods, b/c more of such types of scenarios are more appropriate with quantitative methods if it is possible, please clearly articulate the reason for why.

2. The qualitative and quantitative methods are different aspects, so how do design at a time?

3. Is the given study population representative?

Part 3: Statistical methods

1. Qualitative and quantitative analysis is much more limited, regarding data type, purpose focuses, and others will not provide.

Part 4: Novelty and Contribution

The paper claims to contribute to improving the LLM. The novelty of this contribution is not fully established. The manuscript should better situate its work within the existing literature on LLM, explaining how it advances the state-of-the-art.

Minor Points of Review:

1. Clarity and Readability: the manuscript contains some typographical errors that need corrections for better readability. The structure of the manuscript could be improved by clearly delineating the sections on introductions, methods, and discussions.

Subheadings and bullet points can help in organizing the content more logically.

Conclusion: The study addresses an important issue for LLMs in supporting statistical consulting. Therefore, the manuscript requires substantial revisions. Furthermore, the novelty and contribution of the work are not established. Addressing the points raised above will be essential for improving the quality and impact of the manuscript.

Reviewer #2: The authors did an excellent job, and I have great appreciation for their dedication and hard work. Their commitment and effort are truly commendable

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Wudneh Ketema Moges

Reviewer #2: Yes: Teklebirhan Abraha Gebrehiwot

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: Commented.docx

Exploring the Potential of Large language models for integration into an academic Statistical Consulting Service: The EXPOLS Study Protocol

PONE-D-24-24617R1

Dear Dr. Fichtner,

We’re pleased to inform you that your protocol has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter, and your protocol will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible—no later than 48 hours after receiving the formal acceptance. Your protocol will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Bekalu Tadesse Moges

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: