PONE-D-23-36714Fostering Human Learning in Sequential Decision-Making: Understanding the Role of Evaluative FeedbackPLOS ONE

Dear Dr. Gupta,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Editorial Comments:

The manuscript presents an intriguing integration of methodologies traditionally not combined within a single study. It has been observed that reviewer opinions diverge considerably; however, a closer examination reveals a consensus regarding the fundamental critiques. Specifically, the manuscript draws upon methodologies from distinct academic disciplines, a factor which, when evaluated within the confines of these individual fields, might typically warrant a major revision or outright rejection. This interdisciplinary approach, while innovative, poses significant challenges for evaluation, given that most reviewers possess expertise in a singular discipline.

Despite these challenges, the interdisciplinary methodology employed by the authors holds substantial value and represents a meaningful contribution to the field. To enhance the manuscript's impact and accessibility, it is imperative for the authors to revise the manuscript in a manner that renders it comprehensible to a broader audience. Additionally, there is a substantial need for the improvement of the methods employed in the study.

In light of these considerations, the editorial decision necessitates a major revision. The authors are encouraged to undertake a comprehensive revision that addresses the aforementioned concerns, with a particular focus on making the manuscript more accessible to readers from various disciplinary backgrounds and enhancing the rigor and clarity of the methodologies employed.

==============================

Please submit your revised manuscript by Apr 28 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Rei Akaishi

Academic Editor

PLOS ONE

Journal requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. Note from Emily Chenette, Editor in Chief of PLOS ONE, and Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS: Did you know that depositing data in a repository is associated with up to a 25% citation advantage (https://doi.org/10.1371/journal.pone.0230416)? If you’ve not already done so, consider depositing your raw data in a repository to ensure your work is read, appreciated and cited by the largest possible audience. You’ll also earn an Accessible Data icon on your published paper if you deposit your data in any participating repository (https://plos.org/open-science/open-data/#accessible-data).

4. Thank you for stating the following financial disclosure: 

 [This work has been supported in part by the NSF awards IIS-1734272 and ECCS-2024649. The analysis portion of this work is supported in part by the ONR award N00014-22-1-2813].  

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please respond by return e-mail so that we can amend your financial disclosure and competing interests on your behalf.

5. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

[This work has been supported in part by the NSF awards IIS-1734272 and ECCS-2024649. The analysis portion of this work is supported in part by the ONR award N00014-22-1-2813. ]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

 [This work has been supported in part by the NSF awards IIS-1734272 and ECCS-2024649. The analysis portion of this work is supported in part by the ONR award N00014-22-1-2813.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

6. In the online submission form, you indicated that your data will be submitted to a repository upon acceptance.  We strongly recommend all authors deposit their data before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire minimal  dataset will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. 

Additional Editor Comments:

The manuscript presents an intriguing integration of methodologies traditionally not combined within a single study. It has been observed that reviewer opinions diverge considerably; however, a closer examination reveals a consensus regarding the fundamental critiques. Specifically, the manuscript draws upon methodologies from distinct academic disciplines, a factor which, when evaluated within the confines of these individual fields, might typically warrant a major revision or outright rejection. This interdisciplinary approach, while innovative, poses significant challenges for evaluation, given that most reviewers possess expertise in a singular discipline.

Despite these challenges, the interdisciplinary methodology employed by the authors holds substantial value and represents a meaningful contribution to the field. To enhance the manuscript's impact and accessibility, it is imperative for the authors to revise the manuscript in a manner that renders it comprehensible to a broader audience. Additionally, there is a substantial need for the improvement of the methods employed in the study.

In light of these considerations, the editorial decision necessitates a major revision. The authors are encouraged to undertake a comprehensive revision that addresses the aforementioned concerns, with a particular focus on making the manuscript more accessible to readers from various disciplinary backgrounds and enhancing the rigor and clarity of the methodologies employed.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper connects RL to education. I've personally been looking at connecting LLMs to education and so I enjoyed seeing this paper make these connections. From my conversations, this paper addresses a "hot topic" of mathematical thinking in education.

The statistics and experiments are clearly discussed. Personally I like using Bonferoni like conditions for the model selection (say RIC, or unbiased estimates of risk) but the methods used are fine for making the point for the paper.

Reviewer #2: Thanks for inviting me to comment on the manuscript “Fostering Human Learning in Sequential Decision-Making: Understanding the Role of Evaluative Feedback“. I am an expert in human decision making in the context of agent/individual based simulation ecological and economic models. I am not familiar with the “Tower of Hanoi” game and its substantial literature. I am also not familiar “the application of maximum entropy IRL for learning human rewards”. Therefore, I will not comment on the “maximum entropy IRL” part. I think authors should refer to “maximum entropy IRL” already in the abstract.

Summary of the parts that I address in my review: The authors study the “influence of evaluative feedback on human decision-making in sequential tasks.”. Therefore, participants solved the “Tower of Hanoi” puzzle in “five separate experiments, each featuring a different type of feedback.” “Each participant first solved a 4-disk ToH task ten times (training task) and then a 5-disk ToH task five times (transfer task) to evaluate their skill transfer to a more challenging task.”

I think the experiment could be better explained. Most of my comments below are about the experiment. Maybe a screenshot in the appendix would help to have a better intuition of the experiment. Was there a time constrain? Was time measured at all? And if so did it correlate with the success?

The feedback that participants received while playing is often called AI driven – is that appropriate for the given example? I have not really thought about it, but to me contrasting the state of the player with the optimal path is a simple calculation? I may have missed something completely here. If so the authors may want to make some less technical statements for the broad reader ship of plos one. Maybe my school of thought is also a bit restrictive identifying AI mainly with deep neural networks.

Is the sample of participants unbiased given the entry conditions: "completed a minimum of 500 prior studies and maintaining a

98% approval rate".

The authors provide no discussion. Maybe this is not to be expected in the specific research domain? Or is the discussion merged in the other parts – please comment or revise.

Specific comments:

Please explain abbreviations at their first occurrence, e.g. STEM, IoT (internet of things), RL agent…

The abstract should contain the key results of the study.

“3 Human experiments”:

Please provide numbers for m_min and m_max to allow intuition for the condition the participants have been. Did participants know the restriction in the number of allowed moves.

Is the score S visible for the participants?

The range of S is different for each experiment?

Line: 246 – how is “value of the state” defined – maybe it has been stated before, but it would be helpful to read it here again.

“3.2 Methods”: That means N = 160 and 32 participants for each experiment? Or were participants spread heterogeneously between experiments?

Lines 289 – o.k. here is part of the answer given by the authors “we collect the data of 20 participants each for the 5 set of experiments” – what happened with the remaining 60 participants?

“Results” – “percentage score” is not computed using the scores S from section “3 Human experiments” – “percentage score” may have values between 100 and ~12 assuming m_min = 15 and m_used <=m_allowed = 1.5 * m_min… I guess S_percentage is set to zero again if the participants fail to solve the puzzle? Still the visualisation by boxplots is not ideal, because they suggest that the response is continuos.

Figure 6: The authors should explain the color coding in the caption or add a label to the legend.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Fostering Human Learning in Sequential Decision-Making: Understanding the Role of Evaluative Feedback

PONE-D-23-36714R1

Dear Dr. Gupta,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Rei Akaishi

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All the issues raised by reviewers have been addressed. The careful and detailed nature of these revisions has significantly improved the manuscript, making it ready for publication.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I'm happy with this version and I have no further comments. I didn't have any major complaints on the first round, so this revision mostly addresses the other reviewer and editor's comments.

Reviewer #2: Thanks for allowing me to have a look at the revision of the manuscript “Fostering human learning in sequential decision-making: Understanding the role of evaluative feedback”. Reading through the manuscript again and the comments by the authors and the editor was really insightful. All my comments have been treated appropriately. I had a very quick look through the repository and was wondering whether the authors want to state in the main text that matlab has been used for the analysis. Thanks to everyone involved in this process.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********