PCOMPBIOL-D-24-01559

Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning

PLOS Computational Biology

Dear Dr. Hoang,

Thank you for submitting your manuscript to PLOS Computational Biology. After careful consideration, we feel that it has merit but does not fully meet PLOS Computational Biology's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The reviewers provide a variety of constructive suggestions on how to improve the communication of your paper, particularly with respect to the interpretation of the results and the clear demarcation from your earlier studies.

Please submit your revised manuscript within 30 days Mar 24 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at ploscompbiol@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcompbiol/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Abigail Morrison

Academic Editor

PLOS Computational Biology

Lyle Graham

Section Editor

PLOS Computational Biology

Journal Requirements:

1) Please ensure that the CRediT author contributions listed for every co-author are completed accurately and in full.

At this stage, the following Authors/Authors require contributions: Huu Hoang. Please ensure that the full contributions of each author are acknowledged in the "Add/Edit/Remove Authors" section of our submission form.

The list of CRediT author contributions may be found here: https://journals.plos.org/ploscompbiol/s/authorship#loc-author-contributions

2) Please provide an Author Summary. This should appear in your manuscript between the Abstract (if applicable) and the Introduction, and should be 150-200 words long. The aim should be to make your findings accessible to a wide audience that includes both scientists and non-scientists. Sample summaries can be found on our website under Submission Guidelines:

https://journals.plos.org/ploscompbiol/s/submission-guidelines#loc-parts-of-a-submission

3) Please insert an Ethics Statement at the beginning of your Methods section, under a subheading 'Ethics Statement'. It must include:

i) The full name(s) of the Institutional Review Board(s) or Ethics Committee(s)

ii) The approval number(s), or a statement that approval was granted by the named board(s).

4) Please upload all main figures as separate Figure files in .tif or .eps format. For more information about how to convert and format your figure files please see our guidelines: 

https://journals.plos.org/ploscompbiol/s/figures

5) We notice that your supplementary Figures are included in the manuscript file. Please remove them and upload them with the file type 'Supporting Information'. Please ensure that each Supporting Information file has a legend listed in the manuscript after the references list.

6) Some material included in your submission may be copyrighted. According to PLOSu2019s copyright policy, authors who use figures or other material (e.g., graphics, clipart, maps) from another author or copyright holder must demonstrate or obtain permission to publish this material under the Creative Commons Attribution 4.0 International (CC BY 4.0) License used by PLOS journals. Please closely review the details of PLOSu2019s copyright requirements here: PLOS Licenses and Copyright. If you need to request permissions from a copyright holder, you may use PLOS's Copyright Content Permission form.

Please respond directly to this email and provide any known details concerning your material's license terms and permissions required for reuse, even if you have not yet obtained copyright permissions or are unsure of your material's copyright compatibility. Once you have responded and addressed all other outstanding technical requirements, you may resubmit your manuscript within Editorial Manager. 

Potential Copyright Issues:

i) Figure 1A. Please confirm whether you drew the images / clip-art within the figure panels by hand. If you did not draw the images, please provide (a) a link to the source of the images or icons and their license / terms of use; or (b) written permission from the copyright holder to publish the images or icons under our CC BY 4.0 license. Alternatively, you may replace the images with open source alternatives. See these open source resources you may use to replace images / clip-art:

- https://commons.wikimedia.org

- https://openclipart.org/.

7) Please amend your detailed Financial Disclosure statement. This is published with the article. It must therefore be completed in full sentences and contain the exact wording you wish to be published.

1) State what role the funders took in the study. If the funders had no role in your study, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

8) Your current Financial Disclosure states, " HH, KK and KT were supported by Grants-in-Aid for Scientific Research in Innovative Areas (17H06313) and for Transformative Research Areas (22H05160 to M.M., 22H05156 to M Kawato, and 22H05161 to KK). HH and KT were partially supported by JST ERATO (JPMJER1801, "Brain-AI hybrid"). HH, M Kawato, and KK were partially supported by the Grant Number JP21dm0307002, JP21dm0307008, and JP19dm0207080, respectively, Japan Agency for Medical Research and Development (AMED). M Kawato was partially supported by Innovative Science and Technology Initiative for Security Grant Number JP004596, Acquisition, Technology & Logistics Agency (ATLA), Japan. M Kano and KK were partially supported by Grants-in-Aid for Scientific Research (JP18H04012, JP20H05915, JP21H04785 to M.Kano and JP22H00460 to KK) from the Japan Society for the Promotion of Science (JSPS). ".

However, your funding information on the submission form indicates receiving only one fund. Please ensure that the funders and grant numbers match between the Financial Disclosure field and the Funding Information tab in your submission form. Note that the funders must be provided in the same order in both places as well.. 

Please indicate by return email the full and correct funding information for your study and confirm the order in which funding contributions should appear. Please be sure to indicate whether the funders played any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Reviewers' comments:

Reviewer's Responses to Questions

Reviewer #1: The manuscript describes a combined experimental and computational study of reinforcement learning in the cerebellum. The research integrates a Q-learning model with experimental spike detection, Tensor Component Analysis (TCA) of the neuronal activity, correlation analyses of the neuronal activity against reinforcement learning and sensorimotor control variables, and a spiking neural network model of the olivo-cerebellar system. This is excellent work and beautifully presented – I recommend publication in PLOS Computational Biology subject to minor revisions.

Minor comments:

1. In the neural network model, the lick rate is modelled as a sigmoid function of the combined spike rate of cerebellar nucleus (CN) neurons, with decreasing lick rates for increasing CN spike rates (and a maximum lick rate at a CN spike rate of zero). This differs from many other cerebellar models (e.g. Medina et al. J Neurosci. 2001) that assume movements are triggered by increased output from the CN. It would be good to discuss the neural circuitry downstream of the CN that could implement this mapping of decreased (including zero) CN output on increased lick rates.

2. Although this is already briefly mentioned in the discussion, the network model doesn’t represent a few important anatomical features, in particular, the convergence of many Purkinje cells on CN neurons. It would be good to discuss the expected effect of these features in more detail.

3. The neural network only represents TC1 and TC2, but not TC3 and TC4, although they are also (positively / negatively) correlated with the reward and licking. Would it be possible to speculate about a neural network (without additional simulations) that includes TC3 and TC4?

4. Line 155: “CSs in HIT trials (n = 3,788) were large and distributed across the entire medial hemisphere at the initial learning stage, but later on (2nd and 3rd stages), they became stronger and compartmentally focused on positive zones.” – Figure 2 A shows that positive zones (in particular, 5+ and 5a+) already stand out in the initial learning stage.

5. Figure 2 A also shows that the CS activity increases during learning (from stage 1 to stage 3) in CR trials?

6. Line 180: “We found that neurons in the lateral hemisphere were associated with Q and dQ, but not the reward R per se (Fig 2B).” – Figure 2 B indicates that the lateral zone 7+ is also strongly associated with R.

7. Line 334: “two groups of neurons, corresponding to TC1 and TC2, operating within a modular architecture where each group independently processes Go and No-go cues” - strictly speaking, only the climbing fibre input represents Go and No-go cues independently, the mossy fibre input to the TC1 and TC2 groups seems to be shared.

8. Please check the grammar of a few sentences, e.g. line 119 “As temporal evolution …”, line 576 “while the probability …”.

9. Line 394: “IO firings” --> IO spiking or IO firing.

Reviewer #2: This is a nice study from Dr Hoang and colleagues examining the information content of complex spike Ca responses across large populations of Purkinje neurons during a go-no-go task. The data have been previously published, but the study here extends the analyses employed in those previous studies to examine the whether the observed patterns of CS activity can be used in a Q-learning reinforcement learning algorithm. Further, they advance a straightforward computational model that accounts for the distinct modules they observe with CS tuning. I think the study represents a nice advance. I do not have any substantive scientific suggestions. However, I think it is important for the authors to explicitly spell out in the main text what is new and what has previously been published. Some of the figure panels are identical (or practically identical, with different color lookup tables applied) to previously published work for the same group (e.g. Hoang et al., 2023). In reading that manuscript there is significant overlap with the present study, but nevertheless the present study is distinct and worthy of separate publication. That said, a more explicit enumeration of the distinctions between the two studies will be a service to the readership.

Minor

1. In lines 156-159 I would suggest that the authors call the responses CSs instead of PSTHs.

2. In Hoang et al., 2023 the authors describe ‘error’ responsive CSs and reward-responsive CSs. I found this language helpful to classify different types of responses. Here the description of CS tuning does not use these descriptions, even for CSs that are negatively correlated with reward, or responsive to false alarms. It would be helpful to better match the descriptions in the two papers or to provide an explanation as to why the previous nomenclature no longer holds.

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: None

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: Yes:  Volker Steuber

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. If there are other versions of figure files still present in your submission file inventory at resubmission, please replace them with the PACE-processed versions.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Dear Dr. Hoang,

We are pleased to inform you that your manuscript 'Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology. 

Best regards,

Abigail Morrison

Academic Editor

PLOS Computational Biology

Lyle Graham

Section Editor

PLOS Computational Biology

***********************************************************

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The authors have addressed all of my comments.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: Yes:  Volker Steuber