Peer Review History

Original SubmissionJanuary 14, 2020

Attachments
Attachment
Submitted filename: KulkarniTroyerResponse to reviewers.docx
Decision Letter - Michael Smotherman, Editor

PONE-D-20-01215

Optimal spectral templates for triggered feedback experiments

PLOS ONE

Dear Dr. Troyer,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Apr 18 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Michael Smotherman

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Dear Todd, Both reviewers were very positive. One of the reviewers had a couple of questions that I suspect should be easy to address. The responses to the prior round of reviews appears thoroughly satisfactory, but we couldn't get the same reviewers from last time which requires that we go through the system one more time before a final acceptance can be made.  

Journal Requirements:

When submitting your revision, we need you to address these additional requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.plosone.org/attachments/PLOSOne_formatting_sample_main_body.pdf and http://www.plosone.org/attachments/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for including your ethics statement:

All procedures were performed in accordance with established animal care protocols approved by the University of Texas at San Antonio’s Institutional Animal Care and Use Committee (IACUC).

Please amend your current ethics statement to confirm that your named ethics committee specifically approved this study.

For additional information about PLOS ONE submissions requirements for ethics oversight of animal work, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-animal-research  

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Writing and organization of the article:

1. The organization of Results section appears to be confusing. The first subsection of Results “Template optimization using gradient descent” appears to be a mixture of methods and results.

2. In “Calculating averaged templates” in Methods section, the authors explained how the second slice was calculated as a weighted sum of the first (1/15) and the second slice (14/15). Should the second slice be 2/15 or 14/15?

3. In the caption of Fig 1., “and” is redundant in the sentence: For visual clarity, inter-syllable gaps and are not shown.

4. The authors may identify more literature that is similar to [3] and include those in the citation in the sentence: Our work is based on a popular implementation of TF…birdsong field, to demonstrate this specific field is actually popular and thus enhance the significance of this work.

Content and scientific value:

Method:

5. Instead of stretching the shorter instances and compressing longer instances to match the majority - those as long as 16 slices, will averaging only instances of 16 slices enhance or decrease the performance? One may suspect that averaging over different lengths of instances will distort the data, though intuitively only averaging one length of instances may increase the error to detect the same syllable of other lengths.

Discussion:

6. The offline success of this work may be compromised in the online process in actual experiments due to different asparagus, software, e.g., LabVIEW vs. MATLAB, and the requirement to deliver the triggered stimulus as soon as possible, i.e., balancing accuracy and efficiency. What will the detection performance be once the optimal template is applied in an experiment? Could it be the technical limitations after all to decide the hit rate and eventual delay in delivering the stimulus?

7. How long does it take to acquire an optimized template from raw recordings for one syllable? The authors may indicate this in the discussion as well.

Reviewer #2: The manuscript introduces a new method for template optimizations in operant feedback experiments that rely upon on feedback triggered by a specific set of acoustic parameters, such as a note or syllable containing a predetermined set of acoustic features. In these experiments the animals are rewarded for expressing a particular sound/syllable or altering the spectrotemporal properties of the syllables or songs. The challenge is for the automated system to rapidly and accurately detect when the animal has produced the desired output and provide the reward quickly enough to ensure reinforcement. These types of studies are common in songbird research, which is what the paper focuses on, but one can see how this would be a useful improvement for vocal learning studies in other model systems such as marmosets, and might even offer improvements for automated tracking of speech development or efficacy of speech therapies. I’m familiar with this literature and the techniques described here, but cannot offer much in the way of technical suggestions about the methods. In general, however the computational approach is clearly described and presented in a logical way, and would seem to offer a clear benefit over the other methods currently in use. The manuscript is well organized, appears clear of errors and could be published as in. I confirmed that the software package from Github appears to be sufficient and also found the supporting data repository to be helpful.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

Revision 1

We thank the reviewers for their positive comments on our manuscript. Here we address the specific concerns raised by reviewer #1.

1. 'The organization of Results section appears to be confusing. The first subsection of Results “Template optimization using gradient descent” appears to be a mixture of methods and results.'

It is often uncertain how to divide models or theoretical analysis into “methods” and “results.” We have reorganized the manuscript so that the derivation of our gradient algorithm is included in the Methods section. This section includes some example data to illustrate the method, but all the analysis of performance and group data is presented in Results. We have re-edited the beginning of the Results section to smooth the transition and present our overall approach.

2. 'In “Calculating averaged templates” in Methods section, the authors explained how the second slice was calculated as a weighted sum of the first (1/15) and the second slice (14/15). Should the second slice be 2/15 or 14/15?'

We have tried several iterations of detailing the calculations that underlie alignment by “linear stretching of time.” At this point we believe that the intuition of linear stretching is sufficiently clear, and have dropped the sentences attempting to detail the math.

3. 'In the caption of Fig 1., “and” is redundant in the sentence: For visual clarity, inter-syllable gaps and are not shown.'

Done.



4. 'The authors may identify more literature that is similar to [3] and include those in the citation in the sentence:

Our work is based on a popular implementation of TF…birdsong field, to demonstrate this specific field is actually popular and thus enhance the significance of this work.'

The references describing the use of triggered feedback in the songbird field were presented in the previous paragraph. We now re-reference the articles that use an averaging approach to constructing templates (most of the references). We retain the single reference [3] where we talk about exactly mimicking the real-time algorithm known as EvTAF, first published in [3].

5. 'Instead of stretching the shorter instances and compressing longer instances to match the majority - those as long as 16 slices, will averaging only instances of 16 slices enhance or decrease the performance? One may suspect that averaging over different lengths of instances will distort the data, though intuitively only averaging one length of instances may increase the error to detect the same syllable of other lengths.
'

We share the reviewer’s intuition that restricting the sample size may have the effect of increasing the error. Also, we remind the reviewer that even single spectral slices are averages over some length of the acoustic signal, and that that the windowing inherent in calculating a non-overlapping short time spectrum is not precisely aligned with syllable onsets or offsets. However, to check, we reran our algorithm on a several syllables using only the slices from the modal length syllables as targets. As expected, there was no significant change in the performance of the algorithm.


6. 'The offline success of this work may be compromised in the online process in actual experiments due to different asparagus, software, e.g., LabVIEW vs. MATLAB, and the requirement to deliver the triggered stimulus as soon as possible, i.e., balancing accuracy and efficiency. What will the detection performance be once the optimal template is applied in an experiment? Could it be the technical limitations after all to decide the hit rate and eventual delay in delivering the stimulus?'

We remind the reviewer that our algorithm is an offline algorithm whose sole purpose is to provide a better template to use in whatever online algorithm is being used within a real-time experiment. Any technicalities of real time implementation are shared by the original averaged template and the new optimized version. In general, simple online matching algorithms - such as the one we use - have a low computational cost and take less than 1-2 msec on a moderate speed computer. As for performance evaluations, our Matlab algorithm exactly mimics the popular real-time algorithm EvTAF, written in LabView. We have confirmed this match by running our triggering algorithm on data recorded during an EvTAF session. 


7. 'How long does it take to acquire an optimized template from raw recordings for one syllable? The authors may indicate this in the discussion as well.'

It generally takes less than 5 minutes to run the template optimization algorithm after the songs have been annotated. We have included this fact in our discussion.

Attachments
Attachment
Submitted filename: Response_to_reviewersRev.docx
Decision Letter - Michael Smotherman, Editor

Optimal spectral templates for triggered feedback experiments

PONE-D-20-01215R1

Dear Dr. Troyer,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Michael Smotherman

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Formally Accepted
Acceptance Letter - Michael Smotherman, Editor

PONE-D-20-01215R1

Optimal spectral templates for triggered feedback experiments

Dear Dr. Troyer:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Michael Smotherman

Academic Editor

PLOS ONE

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .