Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data

Charles R. Heller; Stephen V. David

doi:10.1371/journal.pone.0271136

Peer Review History

Original SubmissionJanuary 23, 2022
19 Apr 2022 Decision Letter - David S Vicario, Editor PONE-D-22-02203Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited dataPLOS ONE Dear Dr. Heller, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. This manuscript describes a powerful new technique for the analysis of neural data that has potential applications in sensory physiology. However, the presentation could be improved by a clearer description of the methods. Furthermore, the use of additional principal components (not just the first) should be explored and its possible advantages discussed, as requested by Reviewer #1. Please address the other comments by the Reviewers. Please submit your revised manuscript by Jun 03 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, David S Vicario, Ph.D. Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical. 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: “This work was supported by a National Science Foundation Graduate Research Fellowship (NSF GRFP, GVPRS0015A2) (CRH), the National Institute of Health (NIH, R01 DC0495) (SVD), Achievement Rewards for College Scientists (ARCS) Portland chapter (CRH), and by the Tartar Trust at Oregon Health and Science University (CRH).” Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: “This work was supported by a National Science Foundation Graduate Research Fellowship (NSF GRFP, GVPRS0015A2) (CRH), the National Institute of Health (NIH, R01 DC0495) (SVD), Achievement Rewards for College Scientists (ARCS) Portland chapter (CRH), and by the Tartar Trust at Oregon Health and Science University (CRH). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: No Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: In the current work, the authors proposed a novel computation approach to decode high-dimensional neural data. In this approach, the authors proposed to use the mean of different experimental conditions and the first eigenvector of the population covariance to determine the decoding axis, which has the potential to achieve better decoding performance with limited trials than the traditional PCA. The work is technically solid and interesting. A major issue is the very abbreviated description of the rationale and technical details. For example, the first section of Results described some very basic concepts of this approach. But for anyone who has not read Averbeck and Lee 2006, it would be very hard to understand. Other major concerns: 1. To show the advantage of the new method over traditional PCA, authors applied each method to neural data collected from auditory cortices. The general result did show a clear advantage of the new method over PCA. However, in this comparison, only the first two PCs were used to decode stimulus conditions that have two dimensions. In analyses of auditory neural data, the first PC of PCA usually reflects the rise and fall of the sound, while the second and third PCs contain information about stimulus conditions. Therefore, to decode stimulus conditions with two dimensions, one should at least include three PCs. If the authors can include three PCs in the comparison and still show better performance with the new approach, it will make the conclusion stronger. 2. Demixed PCA has been growing popular in recent years. It is a similar approach to the proposed method. I would appreciate it if the authors can describe the differences, or even make a comparison, between the two approaches. Minor concerns: 1. Page 4. Last paragraph. The authors should add the letter of panels in Fig 2. Same for the text related to Figure 4. 2. Figure 4a. What is the x-axis? 3. Figure 5g. Why was the performance of taPCA reduced when the trial number increased? Reviewer #2: In the manuscript " Dimensionality reduction for neural population decoding", Heller et al. report a new method for dimensionality reduction of neural population data. This approach projects high-dimensional neural activity into a two-dimensional space by capturing the variance of stimulus-evoked activity (signal axis) and the stimulus-independent trial to trial variability (noise axis) separately. It shows a significant advantage over standard principal component analysis in stimulus discriminations, especially in conditions with a fewer number of observations. The outcome is easy to interpret since it visualizes the signal and noise information separately in a 2-D space. Although the approach is limited by only working in a pairwise way and only capturing the 1st primary dimension of noise correlation variability, it is still a simple but effective method that could serve as an alternative approach to decoding analysis with fewer observations. The approach could be of interest to the field, and I would recommend the publication of it in PLOS One. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Bruno Averbeck [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0271136.r001
Revision 1
31 May 2022 Author Response We have included a document, "Response to Reviewers.pdf" that addresses all specific editor and reviewer comments. The text of the document is also pasted below, if needed. Thank you for the positive feedback on our manuscript “Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data''. We have addressed all points contained in the decision letter and outlined the details below. Editor comments / journal requirements 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. We have made all necessary changes to comply with journal requirements for formatting and file naming. These changes are reflected in the submitted documents. 2. Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical. The title of the manuscript has been updated to match the full title included in the online submission form. 3. Thank you for stating the following in the Acknowledgments Section of your manuscript: “This work was supported by a National Science Foundation Graduate Research Fellowship (NSF GRFP, GVPRS0015A2) (CRH), the National Institute of Health (NIH, R01 DC0495) (SVD), Achievement Rewards for College Scientists (ARCS) Portland chapter (CRH), and by the Tartar Trust at Oregon Health and Science University (CRH).” Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: “This work was supported by a National Science Foundation Graduate Research Fellowship (NSF GRFP, GVPRS0015A2) (CRH), the National Institute of Health (NIH, R01 DC0495) (SVD), Achievement Rewards for College Scientists (ARCS) Portland chapter (CRH), and by the Tartar Trust at Oregon Health and Science University (CRH). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.” Please include your amended statements within your cover letter; we will change the online submission form on your behalf. We have removed the funding statement from our manuscript. The funding statement above is correct. There is no need to amend it. 4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. We have included the following ethics statement in our Methods section (lines 46-49) in the revised manuscript): “All procedures were performed in accordance with the Oregon Health and Science University Institutional Animal Care and Use Committee (IACUC) and conform to standards of the Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC).” Please let us know if this needs to be changed in any way. Reviewer comments We appreciate the reviewers’ positive and thoughtful comments. We have revised the manuscript in a way that we hope addresses their concerns. A detailed response to each specific point is provided below. We have also included a marked-up copy of the manuscript that shows all changes from the previous version. Reviewer #1: In the current work, the authors proposed a novel computation approach to decode high-dimensional neural data. In this approach, the authors proposed to use the mean of different experimental conditions and the first eigenvector of the population covariance to determine the decoding axis, which has the potential to achieve better decoding performance with limited trials than the traditional PCA. The work is technically solid and interesting. A major issue is the very abbreviated description of the rationale and technical details. For example, the first section of Results described some very basic concepts of this approach. But for anyone who has not read Averbeck and Lee 2006, it would be very hard to understand. We agree that there were important points that required further clarification. In particular, a more careful treatment of the rationale behind our method was missing. For anyone not familiar with the aforementioned work, the terminology and concepts could be challenging. Therefore, we have added a new section at the beginning of the Results, “Neural population decoding and noise correlations,” in which we introduce the main principles that form the conceptual basis for our work. We define the terms “noise correlation” and “state space” and discuss neural decoding conceptually before diving more deeply into the math in the subsequent results. We also revised the remaining text in several places to ensure that we are consistent in our use of this terminology. Finally, we added a new figure (Fig. 1 in the revision), adapted from Averbeck & Lee, that graphically illustrates the conceptual basis that this work builds on. We hope these changes help to orient readers sufficiently before describing our new method. The new results section is located at lines 114-156 in the revised manuscript. Other major concerns: 1. To show the advantage of the new method over traditional PCA, authors applied each method to neural data collected from auditory cortices. The general result did show a clear advantage of the new method over PCA. However, in this comparison, only the first two PCs were used to decode stimulus conditions that have two dimensions. In analyses of auditory neural data, the first PC of PCA usually reflects the rise and fall of the sound, while the second and third PCs contain information about stimulus conditions. Therefore, to decode stimulus conditions with two dimensions, one should at least include three PCs. If the authors can include three PCs in the comparison and still show better performance with the new approach, it will make the conclusion stronger. We would like to thank the reviewer for this insightful comment and offer some additional clarification. In the analyses presented in this manuscript, we collapsed the sound evoked neural response for each stimulus into a single time bin, so the dynamics of the response were not considered in this example. We have added the following sentence to the final results section to help clarify this point: “The neural response to each stimulus was defined as the total number of spikes detected during the 300 ms stimulus duration for each neuron.” (lines 316 – 318) Therefore, adding additional PCs cannot provide any more information about the stimulus condition. We made the choice to collapse activity over time to keep our analysis as simple and general as possible. We concede that this is a simplified approach and that we are likely missing important time-varying, sound-evoked activity. However, our method can easily be adapted to treat each different time point as its own stimulus condition – in other words, a separate dDR decomposition could be performed for each time bin. This would allow one to evaluate how optimal sound decoding changes as a function of time from e.g., stimulus onset. As response dynamics were not the focus of this manuscript, per se, we did not include this analysis. 2. Demixed PCA has been growing popular in recent years. It is a similar approach to the proposed method. I would appreciate it if the authors can describe the differences, or even make a comparison, between the two approaches. This is an important point that we neglected to fully address in the original manuscript. Thank you for bringing it to our attention. There are at least two critical distinctions between our method and dPCA. First, our method is strictly focused on optimal decoding. It should be thought of as a preprocessing, dimensionality reduction, step prior to applying a method such as Linear Discriminant Analysis (LDA). By using dDR before LDA, one can mitigate overfitting issues that make application of standard LDA to single-unit population data challenging in practice. While one goal of dPCA is indeed to provide a low dimensional representation for decoding, another goal is to maintain a faithful representation of the true geometric structure of the data. This latter goal does not need to be compatible with optimal decoding; therefore, the low-D projection found with dPCA will not necessarily provide optimal decoding of experimental conditions. This trade-off is illustrated very nicely in Figure 2 of Kobak et al., 2016. Second, our method is developed for pairwise experimental conditions. That is, a different dDR projection exists for each pair of stimuli. dPCA is useful when there are more than two. In particular, dPCA can provide a useful tool for marginalization when the data spans multiple dimensions (e.g., stimulus X time X neurons or stim X behavioral state X neurons) and an interpretable, constant low-D space across all dimensions is desired. To address this point, we have added a section to the Discussion, titled “Related methods for dimensionality reduction and neural decoding,” which reviews the key differences between our method and other related approaches for dimensionality reduction and neural decoding, namely standard PCA, LDA, and dPCA. This is located at lines 418-451 in the revised text. Minor concerns: 1. Page 4. Last paragraph. The authors should add the letter of panels in Fig 2. Same for the text related to Figure 4. We have edited the text to specify figure panels in all cases where a figure is cited. These changes are reflected in the marked-up copy of our revised manuscript. 2. Figure 4a. What is the x-axis? We believe the reviewer is referring to Fig 5a (now 6a, after revision), where the x-axis label was omitted and is shared with the panel below (Fig 6b). We have added a label to the x-axis in panel a (“Noise Center Frequency”) and revised the figure legend to explicitly state this as well. 3. Figure 5g. Why was the performance of taPCA reduced when the trial number increased? Thank you for noticing this. There was a mistake in the way we were calculating our bootstrapped estimates of standard error for each sample size. Briefly, we did not correctly control how we randomly resampled our dataset for cross-validated estimates of decoding accuracy. This introduced variability between the data included in the different sample sizes, leading to a spurious apparent drop in decoding performance for taPCA at high repetition counts. This error did not affect any of the conclusions we present in the manuscript, but did make the results confusing. We have corrected this mistake and added a section to the Materials and Methods titled “Bootstrapped estimates of decoding performance for different sample sizes” (lines 91 - 112). Here we describe the bootstrapping procedure used for estimating decoding performance across different sample sizes in detail. Additionally, we have modified the statistic plotted on the y-axis in Fig. 6 panels d and g to more clearly demonstrate the benefit of dDR over PCA methods. Previously, we reported absolute decoding performance for each method in units of d’2. To more directly compare the methods, we now instead report performance of PCA methods as a fraction of dDR performance. We believe this unit-less quantity provides a more interpretable and direct illustration of the benefit of dDR when applied to real neural data. These changes are reflected in Fig. 6 of the revised manuscript. Reviewer #2: In the manuscript " Dimensionality reduction for neural population decoding", Heller et al. report a new method for dimensionality reduction of neural population data. This approach projects high-dimensional neural activity into a two-dimensional space by capturing the variance of stimulus-evoked activity (signal axis) and the stimulus-independent trial to trial variability (noise axis) separately. It shows a significant advantage over standard principal component analysis in stimulus discriminations, especially in conditions with a fewer number of observations. The outcome is easy to interpret since it visualizes the signal and noise information separately in a 2-D space. Although the approach is limited by only working in a pairwise way and only capturing the 1st primary dimension of noise correlation variability, it is still a simple but effective method that could serve as an alternative approach to decoding analysis with fewer observations. The approach could be of interest to the field, and I would recommend the publication of it in PLOS One. We thank the reviewer for these supportive comments. Attachments Attachment Submitted filename: Response to Reviewers 2.pdf https://doi.org/10.1371/journal.pone.0271136.r002
24 Jun 2022 Decision Letter - David S Vicario, Editor Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data PONE-D-22-02203R1 Dear Dr. Heller, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Thank you for your thorough response to the reviewer's comments. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, David S Vicario, Ph.D. Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The revised version is significantly improved. All reviewers’ comments were properly addressed. The authors added an extra section to describe the rationale of the work. The example with the dDR analysis was also improved and more convincing. I also appreciate that the authors compared the new approach with other similar techniques in the discussion. The article is clear and much easier to read now. I would recommend it for publication. Reviewer #2: The authors have addressed the main comments. I am happy to recommend this manuscript for publication. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Bruno B. Averbeck ******** https://doi.org/10.1371/journal.pone.0271136.r003
Formally Accepted
28 Jun 2022 Acceptance Letter - David S Vicario, Editor PONE-D-22-02203R1 Targeted dimensionality reduction enables reliable estimation of neural population coding accuracy from trial-limited data Dear Dr. Heller: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. David S Vicario Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0271136.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .