Robust and consistent measures of pattern separation based on information theory and demonstrated in the dentate gyrus

Pattern separation is a valuable computational function performed by neuronal circuits, such as the dentate gyrus, where dissimilarity between inputs is increased, reducing noise and increasing the storage capacity of downstream networks. Pattern separation is studied from both in vivo experimental and computational perspectives and, a number of different measures (such as orthogonalisation, decorrelation, or spike train distance) have been applied to quantify the process of pattern separation. However, these are known to give conclusions that can differ qualitatively depending on the choice of measure and the parameters used to calculate it. We here demonstrate that arbitrarily increasing sparsity, a noticeable feature of dentate granule cell firing and one that is believed to be key to pattern separation, typically leads to improved classical measures for pattern separation even, inappropriately, up to the point where almost all information about the inputs is lost. Standard measures therefore both cannot differentiate between pattern separation and pattern destruction, and give results that may depend on arbitrary parameter choices. We propose that techniques from information theory, in particular mutual information, transfer entropy, and redundancy, should be applied to penalise the potential for lost information (often due to increased sparsity) that is neglected by existing measures. We compare five commonly-used measures of pattern separation with three novel techniques based on information theory, showing that the latter can be applied in a principled way and provide a robust and reliable measure for comparing the pattern separation performance of different neurons and networks. We demonstrate our new measures on detailed compartmental models of individual dentate granule cells and a dentate microcircuit, and show how structural changes associated with epilepsy affect pattern separation performance. We also demonstrate how our measures of pattern separation can predict pattern completion accuracy. Overall, our measures solve a widely acknowledged problem in assessing the pattern separation of neural circuits such as the dentate gyrus, as well as the cerebellum and mushroom body. Finally we provide a publicly available toolbox allowing for easy analysis of pattern separation in spike train ensembles.


Bird et al
We would like to thank the reviewer for their careful reading of our manuscript and many helpful comments, which we address individually below.
Reviewer 2: 1) Upsilon_M as a measure of Pattern Separation (Line 256 and 291 in the marked-up manuscript): Despite the authors' response to my comments #2 and #3, I still think it is erroneous to call Upsilon_M a measure of pattern separation, as it does not directly compare measures of pattern similarity.Even if it necessarily correlates with pattern separation (which is not demonstrated in the paper), that's not what it measures.It measures efficient coding (sparse and informative).The argument that sparse information transmission is equivalent to pattern separation in a high dimensional space is a very interesting idea, but this argument is not explicitly laid out in the manuscript (only partially in the rebuttal).A citation of Ganguli and Sompolinski 2012 (which does not directly talk about pattern separation) is not enough: if the authors want to claim (and convince) that their metric is the better way to think about pattern separation, they need to make a rigorous demonstration that there is an equivalence or a correlation.
And, at the very least, instead of calling it a "measure of pattern separation", I think it would be safer to call Upsilon_M a "proxy" for pattern separation.The field of pattern separation is already ridden with confusing terminology applied differently in different context, best not to add to the confusion.
We agree that Upsilon_M is a less direct measure of pattern separation than many classical measures.We have made the following changes to the text to highlight this: • 'We note again here that sparsity on its own is not a measure of pattern separation, but can be combined with mutual information I(X,Y) to produce one.''We note again here that sparsity → on its own is not a measure of pattern separation, but can be combined with mutual information I(X,Y) to produce a useful proxy.' • 'Upsilon_M is a simple to understand and easy to apply measure of pattern separation' → 'Upsilon_M is a simple to understand and easy to apply proxy for pattern separation' 1

Response to review
Bird et al

2)
Reporting the codes and binsizes selected by the optimization procedure (response to comment 1d in previous review + see line 284 of the marked-up manuscript): I understand that this information is not easy to summarize.Still important to report here, to provide the reader with a full understanding of the data presented in the manuscript.The full info (not summarized) could be made available as a supplementary table.For a more palatable summary, one could imagine a plot of the distribution of binsizes over all analyses (one plot for inputs, one for outputs), to see if they vary widely or if we always end up in the same ballpark.And for some of the most important analyses (e.g.Fig 3 and 6), the authors could at least report the neural codes and binsizes that maximized the used metric.These are just ideas, there might be better ones, the goal being to make the optimization procedure less of a black-box to the reader.There is some interesting info there.
We are still in the process of compiling the full raw codes and sizes into .csvtables and are uploading them to the Zenodo data repository (doi 10.5281/zenodo.10233025).Unfortunately, the processing is now much slower due to saving these extra values and the full raw data output has not finished yet.We hope not to delay publication of this manuscript further and will continue uploading the remaining data as it finishes.In uncompressed form, the data tables take up several gigabytes of storage space and we do not think it would be appropriate to include this as a printed table.Summarising bin sizes can be difficult, particularly in the later analyses as the optimal codes can differ and, for example, a rate-based code with 100ms bins is not obviously comparable to a spike-timing code with 100ms bins.

Minor comment:
Line 602-4 (in marked-up manuscript), the authors added more comparison on methods used to measure pattern separation in past studies (use of a ratio vs a difference between input and output similarities).I think some terms used are too vague (it is hard to relate to the problem of ratio vs difference of pattern separation) and sometimes inaccurate: 1) not sure what is meant by "overlap" (in reference to the Stark review).Overlap is a