Reviewer #1's Review (Michael Breakspear, University of Sydney & The Brain Dynamics Centre)

“This manuscript reports the comparative performance of four different data analysis methods used to characterize and sub-classify ensembles of neurons (both real and simulated) firing in response to a variety of experimental manipulations. The authors conclude that those methods which rely upon dimension reduction (i.e. subspace projection) outperform those which do not (MGD and ANN), and furthermore a supervised subspace projector (MDA) outperforms an unsupervised one (PCA), at least when the underlying system is known to contain elements belonging to distinct classesAs a non-statistician, I have focused on the overall impact of the manuscript and its potential neuroscience applications rather than the accuracy of the statistical techniques.

Overall the manuscript is well written, nicely complimented by the illustrations, informative and relevant. In particular, the authors are able to make some interesting connections between the performance of the techniques and the properties of the underlying system that seem well supported by their analysis. In particular, they use observations regarding the properties of the ranked dimensions to discriminate classes to infer possible hierarchical classes of neurons. I also found the observation of a plateau in the performance of the MDA technique as the signal-noise ratio was reduced in the real versus simulated data of interest, although only limited discussion was given to this point. Some suggestions are outlined below.

1. A central point of the manuscript involves performing dimension reduction in the familiar setting where the dimensionality of the data greatly outweighs the number of samples/trials available. The experimental data provides a single such example. The use of simulated data would allow a comparison of how the different techniques perform when the ratio of data dimension to samples/trials available is manipulated. For example, the techniques which perform poorly here (e.g. ANN) may perform better when the number of repeated trials is closer to the data dimensions than in the present case, or when the number of underlying classes is greater. The manuscript is already quite long, but such an additional analysis may be quite informative.

2. There is a little redundancy in the use of two simulated data sets from the perspective that (1) Each is modeled somewhat to mirror the core properties of the real data, and (2) Each has the same numbers of neurons, data points and Gaussian properties. The second simulation differs from the first in that the function (arm angle) is drawn from a continuous variable rather than a categorical function (different faces). Perhaps more could/should be made of this difference.”

N.B. These are the general comments made by the reviewer when reviewing this paper. It was revised before publication. Specific points, addressed during revision of the paper, are not shown.