https://orcid.org/0000-0001-6047-8100
Figures
Abstract
Neuronal activity is organized in collective patterns that are critical for information coding, generation, and communication between neural populations. These patterns are often described in terms of synchrony, oscillations, and phase relationships. Many methods have been proposed for the quantification of these collective states of dynamic neuronal organization. However, it is difficult to determine which method is best suited for which experimental setting and research question. This choice is further complicated by the fact that most methods are sensitive to a combination of synchrony, oscillations, and other factors; in addition, some of them display systematic biases that can complicate their interpretation. To address these challenges, we adopt a highly comparative approach, whereby spike trains are represented by a diverse library of measures. This enables unsupervised or supervised analysis in the space of measures, or in that of spike trains. We compile a battery of 122 measures of synchrony, oscillations, and phase relationships, complemented with 9 measures of spiking intensity and variability. We first apply them to sets of synthetic spike trains with known statistical properties, and show that all measures are confounded by extraneous factors such as firing rate or population frequency, but to different extents. Then, we analyze spike trains recorded in different species—rat, mouse, and monkey—and brain areas—primary sensory cortices and hippocampus—and show that our highly comparative approach provides a high-dimensional quantification of collective network activity that can be leveraged for both unsupervised and supervised characterization of firing patterns. Overall, the highly comparative approach provides a detailed description of the empirical properties of multineuron spike train analysis methods, including practical guidelines for their use in experimental settings, and advances our understanding of neuronal coordination and coding.
Author summary
Cognition and brain–body regulation rely on collective patterns of neural activity, which are typically described in terms of synchrony, oscillations and phase relationships. Many methods have been proposed for measuring these properties, and selecting the most appropriate method for a given research question can be a daunting task. To address this issue, we assembled a broad range of statistical measures and tested them on both synthetic and biological spike trains. Our analyses indicate that there is no overall “best” measure, and inform on the relative advantages and drawbacks of a broad range of measures with respect to several criteria of interest for their empirical application, including their modulation by firing rate or spike failures, population frequency, sequentialness and rhythmicity, as well as their bias and precision resulting from finite time window length and number of neurons. Our results provide a comprehensive picture of the range of available methods for the quantification of collective patterns of neural activity, enabling researchers to make better informed decisions and avoid interpretational pitfalls.
Citation: Baroni F, Fulcher BD (2025) Synchrony, oscillations, and phase relationships in collective neuronal activity: A highly comparative overview of methods. PLoS Comput Biol 21(10): e1013597. https://doi.org/10.1371/journal.pcbi.1013597
Editor: Daniel Bush, University College London, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: August 15, 2024; Accepted: October 7, 2025; Published: October 24, 2025
Copyright: © 2025 Baroni, Fulcher. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are available from public repositories: the rat auditory cortex dataset is available from CRCNS.org (doi:10.6080/K09021X1), the mouse hippocampus dataset is available from G-Node (doi:10.12751/g-node.lkx6kk), and the monkey visual cortex dataset is available from CRCNS.org (doi:10.6080/K0NC5Z4X). The code is available at GitHub.com/GNB-UAM/sync_osc_MSTMs.
Funding: This research was supported by grants PID2024-155923NB-I00 and PID2021-122347NB-I00 (PI: Pablo Varona Martinez), PID2023-149669NBI00 and PID2020-114867RB-I00 (PI: Francisco de Borja Rodríguez Ortiz) (MCIN/AEI and ERDF- “A way of making Europe”). BDF acknowledges support from the Australian Research Council (FT240100418). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Neuronal ensembles self-organize into collective activity patterns that are considered the building blocks of neuronal syntax [1,2]. Hence, the identification and categorization of collective activity patterns from large-scale neuronal recordings is a key prerequisite towards assessing their relationships with behavior and their putative coding properties.
The type of collective activity pattern that has received the most attention is oscillatory synchrony [3]. When neurons synchronize their spiking activity, their effects on downstream targets are greatly enhanced [4,5] and the temporal summation of action potential related currents in the extracellular medium yields large amplitude signals that can be easily detected in extracellular recording [6,7] and even non-invasively [8,9]. Correspondingly, oscillatory synchrony has been proposed as a key mechanism in the selective routing of neural signals [10–12] and the neurodynamical substrate of cognitive phenomena such as perceptual binding [13] and attention [14].
However, it has been recognized that collective activity patterns can vary along a number of different dimensions. In addition to synchronous spiking, precisely timed sequences of spiking activity across neuronal ensembles have also shown to occur non-randomly, and to be related to stimuli and behavior [15–17]. In addition, synchronous population events as well as non-random sequences can occur with or without oscillations [18,19]. In some cases the same behavior is causally related to synchronous activity with either oscillatory or non-oscillatory dynamics in different species. For example, while in rodents and humans spatial navigation is accompanied by hippocampal theta oscillations, with the phase of firing supporting a code for space cognition [20–24], in bats the same behavior does not involve oscillations, but non-rhythmic synchronous events which similarly support an allocentric space code [25]. This evidence indicates that a multidimensional quantification of collective activity patterns is required for an appropriate characterization of network states.
Consistently with their importance in neurophysiology and neural coding, many statistical measures have been developed for the quantification of synchrony and/or phase relationships. However, the choice of the most appropriate measures for each experimental condition and research question is difficult for several reasons. Among other factors, the measures vary in: (i) their sensitivity to concomitant pseudo-rhythmic oscillations; (ii) their sensitivity to potential covariates such as firing rate, population frequency, and the number of participating neurons; and (iii) the amount of data they require for obtaining a meaningful estimate. These issues complicate the selection and interpretation of these measures, especially when different conditions yield only subtle (but perhaps physiologically meaningful) changes in collective coordination, and suggest that spike train synchrony is not a monolithic concept, but one that varies across several dimensions. Different measures capture different aspects of spike train synchrony and, more generally, spike train coordination.
Here, we address these challenges by compiling a large battery of statistical measures and applying them to a large set of synthetic and biological spike trains in a systematic fashion, adopting a highly comparative methodology. In the last decade, the highly comparative approach has emerged as a powerful tool for unveiling the empirical structure of analysis methods in a systematic and data-driven manner [26]. It consists in applying a large library of analysis methods to a large set of items of a given data type (e.g., time series), obtaining a data matrix where each row represents a data item and each column represents an analysis method. This matrix constitutes an empirical description of both sets of methods and data items, jointly considered, and its structure can be explored with both unsupervised and supervised techniques [27]. While initially proposed for univariate time series analysis, the highly comparative approach has recently been extended to pairwise interactions in multivariate time series [28] and graph-theoretic measures in networks [29]. The results can recapitulate known theoretical relationships between methods using a data-driven approach, and can also offer unexpected insights into both the methods and the datasets being analyzed. For example, they can reveal surprising similarities in the behavior of different analysis methods that could not be inferred from their formal definition. A systematic empirical comparison between statistical measures is required to obtain a broad and general understanding of their behavior, since theoretical relationships can be established only in very few cases and under specific assumptions (e.g., [30]).
Typically, explorative studies consider a number of statistical measures that are thought to be relevant for the characterization of the processes under examination. Then, it is not uncommon that only those that “worked” (e.g., by exhibiting a statistically significant difference between groups) are reported in scientific publications. By contrast, in the highly comparative approach, a sheer number of methods are employed and reported, hence greatly ameliorating publication biases and related issues such as p-hacking. The highly comparative approach has only recently started to be applied to the neurosciences, but it is already showing its potential to refine and organize knowledge in these fields, including the ability to highlight novel high performing methods for a given analysis task and detect new types of predictive patterns in data [31–37].
Following this methodological approach, we compile a battery of 37 distinct measures that have been proposed for the quantification of synchrony, oscillations, or phase relationships, extended to a set of 122 measures by considering multiple timescales for the timescale-dependent measures, along with 9 univariate measures for the assessment of firing intensity and variability, yielding a total of 131 measures, which we collectively refer to as multineuron spike train measures (MSTMs). We apply MSTMs to sets of synthetic spike trains with known statistical properties and mathematical parameterization, and assess the relationships between each MSTM and the parameter determining synchrony, as well as other parameters, such as firing rate and population frequency, which can be considered as confounding factors. Additionally, we analyze MSTM properties for a range of analysis time window lengths and number of participating neurons in terms of bias and precision. Finally, we apply this MSTM battery to biological spike trains recorded in different species—rat, mouse and monkey—and brain areas—primary sensory cortices and hippocampus—and show that the highly comparative approach provides a high-dimensional quantification of collective network activity that is distinctive of each recording and informative of brain states such as wake and sleep. We provide this set of MSTMs, along with the synthetic spike train generative models and all analysis code, as a software accompaniment to this manuscript.
Methods
This section opens with the description of the synthetic spike train formalisms used for the validation of the MSTMs in a controlled setting with known ground truth, before presenting the MSTMs themselves. Next, we describe the measures of bias and variability employed to quantify of the effects of varying time window length and number of neurons on each MSTM. We proceed by presenting the biological datasets used to illustrate the benefits of a highly comparative analysis for the visualization and classification of firing patterns. Further details on the definition of the MSTMs, as well as a description of the analyses conducted on biological spike trains—including the decoding analyses employing MSTMs as input features—are provided in the Supplementary Methods (S1 Text). This document also describes the analysis methods used to characterize the relationships between MSTMs and between time windows, including clustering, recording fingerprinting, dimensionality reduction and estimation.
Synthetic spike train generation
In order to obtain a diverse repertoire of multineuron spike train data with known ground truth encapsulated in their mathematical formalism (including values of the parameters determining synchrony, as well as of non-synchrony parameters, such as firing rate or population frequency), we generated synthetic multineuron spike trains as multivariate point processes [38] that vary along several independent dimensions. The hierarchically higher distinction is between single-scale and dual-scale formalisms. The former is based on inhomogeneous Poisson processes, while the latter on a doubly stochastic formalism, whereby population events are generated at a higher level, and spikes are distributed around them according to a Gaussian distribution at a lower level. For both formalisms, we considered both pseudo-rhythmic and non-rhythmic spike trains. An additional and independent distinction reflects the extent of sequential structure. While some of these distinctions (i.e., single-scale vs. dual-scale trains, and pseudo-rhythmic vs. non-rhythmic trains) are categorical, parameter values have been varied in a way that results in a smooth transition between categories, as detailed below.
Single-scale spike train generation.
Single-scale spike trains were generated according to an inhomogeneous Poisson process. Poisson processes, both homogeneous and inhomogeneous, are a classic model for the generation of synthetic spike trains ([39,40]; see [41] for a more recent treatment), as firing variability often resembles that of a Poisson process [42]. In particular, sinusoidally modulated inhomogeneous Poisson processes have been used in several studies as a model of rhythmic firing [43–46]. We considered two families of spike trains: pseudo-rhythmic and non-rhythmic.
Pseudo-rhythmic spike trains were generated according to an inhomogeneous Poisson process with an instantaneous rate evolving as a sinusoidally modulated function with linear phase:
where r0 is the average firing rate, m is the modulation amplitude, f0 is the modulation frequency, and is a phase noise term that evolves as an Ornstein–Uhlenbeck process:
where is the autocorrelation time constant,
is the standard deviation, and
is δ-correlated noise.
Non-rhythmic spike trains were generated according to an inhomogeneous Poisson process with an instantaneous rate evolving as a sinusoidally modulated function with piecewise linear phase:
where indicates the initial phase, and
is a piecewise linear phase evolving as
for , where
is the time of the previous node and
is the current Inter-Node Interval, so that
. Inter-Node Intervals are distributed according to a refractory exponential distribution:
where and
is a population refractory period. A factor of 2 has been included in the expression for λ to obtain the same average rate of peak firing for both pseudo-rhythmic and non-rhythmic spike trains. For both pseudo-rhythmic and non-rhythmic spike trains, we varied the average firing rate r0, the population rate f0, and the modulation amplitude m.
For both pseudo-rhythmic and non-rhythmic spike trains, we considered an additional distinction regarding the presence or absence of sequential structure. For non-sequential spike trains, all single-neuron spike trains comprising a multineuron spike train were generated by sampling from the same inhomogeneous Poisson process with rate or
for either pseudo-rhythmic or non-rhythmic trains, respectively. For sequential pseudo-rhythmic spike trains, we added a neuron-specific phase shift which implemented a linear progression with neuron index j:
where Dc is the duty cycle (i.e., the active fraction of a cycle), N is the number of neurons and is the neuron index, so that
for j = 1, the most phase-advanced neuron, and
for j = N, the most phase-delayed neuron.
In the case of sequential non-rhythmic spike trains, a neuron-specific phase shift was similarly implemented:
where the population-level was updated as soon as
(with neuron with index j = 1 being the most phase-advanced neuron), and then the neuron-specific node times
were updated as soon as
for each
. Illustrative examples of single-scale spike trains where both rhythmicity and sequentialness are varied independently are shown in Fig 1A.
Examples of single-scale (A) and dual-scale (B) spike trains either without (upper panels) or with (lower panels) sequential structure. The left (right) column shows pseudo-rhythmic (non-rhythmic) trains. In each panel, the top subpanel shows the dynamic variables that govern the population activity (r for single-scale trains, for dual-scale trains), the middle subpanel shows the instantaneous population firing rate (obtained by convolving each spike train with a Gaussian kernel of bandwidth 0.1 ms and then averaging across neurons), and the bottom subpanel displays the spike train as a raster plot. In the case of sequential single-scale trains, the instantaneous rate variable r is shown for 5 neurons that span the active phase of the cycle, equally spaced with respect to relative phase; shades from blue to green indicate greater phase lag. Parameter values are set as follows:
Hz, m = 1, Dc = 0 or 0.4 (A);
,
, Dc = 0 or 0.4 (B). In all cases,
Hz.
Dual-scale spike train generation.
Dual-scale spike trains were generated according to a doubly stochastic process where a hierarchically higher process determines population events, and a hierarchically lower process determines spike times within each population event. As in the case of single-scale trains, we considered two independent distinctions in the generation of dual-scale spike trains: they could be pseudo-rhythmic or non-rhythmic, as well as sequential or non-sequential.
For the generation of dual-scale pseudo-rhythmic spike trains, we considered an angular variable which evolves according to
where f0 is the modulation frequency and is a phase noise term evolving as an Ornstein–Uhlenbeck process (Eq (2)). Population events
were defined as time points when
crosses from below a reference phase
which occurred at least
after the previous population event
.
For the generation of dual-scale non-rhythmic spike trains, population events were generated according to a refractory exponential distribution of Inter-Event Intervals (IEIs):
where and
is a population refractory period. For both pseudo-rhythmic and non-rhythmic spike trains, spikes were generated around population events according to a Gaussian distribution.
For both pseudo-rhythmic and non-rhythmic spike trains, an additional dimension reflects the extent of sequential structure. For non-sequential spike trains, spikes were generated around population events according to a Gaussian distribution with standard deviation centered on the population event. More precisely, for each population event
, the spike time ti,j for neuron j was determined as
where are i.i.d samples drawn from a zero-mean Gaussian distribution
with standard deviation σ:
For sequential spike trains, we added a neuron-specific bias in the expression for ti,j which included a linear progression with neuron index j:
where Dc is the duty cycle (i.e., the active fraction of the average IEI), N is the number of neurons and is the neuron index. For all dual-scale spike trains, a spike could be deleted with probability
.
For all dual-scale spike trains, we varied the population rate f0, the Gaussian width Σ (expressed as a percentage of the average IEI 1/f0), and the spike deletion probability . For sequential spike trains, we also varied the duty cycle Dc. For non-sequential spike trains, the inverse of the Gaussian width can be considered a measure of synchrony tightness; for sequential spike trains, the relation between Dc and Σ determines the degree of sequentialness, with large
resulting in strongly sequential activity. Note that, for both single-scale and dual-scale formalisms, non-sequential trains can be considered as a special case of sequential trains with Dc = 0. Illustrative examples of dual-scale spike trains where both rhythmicity and sequentialness are varied independently are shown in Fig 1B.
For all spike train formalisms, a neuronal refractory period was adopted, whereby a neuron could not emit two consecutive spikes separated by an inter-spike interval (ISI) shorter than or equal to
. Model equations were evaluated with a simulation time step
ms. The phase noise term
(Eq (2)) was updated at each time step using the properties of Ornstein–Uhlenbeck processes with zero mean. That is,
+
is normally distributed with mean
and standard deviation
. The differential equation governing the angular variable ϕ, representing population phase in dual-scale spike trains (Eq (8)), was integrated using the Euler method. Parameter ranges were chosen to yield approximately the same range of firing rate and synchrony values (as assessed by most MSTMs) for both spike train formalisms. Parameter values and descriptions are provided in Table 1.
Multineuron spike train measures
We compiled a library of 37 distinct measures to assess synchrony, oscillations, or phase relationships that have been proposed for the analysis of spike patterns. We then applied them to an array of synthetic multineuron firing patterns, where the ground truth of the underlying generative model is known, as well as to biological firing patterns, with the aim of comparing them systematically. We did not intend to include all the measures that have been proposed, but rather to include a broad range of measures which includes all of the most commonly employed measures.
Some of these measures are bivariate (i.e., they quantify the relationship between two single-neuron spike trains), and have been applied to every pair of non-identical single-neuron spike trains, and then averaged to obtain a single estimate for the whole multineuron spike train. Others are multivariate, and return a single estimate for each multineuron spike train by examining the relationships between all its constituent single-neuron spike trains simultaneously. In addition, we also included 9 univariate measures (i.e., quantifying individual single-neuron spike train properties) that quantify firing intensity or variability, which have been applied to every single-neuron spike train comprising a multineuron spike train, and then averaged across neurons. These measures have been included to evaluate the influence of univariate firing statistics on the measures of collective synchrony, oscillations, and phase relationships.
Some of the measures derive from the power spectrum of the instantaneous population firing rate. In particular, we extracted the amplitude and frequency of the log power spectrum maximum. We also performed a FOOOF (fitting oscillations & one over f) parameterization of the power spectrum of the instantaneous population firing rate [47], whereby the log power spectral density was described by the sum of an aperiodic component and either one (single-peak model) or two (dual-peak model) periodic components.
This set of 46 multineuron spike train measures (MSTMs, Table 2 and 3), which we refer to as the core set, comprises 12 MSTMs (11 bivariate and 1 multivariate MSTMs, indicated with an asterisk in Table 2) that are timescale-dependent: that is, they include a timescale parameter that determines the temporal range defining synchrony, which was set to 1 ms. We also extended this core set by including a range of timescales for the timescale-dependent measures, yielding a library of 131 MSTMs, which we refer to as the extended set. Specifically, we considered 7 timescales logarithmically spaced between 1 and 64 ms. The extended set also expanded on the core set by including dual peak FOOOF model parameters, which were not part of the core set. Further details are reported in the Supplementary Methods (S1 Text).
Estimates of variability and bias
Measures of synchrony or oscillations are typically applied to datasets or data segments that are limited in both time and space. From a statistical perspective, these data segments can be considered as finite realizations xt of an underlying stochastic generative process Xt, which is a collection of random variables encapsulating the complete statistical description required for the generation of any of its possible realizations [77]. It is well known that the statistical properties of finite realizations can differ significantly from those of the stochastic process they originate from due to several factors, and this difference depends on the statistical estimator considered. These factors include nonstationarity, whereby the statistical properties of the underlying process change over time; sample limitations in time and space, and the variability they induce in the corresponding estimates; and the bias inherent in each statistical estimator, whereby a given estimator (such as any of the MSTMs we consider), when applied to finite samples, might systematically under- or over-estimate the “true” value corresponding to the underlying generative process.
Acknowledging the extent of variability and bias is a necessary prerequisite for the informed application of MSTMs and for an appropriate interpretation of the collective activity states they capture, especially when comparing between conditions that are not equal in terms of time window length and number of participating neurons, or when a temporally and/or spatially resolved description is desired. Conspicuous biases can be particularly detrimental if the analysis is conducted at multiple temporal and/or spatial scales, for example with the aim of determining the relevant scales of population activity and coding [78–82], since they can lead to marked changes in MSTM output due only to variations in the temporal or spatial analysis scale, in the absence of any difference in the underlying generative process. While biological processes are inherently nonstationary, the adoption of synthetic spike trains enables us to characterize the variability and bias of each MSTM in conditions of stationarity and ergodicity, where a sufficiently extended sample completely represents the statistical properties of the underlying process. In particular, we characterize the variability and bias due to finite time window length and number of neurons, two ubiquitous limitations in empirical applications.
To quantify the variability and bias that result from adopting a finite time window , we considered a range of time windows approximately logarithmically spaced between 500 ms and 100 s. For each time window length, we generated
independent spike trains, each composed of
neurons. We applied the MSTMs to each spike train, and for each MSTM we considered the average across the
realizations at the longest time window as an approximation of the “true” but unknown value corresponding to an infinite spike train obtained from the same process (generally indicated as
). For each time window length and each MSTM, variability
was defined as the absolute value of the coefficient of variation
across windows, while the bias
was defined as the difference between the average value
and the “true” value (approximated by
), normalized by the standard deviation:
We quantified the variability and bias that result from adopting a finite number of neurons similarly: we considered a range of
approximately logarithmically spaced between 4 and 100. For each
value, we generated
independent spike trains, each 100 s long. Variability and bias were defined similarly as above, but in this case the “ground truth” value for each MSTM was approximated as the average across the
realizations at the highest
value. Variability and bias were estimated independently for each spike train family and for a range of synchrony values. Purely asynchronous trains (i.e., single-scale trains with m = 0) were not included in this analysis. Both the individual values as well as the average across spike train families, synchrony values and either
or
values are presented in the subsection “Effects of time window length and number of neurons”.
Biological spike train analysis
To ensure that the results obtained using synthetic spike trains do not depend critically on idiosyncratic aspects of the spike train generation algorithms, and to illustrate the benefits of a highly comparative analysis for visualizing, organizing and classifying real spike trains, we applied the same battery of MSTMs to biological spike train datasets. In particular, we analyzed: (i) a rat auditory cortex dataset, obtained during the presentation of dynamic broadband stimuli under ketamine/xylazine anaesthesia [83,84]; (ii) a mouse dorsal hippocampus dataset, recorded during natural sleep and wakefulness [85,86]; and (iii) a monkey visual cortex dataset, acquired during the presentation of a uniform gray screen under sufentanil anaesthesia [87,88]. Further information on each dataset is available in the Supplementary Methods (S1 Text).
Results
In the first part of this section, we applied the MSTM battery to synthetic spike trains to assess their properties in a controlled setting with known ground truth generative processes. This included a systematic comparison between MSTMs and an assessment of their bias and variability. In the second part, we applied the MSTMs to biological datasets to illustrate the benefits of a highly comparative analysis for data visualization and classification over a more traditional approach, whereby only a few MSTMs are selected, with minimal comparison to potential alternatives.
Validating MSTMs on synthetic spike trains
We first validated the MSTM library using synthetic multineuron spike trains with varying levels of synchrony, rhythmicity, and sequentialness, as described above in subsection “Synthetic spike train generation”. These spike trains originate from mathematically defined generative models with known ground truth, and hence enable the direct assessment of the extent to which each MSTM is affected by the parameter determining synchrony as well as by any other parameter included in the generative models. Furthermore, by construction, they are stationary in time and either homogeneous in space, with zero phase shift across neurons (in the case of non-sequential trains), or present a constant phase shift across the neuronal population (in the case of sequential trains), which enables the quantification of bias and variability resulting from time window length and number of neurons.
Effects of synchrony and oscillations.
For both the single-scale (based on inhomogeneous Poisson processes) and the dual-scale (derived from doubly stochastic processes) families of synthetic spike trains, increasing levels of synchrony could be detected by multiple MSTMs, as shown in Fig 2 for an illustrative set of MSTMs applied to pseudo-rhythmic and non-rhythmic spike trains with varying population frequency f0. In these plots, an ideal MSTM that is only sensitive to the parameter determining synchrony, and changes its output value proportionally to changes in that parameter, would result in a straight line, with perfect overlap of the data points corresponding to the same value of the parameter determining synchrony (the modulation amplitude m for single-scale trains, and the population width Σ for dual-scale trains) but different values of population frequency or pseudo-rhythmic properties. None of the MSTMs shown in Fig 2 complies with this ideal behavior in both single-scale and dual-scale spike trains, even if Golomb–Rinzel synchrony SGR and the correlation index Ci approach it in single-scale spike trains. Overall, the MSTM that most closely approximates this ideal behavior in both single-scale and dual-scale spike trains is the correlation index Ci.
Assessment of the level of synchrony on single-scale (A) and dual-scale (B) synthetic spike trains using 5 illustrative MSTMs. A: Synchrony is plotted as a function of the modulation amplitude m for average firing rate Hz. B: Synchrony is plotted as a function of the population width Σ for spike deletion probability
. Synchrony increases with m in single-scale trains, while it decreases with Σ in dual-scale trains. In all panels, the population rate f0 varies in the grid [4,12,36] Hz and is color-coded, with warmer colors indicating higher f0. Solid lines: pseudo-rhythmic spike trains; dash lines: non-rhythmic spike trains. Corresponding results for a broader set of MSTMs and parameter values are shown in S1 Fig.
In single-scale spike trains, the Tiesinga–Sejnowski synchrony measure STS is relatively unaffected by pseudo-rhythmicity, but decreases markedly with population frequency; conversely, in dual-scale spike trains, it is relatively unaffected by population frequency, but decreases markedly with pseudo-rhythmicity. Some bivariate MSTMs, such as the Mean Phase Coherence (MPC) and SPIKE-distance DS, perform poorly if the population frequency is greater than the average firing rate in single-scale trains, or if the proportion of missing spikes is high in dual-scale trains, as in Fig 2B. Both situations correspond to a scenario where individual neurons fire only in a fraction of the population oscillation cycles, a regime commonly reported in experimental studies, especially for high-frequency oscillations (from the beta range—around 20 Hz—upwards, [89–93]). In summary, the variability across MSTM behavior can be characterized based on several factors, including: (i) the linearity of their response; (ii) their capacity to detect small modulations of synchrony; (iii) how they are affected by the presence or absence of rhythmicity; and (iv) their dependence on non-synchrony parameters, such as the average firing rates r0, the population oscillation frequency f0, and the amount of data.
To start quantifying some of these effects, we calculated the absolute value of the Spearman correlation between each MSTM and each generative parameter for both synthetic spike train families (Fig 3). For each spike train family, the correlation with the parameter that determines the level of synchrony is shown in the top row for each panel in Fig 3. The correlations with each of the remaining generative parameters are also shown. These can be interpreted as potential confounding factors, as in the case of the correlation with the mean firing rate r0 or the population frequency f0. The correlation with the presence or absence of pseudo-rhythmic oscillations, indicated in the bottom row for each panel, can also be considered as a potential confounding factor, or at least a factor worth being taken into account, especially for those applications where the distinction between pseudo-rhythmic vs. non-rhythmic synchronization is important.
The absolute value of the Spearman correlation coefficient between each MSTM and each generative parameter is color-coded for the single-scale (A) and dual-scale (B) synthetic spike train families. Correlation values that are not significant (at , uncorrected) are grayed-out. For each spike train family, MSTMs are ordered according to decreasing absolute values of correlation with the generative parameter that determines synchrony, which is shown in the top row. The bottom row for each panel corresponds to the presence or absence of pseudo-rhythmic oscillations. The intermediate rows correspond to potentially confounding factors such as firing rate r0 and population frequency f0.
While nearly all MSTMs correlated with the parameter determining synchrony in at least one spike train family, none of them did so selectively in both spike train families. Overall, the correlation index Ci is the MSTM that resulted in the highest correlation with the generative parameter determining synchrony across both spike-train families. In addition, Ci exhibited a relatively selective profile of correlations: it did not correlate with the remaining generative parameters in single-scale spike trains, and correlated only with pseudo-rhythmicity in dual-scale spike trains, yielding lower values for pseudo-rhythmic spike trains (Figs 1 and S1). However, Ci is quite noisy for low spike numbers (as we will quantify later, in subsection “Effects of time window length and number of neurons”). Other MSTMs that displayed high correlation with the parameter determining synchrony are STS, SGR, CK, STTC and PPC. But note that some of these MSTMs exhibit high correlation with, and are hence potentially confounded by, other generative parameters. In particular, in dual-scale spike trains, STS exhibits a high correlation with the presence or absence of pseudo-rhythmic oscillations, which is even higher than the correlation with the synchrony-determining parameter Σ, suggesting that this MSTM can be more appropriately regarded as a measure of non-rhythmic synchrony rather than a measure of synchrony per se. SDR, CK, and STTC share a similar pattern of confounding influences, being sensitive to firing rate in single-scale trains, and to population frequency and pseudo-rhythmicity in dual-scale trains. The MSTM that most strongly correlates with the parameter determining synchrony in dual-scale trains is , a measure of spectral power. However, this effect is only observed in pseudo-rhythmic spike trains. In fact, FOOOF parameterization did not yield a spectral peak for most non-rhythmic trains, as expected (S1 Fig).
Empirical organization of spike train measures.
To organize MSTMs according to their empirical behavior and to detect any cluster structure in the space of MSTMs, we conducted hierarchical agglomerative clustering analyses (Fig 4). Each MSTM was represented by a vector of values resulting from its application to spike trains from each synthetic spike train family, resulting in a total of 900 spike trains (150 trains from each of the non-sequential single-scale and dual-scale families, and 300 trains from each of the sequential single-scale and dual-scales families). In addition to the measures of synchrony, oscillations, and phase relationships listed above, we also included a battery of univariate measures of firing intensity or variability to detect potentially confounding effects on the measures gauging collective coordination.
A: Dendrogram showing the distances between MSTMs as a hierarchical cluster tree. Measures are sorted along the x axis in increasing order of the dendrogrammatic distance of the first nonsingleton cluster they are grouped in. B: Similarity matrix showing the absolute value of the Spearman correlation coefficient between each pair of MSTMs. Measure labels are color-coded to indicate measure type (green: univariate; blue: bivariate; red: multivariate). Corresponding results for the extended set of 131 MSTMs are shown in S2 Fig.
While most synchrony measures increased with increasing synchrony, some decreased (e.g., DISI, DS and other MSTMs based on spike train distance such as DVP, DvR and their normalized counterparts, Figs 2 and S1). Our main focus is on characterizing synchrony measures based on their patterns of variation on a broad and diverse set of spike trains, which differ in aspects such as firing rate, population frequency, pseudo-rhythmicity and sequentialness, regardless of the sign of the variation. Hence, we adopted D = 1 − as the distance metric, where ρ is the Spearman correlation value between each pair of MSTMs.
The inter-measure similarity matrix shown in Fig 4B displays a complex pattern of similarities and differences. Even so, the similarity matrix and the corresponding dendrogram (Fig 4A) show that most MSTMs can be grouped into 3 distinct clusters. The first cluster includes the mean firing rate r and the measures of synchrony that are strongly dependent on firing rate, such as the Victor–Purpura DVP, the Van Rossum distance DvR and, to a lesser extent, their normalized counterparts, and other MSTMs such as Schreiber correlation CS, Hunter–Milton similarity SHM, and Quian Quiroga event synchronization SQQ. The second cluster includes measures of ISI variability such as CVISI. The third cluster groups measures of synchrony that are not as strongly confounded by firing rate, including the Correlation index Ci, Tiesinga–Sejnowski synchrony STS and Golomb–Rinzel synchrony SGR. A group of related MSTMs proposed by Thomas Kreuz and coworkers, which include ISI-distance DISI, SPIKE-distance DS, SPIKE-synchronization SS and the timescale adaptive version of Quian Quiroga event synchronization SQQA (the first 3 of which are available in the SPIKY software package, [50]), are similar to those of this cluster, but also exhibit high similarity with measures of ISI variability, sitting somewhere in between the second and third cluster. The remaining MSTMs are more distinctive and cannot be clearly grouped into a cluster. These include measures derived from the FOOOF spectral analysis and measures of synchrony that are confounded by the presence or absence of oscillations, such as Spike-Contrast SC.
Effects of time window length and number of neurons.
In empirical studies, measures of synchrony or oscillations are applied to datasets or data segments that are limited in both time and space. In this condition, the estimated MSTM values can exhibit some variability around the “true” value that would be obtained from an infinite dataset derived from the same process, i.e., a process with the same statistical properties. Even more concerningly, the estimated values can exhibit a bias with respect to the “true” value: the average across many realizations does not converge to the “true” value, but systematically over- or under-estimates it. While variability is readily apparent, can be precisely characterized if enough samples are available, and does not undermine the interpretation of measures if unbiased, systematic biases often go undetected and can easily lead to misinterpretation.
This problem is especially severe when these measures are applied to short sliding windows to obtain a time-resolved description of network activity, for example along the course of a multistep task. If the window length is varied in order to obtain a multiple-timescale description of collective activity, strongly biased measures can exhibit conspicuous changes only as a result of varying window length, in the absence of any change in the statistical properties of the underlying generative process. Hence, lack of acknowledging of the extent of variability and bias can lead to a misunderstanding of the collective activity states that these measures aim to capture.
To address this, in this subsection we applied the battery of MSTMs described above to datasets of varying length , and assessed the bias and variability for each MSTM. We observed a broad range of bias and variability across MSTMs as the time window length is varied, as shown in Fig 5A for a selection of measures (and in S4A and S5 Fig for all MSTMs in the core set). Some of the synchrony measures that display the largest bias are DVP, DvR, Sqq and MPC; while DS, SHM and PPC display a small bias (S4Aa Fig). In terms of variability, DVPN, DvRn and the SPIKY measures DISI, DS, SS, and SGR, are among those that exhibit the lowest variability across windows, hence the highest precision (S4Ab Fig). The largest variability, hence lowest precision, is observed for PPC, STTC, CK, FS, and some of the MSTMs that are derived from the power spectrum or its FOOOF parametrization. Considering both bias and variability, there is a tradeoff whereby the measures with the highest bias tend to exhibit the lowest variability, and vice versa (S4Ac Fig). This is an expected trend: if a measure is consistently biased in every time window it will show low variability across windows, hence high precision.
Effects of sample size variations in time (A) or space (B) for 5 illustrative MSTMs in the case of single-scale non-sequential spike trains with Hz and m = 0.5. A: For each MSTM, the mean value across windows is plotted as a function of window length. Shaded areas indicate the SD across windows. B: As in (A), but sample size varies in space instead of time. Corresponding results for a broader set of MSTMs and parameter values are shown in S3 Fig.
As in the case of subsampling in time, subsampling in space is also an ubiquitous feature of nearly all empirical studies: with the exception of whole-brain imaging at cellular resolution recently achieved in simple organisms such as larval zebrafish and C. elegans, or outputs from computational simulations, we only record from a small subpopulation from the total set of relevant neurons. Most measures of synchrony exhibit non-negligible biases as a result of subsampling in space (Figs 5B, S4B and S6); if these are not acknowledged, a change in one of these measures could be erroneously interpreted as a change in synchrony, whereas only a change in the number of participating neurons occurred. The effects of sample size variations in space and time are drastically different, to the extent that some of the measures that display the smallest bias when subsampling in space are among those that exhibit the largest bias when subsampling in time, and vice versa (Figs 5 and S4, compare A with B). For example, MPC shows only a small bias when subsampling in space, but a large bias when subsampling in time. Conversely, SGR and STS exhibit a large bias when subsampling in space, but only a moderate bias when subsampling in time.
Analyzing experimental spike train data through the lens of the highly comparative approach
The application of the highly comparative approach—systematically comparing the behavior of a wide range of spike train measures—to synthetic data is essential for a thorough understanding of the statistical properties of each individual measure in idealized conditions, and it provides a basis for interpreting the results of a highly comparative analysis of real datasets. However, the practical power of the method is best illustrated when applied to real datasets, where it can be used for data visualization and ordering, and it can be combined with both unsupervised and supervised techniques for gaining novel insights on important biological questions. In this section, we illustrate these advantages by applying our MSTM library to biological spike trains recorded in different species—rat, mouse, and monkey—and brain areas—primary sensory cortices and hippocampus, addressing two major neuroscientific questions: (i) the characterization of spike train variability within and across recordings, and (ii) brain state classification.
Spike train variability and recording-wise distinctiveness.
We applied our library of MSTMs to multineuron spike trains obtained from the primary auditory cortex (A1) of anesthetized rats during the presentation of dynamic broadband stimuli. The results can be conveniently visualized in a color-coded matrix form to obtain a global perspective on multineuron activity and coordination across experimental sessions and, for each session, on the structure of multineuron coordination variability and nonstationarity (Fig 6A). For this analysis, the battery of MSTMs was applied to each 30 s non-overlapping window, yielding 20 time windows per recording. This representation shows that different MSTMs tend to capture different aspects of multineuron activity and coordination: in general, the decomposition afforded by the highly comparative approach is truly multidimensional (as we will quantify in the following subsection “Comparing the geometry of inter-MSTM structures obtained from synthetic and biological spike trains”). Frequently, each session displays a unique high-dimensional signature that characterizes it and distinguishes it from other sessions (see also Fig 7).
A: A dataset comprising 14 recordings is visualized through the highly comparative description, highlighting nonstationarities, pseudo-rhythmic patterns, and recording-wise idiosyncrasies. Each row corresponds to a MSTM, each column to a 30 s time window; values are z-scored across windows. MSTMs are ordered according to the outcome of an average linkage hierarchical clustering procedure over MSTMs. Time windows are ordered chronologically and recordings are concatenated and indicated by the color-coded labels on top, using the same identifiers as in the original dataset [84]. Gray cells indicate time windows where the FOOOF spectral peak parameter was below the significance threshold, hence the corresponding parameters were not assigned. B: The same data shown in A is replotted by ordering time windows according to the outcome of an average linkage hierarchical clustering procedure over time windows as in Fig 7A. Vertical lines on top color-code for recording session as in (A); their length codes for chronological order within each session. In both (A) and (B), values greater than , where SZ indicates a generic MSTM value after z-scoring and the maximum value is taken over MSTMs and time windows, are clipped to that value for improved visualization. Corresponding results for the mouse and monkey datasets are shown in S7 Fig.
A: Dendrogram showing the distances between time windows as a hierarchical cluster tree. B: Similarity matrix showing the Spearman correlation coefficient between any pair of time windows. Vertical lines on top color-code for recording session and their length codes for chronological order within each session as in Fig 6B. C: MDS representation of the distance matrix obtained from the values shown in B by taking D = 1 − ρ. Each circle represents a time window, color-coded as in Fig 6. D: Silhouette plot showing for each time window, its mean value in the dataset (red vertical line), and the
significance threshold (gray dashed vertical line). Corresponding results for the mouse and monkey datasets are shown in S8 Fig.
This representation provides a global depiction of a whole dataset, offering a direct impression of the extent of similarity and variability within- and across-recordings. It also illustrates the types and prominence of dynamical features of the data, including nonstationary and pseudo-periodic aspects, and the MSTMs that reveal them. For example, site9 shows a nonstationary firing rate profile, which decreases slightly in the middle and increases markedly near the end of the session. A similar profile is observed for a number of other MSTMs, notably for vR and VP, which have been shown to depend markedly on firing rate. As another example, site3 and site7 display peaks and troughs across many MSTMs which tend to recur every 3–4 time windows.
To organize the structure of neuronal activity in biological spiking data, as captured by our MSTM library, we conducted hierarchical agglomerative clustering analyses on time windows, with each time window represented by a feature vector derived from the application of each MSTM. Reordering the time windows according to the outcome of a hierarchical clustering procedure over time windows further highlights the multidimensionality of our MSTM library (Figs 6B and 7) and characteristic patterns of multineuron activity and coordination which tend to distinguish individual recordings, whereby time windows from the same recording tend to be grouped together by the clustering procedure and appear next to each other in Fig 6B.
Projecting the high-dimensional vectors resulting from the application of the highly comparative analysis to a 2D space by multidimensional scaling tends to preserve the distinctiveness of individual recordings: nearby time windows tend to originate from the same recording more often than would be expected by chance (Fig 7C). To quantify the extent to which time windows from the same recording tend to be more similar to each other than to windows from different recordings, we calculated the silhouette score for each time window and its mean value in the dataset (Fig 7D). This analysis shows that time windows from the same recording are strongly clustered in the high-dimensional space defined by the core set of MSTMs for 9 of the 14 recordings. The average silhouette score
is 0.133 (significant at
, permutation test).
If the same analysis is conducted in the 2D MDS space shown in Fig 7C, time windows from the same recording still tend to be strongly clustered, but decreases to –0.062 (significant at
), indicating that the 2D MDS projection preserves only a fraction of the recording-wise distinctiveness present in the original high-dimensional space. Similar results are obtained in the mouse and monkey datasets (Table 4 and S7 and S8 Figs); however,
does not decrease when considering the 2D MDS projection in the mouse dataset but actually increases, suggesting that, in this dataset, recording-wise distinctiveness is not only preserved but enhanced when a low-dimensional projection of the highly comparative data matrix is considered. It is worth noting that, while the rat and the monkey datasets were recorded under anesthesia, the mouse dataset was recorded in non-sedated animals for longer time periods, comprising intervals of wake, sleep, and multiple behaviors. Hence, the 2D MDS projection might have attenuated the variability related to varying brain state within each recording, which is conspicuous in this dataset but not in the other two and overshadows recording-wise distinctiveness, hence yielding 2D spike train representations that are more distinctive of each recording in comparison to the original high-dimensional representations.
Silhouette scores are reported for each dataset for both the core(46 MSTMs) and the extended (131 MSTMs) sets of MSTMs (unshaded rows), as well as for their 2D MDS projections (shaded rows), along with their corresponding
significance thresholds
.
Brain state classification from spike train measures.
To characterize the information conveyed by each MSTM with respect to brain state, we discriminated between wake and NREM sleep in mouse hippocampal data [85] using each MSTM either individually or in pairs. Wake and NREM sleep can be reliably discriminated in 3/4 recordings on the basis of several MSTMs even when considered individually, with values reaching 0.982 in the case of Tiesinga–Sejnowski synchrony STS in one recording (M1R1, Figs 8A and S9A). For this recording, several other MSTMs achieve
values above 0.95, including the SPIKE-distance, PPC and some timescale-dependent synchrony measures (Golomb–Rinzel synchrony SGR, Kruskal correlation CK and STTC) with intermediate-long timescales (
ms).
A: Univariate decoding accuracies for one recording for each MSTM of the core set of 46 MSTMs. MSTMs are ordered according to decreasing decoding accuracy. Gray dotted lines indicate
significance thresholds, with darker shades of gray indicating more stringent significance levels. Corresponding results for each individual recording and for each MSTM of the extended set of 131 MSTMs are shown in S9 Fig. B: Median univariate decoding accuracies
across recordings for each MSTM (blue bars) are shown together with single-recording decoding accuracies (colored symbols). MSTMs are ordered according to decreasing median decoding accuracy. C: As in B, but across-recording (with LORO cross-validation) decoding accuracies
are shown. Colored symbols show average values for each test recording (with the remaining recordings used for training), blue bars indicate their median values across recordings. D: Violin-scatter plots of selected MSTMs for each recording and class. Gray and black circles indicate median values for wake and NREM respectively, vertical gray lines show the interquartile range.
However, within-recording decoding results are quite heterogeneous across recordings, with one recording (M1R1) yielding nearly perfect accuracy for the best MSTM, and another (M4R1) returning marginally significant results (at , uncorrected) for just one MSTM out of 131, which is compatible with chance (Figs 8B and S9). Consequently, median decoding results across the 4 recordings are considerably lower than within-recording results in the best recording, with the maximal median
value reaching 0.789 in the case of LvR, a measure of instantaneous firing variability (Fig 8B). At the population level, several other MSTMs achieve
values exceeding 0.7, including R2 (a measure of FOOOF spectral fitting quality), Tiesinga–Sejnowski synchrony, and some other measures of instantaneous firing variability such as Lv, CV2ISI, IR and SM.
In interpreting these results, it is worth acknowledging that sleep scoring was conducted in 2 steps, first using a rule-based approach to classify epochs based on LFP power in canonical frequency bands, EMG broadband power, and mouse movement, which left about 49% of the epochs as unclassified, and then training a k-nearest neighbors classifier for labeling the remaining epochs based on the same features (see [94] for details). Crucially, the rule-based approach involved percentiles of band-limited power of electrophysiological data, independently calculated for each recording, leaving the possibility that some time window could be classified as NREM even if the animal just fluctuated across different levels of arousal, without ever entering fully into the NREM state. Hence, it is possible that a different dataset with a more robustly established ground truth could yield more consistent results across recordings.
Surprisingly, across-recording decoding (where one recording was used for testing, and the rest for training, in a leave-one-recording-out—LORO—cross-validation scheme) reached values that are nearly as high for the top 7 MSTMs (
), which are the same as for the median within-recording results and appear in almost the same ranking (Fig 8C). Decoding accuracy is slightly lower for the remaining MSTMs, but at least as significant in many cases, since across-recording decoding yields lower significance thresholds due to the higher number of available samples. This indicates that not only the direction of the effect, but also the value ranges for each class are preserved to a large extent across recordings for the most discriminative MSTMs (see also Fig 8D).
Decoding accuracy increases further if MSTM pairs are considered, as expected (Figs 9, S10 and S11). Single-recording results for M1R1 reached values close to 0.99 for several MSTM pairs (
). In this recording, out of the 8515 MSTM pairs in the extended set of 131 MSTMs, 49 of them (the 0.58%) achieved
values above 0.98. At the population level, median
values across recordings attained 0.825 for the pair (STS,
), combining a measure of synchrony with one of spectral power, and were above 0.8 for 32 MSTM pairs (0.19% of all pairs in the extended set of MSTMs). Many of these pairs combined measures from different categories. For example, the second most discriminative pair was (LvR,Sqq,τ=2ms), comprising a measure of variability and a measure of sequentialness, and the third was (STS,
), again combining a measure of synchrony with one of spectral power.
Bivariate decoding accuracies (A) and synergies
(B) for one recording (M1R1) for each MSTM pair in the core set. MSTMs are ordered as in Fig 8A. Corresponding results for each individual recording and for each MSTM of the extended set of 131 MSTMs are shown in S10 Fig; median results across the 4 recordings are shown in S11 Fig.
To quantify the extent to which MSTM pair decoding improves accuracy with respect to single MSTM decoding, we computed decoding synergy for each MSTM pair. This analysis revealed that MSTM pair decoding can boost accuracy up to 53% in the case of M1R2, and up to 26% at the population level. The most synergistic pairs comprise MSTMs that do not perform particularly well at the individual level, but provide synergistic information when considered jointly (Fig 9 and Figs S10 and S11, compare top and bottom panels, and Fig 10C). For example, in M1R2, the most synergistic pair (SC,
) yields an
of 72%, when neither MSTM exhibits significant decoding accuracy when considered individually.
A: Cumulative probability density functions of median decoding accuracy for each MSTM pair kind of the extended set of MSTMs. The range of shown corresponds to significant decoding accuracy at
, uncorrected; vertical gray dotted lines indicate more stringent
decoding thresholds. The inset shows the
range corresponding to the most discriminating pairs. B: As in (A), for decoding synergies
. C: Decoding synergies are plotted against accuracies for each MSTM pair in the extended set. Pairs in the top 1% of the most synergistic pairs are highlighted in red. D: Wordcloud plots of the MSTMs included in the top 1% most discriminative (top) and synergistic (bottom) pairs. Numbers displayed at the end of timescale-dependent measures indicate the timescale in milliseconds. For FOOOF spectral measures, 1p (2p) refers to the single-peak (dual-peak) model; in the dual-peak model, the number at the end of the parameter name indicates the corresponding peak.
However, certain MSTM pairs exhibit negative synergies, indicating impaired MSTM pair decoding in comparison to the most performing of the individual MSTMs comprising the pair. This can occur when a MSTM with intermediate performance is paired with a poorly performing MSTM. Note that the linear classifiers we considered are intended to capture only the information that can be linearly read out from MSTM patterns; more complex classifiers can be expected to deal more efficiently with poorly informative MSTMs and hence yield less negative synergy values.
At the population level, MSTM pairs composed of two simple univariate measures always discriminate above chance level, and significantly so (at , uncorrected) for 97.2% of the MSTM pairs of this kind (that is, all but (r,LCVISI)), which is above the value obtained for other MSTM pair kinds (Fig 10A). However, the highest discriminability is obtained with MSTM pairs composed of two multivariate measures, followed by MSTM pairs comprising one bivariate and one univariate measure. The MSTMs included in the top 1% most discriminating and most synergistic pairs are shown in word cloud plots in Fig 10D and reported in S1 and S2 Files. Across-recording bivariate decoding yielded similar results, with slightly lower values for decoding accuracy (
), but with lower significance thresholds.
Comparing the geometry of inter-MSTM structures obtained from synthetic and biological spike trains
In previous subsections, we applied a battery of MSTMs to multineuron synthetic spiking data obtained from two distinct generative models, as well as to biological data recorded from three different species and brain areas. By treating each MSTM as a point in a high-dimensional space—where each coordinate corresponds to its evaluation on a particular spike train—we constructed distance matrices representing pairwise dissimilarities between MSTMs. However, the high-dimensional representation provided by the MSTM library offers added value over a low-dimensional subset only if the outputs of different MSTMs are sufficiently independent, rather than being constrained to a low-dimensional manifold due to dependencies between MSTMs. Thus, a comprehensive analysis of a highly comparative MSTM library requires examining the geometry of inter-MSTM relationships. In this section, we compare the geometry of these inter-MSTM structures—defined by the patterns of similarity between each pair of MSTMs—obtained by considering each of the two synthetic spike train families and each of the three biological spike train datasets. We also considered the aggregate family obtained by lumping together all synthetic spike trains (the global synthetic family).
For a given spike train dataset, we could estimate the intrinsic dimensionality (ID) of the highly comparative decomposition by considering the cumulative variance explained by the first N components resulting from a PCA decomposition of the highly comparative data matrix (where each row corresponds to a MSTM and each column to a spike train, as in Fig 6) as a function of N. This analysis shows that the highly comparative decomposition is high dimensional in every case, with the number of components required to explain 95% of the variance ranging from 7 to 17 when considering the core set of 46 MSTMs (9 when considering the global synthetic family, Fig 11A).
A: Number of components needed to explain 95% and 99% of the variance for the core (46 MSTMs, left) and the extended (131 MSTMs, right) sets of MSTMs as obtained from a PCA decomposition. In general, the inclusion of additional MSTMs in the extended set increases the intrinsic dimensionality as assessed by PCA only modestly at the 95% threshold, but more notably so at the 99% threshold. The higher increase at the 99% threshold is more conspicuous in biological datasets. B: Intrinsic dimension estimated with a nearest neighbor based method. The intrinsic dimension estimates are shown as a function of distance range for synthetic spike train families (top) and biological datasets (bottom), considering the core set of 46 MSTMs (left) or the extended set of 131 MSTMs (right). Shaded areas indicate s.e.m. as returned by a maximum likelihood estimator. The results shown here were obtained with Gride; TWO-NN returned similar estimates.
Linear methods such as those based on PCA are likely to overestimate the number of relevant dimensions when the data points are embedded in a nonlinear manifold. Hence, we also adopted two geometrical measures based on nearest neighbor statistics: TWO-NN [95] and Gride [96], implemented in [97]. Unlike PCA-based approaches, these methods do not return a scalar value but a range of values as the relevant scale is varied either by subsampling the data points in the case of TWO-NN, or by varying the neighbors’ rank in the case of Gride. These methods yield ID estimates that are considerably lower than those returned by the PCA-based approach (Fig 11B). In particular, the ID of the set of synthetic spike trains is estimated at a value between 3 and 4 at most scales for both the core and the extended sets of MSTMs, with slightly higher values (up to 5.1) for single-scale trains at the shortest scale. This estimation approximates the expected dimensionality of synthetic spike trains, which vary along the dimensions of synchrony, firing rate (in single-scale train) or spike failure probability (in dual-scale trains), population frequency, sequentialness and pseudo-rhythmicity. This analysis reveals that these 5 spanned parameter dimensions result in 5 independent dimensions of MSTM variation only for single-scale trains at the shortest scale (i.e., locally), and otherwise yield MSTM patterns of covariation that result in lower intrinsic dimensionality. However, the ID rank between the different synthetic and biological datasets is similar across the PCA and NN-based methods. In particular, the ID of single-scale and dual-scale synthetic spike train measures is similar, and the ID does not increase if single-scale and dual-scale spike trains are grouped together.
Spike train measures estimated from biological datasets tend to exhibit a similar ID as those estimated from synthetic spike train families when considering the linear PCA-based ID estimation in the case of the rat and monkey datasets, with higher ID estimates for the mouse datasets. However, the nonlinear NN-based approaches return higher estimates for the ID of biological datasets in all cases, and more markedly so in the case of the mouse dataset. These results are consistent with the fact that the rat and monkey datasets were recorded in a specific brain state, namely under anesthesia and stationary stimulation; conversely, the mouse dataset includes multiple brain states as the animals transition from wake to NREM and REM sleep and through multiple behaviors. However, several other differences exist among these datasets, including species and brain region, that could underlie these differences in ID.
We can further compare the geometry of inter-MSTM structures obtained in each spike train dataset by considering the Pearson correlation between the inter-MSTM distance matrices obtained from each synthetic spike train family and biological spike train dataset (S12 Fig). As expected, the highest correlations are observed between the global synthetic family and each of the synthetic spike train families, with similarly high values obtained for the single-scale and the dual-scale families (0.8 and 0.82 for the core set of MSTMs, 0.88 and 0.91 for the extended set, respectively). The similarity between the inter-MSTM structures obtained from synthetic and biological spike trains is lower, albeit still numerically high, especially in the case of the mouse and monkey datasets. The highest inter-MSTM structure correlations between synthetic and biological datasets are obtained for the mouse dataset, where they attain 0.48 for the core set and 0.62 for the extended set. In this analysis, the rat dataset appears as the most dissimilar to synthetic datasets; however, it is relatively similar to the mouse dataset, with correlation values of 0.45 for both the core and the extended set. These patterns of similarities and differences might reflect dynamical characteristics in the collective coordination repertoire of brain networks that vary across different species, brain areas and brain states.
Discussion
The advancement of science relies heavily on the development of analysis methods that are appropriate for quantifying the phenomena of interest. In the neurosciences, these phenomena often involve the dynamic coordination between a large number of interacting elements, be they ionic channels, spiking neurons, or brain areas, depending upon the spatial and temporal scales considered. Dynamic coordination phenomena are often characterized in terms of synchronous activations or deactivations, which can occur in the presence or absence of oscillatory or pseudo-rhythmic features [1,5,14,25]. Given the sheer number of methods for their quantification, and the key role they play in the interpretation of experimental data and sometimes even in the experimental design and execution, it is essential to provide tools that enable the comparison between methods that aim to capture the same or a similar phenomenon. This can greatly facilitate the selection of the most suitable methods for specific experimental needs, and can readily provide a benchmark against which a newly proposed method can be compared in terms of its empirical properties. Here, we provide such a tool by presenting a highly comparative analysis of methods for the quantification of synchrony and, more generally, phase relationships. The methods are applied to neuronal firing patterns, but they can be readily applied to any other data which can be represented as a point process, either directly or through the application of a suitable transformation such as thresholding or any other method that can be used to extract events of interest. For example, thresholding in fMRI data can be used to obtain a point process from the continuous BOLD signal [98]. In the case of M/EEG, events can be extracted by considering prominent local maxima and/or minima for each channel or reconstructed source [66,99]. Comparisons between spike train synchrony measures have been presented in several previous studies [54,100–103]; however, the MSTM library introduced here constitutes the most extended collection of spike train measures assembled to date, and has been evaluated with respect to the most extensive and diverse set of performance criteria.
We benchmarked our MSTM library on a broad array of synthetic and biological multineuron spike trains. The adoption of synthetic spike trains obtained from multiple generative models enables a precise quantification of each measure’s ability to detect changes in synchrony, along with its sensitivity to pseudo-rhythmic oscillations and to confounding factors such as firing rate or population frequency. In addition, it enables the quantification of finite-size effects in both time (window length) and space (number of neurons). Our results show that these characteristics vary greatly across measures. There is no “best” measure though, hence the choice of which measures to adopt will depend on the experimental settings and the research questions being examined. We considered two different synthetic spike train formalisms, simple enough to be described by just 5 parameters and adopting general assumptions based on simple and widely used statistical distributions, to increase the generality of our results. However, it is still possible that some of the observed performance differences across MSTMs could be partly dependent on these specific formalisms. In particular, it could be possible to design spike train formalisms that favor a particular MSTM, potentially yielding top performance for a MSTM that is not among those that correlate more closely and selectively with the parameters determining synchrony in our analyses.
Differences across synchrony measures can be first characterized on the basis of their formal definition. In particular, some measures are bivariate in nature, and operate by considering all pairs of non-identical single-neuron spike trains, and then averaging across pairs. Other measures are multivariate, and operate by examining the relationships between all single-neuron spike trains collectively. Our analysis shows that this distinction explains only a small part of the empirical variability across measures. For example, bivariate and multivariate measures often displayed highly correlated behavior across spike trains and were consequently grouped together at low linkage distances in our hierarchical clustering analysis (Fig 4).
Comparison between MSTMs’ behavior on synthetic spike trains
Overall, the MSTM that most closely correlates with synchrony in our experiments is the Correlation index Ci [55] (Fig 3). This MSTM is only moderately affected by pseudo-rhythmic oscillations and is relatively insensitive to potential confounds related to differences in firing rate or population frequency. It also displays low temporal finite-size bias, but it is not among the best MSTMs in terms of temporal finite-size precision and spatial finite-size effects.
A measure derived from the FOOOF spectral estimation, FOOOF power (), achieves a high correlation with the parameter determining synchrony in dual-scale trains. However, a numerical value for that MSTM cannot be obtained for every spike train, since a spectral peak with an amplitude exceeding a certain threshold is required, as explained in the Methods subsection “Multineuron spike train measures”.
Tiesinga–Sejnowski synchrony STS is also closely correlated with the parameter determining synchrony for all synthetic spike train families (Fig 3). However, it is highly sensitive to pseudo-rhythmic oscillations, especially for the dual-scale synthetic families. It is worth noting that STS markedly decreases with oscillations in dual-scale synthetic spike trains, while it is relatively unaffected (or can even display small increases, depending on parameter values) in single-scale spike trains (Figs 2 and S1), indicating that oscillations affect this MSTM in a way that depends on the specific structure of the synthetic spike train families considered.
Another measure that achieves good performance is the spike time tiling coefficient STTC [54]. This MSTM correlates closely and relatively selectively with synchrony in both single-scale and dual-scale trains, and is only modestly biased when varying or
. However, it is highly variable across different realizations of the same generative model (S4Ab and S4Bb Fig).
Other MSTMs that perform relatively well are Golomb–Rinzel synchrony SGR [49], pairwise phase consistency PPC [58], and Kruskal correlation CK [64]. These measures exhibit relatively high and selective correlation with synchrony, but they are not among the best measures in terms of bias. It is worth noting that PPC is a measure of phase consistency irrespective of lag, and will hence yield a high value also in the case of consistent phase relationships with non-zero lag. As expected from a phase-based measure, it is sensitive to pseudo-rhythmicity in dual-scale trains.
The Victor–Purpura and the van Rossum distances [59,61], which are classic measures of spike train distance that have been used extensively as measures of synchrony, are mostly sensitive to firing rate (Fig 3). Correspondingly, in the hierarchical clustering analysis, they are clustered with the univariate firing rate at a very low linkage distance (Fig 4). Hence, they are meaningful for the evaluation of synchrony only in those cases where the spike trains to be compared have equal firing rates. Normalized versions that take into account the linear component of their dependency on the number of spikes have been proposed here and in previous work [60]; however, while normalization greatly ameliorates this issue, they still retain a strong relationship with firing rate, stronger than their relationship with the parameter determining synchrony. In the hierarchical clustering analysis, these normalized measures are clustered together at a very low linkage distance with Schreiber correlation [63], Hunter–Milton similarity [65] and Quian Quiroga event synchronization [66], all MSTMs based on a similar correlation-based approach and sharing a marked dependence on firing rate.
Interestingly, Sqq, a measure proposed for the quantification of delay asymmetry [66], is among the MSTMs that best correlate with Σ, the parameter determining synchrony in the dual-scale synthetic families. This is observed even in the case of non-sequential dual-scale spike trains, where the expected value of Sqq as the time window duration is increased is 0. Hence, in this case this MSTM closely follows the level of synchrony, as parameterized by Σ, by virtue of its finite-size bias only.
Thomas Kreuz and coworkers proposed several measures of synchrony based on nearest spike approaches which have some desirable properties, including the fact of being parameter free and timescale independent [50,56,57]. These MSTMs are similar to other parameter-free and timescale independent MSTMs such as SQQA [66], PPC [58] and SC [52]. However, these MSTMs are not among those that correlate most closely and most selectively with the parameter determining synchrony in the synthetic spike train families (Fig 3). In particular, they tend to be very sensitive to , the spike failure probability, as expected from their nearest spike based approach.
In our approach, MSTMs are evaluated based on how their output values change in response to changes in the generative parameters. We believe that this is the most important MSTM property for experimental applications, which often involve the assessment of statistical differences between conditions. In particular, we did not consider absolute MSTM output values. We note, however, that other authors designed or evaluated MSTMs based also on the expected numeric values they yield for reference spike trains, such as perfectly correlated or anticorrelated trains, or completely random trains [48,49,53,54,56–58,66,76].
When deciding on the best MSTMs to consider for a given experimental setting, it is also important to take into account their finite-size variability and bias in both time and space (Figs 5 and S4). Temporal finite-size effects are going to be especially critical in the common scenarios where a time-resolved measure is desired, or where an experimental task of interest is comprised of different phases with relatively short duration. Spatial finite-size effects are also ubiquitous, and can complicate the assessment of differences in synchrony between recordings with different numbers of neurons, and even between conditions within the same recording, if the number of participating neurons differs.
While some MSTMs can be positively or negatively biased depending on spike generation parameters, other MSTMs are consistently biased. With respect to varying time window length, these include MSTMs that strongly depend on firing rate such as DVP and DvR, which exhibit a marked negative bias (as expected), but also other MSTMs that are not as sensitive to firing rate, such as Sqq, Sqqa and FS, which display a consistent positive bias. Finite-size biases can be substantial also when varying the number of neurons, with multivariate measures being most strongly affected, as expected. In particular, two of the multivariate measures that exhibited the strongest and most selective correlation with the parameter determining synchrony (Fig 3), SGR and STS, are those with the greatest positive bias as the number of neurons is varied. Hence, these MSTMs might signal a decrease in synchrony when only the number of participating neurons has increased, or vice versa.
Comparison between MSTMs’ behavior on biological spike trains
Finally, we applied this battery of MSTMs to biological spike trains recorded in different species—rat, mouse and monkey—and brain areas—primary sensory cortices and hippocampus, and illustrated how the highly comparative approach presents distinct advantages in terms of visualization and classification of multineuron firing patterns.
In particular, the highly comparative approach enables the visualization of multiple aspects of neuronal activity and collective coordination simultaneously (Fig 6). The evolution of neuronal activity along time and recording sessions can be conveniently visualized after ordering time windows chronologically and MSTMs according to a hierarchical clustering procedure across MSTMs, whereby empirically similar MSTMs are displayed nearby, while the prominence and distinctiveness of specific coordination states can be conveniently visualized if time windows are also ordered according to a hierarchical clustering procedure (across time windows).
We also illustrated how this approach can be leveraged for supervised classification, in particular for the estimation of recording-wise distinctiveness (Fig 7) and for discriminating between awake and NREM sleep (Figs 8, 9, and 10, S9, S10 and S11). Among the most discriminating MSTMs for the latter, we found some MSTM that has already been emphasized as displaying a high correlation with the parameter determining synchrony in synthetic spike trains, such as STS. However, we also found univariate measures of instantaneous ISI variability such as LvR, and measures of spectral fitting quality such as the FOOOF R-squared, which are rarely considered in studies on neuronal synchrony and oscillations. In fact, among the top 7 MSTMs at the population level, 5 of them are univariate measures of instantaneous ISI variability. This is a surprising result, given that variations in consciousness state are thought to be mediated by changes in collective neuronal coordination patterns. Other univariate measures yielded only marginally significant results (as in the case of CVISI, a classic measure of global ISI variability which is sensitive to nonstationarity and firing rate) or non-significant results (as in the case of the mean firing rate r). This further emphasizes the utility of adopting a highly comparative approach where multiple MSTMs at various levels of description are contrasted. Among the bivariate and multivariate measures, the most discriminative ones do not correspond with those that exhibit the most selective correlation with the parameter determining synchrony in synthetic spike trains (Fig 3), but tend to capture a combination of features. For example, STS is sensitive to both synchrony and pseudo-rhythmic properties of input trains. Hence, while measure selectivity is surely a desirable property, and chiefly so if interpretability is key, it might not be a decisive factor if decoding accuracy is the main objective.
In the particular dataset we considered [85], decoding accuracy is high even when MSTMs are considered individually, and increases only slightly when considering MSTM pairs (from 0.982 to 0.989 in the best recording, and from 0.789 to 0.825 at the population level for within-recording decoding; and from 0.783 to 0.805 for across-recording decoding). There is a high heterogeneity across recordings, with one recording yielding nearly perfect accuracy for the best MSTM, and another yielding results that are compatible with chance. The decoding accuracy that results from combining every single MSTM with every other MSTM can also be used as a criterion for analyzing the space of MSTMs and identifying synergistic or redundant relationships between MSTMs. We observed that the most synergistic pairs tend to combine measures of different kinds, such as a bivariate measure of synchrony and a multivariate spectral measure. In particular, spectral measures figured prominently among the most synergistic pairs.
It is worth noting that, even in the best recording—exhibiting nearly perfect decoding accuracy for the top-performing MSTM—many MSTMs performed at chance level. This further highlights the value of the highly comparative methodology: if two separate research teams were to analyze this recording to determine the relationship between consciousness state and spike synchrony, with one team employing a top-performing MSTM (e.g., STS), and the another using one that performs poorly (e.g., CS), they would reach completely different results. If they adopted only a minimal level of comparison to potential alternatives, by considering a single additional MSTM that in each case performs similarly to their initially chosen MSTM (say, DS for the first team, and SQQA for the second), their confidence in their respective results would likely be strengthened. This scenario emphasizes the benefit of highly comparative analyses in obtaining robust and reliable research outcomes.
Another important message obtained from a highly comparative analysis reflects the interaction of recording variability and measure variability. Only a few measures performed consistently well across the majority of the recordings, while many others performed well only on one recording, yielding non-significant results for the other recordings. Only a broad comparison can inform on the relative attributes of each measure and guide the selection of the most appropriate measure for capturing the effect of interest within each specific application and dataset, depending on the desired balance between generalizability with lower accuracy, versus specificity with potentially higher accuracy. Considering a range of decoding analyses across multiple datasets could expose this kind of variability to an even greater extent, and deepen our understanding of both measures as well as datasets, and the relevant physiological phenomena they reflect. This perspective can foster the development of more refined research methodologies, and aid in the integration of scientific results towards a more coherent understanding.
Synchrony as a multidimensional concept
To assess the extent to which the highly comparative analysis yielded relatively independent MSTMs, hence providing an intrinsically high-dimensional description of multineuron spike trains, we estimated the intrinsic dimensionality of the highly comparative data matrix, where each row corresponds to a MSTM and each column to a spike train. While synthetic spike trains yielded relatively low values (between 3 and 4 at most scales), with little difference between the core and the extended sets of MSTMs, consistent with the mathematical formalisms they originate from, biological spike trains could result in considerably higher values, especially in the case of the mouse dataset, where intrinsic dimension reached a value of 12.9 (11) at the shortest scale for the extended (core) set of MSTMs. It is worth noting that this is the only dataset where animals were awake, behaving, and transiting through different arousal levels, while the other two datasets were acquired under anesthesia and did not yield ID estimates above 9. More stereotyped network activity under anesthesia has also been reported in previous work [104]. These results suggest a view of spike train coordination as a multidimensional concept that is best quantified by considering a battery of MSTMs, instead of one or a few measures, preselected with only minimal comparison to potential alternatives.
Some spike train features that are not directly related to synchrony but can affect certain synchrony measures in a systematic manner have been considered in synthetic spike trains, where firing rate, the probability of missing spikes, collective frequency, the degree of sequentialness and the presence or absence of pseudo-rhythmic oscillations have been systematically varied. Other aspects that could underlie the diversity across MSTMs’ behavior are the heterogeneity across spike trains (which is itself a multidimensional concept) and the relative proportion of synchronous silence vs. synchronous activity, or, more generally, the shape of the instantaneous spike probability waveform.
All these aspects potentially contribute to the high dimensionality of the highly comparative matrices obtained from biological datasets. Some of these aspects can be important for a thorough understanding of the physiological mechanisms underlying neuronal network activity. For example, a study addressing spiking variability in the cortex demonstrated that neuronal noise correlations (i.e., common fluctuations among neurons that occur in conditions of constant sensory input and motor output) are mostly contributed by neuronal coinactivation, rather than coactivation [105]. A measure that selectively captures coactivation (and can be easily modified to capture coinactivation) has already been proposed for the characterization of fMRI functional connectivity [106]; adaptations of this measure to multivariate point processes could be included in future extensions of this MSTM library. While the importance of waveform shape in lumped electrophysiological signals such as LFP, ECoG and M/EEG recordings is increasingly recognized [107], this aspect is often overlooked in the analysis of multineuron firing patterns. However, it is likely to be at least as informative as in the formers, given that these are mostly contributed by the summation of extracellular fields resulting from neuronal spiking. These observations further emphasize the need to enrich our understanding of synchrony, and our operational capability to quantify it, by developing measures that selectively capture each of the different aspects that comprise the multidimensional concept of synchrony or, even more generally, spike train coordination.
The highly comparative approach offers a promising tool for the design of such measures. Comparing each measure with a library of already established measures is expected to greatly facilitate the design of new measures that selectively capture a specific aspect of collective spike train coordination which is not already captured by the library, or not selectively captured by any of its constituent measures. This endeavor will also be facilitated by the concomitant design of suitable synthetic spike train formalisms, enabling the validation of candidate measures in conditions where the ground truth can be precisely controlled. Potential directions for future research are proposed in the Supplementary Discussion (S1 Text).
Conclusion
We presented a highly comparative analysis of measures of multineuron spike train synchrony, oscillations, and phase relationships. We first validated the battery of measures on synthetic spike trains generated with two different formalisms, where the ground truth in terms of synchrony and potentially confounding factors such as firing rate, population frequency, spike failure probability, pseudo-rhythmicity and sequentialness can be precisely controlled. The analysis of synthetic spike trains enabled the quantification of the correlation between each measure and each generative parameter, profiling each measure in terms of its dependance with the parameter determining synchrony as well as with the other, non-synchrony related parameters. It also enabled the analysis of bias and variability resulting from finite time window length and number of neurons, crucial but often overlooked factors for empirical applicability. Then, we applied the battery of measures to biological multineuron spike trains obtained from three different species and brain areas, and showed how the highly comparative approach can be leveraged for both unsupervised and supervised applications. While best measures for each considered criterion are clearly highlighted, there is no overall “best” measure. Hence, the selection of a measure or a set of measures is left to the criterion of each individual researcher. However, this decision no longer needs to be based mostly on habit or familiarity, or on comparison among a narrow set of candidates. Instead, it can now be made with the cautious awareness of the advantages and drawbacks of each measure as compared to a broad range of alternatives. This includes considering the desired or admissible balance between strength and selectivity of the correlation with the parameter determining synchrony, bias and variability in time and space, interpretability and decoding performance. Importantly, the code that accompanies this article (available at https://github.com/GNB-UAM/sync_osc_MSTMs) will enable researchers to select suitable measures for their specific criteria and research goals, and facilitates the extension of the current framework to other measures of spike train synchrony, oscillations, or phase relationships, synthetic spike generation formalisms, and biological datasets. These could eventually be included in a self-organizing, community-driven, evolving library of spike train methods, generative models and biological datasets, in the spirit of the living library of univariate time series CompEngine [108], which could greatly foster the structured organization of accumulated knowledge and facilitate novel insights in this field.
Supporting information
The supporting information is also available as a single file at https://doi.org/10.17605/OSF.IO/BXT6V.
S1 Text. Supplementary Methods and Supplementary Discussion.
https://doi.org/10.1371/journal.pcbi.1013597.s001
(PDF)
S1 Fig. Assessment of synchrony on synthetic spike trains.
Assessment of the level of synchrony on single-scale (left) and dual-scale (right) synthetic spike trains, considering multiple MSTMs. Left: Synchrony is plotted as a function of the modulation amplitude m for average firing rates r0 and population rates f0 varying in the grid [4,12,36] Hz × [4,12,36] Hz. Results for different average firing rates r0 are plotted in separate columns. Right: Synchrony is plotted as a function of the population width Σ for population rates f0 and spike deletion probability varying in the grid [4,12,36] Hz × [0.8,0.4,0]. Results for different spike deletion probabilities
are plotted in separate columns. In all panels, the population rate f0 varies in the grid [4,12,36] Hz and is color-coded, with warmer colors indicating higher f0. Solid lines: pseudo-rhythmic spike trains; dash lines: non-rhythmic spike trains.
https://doi.org/10.1371/journal.pcbi.1013597.s002
(PNG)
S2 Fig. Hierarchical clustering of measures, considering multiple timescales for the timescale-specific measures.
A: Dendrogram showing the distances between MSTMs as a hierarchical cluster tree. Measures are sorted along the x axis in increasing order of the dendrogrammatic distance of the first nonsingleton cluster they are grouped in. B: Similarity matrix showing the absolute value of the Spearman correlation coefficient between each pair of MSTMs. Measure labels are color-coded to indicate measure type (green: univariate; blue: bivariate; red: multivariate). Numbers displayed at the end of timescale-dependent measures indicate the timescale in milliseconds. For FOOOF spectral measures, 1p (2p) refers to the single-peak (dual-peak) model; in the dual-peak model, the number at the end of the parameter name indicates the corresponding peak.
https://doi.org/10.1371/journal.pcbi.1013597.s003
(PNG)
S3 Fig. Effects of time window length and number of neurons for a subset of selected measures.
Effects of sample size variations in time (A) or space (B) for a subset of selected MSTMs. A: For each MSTM, the mean value across windows is plotted as a function of window length. Shaded areas indicate the SD across windows. Different columns correspond to different spike train families and generative parameter values: single-scale pseudo-rhythmic spike train with Hz, m = 0.5 (non-sequential, left; sequential with Dc = 0.2, center-left); dual-scale pseudo-rhythmic spike train with
Hz,
,
, (non-sequential, center-right; sequential with Dc = 0.2, right). MPC and PPC could not be estimated at the shortest time window length considered due to insufficient number of spikes. B: As in (A), but sample size varies in space instead of time.
https://doi.org/10.1371/journal.pcbi.1013597.s004
(PNG)
S4 Fig. Bias and variability resulting from a finite time window length or a finite number of neurons.
A: Bias and variability resulting from a finite time window length. Measures are ordered accordingly to increasing bias (a) or variability (b). Bars indicate mean values across synthetic spike train families, synchrony values and time window length , with individual values shown by symbols as indicated in the legend. Other parameters are fixed at intermediate values:
Hz for the single-scale family,
Hz and
for the dual-scale family, Dc = 0 or 0.2 for non-sequential and sequential trains, respectively. Results corresponding to time windows of increasing length are plotted from left to right for each MSTM. Variability is plotted against bias in (c); best behavior corresponds to minimum
and
. Some measures are omitted for clarity. In (a), the gray background indicates MSTMs with bias
> 0.1 (light gray) or
> 1 (dark gray). In (b), the gray background indicates MSTMs with variability
> 0.1 (light gray) or
> 1 (dark gray). Note the use of a symmetric log scale for bias in (a) and (c). B: As in (A), but sample size varies in space instead of time. Corresponding results are plotted for each individual
(
) value separately in S5 Fig (S6). This figure conveys information on temporal and spatial finite-size effects at multiple levels of detail, from average values (bars) to individual results for each spike train formalism (symbols), with symbol color coding for the level of synchrony and symbol type coding for spike train formalism.
https://doi.org/10.1371/journal.pcbi.1013597.s005
(PNG)
S5 Fig. Bias and variability resulting from a finite time window length at different timescales.
The same information plotted in S4A Fig is replotted separately for each value of the time window length : results corresponding to short (
ms), intermediate (
s) and long (
s) time windows are plotted in the top, middle, and bottom row, respectively.
https://doi.org/10.1371/journal.pcbi.1013597.s006
(PNG)
S6 Fig. Bias and variability resulting from a finite number of neurons at different spatial scales.
The same information plotted in S4B Fig is replotted separately for each value of the number of neurons : results corresponding to a low (
), intermediate (
) and high (
) number of neurons are plotted in the top, middle, and bottom row, respectively.
https://doi.org/10.1371/journal.pcbi.1013597.s007
(PNG)
S7 Fig. Visualizing collective dynamic coordination in spiking activity through the lens of the highly comparative approach.
As in Fig 6, for the mouse (A) and monkey (B) datasets.
https://doi.org/10.1371/journal.pcbi.1013597.s008
(PNG)
S8 Fig. Spiking activity exhibits structured variability that is distinctive of each recording.
As in Fig 7, for the mouse (A) and monkey (B) datasets.
https://doi.org/10.1371/journal.pcbi.1013597.s009
(PNG)
S9 Fig. Univariate decoding results for each individual recording and for each MSTM of the extended set of 131 MSTMs.
As in Fig 8A, for each individual recording and for each MSTM of the extended set of 131 MSTMs.
https://doi.org/10.1371/journal.pcbi.1013597.s010
(PNG)
S10 Fig. Bivariate decoding results for each individual recording and for each MSTM of the extended set of 131 MSTMs.
https://doi.org/10.1371/journal.pcbi.1013597.s011
As in Fig 9, for each individual recording and for each MSTM of the extended set of 131 MSTMs. The same results are shown with the MSTMs ordered according to decreasing univariate decoding accuracy (left column), and with the MSTMs ordered according to the results of a hierarchical agglomerative clustering analysis over bivariate decoding accuracies (right column), hence displaying nearby MSTMs that result in similar patterns of decoding accuracies when combined with every other MSTMs.
S11 Fig. Discriminating wake vs. NREM sleep from spike train measure pairs: median results across recordings.
As in Fig 9, but median results across recordings are shown.
https://doi.org/10.1371/journal.pcbi.1013597.s012
(PNG)
S12 Fig. Correlation between the inter-MSTM distance matrices obtained from synthetic spike train families and biological spike train datasets.
Correlation between pairs of inter-MSTM distance matrices obtained from synthetic spike train families and biological spike train datasets considering either the core set of 46 MSTMs (A) or the full set of 131 MSTMs (B).
https://doi.org/10.1371/journal.pcbi.1013597.s013
(PNG)
S1 File. MSTMs included in the top 1% most discriminating MSTM pairs along with the corresponding number of such pairs they were included in and their relative frequencies.
https://doi.org/10.1371/journal.pcbi.1013597.s014
(CSV)
S2 File. MSTMs included in the top 1% most synergistic MSTM pairs along with the corresponding number of such pairs they were included in and their relative frequencies.
https://doi.org/10.1371/journal.pcbi.1013597.s015
(CSV)
References
- 1. Buzsáki G. Neural syntax: Cell assemblies, synapsembles, and readers. Neuron. 2010;68(3):362–85. pmid:21040841
- 2. Luczak A, McNaughton BL, Harris KD. Packet-based communication in the cortex. Nat Rev Neurosci. 2015;16(12):745–55. pmid:26507295
- 3. Gray CM, Singer W. Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci U S A. 1989;86(5):1698–702. pmid:2922407
- 4. Salinas E, Sejnowski TJ. Correlated neuronal activity and the flow of neural information. Nat Rev Neurosci. 2001;2(8):539–50. pmid:11483997
- 5. Wang H-P, Spencer D, Fellous J-M, Sejnowski TJ. Synchrony of thalamocortical inputs maximizes cortical reliability. Science. 2010;328(5974):106–9. pmid:20360111
- 6. Buzsáki G, Anastassiou CA, Koch C. The origin of extracellular fields and currents—EEG, ECoG, LFP and spikes. Nat Rev Neurosci. 2012;13(6):407–20. pmid:22595786
- 7. Herreras O. Local field potentials: Myths and misunderstandings. Front Neural Circuits. 2016;10:101. pmid:28018180
- 8. Michel CM, Murray MM. Towards the utilization of EEG as a brain imaging tool. Neuroimage. 2012;61(2):371–85. pmid:22227136
- 9. Lopes da Silva F. EEG and MEG: Relevance to neuroscience. Neuron. 2013;80(5):1112–28. pmid:24314724
- 10. Fries P. A mechanism for cognitive dynamics: Neuronal communication through neuronal coherence. Trends Cogn Sci. 2005;9(10):474–80. pmid:16150631
- 11. Palmigiano A, Geisel T, Wolf F, Battaglia D. Flexible information routing by transient synchrony. Nat Neurosci. 2017;20(7):1014–22. pmid:28530664
- 12. Bastos AM, Lundqvist M, Waite AS, Kopell N, Miller EK. Layer and rhythm specificity for predictive routing. Proc Natl Acad Sci U S A. 2020;117(49):31459–69. pmid:33229572
- 13. Singer W. Neuronal synchrony: A versatile code for the definition of relations?. Neuron. 1999;24(1):49–65, 111–25. pmid:10677026
- 14. Fries P, Reynolds JH, Rorie AE, Desimone R. Modulation of oscillatory neuronal synchronization by selective visual attention. Science. 2001;291(5508):1560–3. pmid:11222864
- 15. Luczak A, Barthó P, Marguet SL, Buzsáki G, Harris KD. Sequential structure of neocortical spontaneous activity in vivo. Proc Natl Acad Sci U S A. 2007;104(1):347–52. pmid:17185420
- 16. Kerschensteiner D, Wong ROL. A precisely timed asynchronous pattern of ON and OFF retinal ganglion cell activity during propagation of retinal waves. Neuron. 2008;58(6):851–8. pmid:18579076
- 17. Havenith MN, Yu S, Biederlack J, Chen N-H, Singer W, Nikolić D. Synchrony makes neurons fire in sequence, and stimulus properties determine who is ahead. J Neurosci. 2011;31(23):8570–84. pmid:21653861
- 18. Eckhorn R. Oscillatory and non-oscillatory synchronizations in the visual cortex and their possible roles in associations of visual features. Prog Brain Res. 1994;102:405–26. pmid:7800830
- 19. König P, Engel AK, Singer W. Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proc Natl Acad Sci U S A. 1995;92(1):290–4. pmid:7816836
- 20. Vanderwolf CH. Hippocampal electrical activity and voluntary movement in the rat. Electroencephalogr Clin Neurophysiol. 1969;26(4):407–18. pmid:4183562
- 21.
O’Keefe J, Nadel L. The hippocampus as a cognitive map. Oxford: Clarendon Press; 1978.
- 22. Ekstrom AD, Caplan JB, Ho E, Shattuck K, Fried I, Kahana MJ. Human hippocampal theta activity during virtual navigation. Hippocampus. 2005;15(7):881–9. pmid:16114040
- 23. Bush D, Bisby JA, Bird CM, Gollwitzer S, Rodionov R, Diehl B, et al. Human hippocampal theta power indicates movement onset and distance travelled. Proc Natl Acad Sci U S A. 2017;114(46):12297–302. pmid:29078334
- 24. Qasim SE, Fried I, Jacobs J. Phase precession in the human hippocampus and entorhinal cortex. Cell. 2021;184(12):3242-3255.e10. pmid:33979655
- 25. Eliav T, Geva-Sagiv M, Yartsev MM, Finkelstein A, Rubin A, Las L, et al. Nonoscillatory phase coding and synchronization in the bat hippocampal formation. Cell. 2018;175(4):1119-1130.e15. pmid:30318145
- 26. Fulcher BD, Little MA, Jones NS. Highly comparative time-series analysis: The empirical structure of time series and their methods. J R Soc Interface. 2013;10(83):20130048. pmid:23554344
- 27. Fulcher BD, Jones NS. Highly comparative feature-based time-series classification. IEEE Trans Knowl Data Eng. 2014;26(12):3026–37.
- 28. Cliff OM, Bryant AG, Lizier JT, Tsuchiya N, Fulcher BD. Unifying pairwise interactions in complex dynamics. Nat Comput Sci. 2023;3(10):883–93. pmid:38177751
- 29. Peach RL, Arnaudon A, Schmidt JA, Palasciano HA, Bernier NR, Jelfs KE, et al. HCGA: Highly comparative graph analysis for network phenotyping. Patterns (N Y). 2021;2(4):100227. pmid:33982022
- 30. Kessler D, Carr CE, Kretzberg J, Ashida G. Theoretical relationship between two measures of spike synchrony: Correlation index and vector strength. Front Neurosci. 2021;15:761826. pmid:34987357
- 31. Shafiei G, Markello RD, Vos de Wael R, Bernhardt BC, Fulcher BD, Misic B. Topographic gradients of intrinsic dynamics across neocortex. Elife. 2020;9:e62116. pmid:33331819
- 32. Decat N, Walter J, Koh ZH, Sribanditmongkol P, Fulcher BD, Windt JM, et al. Beyond traditional sleep scoring: Massive feature extraction and data-driven clustering of sleep time series. Sleep Med. 2022;98:39–52. pmid:35779380
- 33. Shafiei G, Fulcher BD, Voytek B, Satterthwaite TD, Baillet S, Misic B. Neurophysiological signatures of cortical micro-architecture. Nat Commun. 2023;14(1):6000. pmid:37752115
- 34. Bailey NW, Fulcher BD, Caldwell B, Hill AT, Fitzgibbon B, van Dijk H, et al. Uncovering a stability signature of brain dynamics associated with meditation experience using massive time-series feature extraction. Neural Netw. 2024;171:171–85. pmid:38091761
- 35. Bryant AG, Aquino K, Parkes L, Fornito A, Fulcher BD. Extracting interpretable signatures of whole-brain dynamics through systematic comparison. PLoS Comput Biol. 2024;20(12):e1012692. pmid:39715231
- 36. Liu Z-Q, Luppi AI, Hansen JY, Tian YE, Zalesky A, Yeo BTT, et al. Benchmarking methods for mapping functional connectivity in the brain. Nat Methods. 2025;22(7):1593–602. pmid:40481361
- 37. Leung A, Mahmoud A, Jeans T, Fulcher BD, van Swinderen B, Tsuchiya N. Wakefulness can be distinguished from general anesthesia and sleep in flies using a massive library of univariate time series analyses. PLoS Biol. 2025;23(7):e3003217. pmid:40638578
- 38.
Cox DR, Isham V. Point Processes. CRC Press. 1980.
- 39. Perkel DH, Gerstein GL, Moore GP. Neuronal spike trains and stochastic point processes. I. The single spike train. Biophys J. 1967;7(4):391–418. pmid:4292791
- 40. Perkel DH, Gerstein GL, Moore GP. Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys J. 1967;7(4):419–40. pmid:4292792
- 41.
Dayan P, Abbott LF. Theoretical neuroscience: Computational and mathematical modeling of neural systems. Cambridge, MA: MIT Press; 2001.
- 42. Shadlen MN, Newsome WT. The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J Neurosci. 1998;18(10):3870–96. pmid:9570816
- 43. Baroni F, Varona P. Spike timing-dependent plasticity is affected by the interplay of intrinsic and network oscillations. J Physiol Paris. 2010;104(1–2):91–8. pmid:19913095
- 44. Kerr RR, Burkitt AN, Thomas DA, Gilson M, Grayden DB. Delay selection by spike-timing-dependent plasticity in recurrent networks of spiking neurons receiving oscillatory inputs. PLoS Comput Biol. 2013;9(2):e1002897. pmid:23408878
- 45. Matzner A, Bar-Gad I. Quantifying spike train oscillations: Biases, distortions and solutions. PLoS Comput Biol. 2015;11(4):e1004252. pmid:25909328
- 46. Luz Y, Shamir M. Oscillations via spike-timing dependent plasticity in a feed-forward model. PLoS Comput Biol. 2016;12(4):e1004878. pmid:27082118
- 47. Donoghue T, Haller M, Peterson EJ, Varma P, Sebastian P, Gao R, et al. Parameterizing neural power spectra into periodic and aperiodic components. Nat Neurosci. 2020;23(12):1655–65. pmid:33230329
- 48. Tiesinga PHE, Sejnowski TJ. Rapid temporal modulation of synchrony by competition in cortical interneuron networks. Neural Comput. 2004;16(2):251–75. pmid:15006096
- 49. Golomb D, Rinzel J. Dynamics of globally coupled inhibitory neurons with heterogeneity. Phys Rev E Stat Phys Plasmas Fluids Relat Interdiscip Topics. 1993;48(6):4810–4. pmid:9961165
- 50. Kreuz T, Mulansky M, Bozanic N. SPIKY: A graphical user interface for monitoring spike train synchrony. J Neurophysiol. 2015;113(9):3432–45. pmid:25744888
- 51. Kreuz T, Satuvuori E, Pofahl M, Mulansky M. Leaders and followers: quantifying consistency in spatio-temporal propagation patterns. New J Phys. 2017;19(4):043028.
- 52. Ciba M, Isomura T, Jimbo Y, Bahmer A, Thielemann C. Spike-contrast: A novel time scale independent and multivariate measure of spike train synchrony. J Neurosci Methods. 2018;293:136–43. pmid:28935422
- 53. Mormann F, Lehnertz K, David P, E. Elger C. Mean phase coherence as a measure for phase synchronization and its application to the EEG of epilepsy patients. Phys D: Nonlin Phenom. 2000;144(3–4):358–69.
- 54. Cutts CS, Eglen SJ. Detecting pairwise correlations in spike trains: An objective comparison of methods and application to the study of retinal waves. J Neurosci. 2014;34(43):14288–303. pmid:25339742
- 55. Wong RO, Meister M, Shatz CJ. Transient period of correlated bursting activity during development of the mammalian retina. Neuron. 1993;11(5):923–38. pmid:8240814
- 56. Kreuz T, Haas JS, Morelli A, Abarbanel HDI, Politi A. Measuring spike train synchrony. J Neurosci Methods. 2007;165(1):151–61. pmid:17628690
- 57. Kreuz T, Chicharro D, Greschner M, Andrzejak RG. Time-resolved and time-scale adaptive measures of spike train synchrony. J Neurosci Methods. 2011;195(1):92–106. pmid:21129402
- 58. Vinck M, van Wingerden M, Womelsdorf T, Fries P, Pennartz CMA. The pairwise phase consistency: A bias-free measure of rhythmic neuronal synchronization. Neuroimage. 2010;51(1):112–22. pmid:20114076
- 59. Victor JD, Purpura KP. Nature and precision of temporal coding in visual cortex: A metric-space analysis. J Neurophysiol. 1996;76(2):1310–26. pmid:8871238
- 60. Kreiman G, Krahe R, Metzner W, Koch C, Gabbiani F. Robustness and variability of neuronal coding by amplitude-sensitive afferents in the weakly electric fish eigenmannia. J Neurophysiol. 2000;84(1):189–204. pmid:10899196
- 61. van Rossum MC. A novel spike distance. Neural Comput. 2001;13(4):751–63. pmid:11255567
- 62. Christen M, Kohn A, Ott T, Stoop R. Measuring spike pattern reliability with the Lempel-Ziv-distance. J Neurosci Methods. 2006;156(1–2):342–50. pmid:16584787
- 63. Schreiber S, Fellous JM, Whitmer D, Tiesinga P, Sejnowski TJ. A new correlation-based measure of spike timing reliability. Neurocomputing (Amst). 2003;52–54:925–31. pmid:20740049
- 64. Kruskal PB, Stanis JJ, McNaughton BL, Thomas PJ. A binless correlation measure reduces the variability of memory reactivation estimates. Stat Med. 2007;26(21):3997–4008. pmid:17593566
- 65. Hunter JD, Milton JG. Amplitude and frequency dependence of spike timing: Implications for dynamic regulation. J Neurophysiol. 2003;90(1):387–94. pmid:12634276
- 66. Quian Quiroga R, Kreuz T, Grassberger P. Event synchronization: A simple and fast method to measure synchronicity and time delay patterns. Phys Rev E Stat Nonlin Soft Matter Phys. 2002;66(4 Pt 1):041904. pmid:12443232
- 67. Sihn D, Kim S-P. A spike train distance robust to firing rate changes based on the earth mover’s distance. Front Comput Neurosci. 2019;13:82. pmid:31920607
- 68. Rusu CV, Florian RV. A new class of metrics for spike trains. Neural Comput. 2014;26(2):306–48. pmid:24206385
- 69. Dorval AD. Estimating neuronal information: Logarithmic binning of neuronal inter-spike intervals. Entropy (Basel). 2011;13(2):485–501. pmid:24839390
- 70. Holt GR, Softky WR, Koch C, Douglas RJ. Comparison of discharge variability in vitro and in vivo in cat visual cortex neurons. J Neurophysiol. 1996;75(5):1806–14. pmid:8734581
- 71. Shinomoto S, Shima K, Tanji J. Differences in spiking patterns among cortical neurons. Neural Comput. 2003;15(12):2823–42. pmid:14629869
- 72. Shinomoto S, Kim H, Shimokawa T, Matsuno N, Funahashi S, Shima K, et al. Relating neuronal firing patterns to functional differentiation of cerebral cortex. PLoS Comput Biol. 2009;5(7):e1000433. pmid:19593378
- 73. Davies RM, Gerstein GL, Baker SN. Measurement of time-dependent changes in the irregularity of neural spiking. J Neurophysiol. 2006;96(2):906–18. pmid:16554511
- 74. Bhumbra GS, Dyball REJ. Measuring spike coding in the rat supraoptic nucleus. J Physiol. 2004;555(Pt 1):281–96. pmid:14608010
- 75. Miura K, Okada M, Amari S-I. Estimating spiking irregularities under changing environments. Neural Comput. 2006;18(10):2359–86. pmid:16907630
- 76. Kuramoto Y, Nishikawa I. Statistical macrodynamics of large dynamical systems. Case of a phase transition in oscillator communities. J Stat Phys. 1987;49(3–4):569–605.
- 77.
Brockwell PJ, Davis RA. Time series: Theory and methods. Springer New York; 1991. https://doi.org/10.1007/978-1-4419-0320-4
- 78. Machens CK, Schütze H, Franz A, Kolesnikova O, Stemmler MB, Ronacher B, et al. Single auditory neurons rapidly discriminate conspecific communication signals. Nat Neurosci. 2003;6(4):341–2. pmid:12652305
- 79. Narayan R, Graña G, Sen K. Distinct time scales in cortical discrimination of natural sounds in songbirds. J Neurophysiol. 2006;96(1):252–8. pmid:16571738
- 80. Panzeri S, Brunel N, Logothetis NK, Kayser C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 2010;33(3):111–20. pmid:20045201
- 81. Sun W, Barbour DL. Rate, not selectivity, determines neuronal population coding accuracy in auditory cortex. PLoS Biol. 2017;15(11):e2002459. pmid:29091725
- 82. Runyan CA, Piasini E, Panzeri S, Harvey CD. Distinct timescales of population coding across cortex. Nature. 2017;548(7665):92–6. pmid:28723889
- 83. See JZ, Atencio CA, Sohal VS, Schreiner CE. Coordinated neuronal ensembles in primary auditory cortical columns. Elife. 2018;7:e35587. pmid:29869986
- 84. See JZ, Atencio CA, Schreiner CE. High-density extracellular recordings from the primary auditory cortex in anesthetized rats listening to dynamic broadband stimuli. CRCNS.org. 2018.
- 85. Formozov A, Chini M, Dieter A, Yang W, Pöpplau JA, Hanganu-Opatz IL, et al. Calcium Imaging and electrophysiology of hippocampal activity under anesthesia and natural sleep in mice. Sci Data. 2022;9(1):113. pmid:35351935
- 86. Formozov A, Chini M, Dieter A, Yang W, Pöpplau JA, Hanganu-Opatz IL. Calcium imaging and electrophysiology of hippocampal activity under anesthesia and natural sleep in mice. G-Node. 2022.
- 87. Smith MA, Kohn A. Spatial and temporal scales of neuronal correlation in primary visual cortex. J Neurosci. 2008;28(48):12591–603. pmid:19036953
- 88.
Kohn A, Smith MA. Utah array extracellular recordings of spontaneous and visually evoked activity from anesthetized macaque primary visual cortex (V1). CRCNS.org. https://crcns.org/data-sets/v1/utah-array-extracellular-recordings. 2016.
- 89. Eckhorn R, Frien A, Bauer R, Woelbern T, Kehr H. High frequency (60-90 Hz) oscillations in primary visual cortex of awake monkey. Neuroreport. 1993;4(3):243–6. pmid:8477045
- 90. Fisahn A, Pike FG, Buhl EH, Paulsen O. Cholinergic induction of network oscillations at 40 Hz in the hippocampus in vitro. Nature. 1998;394(6689):186–9. pmid:9671302
- 91. Friedrich RW, Habermann CJ, Laurent G. Multiplexing using synchrony in the zebrafish olfactory bulb. Nat Neurosci. 2004;7(8):862–71. pmid:15273692
- 92. van Aerde KI, Mann EO, Canto CB, Heistek TS, Linkenkaer-Hansen K, Mulder AB, et al. Flexible spike timing of layer 5 neurons during dynamic beta oscillation shifts in rat prefrontal cortex. J Physiol. 2009;587(Pt 21):5177–96. pmid:19752121
- 93. Lacey MG, Gooding-Williams G, Prokic EJ, Yamawaki N, Hall SD, Stanford IM, et al. Spike firing and IPSPs in layer V pyramidal neurons during beta oscillations in rat primary motor cortex (M1) in vitro. PLoS One. 2014;9(1):e85109. pmid:24465488
- 94. Yang W, Chini M, Pöpplau JA, Formozov A, Dieter A, Piechocinski P, et al. Anesthetics fragment hippocampal network activity, alter spine dynamics, and affect memory consolidation. PLoS Biol. 2021;19(4):e3001146. pmid:33793545
- 95. Facco E, d’Errico M, Rodriguez A, Laio A. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Sci Rep. 2017;7(1):12140. pmid:28939866
- 96. Denti F, Doimo D, Laio A, Mira A. The generalized ratios intrinsic dimension estimator. Sci Rep. 2022;12(1):20005. pmid:36411305
- 97. Glielmo A, Macocco I, Doimo D, Carli M, Zeni C, Wild R, et al. DADApy: Distance-based analysis of data-manifolds in Python. Patterns (N Y). 2022;3(10):100589. pmid:36277821
- 98. Tagliazucchi E, Balenzuela P, Fraiman D, Chialvo DR. Criticality in large-scale brain FMRI dynamics unveiled by a novel point process analysis. Front Physiol. 2012;3:15. pmid:22347863
- 99. Kreuz T, Chicharro D, Houghton C, Andrzejak RG, Mormann F. Monitoring spike train synchrony. J Neurophysiol. 2013;109(5):1457–72. pmid:23221419
- 100.
Pinsky PF, Rinzel J. Synchrony Measures for Biological Neural Networks. Biological Cybernetics. 1995 ;73(2):129–137. https://doi.org/10.1007/BF00204051.
- 101. Kreuz T, Mormann F, Andrzejak RG, Kraskov A, Lehnertz K, Grassberger P. Measuring synchronization in coupled model systems: A comparison of different approaches. Physica D: Nonlinear Phenomena. 2007;225(1):29–42.
- 102. Satuvuori E, Kreuz T. Which spike train distance is most suitable for distinguishing rate and temporal coding?. J Neurosci Methods. 2018;299:22–33. pmid:29462713
- 103. Ciba M, Bestel R, Nick C, de Arruda GF, Peron T, Henrique CC, et al. Comparison of different spike train synchrony measures regarding their robustness to erroneous data from bicuculline-induced epileptiform activity. Neural Comput. 2020;32(5):887–911. pmid:32187002
- 104. Dasilva M, Camassa A, Navarro-Guzman A, Pazienti A, Perez-Mendez L, Zamora-López G, et al. Modulation of cortical slow oscillations and complexity across anesthesia levels. Neuroimage. 2021;224:117415. pmid:33011419
- 105. Mochol G, Hermoso-Mendizabal A, Sakata S, Harris KD, de la Rocha J. Stochastic transitions into silence cause noise correlations in cortical circuits. Proc Natl Acad Sci U S A. 2015;112(11):3529–34. pmid:25739962
- 106.
Meskaldji D-E, Morgenthaler S, Van De Ville D. New measures of brain functional connectivity by temporal analysis of extreme events. In: 2015 IEEE 12th international symposium on biomedical imaging (ISBI); 2015. p. 26–9. https://doi.org/10.1109/isbi.2015.7163808
- 107. Cole SR, Voytek B. Brain oscillations and the importance of waveform shape. Trends Cogn Sci. 2017;21(2):137–49. pmid:28063662
- 108. Fulcher BD, Lubba CH, Sethi SS, Jones NS. A self-organizing, living library of time-series data. Sci Data. 2020;7(1):213. pmid:32636393