Fig 1.
Copula families used in the mixture models in our framework.
Each panel shows a scatter plot of samples drawn from a parametric copula family (named in the title of each plot) with a fixed parameter θ (shown in the bottom right corner). In total, we used 10 different copula elements—Gaussian + Frank + 4 × Clayton + 4 × Gumbel—to construct copula mixtures. The rightmost figure shows a mixture of two copulas from different copula families taken with equal mixing weights (0.5). Blue points here correspond to the samples drawn from a Clayton copula, orange points—to a 90°-rotated Gumbel copula. Note, that a mixture of copulas is a copula itself.
Fig 2.
Copula-GP finds that uncoupled neurons are independent given the stimulus.
A GLM model of two identical uncoupled neurons that receive the same time-dependent input x(t); B simulated calcium transients (fluorescence across time) showing dynamic responses to the stimulus x(t) for one of the neurons; C calcium transients of two neurons (y1(t), y2(t)) projected onto a unit cube by the probability integral transform based on unconditional marginals; colored points show transformed samples (u1, u2) corresponding to times t (color-coded). The clusters of similarly colored points (e.g. green) illustrate that the copula c(u) depends on time t; the particular shape and the location of the clusters depends on the function x(t); only 10% of data-points are shown (selected randomly). D same as C, but based on conditional marginals Fi(yi|t). The resulting empirical copula describes ‘noise correlations’ between two neurons. The colored data-points (,
) are uniformly distributed on the unit square, which suggests that there is no noise correlation between these neurons, the copula c(ut) is independent of time t, and the neurons are independent given the time-dependent stimulus.
Fig 3.
Copula-GP describes the noise correlations between dynamically coupled neurons with a Clayton copula.
A i. GLM model of two coupled neurons (excitatory and inhibitory) that receive the same time-dependent input x(t); ii. the spike history coupling filters h12 and h21; B-C simulated calcium transients (fluorescence across time) showing dynamic responses to the stimulus x(t) for excitatory and inhibitory neurons, respectively; D calcium transients of two neurons (y1(t), y2(t)) projected onto a unit cube by the probability integral transform based on unconditional marginals; colored points show transformed samples (u1, u2) corresponding to times t (color-coded). The clusters of similarly colored points (e.g. green) illustrate that the copula c(u) depends on time t; the particular shape and the location of the clusters depends on the function x(t). E same as D, but based on conditional marginals Fi(yi|t). The resulting copula describes ‘noise correlations’ between two neurons. The colored data-points (,
) are not uniformly distributed on the unit square, which suggests that the noise correlation between these neurons and the copula c(ut) itself depends on time t. F Clayton copula parameter (θ) that characterizes the strength of the non-linear noise correlation between neurons (see Methods for details); G probability density plots illustrating the stimulus-dependent shape of noise correlations. The empirical dependence estimated from data samples is shown with black outlines, while the predictions of the Clayton copula model are shown in shades of blue. The proportion of the variance explained
is indicated in the upper-right corner for each time interval. Orange circles indicate the heavy tail of the distribution, which can be best seen in the range t ∈ [0.6, 1.0] where the variables are stronger correlated.
Fig 4.
Comparison of the Copula-GP model against the non-parametric information estimators, performed on three benchmarking datasets A Multivariate Gaussian. B Multivariate Student T. C Multivariate Gaussian y (same as A), morphed into another distribution with a tail dependence, while
. In each row, the plots show: i. the probability density plots from each dataset: the unconditional dependency structure p(u) (left) and conditional dependency structures at the beginning and the end of the parameter domain dom x = [0, 1] (middle and right, respectively). ii. conditional entropy H(y|x); the black line shows the true values, the red line—Copula-GP, the orange line—BI-KSG; Note, that MINE is not included in this comparison, as it does not produce estimates of H(y|x). iii. mutual information I(x, y); black line—true value; red—Copula-GP (solid: MC integration (12); dashed: estimated MI (13)); orange—BI-KSG; green—KSG; blue—MINE (dotted: 100 HU, dashed: 200 HU, solid: 500 HU). Gray intervals show either standard error of mean (SE, 6 repetitions), or
for integrated variables. Note, that MINE estimates are sensitive to the choice of hyper-parameters (e.g. number of hidden units, shown in different line styles).
Fig 5.
Validation of Copula-GP method on neuronal population activity and behavioral variables from awake mice.
Copula-GP accurately models the neuronal and behavioral heavy tailed dependencies in the data from the visual cortex of awake mice, and quantifies more mutual information between various combinations of variables than the alternative methods. A Schematic of the navigational experimental task [5, 16] in virtual reality; B Example traces from ten example trials: x is a position in virtual reality, y is a vector of neuronal (blue) and behavioral (red) variables; these traces show that variables have different timescales and different signal-to-noise ratios, which result in different distributions of single variables yi (i.e. different marginal statistics). C-D Copula probability density plots for: the noise correlation between two neurons (number 3 and 63) (C) and for the correlation between one neuronal activity (60) vs. one behavioral variable (licks) (D); Black outlines show empirical copula, shades of blue—the best fitting Copula-GP model: a mixture of Gaussian + 90°-rotated Clayton copula in (C) and a mixture of Frank + 0°-rotated Clayton + 270°-rotated Gumbel copula in (D) (see S4 Text for model parameters). Similarly to the example with two dynamically coupled neurons (Fig 3G), these copulas are heavy-tailed. The goodness-of-fit for these models is measured with the proportion of the variance explained , which is indicated in the upper-right corner of each plot corresponding to a range of positions in virtual reality; E-G Conditional entropy for the bivariate examples (E-F) and the population-wide statistics (G) all peak in the reward zone; this entropy is equivalent to the mutual information between variables, given the position x, which means that the variables carry the most information about each other when the animal is in the reward zone. H Comparison of Copula-GP method (“integrated”) vs. non-parametric MI estimators (MINE [55] and KSG [53]) on estimating the amount of information about the location x from the subsets of variables ux. While the true I(x, ux) is unknown, the validation on synthetic data (Fig 4) suggest that Copula-GP “integrated” does not overestimate the amount of mutual information. Yet, Copula-GP “integrated” quantifies more information about the position x from the large subsets of data ux than MINE and KSG methods.
Table 1.
A list of variables in the navigational task dataset from Pakan et al.; for a detailed description of the task and the reward, see the original paper [5].
The variables are grouped according to their type; their order does not correspond to the order of variables y1…y109 in vine models (see text).
Table 2.
Bivariate copula families and their GPLink functions.
Table 3.
Hyper-parameters of the bivariate Copula-GP model.