Figure 1.
Geometric view of recursive filtering in subspace .
Each point in this figure represents a probability distribution, , of an
-tuple binary variable,
. The underlying time-dependent model is represented by white circles in the space of
. The dashed lines indicate projections of the underlying models to the model subspace,
. The maximum a posteriori (MAP) estimates of the underlying models projected on subspace
were obtained recursively: Starting from the MAP estimate at time
(filter estimate, red circle), the model at time
is predicted based on the prior knowledge of the state transition, Eq. 8 (blue arrow, prediction; black cross, a predicted distribution). The maximum likelihood estimate (MLE, black circle) for the spike data at time
derived by Eq. 4 is expected to appear near the projection point of the underlying model at time
in
. The filter distribution at time
is obtained by correcting the prediction by the observation of data at time
(black arrow). The filter estimation at time
is used for predicting the model at time
and so on. This recursive procedure allows us to retain past information while tracking the underlying time-dependent model based on the current observation.
Table 1.
Method for estimating dynamic spike interactions.
Figure 2.
Estimation of pairwise interactions in two simulated parallel spike sequences.
(A) Application of the state-space log-linear model to parallel spike sequences with time-varying spike interaction. (Left) Based on a time-dependent formulation of the log-linear model (dashed lines in the right panels represent the model parameters), parallel spike sequences,
, are simulated repeatedly for
trials (duration:
bins). The two panels show dot displays of the spike events of the variables,
or
(
and
). The observed synchronous spike events across the two spike sequences within the same trials are marked by blue circles. (Right) Smoothed estimates of the log-linear parameters,
(solid lines, red: pairwise interaction; blue and green: the first order), estimated from the data shown in the left panels. The gray bands indicate the 99% credible interval from the posterior density of the log-linear parameters. The dashed lines are the underlying time-dependent model parameters used for the generation of the spike sequences in the left panels. (B) Application of the state-space log-linear model to independent parallel spike sequences with time-varying spike rates. Each panel retains the same presentation format as in A.
Figure 3.
Simultaneous estimation of pairwise interactions of 8 simulated neurons.
(A) Snapshots of the underlying model parameters of a time-dependent log-linear model of neurons containing up to pairwise interactions (duration:
bins) at
bins. No higher-order interactions are included in the model. Each node represents a single neuron. The strength of a pairwise interaction between the
th and
th neurons,
(
), is expressed by the color as well as the thickness of the link between the neurons (see legend at the right of panel B). A red solid line indicates a positive pairwise interaction, whereas a blue dashed line represents a negative pairwise interaction. The underlying spike rates of the individual neurons,
(
), are coded by the color of the nodes (see color bar to the right of panel A). (B) Dot displays of the simulated parallel spike sequences of 8 neurons,
, sampled repeatedly for
trials from the time-dependent log-linear model shown in A. For better visibility, only the first
trials are displayed (
). Synchronous spike events between any two neurons (28 pairs in total) are marked by blue circles. (C) Pairwise analysis of the data illustrated in B (using all
trials) assuming a pairwise model (
) of 8 neurons. The snapshots at the
bins show smoothed estimates of the time-varying pairwise interactions,
(
), and the spike rates,
(
). For this estimation, we use
for the prior density of initial parameters. The scales are identical to the one in panel A.
Figure 4.
Estimation of triple-wise interaction from simulated parallel spike sequences of 3 neurons.
(A) Dot displays of the simulated spike sequences, , which are sampled repeatedly for
trials from a time-dependent log-linear model containing time varying pairwise and triple-wise interactions (duration:
bins; see the dashed lines in C for the model parameters). Each of the 3 panels shows the spike events for each of the 3 variables,
(
and
), as black dots. Synchronous spike events across the 3 neurons as detected in individual trials are marked by blue circles. (B) Observed rates of joint spike events,
(
). (Top) Observed rates of the synchronous spike events between all possible pair constellations as specified by index
(
). (Bottom) Observed rate of the synchronous spikes across all 3 neurons,
. (C) Smoothed estimates of the time-varying log-linear parameters,
. The three panels depict the smoothed estimates (solid lines) of the log-linear parameters,
, of the different orders (
), as obtained from the data shown in A and B (top and middle: the first and second order log-linear parameters; bottom: triple-wise spike interaction,
). The gray bands indicate the 99% credible interval of the marginal posterior densities of the log-linear parameters. The dashed lines indicate the underlying time-dependent parameters used for the generation of the spike sequences.
Table 2.
AICs for different numbers of trials.
Table 3.
Models selected using different information criteria.
Table 4.
Model orders selected by different information criteria for different numbers of trials.
Table 5.
Model orders selected using different information criteria for model without triple-wise spike interaction.
Figure 5.
Detection of spike correlations and their relation to pseudo experimental protocol (simulation study).
(A) Sketch of an assumed experimental time course composed of four epochs (I–IV), e.g., of different behavioral task conditions. Epochs I–III have a duration of 100 bins, period IV has a duration of 200 bins. (B) Time-varying triple-wise spike interaction parameter of the underlying model (cf. Figure 4C, ) used for the simulation of spike data (
neurons,
trials) during the time course outlined in A. The gray areas indicate the time intervals in which the triple-wise interaction is positive:
. (C) Hypothesis testing for a triple-wise spike correlation based on surrogates. In each time period (I–IV), we perform a test on the Bayes factor (BF) resulting from the original data. The observed BF (Eq. 12), marked by a red line and triangle, is computed as evidence of a positive triple-wise interaction,
:
, as opposed to a zero or negative triple-wise interaction,
:
. We then compare the ‘observed BF’ with cumulative distribution functions (CDFs, solid lines) for the BFs derived from surrogate data sets generated from a model containing only up to pairwise interactions (
). In all of the CDFs, the gray area indicates the 95% confidence interval of the distribution. If the observed BF falls into the lower tail of the distribution (blue area),
is supported; if it falls into the upper tail (red marked area),
is supported. (D) Time courses of the underlying pair-interaction parameters of a pairwise log-linear model (
),
(
). These underlying parameters are obtained by projecting the full underlying model (
) in Figure 4 to the pairwise model space,
. The gray areas indicate the time periods in which all of the pairwise interactions are positive. (E) Similar tests as in C, but for the BF computed as evidence for the presence of simultaneous positive pairwise interactions,
, as opposed to the absence of such an assembly,
. (F) Compact visualization of the test results from C and E. The colored bars show which hypotheses are supported in the different time segments (red and blue) and where the null hypothesis cannot be rejected.
Figure 6.
Analysis of experimental spike data using the state-space log-linear model.
(A) Experimental time table of delayed-response hand movement task. The experiments were designed and conducted by Riehle and her colleagues [8], [84]. During the preparatory period (PP, 1500 ms) that starts with the preparatory signal (PS), the presentation of the response signal (RS) was expected at three distinct moments at 600, 900, and 1200 ms (expected signals, ESs). Here the RS occurs finally after the longest possible delay of 1500 ms. After the RS, the requested movement was executed (reaction and movement time, RT-MT). See [8], [84] and the text for the detailed experimental protocol. (B) Dot displays of the spike sequences (duration: 2 s, sampling resolution 1 ms) of three neurons simultaneously recorded from the primary motor cortex (MI). The spike sequences are aligned at the onset of the PS ( trials). The synchronous spike events across the 3 neurons detected in individual trials (detection in bins of
width) are marked by blue circles. (C) Observed rates of joint spike events,
(
). All of the events are detected in bins with a width of
. (Top) Observed rates of the spike occurrence of the individual neurons (
). (Middle) Observed rates of the synchronous spike events between two neurons specified by index
(
). (Bottom) Observed rate of the synchronous spike events of all 3 neurons,
. (D) Estimation of the time-varying log-linear parameters of 3 neurons,
(
), from the binary data shown in C, according to the method summarized in Table 1. The panels depict the MAP estimates of the log-linear interaction parameters (solid lines; from top to bottom, the first and second order log-linear parameters and a triple-wise spike interaction,
). The gray bands indicate the 95% credible interval. In this analysis, we used an identity matrix as an autoregressive parameter, i.e.
, in the state model. The covariance matrix of the initial parameter was fixed to
. (E) The smoothed estimate of a triple-wise spike interaction,
, is computed from binary data constructed using a bin-width of 2 ms (Top) and a bin-width of 5 ms (Bottom). The top of each panel shows the timing of the synchronous spike events of all three neurons.
Figure 7.
Detection of triple-wise spike correlation of MI neurons.
(A) (Top) The bin-by-bin Bayes factor (BF) for a model of triple-wise spike interaction computed locally in time in bit units, Eq. 13. The bin-by-bin BF is computed from Eq. 13, using the state-space log-linear model fitted to the spike data (a total of 36 trials, 2 s binned using 3 ms bin-width; cf. Figure 6C). The BF computes evidence of a positive triple-wise spike interaction, , as opposed to a zero or negative triple-wise interaction,
. The evidence for model
as opposed to
in a behaviorally relevant time period is obtained by summing the log of the bin-by-bin BF in that period (cf. Eq. 12). (Bottom) Two behavioral periods (preparatory period, PP, and reaction and movement time, RT-MT) tested for presence of a triple-wise spike correlation. In addition, to examine the evolution of the triple-wise spike correlation in the PP, the PP is divided into early and late stages at the middle of the PP. (B) (Left) The observed BF for an entire PP (marked by a red line and triangle) computed using Eq. 12 (bin-width: 3 ms). We then test the ‘observed BF’ using a distribution function (solid line) of null BFs derived from surrogate data sets generated from a fitted model containing only up to pairwise interaction terms (
). The gray area indicates the 95% confidence interval of the distribution. Vertical dashed lines indicate the 90% confidence interval. If the observed BF falls into the lower tail of the distribution (blue area),
is supported, if it falls into the upper tail (red marked area),
is supported. (Right) The same analysis in the left panel, but with binary data constructed using larger (5 ms, upper panel) and smaller (2 ms, lower panel) bin-widths. (C) (Top) Test of observed BFs (bin-width 3 ms) computed in distinct periods: early PP (0–750 ms), late PP (750–1500 ms), and RT-MT (1500–1800 ms). (Bottom) The same test as in the top panel, except that binary data using a bin-width of 5 ms were used. (D) Test of observed BFs (bin-width: 3 ms) in each period using spike data from only the last half of the 36 trials (trials 19–36).
Figure 8.
Analysis of stationary spike correlations of simulated neurons using Bayes factor.
(A) Sketch of different time periods and the underlying models used for the generation of parallel spike sequences: (I) Model of independent spiking (,
,
for
and
); (II) Model of simultaneously positive pairwise interactions, without a triple-wise interaction (
,
,
for
); (III) Model of triple-wise interaction, with negative pair interactions (
,
,
for
). (B) Raster display of three parallel spike sequences,
, within one example trial. Each spike is colored according to the spike pattern in which it appears: Spikes occurring in triplets are shown in red, spikes within doublets (all types) are marked in blue, and spikes not involved in any synchrony pattern are shown in black. (C) The bar plots demonstrate the Bayes factors (BFs), Eq. 48, for each of the time periods I–III in bit units. The upper panel shows the average BF when testing simultaneously positive pairwise interactions (Test 1), averaged across 200 realizations (
). A positive value for the log of the BF supports the model for the presence of simultaneously positive pairwise interactions,
, while a negative value supports the absence of such an assembly,
. The BFs per sample and time period are computed by applying a pairwise state-space log-linear model (
) independently to the three periods. We use a state model with
. The lower panel shows the BF for the positive triple-wise spike interaction (Test 2),
, as opposed to a zero or negative triple-wise spike interaction,
, in each of the three periods. The BFs are computed from a full state-space log-linear model (
). (D) Bar plot of the bin-by-bin BFs (Eq. 52) sorted by the different spike patterns in the three periods (from top to bottom). The contributions to the BFs (shown in C) from the different spike patterns are sorted and displayed using the indicated spike patterns (000, 100, 110 and 111) as representative examples. The gray bars indicate the average BFs of simultaneously positive pairwise interactions, with the average computed for the spike patterns observed in 200 realizations in the respective periods, while the dark gray bars indicate the average BFs for the triple-wise spike correlation.