Bayesian inference model

The notation of variables in this article is based on the previous work [10]. The differences in the notation from the previous study due to the generalization of the model will be summarized later. Boldface is used to denote a vector or matrix variable unless otherwise specified. Additionally, subscript is used to denote each element in a vector or matrix variable.

In our model, let N be the number of neurons, M be the number of time steps in activity measurement, and A be the number of neuronal ensembles. The generative model in our proposed Bayesian inference method is expressed using three variables: s is neuronal activity; t is the weight of a neuron belonging to a neuronal ensemble; and ω is time-series activity state of a neuronal ensemble. The variable s representing neuronal activity is given as input. Accordingly, the number of neurons N and the number of time steps M are determined from the input. In BIM and our proposed method, the number of neuronal ensembles is changed dynamically. Hence, inferences of ensemble structure and the number of ensembles are performed simultaneously from given activity data. Therefore, the number of neuronal ensembles A is an estimator, which is a natural number between 1 and N for hard clustering in BIM. However, due to the extension of our proposed method to soft clustering, A may be larger than the number of neurons N, and the possible value of A is in the whole natural number. In practice, the number of ensembles is expected to be sufficiently smaller than N if an appropriate ensemble structure is estimated, therefore our proposed method provides an upper bound on A as a parameter. The estimation of A will be described again in the description of the algorithm. Next, the subscripts i, k, and μ used in elementwise notation of each variable represent the label of the neuron, the label of the time step, and the label of the neuronal ensemble, respectively. Therefore, each variable represents the following.

• s_ik: neuronal activity of the ith neuron at the kth time step
• t_μi: membership weight of the ith neuron to the μth ensemble
• ω_kμ: activity of the μth ensemble at the kth time step

The ranges of variables s and ω are limited as 0 ≤ s_ik ≤ 1 ∀i, k and 0 ≤ ω_kμ ≤ 1 ∀k, μ, respectively, to normalize the activity weights of each neuron and ensemble. The variable t_i = {t_1i, ⋯, t_Ai} is one-hot representation or t_μi ∈ {0, 1} in the case of hard clustering, while t_μi ranges from 0 to 1 and satisfies in the case of soft clustering. In comparison with BIM, the input neural activity s and ensemble activity ω are changed from 0-or-1 binary variables to continuous ones in the range [0, 1], and the weight t of neurons is changed from categorical variables to continuous-valued vector variables. The range [0, 1] for s and ω is designed to normalize the amount of information in the activity of each neuron or ensemble. In addition, the number of ensembles initially given in MCMC is used as the upper bound of A, which is denoted by A_init in Table 1. The estimation of the appropriate A and ensemble structure under given upper bound are common both in BIM and our proposed model. The detail of the variation in the number of ensembles will be explained later in the algorithm part. The specific changes in variables are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Variables in BIM and our proposed method.

https://doi.org/10.1371/journal.pone.0287708.t001

In addition to these variables, three parameters n, p, and λ are introduced to each ensemble in the generative model, whose meanings are described in the following. For each parameter, Dirichlet or beta distribution is assumed as prior distribution to facilitate marginalization in Bayesian inference.

• n: affinity parameter to a neuronal ensemble, following Dirichlet distribution
• p: activity parameter of a neuronal ensemble, following beta distribution
• λ: synchronization parameter between the activities of a neuronal ensemble and neurons within the ensemble, following beta distribution

More precisely, two kinds of λ’s are introduced for synchronization; λ₁ or λ₀ is the synchronization when both of ensemble and neuron are active or inactive, respectively. The distribution of parameters in this model is expressed as follows. (1) (2) (3) (4)

Here is the hyperparameter of the prior Dirichlet distribution on neuron assignment, which represents the frequency of neuron assignment to each ensemble. and are the hyperparameters of the prior beta distribution, which represent the frequency of ensemble activity and inactivity, respectively. and are the hyperparameters of the prior beta distribution, which represent the frequency of active and inactive neurons when the belonging ensemble is active, respectively. and are the hyperparameters of the prior beta distribution, which represent the frequency of active and inactive neurons when the belonging ensemble is inactive, respectively.

The likelihood is designed as follows to facilitate marginalization like the design of prior distribution. (5) where (6) (7) (8) (9) (10) (11) (12)

The meanings of these quantities are the same as the hyperparameters, which have already been defined without tildes. Note that they are defined by the variables t, ω, and s. For inference, we need the joint distribution of t, ω, and s under given hyperparameters. In BIM, the above likelihood was expressed in the form of Kronecker delta, since t, ω, and s take discrete values. The correspondence between the variables in BIM and ours is shown in Table 2. As can be seen from the table, the likelihood in our model is the generalization to continuous variables, which also includes binary and categorical cases. By product rule in Bayesian statistics and marginalization of the parameters n, p, λ, the following expression is obtained, (13) where B(x₁, x₂) is beta function and is multivariate beta function represented as (14) (15) with Γ(x) being gamma function. For inference of the structure of the functional neuronal ensemble and activity of the ensemble, namely inference of t and ω, the marginalized expression (13) is used for improvement of inference accuracy and reduction of computational cost. In more detail, Dirichlet and beta distributions are chosen as the conjugate prior both in BIM and in our model so that the forms of the prior and posterior distributions are the same. In addition, the form of prior distribution is designed to remain unchanged after update of the hyperparameters. Such design enables us to integrate out the parameters n, p, and λ analytically as in Eq (13), which yields probability model expressed only by hyperparameters. Consequently, computational cost and estimation error can be reduced, because the parameters n, p and λ can be eliminated by analytical integration, and accordingly the numerical integration of these parameters is not necessary for the inference.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Parameters in the likelihood: The second column from the left is for corresponding hyperparameter without tilde, namely what distribution the hyperparameter describes and the meaning of hyperparameter as mentioned in the text.

https://doi.org/10.1371/journal.pone.0287708.t002

To summarize, we compare the two models: BIM and ours. In our model, the variable range of neuronal state is extended by the continuous s and ω, which leads to more detailed representation of the similarity between activities of neurons and ensemble. Membership weight t is generalized to continuous-valued vector variable for expressing the certainty of membership assignment to target ensemble. By changing t to continuous-valued, our generative model allows soft clustering for expressing overlap in membership assignment, and accordingly multiple roles of neurons can also be expressed. Moreover, the likelihood in BIM is generalized to improve the expressivity of the model. Consequently, our model is very effective not only for stationary activity data with the fixed functional role of neuron but also for the data of non-stationary activity, the data with multiple functional roles for each neuron, or the data under the experiment of low reproducibility. Additionally, MCMC algorithm in BIM is also changed for estimation of continuous variables. The changes in the algorithm will be described in the following.

MCMC and Dirichlet process

In our method, the structure of the neuronal ensemble and activity of ensemble are inferred from the solution of posterior maximization by applying MCMC method to Eq (13) similarly to BIM [10]. In addition, the number of neuronal ensembles is dynamically changed by Dirichlet process in MCMC. Dirichlet process is a stochastic process that can generate arbitrary discrete distributions, and the probability of each event is determined by concentration parameter [15]. For speeding up the inference by MCMC, parallel computation using synchronous update can be implemented.

The variables to be inferred are t and ω in our model, and the variable updates for t and ω are performed alternately until convergence in MCMC. The procedure of variable update is summarized as follows.

1. Ensemble activity ω is updated according to Dirichlet process of order two (or beta process) and Metropolis-Hastings method in MCMC.
2. Value of membership weight t is updated according to Dirichlet process and Metropolis-Hastings method in MCMC.
3. Hyperparameters are updated using updated ω and t by the processes 1 and 2.

The update rule of membership weight t_i = {t_1i, …, t_Ai} is explained in the following, which differs from BIM due to the change of the generative model. Other variables are updated in the same manner as in BIM.

In our model, the element of membership weight t_μi is a continuous variable satisfying normalization condition . Hence, for update of membership weight , all elements in must be updated unlike Dirichlet process for one-hot representation of in BIM. Let be the original membership weight of neuron i before update and be the proposed membership weight for update, which may or may not be accepted by Metropolis-Hastings method. Here we define concentration ensemble for neuron i in each iteration of MCMC. The values in elements of the proposed membership weights will concentrate on the ensemble by Dirichlet process with transition parameter α^(t). More precisely, in Dirichlet process the concentration ensemble is determined first, next the weight is increased by α^(t) and the weights of other ensembles remain unchanged, then finally all elements of the proposed weight are normalized so as to satisfy the condition of probability. In the case of hard clustering as in BIM, such normalization is not necessary because of one-hot representation of the weight.

The concentration ensemble is determined according to the transition probability in Dirichlet process as in the following Eq (16). The probability for the concentration ensemble is proportional to the size of the ensemble , that is, the sum of the proposed weights of the neurons belonging to the ensemble [15, 16]. (16) where . Then, the proposed weights are computed as in Eq (17), which includes the increase of the weight t_G*i and the normalization. (17)

Conversely, to satisfy the detailed balance condition in MCMC, the reverse process for the determination of concentration ensemble must be considered. The probability of concentration ensemble before update, denoted by , is expressed by Eq (18) under given membership weight after update, namely . (18) where backslash means removal of specific element, .

Here we should comment on the difference in the inference of membership weight t between BIM and our model for later convenience. In BIM, the membership weight t is a categorical variable, therefore the concentration ensemble is not necessary in determining the transition destination, and the transition probability to each ensemble is defined in proportion to the size of ensemble. Since there may be a transition to a new ensemble according to Dirichlet process, the cases of transitions to an existing ensemble and to new ensemble should be described separately. Hence, in BIM, the transition probabilities under the weight t in one-hot representation are given as follows. (19) where α_new represents concentration parameter in Dirichlet process, and larger value of α_new results in larger A. In our model, α_new is set to 0 because there is no upper bound for A due to soft clustering.

We go back to the description of our model. The ratio of probabilities for the concentration ensembles is given by (20)

The derived probabilities ratio is represented by the ratio of the ensemble sizes. Remember that is the proposed membership weight, and for acceptance of the proposed weight it is necessary to calculate the acceptance rate in Metropolis-Hastings method. The acceptance rate of the proposed weight under the original weight is given as (21) where hyperparameters in P are omitted and (22)

For the determination of the transition destination of ω, it is not chosen probabilistically but always an inverted value of the current activity state, because ω is a binary variable taking 0 or 1. Furthermore, since the value of ω after the transition is also binary, no special change is required after transition. In contrast, ω takes continuous value in our method, therefore it is necessary to change a decision rule for the transition destination. In addition, for normalization, it is necessary to consider how to change the value of ω after determining the transition destination.

Based on the fact that beta distribution is a special case of Dirichlet distribution, namely Dirichlet distribution of order two, we design the decision rule for the transition destination of ω and its value after the transition by considering the difference of update rule on t between BIM and ours. Here we give the constraint that the number of dimensions does not increase, as for t in BIM in Eq (19). As a result, the transition destination is determined in proportion to the sum of the current activity values, with the value of the concentration parameter being set to 0. (23)

For the value of ω after the transition, normalization is applied so that the sum of the activity values is 1. Here the transition parameter α^(w) is introduced as in the case of t in Eq (18), and the activity value of transition destination is increased by α^(w). (24)

Finally, acceptance rate is expressed by the formula in Eq (25). (25)

Once the variables are updated, the hyperparameters are also updated using updated ω and t. The hyperparameters to be updated are , which is related to membership of neuron, , which are related to ensemble activity, and , which are related to synchronization between activities of ensemble and neurons. The update rules of the hyperparameters are given as follows, (26) (27) (28) (29) (30) (31) (32) where the symbols with tilde are defined in Eqs (6)–(12) and the ones with hat are updated hyperparameters.

Finally, we mention the computational cost of our proposed method. The computational cost of our method is larger than the original BIM by a factor of the number of clusters. However, in our analysis, the number of clusters is supposed to be relatively small. In addition, the computational cost of the original BIM is not so large. Therefore, the computational cost of our method is not a problem in practice.

Algorithm 1 MCMC algorithm

initialize t, ω according to uniform distribution

hyperparameters are set to the given initial values

for execute until convergence of t, ω do

 for each ensemble μ ∈ {1, A}, k ∈ {1, M} (in parallel if possible) do

  draw by Eq (23)

  calculate by Eq (24)

  calculate acceptance rate by Eq (25)

 end for

 for each ensemble μ ∈ {1, A}, k ∈ {1, M} (updating synchronously) do

  Update the value of ω synchronously according to the computed acceptance rate

 end for

 for each neuron i ∈ {1, N} (in parallel if possible) do

  draw by Eq (16)

  for each ensemble μ ∈ {1, A} do

   calculate by Eq (17)

  end for

  calculate acceptance rate by Eq (21)

 end for

 for each neuron i ∈ {1, N} (updating synchronously) do

  update the values of {t_1i, …, t_Ai} synchronously according to the computed acceptance rate

 end for

 update hyperparameter values using updated ω and t by following Eqs (26)–(32)

end for

The model of fluorescence intensity

We conduct numerical analysis for the effectiveness and the validity of our proposed method by applying to continuous-valued data. The synthetic activity data for validation of the proposed method is generated under consideration of experimental fluorescence intensity by calcium imaging. In calcium imaging, fluorescent protein binds to calcium ions and emits light, and its activity is observed as light intensity. There is a chemical nonlinear relation between neuronal activity and fluorescence intensity. This relation depends on calcium ion concentration and the type of fluorescent protein, and nonlinearity is controlled by the parameters in general.

In our study, leaky integrated-and-fire model is used to generate the synthetic data. This mathematical model is for representing time series data of electrical intracellular membrane potentials. In this model, the relation among inflow currents from other neurons via synaptic and gap junctions, the external stimulus currents, and the time variation of membrane potential is expressed as differential equations with additional noise term, (33) where the definitions of variables and parameters are given in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Variables and parameters in leaky integrated-and-fire model.

https://doi.org/10.1371/journal.pone.0287708.t003

The input currents via chemical synaptic junction and gap junction are determined by the chemical synaptic junction weight and the gap junction weight , respectively. The current via chemical synaptic junction is given by the logistic function of membrane potential in Eq (34). For the current via the gap junction, the weight serves as a constant resistance in Eq (35). To summarize, these currents are expressed as (34) (35) where V^(half) and V^(width) represent the position of half value and half width of the logistic function, respectively. For chemical synaptic connection, the sign of is positive for excitatory connection and negative for inhibitory, respectively. In numerical analysis, these equations are implemented by Euler-Maruyama method to generate time series signals of membrane potentials.

Next, the model for converting membrane potential to fluorescence intensity is described. The time derivative of fluorescence intensity in the calcium imaging is given by the following differential equation, (36) where is the time constant of the fluorescent protein, is the intracellular calcium ion concentration of the ith neuron at time τ, and F_iτ is the fluorescence intensity of the ith neuron at time τ. The nonlinear function for calcium ion concentration in Eq (36) is expressed by sigmoid-like function, which is called Hill’s equation as (37) where F^(max) is the maximum of fluorescence intensity, K^(D) and h are parameters for controlling the shape of sigmoid-like function. The chemical response properties of fluorescent proteins to changes in calcium concentration are described in Ref [17]. We used the function in Eq (37) by referring it.

In generating the data of fluorescence intensity, the relation between calcium ion concentration and membrane potential is simplified in our work. Namely, we assume that calcium ion concentration is directly proportional to membrane potential as (we set the constant of proportionality unity in the following), which leads to the differential equation for fluorescence intensity in the following form, (38)

In applying our proposed method to synthetic fluorescence intensity practically, the fluorescence intensity data is rescaled to as follows, whose value is in the range [0, 1]. (39)

Generation of synthetic data

In our numerical analysis, we generate synthetic data with N = 100, M = 2000, A = 8, and with “structural” ensembles E₁ ∼ E₈ defined by the geometry of structural connections. Two structural ensembles E₁ and E₅ have 20 neurons and others have 10 neurons.

The values of parameters in leaky integrated-and-fire model are common to all neurons as in Table 4. For fluorescent protein, we set parameters for chemical reaction by considering specific fluorescent protein. Neurons in the structural ensembles E₁ ∼ E₄ respond faster than the ones in E₅ ∼ E₈ because their time constant is smaller. Between ensembles, structural connections are given probabilistically as synaptic connection and gap junction. Excitatory synaptic connections are given in the ensemble pairs, E₁ → E₂ and E₆ → E₅. Similarly, inhibitory synaptic connections are given in the pairs E₁ → E₃ and E₇ → E₅. There is no synaptic connection between ensembles except for the above-mentioned pairs. In addition, excitatory synaptic connections and symmetric gap junctions are given probabilistically between neurons in the same ensemble. The connection probabilities are given in Table 5. As a result, we obtain the connection matrices of chemical synaptic connection and gap junction between neurons as shown in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

The connection matrices between neurons: A: Weight of chemical synaptic connection . B: Weight of gap junction . C: Weight sum of chemical synaptic connection and gap junction .

https://doi.org/10.1371/journal.pone.0287708.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Values of parameters in leaky integrated-and-fire model: The unit of each quantity can be chosen auxiliary in this numerical analysis.

In practical application, the unit can be chosen appropriately to match the experimental condition.

https://doi.org/10.1371/journal.pone.0287708.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Probability of connections between structural ensembles.

https://doi.org/10.1371/journal.pone.0287708.t005

In out setting, the reason for introducing disconnected structural ensembles E₄ and E₈ is as follows. As mentioned in the introduction, our method estimates functional neuronal ensembles only from synchronization of activity data. Therefore, it cannot directly find the possible background structural connection between neurons. Even though the result of estimation for functional ensembles reflects the geometry of structural ensembles, functional and structural ensembles are not always the same. To demonstrate the difference between functional and structural ensembles, we intentionally use the same input currents to obtain similar neuronal activities for prepared structural ensembles, where some ensembles have inter-ensemble connections and others do not.

In addition, due to the difference between functional and structural ensembles as mentioned above, the exact structural ensembles cannot be found by our method. Hence, it is not necessary to introduce detailed structure of neuronal network geometry in the numerical analysis. For investigating the basic properties of the proposed method, we use simple neuronal network geometry with 8 structural ensembles in our numerical analysis.

For investigating stationary and non-stationary activities, we generate activity data with constant and time-varying input currents. Specifically, we generate three time series data, which are shown in Fig 2: the data under the same and stationary input current to all neurons as in 1 in Fig 3, the data under the same and non-stationary input current to all neurons as in 2 in Fig 3, and the data under different non-stationary input current to each ensemble as in 3 in Fig 3. The non-stationary input currents are the same in the structural ensembles pairs of {E₁, E₅}, {E₂, E₆}, {E₃, E₇}, and {E₄, E₈}. In Fig 2, membrane potential V_iτ under these conditions is shown in A, fluorescence intensity F_iτ in B, and rescaled fluorescence intensity in C. As seen in Fig 2, the activities are similar for neurons in the ensemble pair with the same pattern of input currents.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The signals in leaky integrated-and-fire model: In the identifier of each figure, the alphabet and the number represent the information of the signal and the input, respectively.

The details are as follows. A: Membrane potential V_iτ. B: Fluorescence intensity F_iτ. C: Normalized fluorescence intensity . 1: Identical and stationary input. 2: Identical and non-stationary input. 3: Different and non-stationary input.

https://doi.org/10.1371/journal.pone.0287708.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The ensembles in neuronal network and the input signals in the numerical analysis: X: Structural ensembles.

1: Identical and stationary input current. 2: Identical and non-stationary input current. 3: Different and non-stationary input current.

https://doi.org/10.1371/journal.pone.0287708.g003

We should compare the result of our numerical analysis with BIM, however application of the inference method in BIM is limited to binary data. Therefore, when applying two methods to synthetic continuous-valued data, raw continuous data is used in our proposed method and the binarized one is used in BIM. Binarization is performed using Peakfind function in MATLAB, where local maximum of time-series signal is regarded as spike. More precisely, Peakfind function can find local maximum value of a sample point in a data series, which is larger than the values of two adjacent samples and considered as a local peak. For the detail of Peakfind function, see the reference of MATLAB.

The figures A1, A2, and A3 in Fig 4 show synthetic activity data under the parameters in Tables 4 and 5. In contrast, B1, B2, and B3 are the data after binarization using Peakfind function, which are for the numerical analysis of BIM. In B1, B2, and B3, the activities in the same structural ensemble are also similar as in A1, A2, and A3. Therefore we consider there is no problem with data preprocessing by binarization. Furthermore, it can be observed that the activities of neurons in the downstream structural ensembles, namely E₂, E₃, E₅ in our setting, tend to be disturbed by the neurons in the upstream ensembles, E₁, E₆, E₇. More precisely, synchronization of neuronal activities in the downstream ensemble tends to be disrupted due to the activity of neurons in the upstream ensemble, as can be seen in A1, A2, and A3 in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Synthetic activity data to be analyzed: The figures A1, A2, and A3 describing normalized fluorescence intensity are the same as C1, C2, and C3 in Fig 2, respectively.

The conditions in leaky integrated-and-fire model are summarized as follows. A1: Stationary activity data under the same stationary input current to all neurons. A2: Non-stationary activity data under the same non-stationary input current to all neurons. A3: Non-stationary activity data under different non-stationary input currents to each ensemble. The figures B1, B2, B3 are binarized data of A1, A2, A3, respectively.

https://doi.org/10.1371/journal.pone.0287708.g004

Results of the application and discussion

As mentioned before, the purpose of our method is to infer functional neuronal ensembles, and it does not matter whether the structural neuronal ensembles defined by structural connections can be estimated or not. In other words, there is no ground-truth solution. Therefore, we attempt to capture neuronal ensembles with similar activities under various “granularities”, namely scales of ensembles. We emphasize that our soft clustering approach allows ensemble inference at different granularities, which is not possible in BIM due to hard clustering. Since our goal is to estimate the ensembles at different granularities, we cannot judge the superiority or inferiority of ensemble inference performance using a metric such as the distance from true structural connection. Hence, we display the results visually and discuss them. Additionally, stationary and non-stationary data are used to show that even non-stationary data can be handled by our soft clustering method.

We apply two methods to synthetic data: our proposed method and BIM. For the performance of functional ensemble inference, the average of multiple trials with different initial conditions is taken, because the dependence of MCMC result on initial condition should be removed. In BIM, the number of initial ensembles is set to half the number of neurons as recommended in [10], which is to avoid convergence to an inappropriate local solution. Other conditions are the same in two methods. The detailed conditions of the numerical analysis are summarized in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Conditions of numerical inference by MCMC.

https://doi.org/10.1371/journal.pone.0287708.t006

For the behavior by multiple trials, we observe how often each neuron is classified into the same functional ensemble, for which we define the similarity matrix U. The element of U represents the similarity of activities between neurons, which has the information of functional ensembles averaged over multiple trials. Element of the similarity matrix is expressed as , where superscript r is the label of trial number with initial condition being changed.

By comparing A3, B3, C3, and D3 in Fig 5 under different and non-stationary input current as in 3 in Fig 3, the resulting functional ensembles are changed by the value of the transition parameter α^(t). For example, when focusing on neurons i = 1 ∼ 30 in the structural ensembles E₁ and E₂, the coarse-grained large functional ensemble is obtained under small α^(t) as in A3 in Fig 5, where the original two structural ensembles E₁ and E₂ are merged into one functional ensemble. In contrast, smaller functional ensembles are obtained under large α^(t) as in D3 in Fig 5, where the structural ensembles E₁ and E₂ are separated. Furthermore, in our method, we observe that the neurons in different structural ensembles, for example the neurons in the structural ensemble pair {E₃, E₇} or another pair {E₄, E₈}, are classified into the merged functional ensemble in all cases of α^(t), although there is no structural connection between ensembles in the pair. The neurons in such ensemble pair have similar neuronal activity as in A3 in Fig 4. This is because the same input current is provided into the ensemble pair as in 3 in Fig 3. In our method, the inference of functional ensemble is based only on synchronization of activity. Thus, even if there is no structural connection, these neurons with similar activity are classified into the same functional ensemble as a consequence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Heat map of U by applying our proposed method to stationary and non-stationary activities: In each figure, the alphabet and the number in the identifier represent the information of transition parameter α^(t) and the input signal, respectively. The details are as follows.

A: α^(t) = 0.1. B: α^(t) = 0.3. C: α^(t) = 0.5. D: α^(t) = 0.8. 1: Identical and stationary input. 2: Identical and non-stationary input. 3: Different and non-stationary input.

https://doi.org/10.1371/journal.pone.0287708.g005

In contrast, when the method in BIM is applied to the activity data after binarization, only the course-grained functional ensembles can be found as in A3 in Fig 6. It is difficult to separate the original structural ensembles with similar activity by this method, because this method can only be used as hard clustering and does not have an extra parameter to control hard/soft clustering like the transition parameter in our proposed method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6.

Heat map of U by applying BIM to stationary and non-stationary activities: A1: Identical and stationary input. A2: Identical and non-stationary input. A3: Different and non-stationary input.

https://doi.org/10.1371/journal.pone.0287708.g006

We also observe the dependence on the input current. In our proposed method, when the activities in all neurons are similar as in A1 in Fig 4, the difference in time constants between the ensemble groups, {E₁, E₂, E₃, E₄} and {E₅, E₆, E₇, E₈}, is mainly reflected in resulting course-grained functional ensembles. The differences among ensembles in the same group can be found in more detail as we switch toward hard clustering (or large α^(t)). This behavior can be observed in A1, B1, C1, and D1 in Fig 5. In contrast, in the case of A2 in Fig 5, difference in information flow between ensembles due to non-stationary input current leads to clear difference in activity. In this case, a large functional ensemble is confirmed as the merged structural ensemble groups {E₃, E₄, E₆, E₇, E₈}, which have similar activities. As we switch closer to hard clustering, the hierarchy can be found: the smaller functional ensembles of {E₃, E₆} and {E₄, E₇, E₈} (see D2 in Fig 5). On the other hand, the structural ensembles in the groups of {E₃, E₆} or {E₄, E₇, E₈} cannot be separated anymore, where the activities of neurons highly synchronize in each group.

While BIM can estimate course-grained functional ensemble, it cannot find the hierarchy of ensembles confirmed by our method from the results in A1, A2, and A3 in Fig 6. Another feature is that the presence or absence of stationarity has little influence on the results by BIM. This may be due to hard clustering or ensemble inference by time-averaged activity, because hard clustering or time-averaged activity cannot appropriately capture the temporal change of the feature in the activity.

Both of BIM and our proposed method determine functional ensembles based on synchronization of activity. Therefore, the activation timings must coincide among neurons to be regarded as synchronous activity in both models. Nevertheless, our method has the advantage for functional ensemble inference. There are upstream and downstream information transmissions through connections. The downstream neurons are activated later than the upstream ones, and there is always a time gap even in the activity of synchronous neurons. In binary data, the perfect coincidence of activation timing (namely spike timing) is required for identifying synchronization. Hence, it is difficult to find synchronization due to the presence of time gap. In contrast, in continuous-valued data, the activity data before and after the maximum activity value (or spike) can also be used to identify synchronization. Therefore, the model of continuous activity can identify synchronization more appropriately than the binary model. We guess that this is the reason that the functional ensemble in Fig 6 by BIM becomes always similar regardless of the feature of external input current.

Finally, for comparison between the results by our proposed method and BIM, dendrograms for hierarchical structure of functional ensembles are depicted in Fig 7, where the data under non-stationary input current (3 in Fig 3) is used. For dendrograms, similarity matrices U in Figs 5 and 6 are used for computing distance between neurons. In addition, we also define similarity matrix for input signal, namely neuronal activity s as (40)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Dendrogram of hierarchical structure for functional ensembles: Each dendrogram is depicted using different similarity matrix for computing distance between nodes.

X: Matrix from activity data in A3 in Fig 4. A: Matrix U in A3 (α^(t) = 0.1) in Fig 5. B: Matrix U in B3 (α^(t) = 0.3) in Fig 5. C: Matrix U in C3 (α^(t) = 0.5) in Fig 5. D: Matrix U in D3 (α^(t) = 0.8) in Fig 5. Y: Matrix U in A3 (BIM) in Fig 6. The vertical and horizontal axes indicate distance and neuron label, respectively. The color of neuron label indicates the color of input current shown in 3 in Fig 3. For example, neuron labels in the structural ensembles {E₁, E₅} with the same input current are shown by the same color. The font of neuron label in (ensembles with smaller ) is shown by lightface, while in (with larger ) by boldface.

https://doi.org/10.1371/journal.pone.0287708.g007

From dendrogram X in Fig 7 depicted by from activity data in A3 in Fig 4, it is found that the activities of neurons in structural ensembles under the same input current are similar, even if the structural ensembles are not directly connected. Next, from dendrograms A, B, C, and D depicted by U under different α^(t), hierarchical structure of dendrogram is changed by tuning transition parameter α^(t). Obviously, the structures of dendrograms A, B, C, and D are similar to X. This means that the hierarchical structure of function ensembles by our proposed method reflects the hierarchical structure of input activity data correctly, which validates the result by our proposed method. For small α^(t) or soft clustering case, the distance between neurons tends to be small. This implies that membership weight t of individual neuron spreads over many ensembles, and the number of neurons in one ensemble tends to be large. In contrast, for large α^(t) or hard clustering case, distance between neurons tends to be large. In this case, the membership weight t concentrates on a specific ensemble, and the number of neurons in one ensemble tends to be small. Such difference between membership weights leads to the difference between dendrograms. Namely, hierarchical structure of dendrogram can be controlled by transition parameter α^(t). We also depict the dendrogram Y by the result of BIM, where the hierarchical structure is not similar to X. In addition, the result of BIM cannot be controlled by tuning parameter like α^(t). This implies that BIM may not be able to infer functional ensembles appropriately in some cases. We think that this fact also supports the advantage of our proposed method.