2.1 Simulations

The network considered in this study consists of two populations, one with excitatory neurons and one with inhibitory neurons, of size N_E and N_I respectively. The neurons are modelled as leaky integrate-and-fire neurons, connected with alpha-shaped current-based synapses described by Eq 2 in Table 1. The sub-threshold dynamics are described by a first-order differential equation given by Eq 1 in Table 1. When its membrane potential reaches a firing threshold V_thr, the neuron fires and and the synapses on all its postsynaptic neurons are activated after a time t_d. After firing, the membrane potential of a neuron is clamped to a voltage V_reset for a refractory period t_ref. Each neuron receives synapses from a fraction ϵ of all other local neurons in the network. Post-synaptic currents from excitatory synapses have a peak amplitude J, and post-synaptic currents from inhibitory synapses have a peak amplitude of −gJ. Both types of synapses share a synaptic time constant of τ_syn. Additionally, each neuron receives independent excitatory input from an external population modelled as a Poisson process with fixed rate ν_ext. This rate is determined by the parameter , where ν_thr = V_thrC_m/(Jτ_meτ_syn), the minimum constant input rate that will drive the neuron to its threshold. Since the synaptic strength and synaptic time constant both contribute multiplicatively to the total charge transfer during a synaptic event, we opted to parameterize the synaptic strength in terms of the total charge transfer during a synaptic event Q_s = τ_synJ. A complete description of the point-network model is given in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Description of point-neuron network following the guidelines of [33].

https://doi.org/10.1371/journal.pcbi.1011625.t001

Ten model parameters were uniformly randomly sampled, using Latin Hypercube sampling [27], which is a stratified sampling strategy suitable for sampling in high-dimensional systems. A complete list of model parameters and their values and ranges are given in Table 2. 10000 simulations were run using the NEST simulator [28, 29], comprising both the training data set and the test data set. The regions of the parameter space giving strongly synchronous firing, discussed in [5, 30], were avoided by avoiding unbalanced regimes caused by low inhibition, and high external input (η > 3.5). Additionally, a small number of simulations, which were found to be strongly synchronous were removed, making up roughly 3% of the total simulations. These were found by the following ad hoc criteria: more than 150 time bins of 1 ms in which more than 10% of the neurons fire, as well as more than 500 time bins in which fewer than 0.25% of the neurons fire. See Section 3.1 and Section 4 for further discussion. For 100 of the simulation outputs in the test data set, an additional 50 simulations were run with parameters sampled from the estimated posterior distributions (see Sections 2.4 and 3.4). For each simulation, the LFP was computed as described in the next section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Point-neuron network parameters.

https://doi.org/10.1371/journal.pcbi.1011625.t002

The population spiking activities were saved at a resolution of 1 ms as the sum of all spike trains in the populations. The power spectrum, defined as [ψ](f)*[ψ](f)/T for some signal ψ(t), where [ψ](f) denotes the Fourier transform of ψ, and T is the window length, was estimated using Welch’s method [31] for both the population spiking activities and the LFP. The implementation in the Python SciPy [32] package (scipy.signal.welch) was used, with a Hann window of length 256 and an overlap of 128. The metamodels are trained on the base 10 logarithm of these power spectra.

2.2 Forward-model predictions of LFPs

In order to compute local field potentials (LFPs) from the point-neuron network, we utilized the hybrid LFP scheme [34] (github.com/INM-6/hybridLFPy), which allows for the decoupling of the simulation of spiking dynamics (here computed using point neurons) and predictions of extracellularly recorded LFPs. The latter part relies on reconstructed cell morphologies and multi-compartment modelling in combination with an electrostatic forward model. A complete description of the scheme (including the biophysics-based forward model) can be found in [34], and a concise description of the multi-compartment neuron model is given in Table 3. The parameters used are given in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Description of multi-compartment neuron populations.

https://doi.org/10.1371/journal.pcbi.1011625.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Multi-compartment neuron parameters.

https://doi.org/10.1371/journal.pcbi.1011625.t004

Current-based synapses and morphologies with passive membranes in the multi-compartment neuron models give a linear relationship between any presynaptic spike event and contributions to the LFP resulting from evoked currents in all postsynaptic multi-compartment neurons. Thus the LFP contribution at position r from a single presynaptic point-neuron j in population Y can, in general, be calculated by the convolution of its spike train with a unique kernel as . This kernel encompasses effects of the postsynaptic neuron morphologies and biophysics, the electrostatic forward model, the synaptic connectivity pattern, conduction delay and post-synaptic currents. This kernel can be further decomposed as a convolution between the LFP response of a unit current for a single time step, and the post-synaptic current. The resulting LFP due to spikes in a presynaptic population Y is then given by (9) where is the post-synaptic current given by Eq 7 in Table 3, and .

As shown in [30, 34], a good approximation can be made by convolving the population firing rates with averaged kernels , that is, (10)

As in [30, 34] these averaged kernels were computed using the full hybrid-scheme. This was done by computing the LFP resulting from a fully synchronous activation of all the outgoing synapses from all neurons in the presynaptic population. Thus for the computation of the LFP kernel, we have where t_Y is the timing of the synchronous event in population Y.

After the single time-step kernel has been computed, the full kernel can easily be computed for any post-synaptic current, and hence synaptic parameters. In order to account for the membrane parameters τ_m and c_m, where c_m is the per-area capacitance, we compute the kernel for a single time step on a grid of different values of τ_m and C_m, and adjust this kernel by linear interpolation. We compute the kernels at the values [0.4, 0.6, 0.8, 1.0, 1.2] for c_m and [15, 18, 21, 24, 27, 30] for τ_m. Since the membrane capacitance for morphological neurons is dependent on the area of the neuron, we choose a baseline value of c_m = 1.0, and adjust it by the same proportion as C_m for the point-neuron network. The synaptic parameters can subsequently be accounted for in a separate step.

2.3 Gaussian process regression

Here we give a brief overview of Gaussian process regression models, their training procedures and the extension to deep models.

Gaussian process regression models are probabilistic non-parametric models that provide distributions over arbitrary functions, and are trained using supervised learning. A Gaussian process (GP) is a collection of random variables such that any finite number of them follow a joint Gaussian distribution, and a GP is completely specified by a mean function m(θ) and a covariance function k(θ, θ) [18]. Consider a data set consisting of n inputs of dimension d contained in the matrix Θ, and corresponding outputs x, which are assumed to be one-dimensional for notational simplicity in this section. A Gaussian process regression model assumes a distribution over an underlying latent function f(θ) which is modelled as a Gaussian process. Evaluating this distribution at the set of input points Θ gives a joint Gaussian distribution , where the covariance function k(Θ, Θ) must return a positive semi-definite covariance matrix between all the data points contained in Θ. The mean function is usually assumed to be 0 without loss of expressiveness. The covariance function encodes the properties of the function, and there are many different covariance functions available with different properties. In the present work, we use the Matérn 5/2 kernel (11) where l and σ are learnable parameters.

The model of the observed outputs x is given by the underlying latent function f(Θ) corrupted by some noise such that . The marginal likelihood of the data is obtained as (12) which can be found analytically in the case of a Gaussian likelihood p(x|f, Θ), and is used to optimize the model parameters.

Given a new set of inputs Θ_* on which we would like to evaluate the model, the GP specifies a joint distribution over x and x_*, and the predictions are made as the distribution over x_* conditioned on the training data x [18] (13) (14) where we use the shorthand notation K_nn = k(Θ, Θ), K_n* = k(Θ, Θ_*) and K_** = k(Θ_*, Θ_*). The semicolon indicates the input points corresponding to the distributions, on which the covariance function is evaluated. We can immediately see that the mean value of the distribution at the predicted locations Θ_* are linear combinations of the training values x.

Deep, or composite, GP models are constructed by iteratively defining GPs with the previous GP as inputs. A joint prior distribution over all layers is given by (15) where f⁰ = θ. As in standard GP models, we would like to marginalize out the latent functions in all layers for optimization. For predictions, we would like to condition the latent functions on the training data and then marginalize them. This is obviously not possible analytically, and approximations have to be used. We do not go in to details, but refer the readers to [35] for an excellent tutorial.

For multi-dimensional outputs, one can either assume the outputs to be independent, or correlated. In the former case, a separate GP is indepedently trained for each output dimension. For the latter case, one simple way of introducing correlations between output channels is to make a linear combination of independent GPs. If the number of independent GPs is lower than the number of output dimensions, a low-dimensional representation of the output is achieved, with a linear mapping between the latent space and the output. The model can be written as (16) where g_p(θ) forms a set of independent GPs, and W is an n × p matrix used to linearly construct the output f(θ) [36].

This can be viewed as an input-dependent linear latent space model, where we assume a linear mapping between a latent space and the output, but the coefficients of this mapping can change in a non-linear fashion. If we choose P to be a small number, a low-rank representation of the output is achieved.

We used a DGPR model based on the framework by [23]. We use 2 layers, and do a parameter scan in order to find the number of GPs to use in each layer. The final output is constructed by a linear transformation from the GPs in the second layer as given by Eq 16.

2.4 Posterior distribution over parameters

The metamodels provide a distribution p(x|θ), which is an approximation to the distribution over the outputs given by the simulations, where x is the simulation output, and θ is the simulation inputs. Introducing a prior distribution over the inputs p(θ), which we take to be uniform over the ranges of input values used to train the metamodels, the unnormalized posterior distribution over parameters given a specific observed model output x₀ is given by p(θ|x₀) ∝ p(x₀|θ)p(θ). We sampled from this distribution using an adaptive Metropolis algorithm [37]. The adaptive Metropolis algorithm is a random walk Markov Chain Monte Carlo algorithm, where a multivariate Gaussian proposal distribution is used, and where its covariance matrix is iteratively estimated by the sample covariance. We only estimated the covariance during the burn-in period, and kept it constant during sampling. Additionally, we used a separate scaling factor for the covariance matrix which was tuned during the burn-in period to give an acceptance ratio of around 0.3. Five chains were run in parallel, and run for a total of 40000 evaluations of the proposals, where the samples from the first 12000 proposals were discarded and considered the burn-in period. For each example in the test data set, the posterior distribution was sampled separately for both the metamodel of the population spiking activities and the metamodel for the LFP. The Gelman-Rubin statistic [38] was computed independently for each parameter, for each distribution, in order to assess the convergence properties of the chains.

2.5 Technical details

3.1 Simulations

A total of 10000 simulations were run, with parameters sampled as described in Section 2.1. A network consisting of one excitatory and one inhibitory population can produce a variety of activity states, characterized by whether the neurons fire in synchrony, and how regularly they fire, that is, whether the inter-spike intervals for individual neurons have a wide or narrow distribution. The behaviour of the present two-population network was analyzed in the framework of mean-field theory in [5] for integrate-and-fire neurons where the synaptic currents were modelled as delta functions. Brunel [5] found that the network spiking activity could be described by distinct regimes, classified as synchronous regular (SR), asynchronous irregular (AI) and synchronous irregular (SI, fast and slow), depending on which specific parameter values were used, and arise by different mechanisms. As described in Section 2.1, we avoided activity states characterized by strongly synchronous behaviour, which produces sharp peaks in the power spectrum. Fig 2 shows three examples of different states of network activity. Column A shows the synchronous regular regime, characterized by runaway excitation, where the synaptic strengths of inhibitory neurons are not strong enough to overcome the excitation, and the neurons fire regularly and synchronously with very high rates. Column B shows an example of a synchronous irregular state, produced in some cases when the network receives high amounts of external input and the inhibitory synaptic strengths are high. Column C shows asynchronous irregular activity. This is the regime in which most of the simulations used for training and validation were, although there is of course much variation within this regime. Columns A and B show the type of activity we avoided, with sharply peaked spectra.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Example activity states.

Each column shows, from top to bottom, a raster plot, population firing rates, the power spectrum of the population averaged firing rates, the LFP, and the power spectrum of the LFP. The raster plot and firing rates are taken from the excitatory population. A: Synchronous regular activity. B: Synchronous irregular activity. C: Asynchronous irregular activity.

https://doi.org/10.1371/journal.pcbi.1011625.g002

We will not attempt to give a full description of all possible network states or the effects of all different parameters in this paper, but show these examples to make clear the states we have actively avoided, as described in Section 2.1.

3.2 Forward predictions

The metamodels were tested on a data set containing 1681 examples, jointly sampled with the training data using Latin Hypercube sampling. Models were trained on data sets ranging from 100 to 8000 examples in order to examine the density of samples needed to adequately represent the simulation output space. For all other experiments, the metamodels used were trained on the full 8000 example simulations. A grid search of hyperparameters was run for each metamodel in order to find a suitable model for the problem at hand, which can be found in S1 Supplementary Section. The metamodel obtained from the hyperparameter search was used for all subsequent experiments, and where we trained new metamodels, the hyperparmeters were the kept the same. Fig 3A and 3B show examples of simulated outputs and corresponding outputs from the metamodels for two different sets of parameter values, for the population spiking activities and LFPs respectively. Only the outputs for the excitatory population spiking activity and the topmost LFP channel are shown. 10 simulations run with different seeds are shown in black lines, and the colored lines show the output given by the metamodels. The shaded area shows two standard deviations above and below the mean for the DGPR and MAF metamodels. Visually inspecting the top example in panel A, both metamodels accurately capture the mean of the simulations for all frequencies, but slightly overestimate the variance, particularly for the lower frequencies. In the bottom example, there is a larger discrepancy between the MAF metamodel and the simulator for the middle frequencies, while the DGPR metamodel is accurately able to capture both the mean and variance. Note that the covariance between frequency channels is not considered at all in this figure. In panel B, the LFP for the same simulations and the DGPR prediction are shown. Generally, the LFPs are smoother and the variance is lower compared to the population spiking activity. Fig 3C and 3D show the distribution over maximum (over frequencies) absolute errors on the test data set, where the expectation of the metamodel is used as the prediction value. We show the maximum error over frequencies since the large number of output channels make the mean small even in the presence of a few large deviations. Moreover, as the simulation output itself is random, we are mainly interested in finding any larger deviations, rather than small deviations, which are expected to occur. The expectation of the distribution given by the DGPR metamodel more closesly matches the simulated values compared to the MAF metamodel. Fig 3E and 3F show the mean standard deviation of the metamodel distributions evaluated at the test data parameters. The DGPR metamodel produces narrower distributions over the simulator output compared to the MAF model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3.

A: For two different parameter sets, the power spectrum of the population spiking activity from 10 simulations (black lines), and metamodels (blue and orange lines) are shown. The shaded area shows two standard deviations of the distribution given by the metamodels. B: Same as A, but for the LFP. Only the DGPR metamodel was trained on the LFP. C: Distribution over maximum (over frequencies) absolute errors for both metamodels, on the power spectrum of the population spiking activity. D: Same as for C, but for the LFP. E: Mean standard deviation of the metamodel outputs, evaluated at the parameters in the test data set. F: Same as E, but for the LFP.

https://doi.org/10.1371/journal.pcbi.1011625.g003

In terms of computational cost, evaluating the log probability of 50 example outputs takes around 4 seconds with the DGPR metamodel, and 0.014 seconds with the MAF metamodel. Generating 50 samples from the metamodels takes 1.6 seconds for the MAF metamodel, and 0.07 seconds with the DGPR metamodel. By contrast, running a single simulation took on average 4.4 CPU-minutes.

The training time for the metamodels depends on the model parameters and learning rate, as well as the random seed. A full epoch of the 8000 training examples takes around 4.4 seconds for the DGPR metamodel, and 1.4 seconds for the MAF model. Generally, the training converged after a few hundred epochs for both models, giving a training time (not counting the periodic train/test evaluation) on the scale of an hour for the DGPR model and around a third of that for the MAF model.

3.3 Number of training samples needed to represent the model

The DGPR metamodel was trained on data sets ranging from 100 to 8000 examples of the population spiking activities, and the performances on the same test data set were evaluated. Fig 4 shows the maximum (over frequencies) absolute prediction errors, averaged over the test data set for each trained metamodel. As the number of training samples increases, the prediction errors on the test data set decreases, as the metamodel starts to generalize better. The prediction errors on the training data set increases, as it becomes more heterogeneous and the model is less able to overfit to it. After a sharp decrease in errors as the number increases to a few hundreds, the benefit of increasing the number of training samples decreases significantly after on the order of 1000 is reached, indicating that the parameter space is sufficiently densely sampled.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Mean (over test data set and training data set) of the maximum (over frequencies) errors, in absolute scale, as a function of the number of examples used to train the metamodels on the population spiking activities.

https://doi.org/10.1371/journal.pcbi.1011625.g004

3.4 Inverse modelling

Using the conditional distribution over simulation outputs given parameter values, estimated by the metamodels, posterior distributions for 100 simulations in the test data set were estimated. Separate posterior distributions for both metamodels conditioned on the population spiking activities, and for the DGPR metamodel conditioned on the LFP, were estimated using an adaptive Metropolis algorithm as described in Section 2.4.

The Gelman-Rubin statistic was computed for all the distributions, and was below 1.1 for over 90% of the posterior distributions for the population spiking activities, and over 95% of the posterior distributions for the LFP, indicating that the chains converged for most of the distributions. In the computation of average values over the test data set, all distributions were used, regardless of the value of the Gelman-Rubin statistic. For each of the estimated posterior distributions of both the population spiking activities and the LFP, 50 additional simulations were run with parameters drawn from the posterior distributions in order to evaluate the accuracy of the posterior distribution.

Fig 5A shows an example of a posterior distribution generated by the outputs of one simulation in the test data set. The 1D and 2D posterior marginal distributions of a subset of the parameters are shown. On the left-hand side, the posterior distribution generated by the population spiking activities are shown, and the corresponding posterior distribution from the LFP is shown on the right-hand side. The red bars and dots show the values of the parameters used in the simulation from which the posterior is generated. Qualitatively, the distributions from both the population spiking activities and the LFP appear similar for the first three parameters shown.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5.

In A, C and D, the left-hand and right-hand side show equivalent plots for the population spiking activities and LFPs respectively. The DGPR metamodel is used in both cases. A: 1D and 2D marginal posterior distributions over a subset of the parameters, for an example in the test data set. The red dots and bars show the parameter values corresponding to the simulation output the posterior distribution was computed for. B: Left: mean (across test data set) of standard deviation of the 1D marginal distributions for each parameter. Right: Absolute error of the parameter predictions based on the expectation. C: Metamodel predictions for the power spectra of excitatory population spiking activity (left) and the uppermost channel of the LFP (right) in orange. Black lines show 50 simulation outputs run with parameters drawn from the posterior distribution. D: Distribution of maximum distance between the ground-truth simulation output from which the posterior distribution was computed, and the simulation output from the posterior distributions (black lines in C). Left plot shows the errors from the population spiking activities, right plot shows the errors from the LFP. The black lines show the simulations from the posterior conditioned on the population spiking activities, the gray lines show the simulations from the posterior conditioned on the LFP.

https://doi.org/10.1371/journal.pcbi.1011625.g005

For this particular example, the 1D marginal distributions of both posteriors are wide for the membrane time constant τ_m, quite narrow for g, and somewhere in between for η. Wide marginal distributions entail that, lacking information about all other parameters, the value of the parameter cannot be determined with certainty, while narrow marginal distributions entail that the parameter can be accurately determined even without knowledge of other parameters. There is a positive correlation between η and τ_m in the posterior distribution, i.e. if the value of η is increased the value of τ_m must also be increased for the model to produce similar outputs. Equivalently, if you know the specific value of either η or τ_m, the uncertainty in the other parameter would decrease. Note that because we only see 1D and 2D marginals, there may be higher-order interactions present that cannot be easily visualized. For the synaptic strengths, the distribution is much wider for the model trained on the population spiking activities compared to the model trained on the LFP.

The left-hand side of Fig 5B shows the mean (over all test examples) standard deviations of the 1D marginal distributions. Generally, the parameters related to the synaptic inputs have the narrowest marginal distributions, while the neuron parameters have much wider marginal distributions, indicating that changes to neuron parameters can largely be compensated for by adapting other parameters, while changes to synaptic and input parameters have a larger effect that cannot be compensated for to the same degree. The right-hand side of Fig 5B shows the mean (over all examples in the test data set) absolute distance in standard deviations of the parameters of the simulation used to generate the posterior (i.e. red dots in Fig 5A) and the expectation of the posterior distribution. For all the parameters, the mean absolute distance is smaller than the standard deviation of the approximate marginal posterior distributions, indicating that the approximate posteriors generally covers the parameters of the simulation the posterior distribution is conditioned on.

One notable difference between the posterior distributions computed from the metamodel of the population spiking activities and the posterior distributions computed from the metamodel of the LFP is that the overall synaptic strength, Q_s, and the membrane capacitance, C_m, are significantly more constrained by observing the LFP than by the population spiking activities, with a standard deviation more than three times as large for the population spiking activities. This can also be observed in the example distributions shown in Fig 5A. For the rest of the parameters, the standard deviations of the 1D marginal distributions are roughly the same when computed for the population spiking activities and the LFP. The full posterior distribution from Fig 5A is shown in S1 Supplementary Section.

Fig 5C shows the metamodel output (orange) corresponding to the same simulation as in A, and 50 individual simulation outputs (black) run with parameter values sampled from the posterior distribution of the example shown in A. Only the spectra from the excitatory population spiking activity and the topmost LFP channel are shown. The mean of the metamodel distribution is shown in dashed orange, and the shaded area shows ± 2 standard deviations. The simulations run with parameters from the posterior distribution fit nicely with the distribution over the outputs given by the metamodel, suggesting that the posterior distribution is reasonable. Fig 5D shows the distribution of the maximum (taken over frequencies and simulations for a given posterior) distance between the simulation outputs run with parameters drawn from the posterior distributions, and the simulation output the posterior is conditioned on, for the 100 posterior distributions for which the extra simulations were run.

The black line shows the errors for the simulations run with parameters drawn from the posterior generated by the population spiking activities, while the gray distribution shows the distances for the simulations run with parameters drawn from the posterior generated by LFPs. The left-hand side shows the distances of the population spiking activities, while the right-hand side shows the distances for the LFP. For both the population spiking activities and the LFP, the simulations from the posterior distributions are very similar to the signal they were conditioned on. Simulations from the posterior distributions conditioned on the LFP, also give population spiking activities very similar to the ground-truth. Simulations from the posterior distributions conditioned on the population spiking activities, on the other hand, do not give as similar LFPs to the ground-truth, which again shows that observing the LFP constrains the parameters to a greater degree than observing the population spiking activities.

Fig 6 shows a comparison between the posteriors obtained by the DGPR metamodel and the MAF metamodel. Panel A shows the marginal standard deviations, averaged over all posterior distributions. For both metamodels, they are similar, but the DGPR is generally slightly wider. Panel B shows the same plot, but for the posterior predictive distribution, i.e. the distribution over outputs when parameters are sampled from the posterior distribution. The posterior predictive distributions from the DGPR metamodel are significantly narrower than those from the MAF metamodel, even with wider posterior distributions, indicating that the posterior distributions are more correct compared to those from the MAF metamodel. Panels C and D show the mean absolute distance between the expectation of the posterior distribution and posterior predictive distribution respectively, and the simulations they are conditioned on. For both models, the errors are on the same scale as the standard deviations, indicating that neither systematically misses the simulation it is conditioned on.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6.

A: mean marginal standard deviations of the posterior distributions. B: mean marginal standard deviations of the posterior predictive distributions. C: mean distance between expectation of marginal posterior distribution and the parameters from the simulation the posterior distribution is conditioned on. D: Same as C, but for the posterior predictive distribution.

https://doi.org/10.1371/journal.pcbi.1011625.g006

The Pearson correlation coefficient was computed for all pairwise combinations of parameters for all sampled posterior distributions. Fig 7 shows the mean of the correlation coefficients over all examples in the test data set. The upper triangle shows the coefficients for the samples from the posterior distribution conditioned on the population spiking activities, and the lower triangle shows the coefficients conditioned on the LFP. The correlations between most of the parameters are fairly weak, but since only the average is shown, there could be stronger correlations in individual posterior distributions that the plot does not capture. For the LFP, there are strong average positive correlations between the synaptic parameters, i.e. the synaptic strength Q_s, the synaptic time constant τ_syn and relative inhibitory strength g. Interestingly, the corresponding coefficients for the population spiking activities are close to 0. This again highlights the difference in the posterior distributions for Q_s between the population spiking activities and the LFP. For the distributions conditioned on both the LFP and the population spiking activities, there are positive correlations are present between the parameters η and τ_m, as well as negative correlations between η, V_reset and C_m. As these parameters all impact the amount of input required for a neuron to spike, there are possibly complex interactions between all these parameters yielding manifolds of similar activities and LFPs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Pearson correlation coefficients between samples from posterior distribution over parameters, averaged over the test data set.

Upper triangle shows the correlation coefficients for the posterior distributions from the simulated power spectra from the population spiking activities, while the lower triangle shows the same correlation coefficients from the LFP.

https://doi.org/10.1371/journal.pcbi.1011625.g007

4.1 Model inversion

One key motivation for using probabilistic metamodels is that it allows inverting the model and finding posterior distributions over parameters given observed outputs. We opted to sample the posterior using an adaptive Metropolis algortihm as it is simple to implement and worked well for the present problem. The Gelman-Rubin statistic was computed, and for most of the sampled posterior distributions (> 90%), it was below 1.1, which is often considered to indicate convergence [48]. For some of the posterior distributions, the chains did not converge, although this was likely only for a minority. Particularly for the examples on which the forward predictions are poor, we would not expect the estimated posterior distributions to be accurate, as the metamodel generally will not produce outputs which are similar to the observed output used to generate the posteriors.

The degree to which the different model parameters were constrained by observing an output varied highly (see Fig 5B). Notably the distribution over most neuron parameters were wide, while the distribution over synapse and input parameters were narrower. Interestingly, the synaptic strength (Q_s) and membrane capacitance (C_m) were significantly more constrained by the LFP than by the population spiking activities, illustrating the utility of using different signals for model validation. The effect is likely due to the fact that LFP is directly dependent on both the post-synaptic currents as well as the population spiking activities. The effect was not seen for the synaptic time constant, which also directly influences the post-synaptic current, something that indicates that the extra information may be in the scale of the LFP, as that is directly determined by the scale of the post-synaptic currents. The remaining parameters were similarly constrained by both signals, indicating that for the present situation, both signals may contain similar information about the parameters.

A previous study on model inversion using the LFP of a similar network using convolutional neural network [30] provided more accurate predictions of parameter values. In that study, neuron models with delta synapses were used, and it was limited to only three parameters, which likely makes it an easier task. Particularly, wide marginal posterior distributions might only be possible if there are sufficiently many parameters that are free to vary such that they can compensate for the effects of other parameters. An interesting question which we did not pursue here is to what degree keeping the different parameters fixed constrains the remaining parameters given observed simulation outputs.

Comparing the posterior distributions generated from the DGPR metamodel to the posterior distribution generated from the MAF metamodel, we find that the posterior distributions from the DGPR metamodel are wider, while simultaneously having a significantly narrower posterior predictive distribution. In other words, it is able to find a larger region of the parameter space that gives more similar outputs compared to the MAF metamodel. This shows the importance of accurate forward modelling in order to find good posterior distributions, an area where DGPR metamodels can be useful also in other settings when dealing with data from complex simulations.

Methods such as Approximate Bayesian Computation (ABC) and Simulation-based Inference (SBI) can also be used to estimate posterior distributions over parameters given specific observed model outputs. Our work is closely related to SBI, particularly SNLE [13]. The approach in the present work can be considered the same as SNLE run for a single round, where we use the DGPR metamodel as a likelihood estimator. Thus, in cases where one is interested in estimating posterior distributions, particularly in cases where one has limited data and model flexibility is a concern, DGPR metamodels may be of use in the SNLE framework. Methods such as Sequential Neural Variational Inference [49] for estimating the posterior distribution instead of MCMC sampling can also readily be used with the DGPR metamodel.

4.2 Generalization to more complex network models

The two-population model is fairly simple, and is a useful starting point since the dynamics are well known, and the simulations are cheap to run. A natural question is how well these metamodelling techniques generalize to more complex networks such as a multi-population model of cortical circuitry [50], a multi-area model of the visual system [51, 52], or a large-scale model of mouse primary visual cortex [53]. Models with more populations will have more potential model outputs to capture, and richer dynamics, and a larger number of parameters will yield a higher dimensional input space. The expressiveness of the metamodels may have to increase, and the number of samples needed to train a metamodel will likely be much higher. Of course, the number of parameters one chooses to include is arbitrary, and if one is only interested in a subset of the parameters, that can be modelled more cheaply. It is interesting to note, however, that in the present case, the power spectra mostly in the AI regime can be accurately captured by a low-dimensional latent representation.

In the present case, we model the external spiking input as a constant rate Poisson process, parameterised by the variable η. In more realistic networks, the input to the network will generally be more patterned. The parameters in the network we used were also kept constant, while in more realistic networks the parameters might follow a statistical distribution instead. Similar methods can be used to find the parameters of such distributions instead of the constant value as in the present case. Patterned input could also be represented in a metamodel, either through some continuous input to the model if the input changes in a continuous manner, or possibly discretely, in which case it may be necessary to train multiple metamodels for each input class separately.

The metamodels can easily be made more flexible, for the DGPR metamodel by increasing the number of GPs in either layer, and increasing the dimensionality in the latent space, and for the MAF metamodel by increasing the number of transformations or increasing the number of hidden features in each transformation. This will lead to higher computational costs, but more complex neural networks are of course also more costly to run, so the relative benefit may not change. In the present case, sampling was done using Latin Hypercube sampling, which ensures that each parameter is uniformly sampled. This can potentially lead to a too dense sampling in regions of the parameter space where the metamodel performs well, and too sparse sampling in regions where the metamodel performs poorly. Methods for adaptively sampling only or mostly in regions where additional sampling is needed would be an interesting direction for future research.

4.3 Use of metamodels

Once a metamodel has been trained, it can be evaluated much more rapidly than the model it is trained on. If it is well calibrated, it should be able to interpolate accurately inside the volume of the parameter space which was used to train it, and give a continuous approximation to the model behaviour. For models which are computationally demanding, trained metamodels could be distributed for use by others without requiring large computational resources.

For working with experimental data, metamodels could be useful in different ways. In the context of SBI, metamodels can be used as likelihood estimators in order to find parameters fitting network models to observed experimental data, in the same manner as in this work. Depending on which phenomena one is interested in modelling, this might require larger and more realistic network models than the one used here, but the principle would be the same. In the context of forward modelling, metamodels could be applied to experimental data to guide the choice of experimental parameters and their effects on experimental outcomes. How well this would work in practice, however, is not known. It would require training to specific experimental settings, and could be an interesting direction of future research.

4.4 Conclusion

We have shown that for the present situation, the power spectra of the population spiking activities and the local field potential can be accurately modelled by the DGPR and MAF metamodels, over a domain largely consisting of activity in the AI regime. The DGPR metamodel performed better than the MAF metamodel, but at the cost of higher computational demands for training and evaluation. The trained metamodels can be used to estimate posterior distributions over simulation parameters given observed simulation outputs. The posterior distribution estimated from the DGPR metamodel had higher marginal variances compared to the posterior distributions estimated from the MAF metamodel, while the posterior predictive distributions from the DGPR metamodel were narrower compared to the posterior predictive distributions from the MAF metamodel, indicating more accurate posterior distributions. When dealing with complex models, utilizing DGPR for forward modelling may therefore be a good alternative, particularly when the amount of data is a limiting factor, and the increased training time is not a problem.

Based on the estimated posterior distributions, the LFP was found to contain significantly more information about the synaptic strength and membrane capacitance, while for the remaining eight parameters, both signals were equally informative.