Fig 1.
Viewing mechanistic neuronal circuit models as generative models for tuning curves.
(A) Schematic representation of a generic recurrent circuit model from theoretical neuroscience. The network structure, in this case reduced to the full recurrent connectivity matrix W, is partially random with statistics described by the distribution , which depends on parameters θ. These parameters could, e.g., include the average strength or spatial range of connections between different cell types. The network receives an external input, I(s), that represents stimuli, which can be in one of S different conditions employed in an experiment. The output of the model is taken to be the sustained response,
, of a pre-selected probe neuron (blue arrow). Given a realization of the network structure W (sampled from
), this response can be obtained in each stimulus condition, s, by simulating the network in the presence of external input I(s). The responses,
, for all stimulus conditions are then concatenated into a tuning curve vector
, which is the ultimate output of the network when viewed as a generative model. (B) The deterministic mapping f stands for the simulation process that links the particular realization of the network structure W to its functional (tuning curve) output x. Since the ultimate goal is to use gradient-based methods to learn the model parameters, θ, the process of sampling a realization of W is cast (cyan area) using a parametrized mapping, gθ, that transforms a set of standard noise variables z, sampled from a fixed distribution, into W. The composition of f and gθ yields the full parametrized generator function, Gθ ≡ f ◦ gθ, as used in the GAN framework (bottom row). Given a set of parameters, the generator thus receives a set of noise variables, z, sampled from their standard distribution, and generates a tuning curve, x.
Fig 2.
Structure of the feedforward and recurrent generative models used in our computational experiments.
(A) The feedforward network model of primary motor cortex (M1) is borrowed from Ref. [4] and produces heterogeneous hand-location tuning curves. This heterogeneity is rooted in the random network structure, including the variability in the input layer (modeling a premotor or parietal area) receptive field widths, σi, feedforward weights to the M1 layer, , and the threshold, ϕ, of M1 output neurons which are rectified linear units. (B) The structure of the Stabilized Supralinear Network (SSN) with one-dimensional topography (retinotopy) as a model of the primary visual cortex (V1). The SSN is a recurrent network of excitatory (E) and inhibitory (I) neurons. The visual stimulus (bottom) models the input to V1 due to a grating of diameter bs in condition s. Heterogeneity in model output (size tuning curves) originates in the heterogeneity of feedforward and recurrent horizontal connections. The mean and variance of the horizontal connections between SSN neurons depend on the pre- and postsynaptic cell-types and their retinotopic distance, and for different connection-types falloff over different characteristic length scales. For a full description of models and their parameters see Materials and methods.
Fig 3.
Summary of the feedforward model fit to the M1 tuning curve dataset of Ref. [4].
(A) Data and model tuning curves at different stages of training; the plotted tuning curves are the projections of the 3D tuning curves along the preferred position (PP) vector which is obtained by a linear fit to the 3D tuning curve (see Ref. [4]). (B) Kolmogorov-Smirnov (KS) distance throughout training between the data and model distributions, shown in panels C–F, of four summary statistics characterizing the tuning curves evaluated on held-out data. Vertical dashed lines mark the epochs at which curves in panel A are sampled. (C–F) histograms of four tuning curve statistics (R2, complexity score, firing rates, and coding level) showing the data distribution (black), the initial model distribution (blue) and the final model distribution after fitting (red). Vertical lines show the mean value of each histogram.
Table 1.
The parameter values for the ground-truth SSN used for generating the training set tuning curves.
The columns correspond to different (a, b) possibilities. The feedforward weight heterogeneity parameter is V = 0.1.
Fig 4.
Summary of the recurrent SSN model fit to simulated V1 tuning curve data.
(A) Data and model tuning curves at three different stages of training. (B) KS distance between the data and model distributions of four summary statistics characterizing the tuning curves, throughout training. Vertical dashed lines mark the epochs at which curves in panel A are sampled. (C) Scatter plots of peak rate vs. suppression index (S.I.) for the training data, the WGAN fit, and moment matching fit. (D) Cumulative distribution functions (CDF) for preferred stimulus size and normalized participation ratio of the data, the WGAN fit, and moment matching.
Fig 5.
System identification of the SSN model from simulated tuning curve data using the cWGAN method and moment matching methods.
Top and bottom rows show the results of cWGAN and moment matching fits, respectively. (A) & (D): symmetric mean average percent error (sMAPE), Eq (44), of all parameters throughout training. (B) & (E): the (solid lines) and δJab (shaded areas) parameters throughout training. The box to the right shows the true parameters as points (
) with error bars (δJab). (C) & (F): σab throughout training. Arrows to the right of the plot denote the true values of σab; note that the E → I and I → I arrows occlude each other in panels C and F. In B-C and E-F colors represent connection-types as shown in the legend of B. The input heterogeneity parameter V (not plotted) converged to 0.100 for cWGAN, and to 0.116 for moment matching, compared to the value V = 0.1.
Fig 6.
Comparison of conditional Wasserstein GAN and moment matching.
Distribution of the relative error (symmetric mean average percent error; sMAPE) for our proposed method (cWGAN, in purple) and moment matching (MM, in Green) across different hyperparameters (see Materials and methods). The bars denote the number of trainings, possibly with different hyperparameters, resulting in an sMAPE within the corresponding bin. cWGAN consistently produced lower estimation errors compared to moment matching.
Table 2.
The description of the parameters of the feedforward model fit to tuning curve data.
Table 3.
The description of the parameters of the SSN inferred from tuning curve data (a, b ∈ {E, I}).