Fig 1.
Receptive field and response of 1D network.
(A) The RF of neuron i is peaked at j = i and decays exponentially with . The RF width parameter is here d = 2. (B) Top: The network response to an isolated unit input, here located at j0 = 2, has the same shape and amplitude as a neuron’s RF. It peaks at i = j0. Bottom: Input and feature neurons are shown with color-coded activities rj and xi, respectively. Increasingly dark red color represents higher activity; white squares indicate inactive neurons.
Fig 2.
Schematics of feedforward and cooperatively coding networks.
(A) Top: In the feedforward network, the response (gray solid curve) to an isolated input is fully generated by the neurons’ feedforward inputs (blue lines and dots, line thickness represents input strength). For the displayed RF width d = 2, five neurons receive feedforward input, so that the network response (gray solid curve) represents
of the summed target response (gray dashed curve). Bottom: Feature and input neuron activities as in Fig 1. Outgoing feedforward synapses from the active input neuron j = 2 and incoming feedforward synapses to feature neuron i = 6 are shown in blue. (B) Top: In the cooperatively coding network model, the network response (gray solid curve) is the sum of feedforward input (blue line and dot) and recurrent input (brown-purple dashed curve). For the displayed case of an isolated input, only one neuron receives feedforward input, which induces a part of the stationary response of the most active feature neuron. The rest of the response and all other responses are induced by recurrent input from neighboring neurons. The total recurrent input that each feature neuron receives is the sum of recurrent input from the right (brown solid curve) and left neighbor (purple solid curve). Bottom: Each feature neuron receives one feedforward synapse (blue lines) and two recurrent synapses (black lines, all recurrent connections are bidirectional). (C) The RF of feature neuron i (
for varying j, gray solid curve) is the weighted sum (brown-purple dashed curve) of the RFs of its left (
, purple) and right neighbors (
, brown) plus a contribution from feedforward input (
, blue line and dot). All shown RFs have width d = 2.
Fig 3.
Response formation and activity propagation.
(A) Network activity at different times (shaded curves) after r100 has been set from 0 to 1. For long times, network activity approaches the target response (black curve). (B) Development of the activity of neurons (y axis) with time (x axis), measured relative to their target activities. The diagonal fronts of equal relative activities indicate propagation of activity with constant propagation speed. The points where neurons reach 50% of their final activity are connected by a red dashed line. Parameters: , such that
(see Eq 13), N = 200 neurons.
Fig 4.
Loss evolution and response speed - synapse number trade-off.
(A) Exemplary loss evolution for a network with so that
. Experimentally,
is determined as the time (gray vertical line) at which the loss drops to
(gray horizontal line, blue open circle). (B) Response times
(circles: simulation results; dotted line: analytical solution Eq 14) for target RFs of different widths
. Data was created by scanning
, setting
to yield an RF of size
and determining
from the loss dynamics.
Fig 5.
Loss evolution for different strengths of EI-balance.
(A) Loss evolution (dashed: analytical approximation (cf. Eq S30 and S41 in S1 Appendix, partly occluded; solid: network simulation) for balance strengths that are slightly weaker (orange), equal (blue) or slightly stronger (teal) than the critical balance, on a logarithmic scale. The slope of the decay is given by (see (B)), explicitly highlighted for the overdamped dynamics. The oscillation period of the underdamped dynamics is
. In case of oscillations, the analytic approximation briefly reaches zero loss once in a period (sharp dips in dashed curve). In the network simulation there is also a pronounced oscillation, but there always remains a finite error. (B) Real part (decay rate
, black/gray) and imaginary part (oscillation frequency
times
, red) of the complex frequency of the exponential loss evolution, scaled by
. For weak EI-balance, measured by
, there are two exponentially decaying modes (
, black and gray curve). At the critical balance
(blue dashed vertical line), there is only a single decay rate and no oscillation; the decay rate (in the overdamped case: of the relevant slower-decaying mode) is maximized. For stronger balance, network activity begins to oscillate (nonzero
, red), and diverges once
becomes negative. This happens approximately at
, which is slightly larger than 1 because of the stabilizing effect of the contracting dynamics of the unbalanced network. Dashed vertical lines show the balance strengths scaled by
for the curves in (A) (
). Parameters:
,
,
, and N = 200 for the network simulation.
Fig 6.
Response speed of networks with inhibition and linear MS.
(A) Response times for the 1D network and for the 2D linear MS network. The quadratic scaling of with
for the excitatory networks (blue) can be improved to a linear dependence by introducing balancing, delayed inhibition (red) or SFA (orange). Open (1D network) and filled (MS network) circles display numerical results. Alike-colored dotted (1D network) or continuous (MS network) curves show theoretical estimates (Eqs 14, 31, 37, 38) or, for the SFA network, fit results (monomial fit:
). We use the slowliest-decaying eigenmode to theoretically estimate the response times (see (C) and Eq S46 in S1 Appendix). Since the balanced networks are not initialized in this eigenmode (in contrast to the purely excitatory networks), the numerically measured response times (red markers) lie above the theoretical values (red lines). (B) Schematic of a 2D network with linear MS. Feature neurons are arranged on a two-dimensional grid (labeled “Response x”). Each receives feedforward input from two arrays of input neurons (labeled “Inputs r(1|2)”) and four recurrent inputs. Feedforward and recurrent synapses are shown in blue (exemplarily) and black, respectively. Input and feature neuron activities are color-coded. The (linear) network response is the sum of the responses to input one and input two. (C) Exemplary loss evolution of a 1D network with lagged inhibition. Due to the temporally constant initialization (xi(0) = 0,
), the network activity (solid red curve) converges initially more slowly than the network’s slowest eigenmode (dotted red line). The experimentally measured response time (continuous vertical gray line) is defined as the time when the loss has decayed by 1/e (red open circle, horizontal gray line), see also Fig 4A. It is larger than that of the network’s eigenmode (dotted gray line), which we use as analytical estimate of the response time. We created the data in (A) by scanning
, setting
to yield an RF of size
, setting
to 0 or its critical value, and determining
or
from the loss dynamics. For the SFA network we set
, scanned
and used the value that minimized the temporally integrated loss.
Fig 7.
2D network schematic and response times versus RF size.
(A) Schematic of a two-dimensional network responding to a two-dimensional stimulus. Feature neurons (labeled “Response x”) and input neurons (labeled “Inputs r”) are arranged on two-dimensional grids. In the cooperatively coding network each feature neuron receives one feedforward and four recurrent inputs; activities and shown connections are color-coded as in Fig 2. (B) Response times in the cooperatively coding 2D network increase linearly (without inhibition, blue. Monomial fit: ) or square-root-like (with inhibition, red. Monomial fit:
) with the RF size
. Dotted lines represent the monomial fits. Data was created by scanning
, setting
to 0 or its critical value, and determining
and
or
, respectively, from the response curves after network activity converged.
Fig 8.
Cooperative coding in an excitatory spiking network of leaky integrate-and-fire neurons.
(A) Network wiring diagram. Connections are marked in black if their strength depends on RF size, and in gray otherwise. (B) Single neuron transfer function (black solid) and threshold-linear fit (gray dashed). Inset highlights onset nonlinearity. (C) Top: Simulated vs. target RF size. Light gray crosses: networks with analytically computed weights; black circles: networks with numerically optimized weights. Middle: excitatory synaptic weights (normalized for arbitrary indegree: the voltage increase in response to one spike is given by ) . Bottom: Feedforward input to stimulated population (on) and to others (off), used both for networks with analytically or numerically computed weights. Left and right dashed vertical lines mark example simulation shown in (D) and (E). The target RF peak rate was set here to
Hz. (D) Example simulation for target RF size
. Top: Binned average firing rates of 41 feature populations. Middle: Raster plot showing spikes of 10 exemplary neurons from each feature population. Color codes distance from the stimulated population (population index 21, the population includes neurons 200-209 shown in black). Bottom: Feedforward stimulation. Right: Stationary population rates (gray), exponential fit (black dashed), and target rate profile (red). (E) Example simulation for target RF size
. All panels are as in (D).
Fig 9.
Number of synaptic connections for the feedforward and cooperative coding architecture depending on RF size, for different sizes of feature populations.
Total number of synapses in a network of NF = 41 feature populations as a function of RF size and number of neurons NE per feature population (color-coded). In the feedforward network the number of synapses
(solid lines) depends linearly on RF size with a slope proportional to NE. In the cooperative network the number of synapses
(dotted lines) is independent of RF size but increases quadratically with NE. Vertical lines mark the minimal RF size for which the cooperative network saves synapses compared to the feedforward network (
). Synapse numbers are shown here for
and
.
Fig 10.
Response times of cooperatively coding spiking neural networks.
(A) Network wiring diagram of the balanced excitatory-inhibitory spiking network. Connections are marked in black if their strength depends on RF size, and in gray otherwise. (B) Stationary rate profiles of excitatory (blue) and balanced networks (red) in response to isolated inputs match. There is no balanced network simulation shown for , since there is no recurrent coupling between or within populations in that case. (C) Top: Response times. Blue: purely excitatory network (cf. Fig 8). Red: balanced excitatory-inhibitory network. Dashed horizontal line: membrane time constant. Quadratic and linear scaling is indicated by the dashed and dotted lines, respectively. Bottom: scaling factor s used to scale up the excitatory synaptic coupling strength
in the balanced network. (D) Activity of a balanced network with RF size 11. Panels as in Fig 8D, 8E; top: excitatory, bottom: inhibitory populations. (E) Direct comparison of the rate of the stimulated population (index 21) around stimulus onset in the purely excitatory network (blue, cf. black trace in Fig 8E), and in the balanced network (red, cf. black trace in panel D, top). Rates are shown for a finer binsize of 0.5 ms.
Table 1.
Parameters of spiking network models. Top: excitatory-only network (Fig 8). Bottom: additional parameters for balanced network (Fig 10). Equation numbers refer to S1 Appendix.