< Back to Article

Cooperative coding of continuous variables in networks with sparsity constraint

Fig 6

Response speed of networks with inhibition and linear MS.

(A) Response times for the 1D network and for the 2D linear MS network. The quadratic scaling of with for the excitatory networks (blue) can be improved to a linear dependence by introducing balancing, delayed inhibition (red) or SFA (orange). Open (1D network) and filled (MS network) circles display numerical results. Alike-colored dotted (1D network) or continuous (MS network) curves show theoretical estimates (Eqs 14, 31, 37, 38) or, for the SFA network, fit results (monomial fit: ). We use the slowliest-decaying eigenmode to theoretically estimate the response times (see (C) and Eq S46 in S1 Appendix). Since the balanced networks are not initialized in this eigenmode (in contrast to the purely excitatory networks), the numerically measured response times (red markers) lie above the theoretical values (red lines). (B) Schematic of a 2D network with linear MS. Feature neurons are arranged on a two-dimensional grid (labeled “Response x”). Each receives feedforward input from two arrays of input neurons (labeled “Inputs r^(1|2)”) and four recurrent inputs. Feedforward and recurrent synapses are shown in blue (exemplarily) and black, respectively. Input and feature neuron activities are color-coded. The (linear) network response is the sum of the responses to input one and input two. (C) Exemplary loss evolution of a 1D network with lagged inhibition. Due to the temporally constant initialization (x_i(0) = 0, ), the network activity (solid red curve) converges initially more slowly than the network’s slowest eigenmode (dotted red line). The experimentally measured response time (continuous vertical gray line) is defined as the time when the loss has decayed by 1/e (red open circle, horizontal gray line), see also Fig 4A. It is larger than that of the network’s eigenmode (dotted gray line), which we use as analytical estimate of the response time. We created the data in (A) by scanning , setting to yield an RF of size , setting to 0 or its critical value, and determining or from the loss dynamics. For the SFA network we set , scanned and used the value that minimized the temporally integrated loss.

doi: https://doi.org/10.1371/journal.pcbi.1012156.g006