Fig 1.
Elements explored in the manuscript.
A, Schematic of the elements explored in this manuscript. Top left and clockwise: The connectivity matrix WB; its corresponding Schur upper triangular decomposition ; the eigenspectrum; and the induced dynamics. In WB and
, red dots indicate positive (excitatory) connections while blue dots indicate negative (inhibitory) connections. B, The upper triangular matrix
with the quantities that we alter in this manuscript in pink.
Fig 2.
Eigenspectra and dynamics of the corresponding networks.
A–D, Four cases of eigenspectra and dynamics of the corresponding network of size N = 200. In each panel, clockwise: The spectrum; linear dynamics; non-linear dynamics; the logarithm of the maximum norm of the firing rate per initial condition. The same initial condition that elicits the maximum norm is used for both linear and non-linear dynamics. Pink dotted line indicates the percentage of conditions whose norm is amplified by at least 50%. The feedforward structure is taken from a stability-optimised circuit [8] and its Frobenius norm is fixed to 75. Real and imaginary parts follow an uniform distribution with diameters dim and dre, respectively. A, When dim = dre = 10, only 1% (2 out of 200) of the conditions are slightly amplified. B, When dim = 10 and dre = 1, the system is capable of more amplification. C, Here, dim = 1 and dre = 10, surprisingly creating more amplification compared to the case shown in panel A. D, When dim = dre = 1, the system amplifies almost half of the initial conditions. The dynamics, given an initial condition of norm 1, reach the value of ∼ 105 Hz in the linear case and consequently long-lasting dynamics in the non-linear case.
Fig 3.
Effects of manipulating the spectrum and the feedforward norms.
A, Maximum response norm for the preferred initial condition (i), percentage of directions whose norm is amplified more than 50% (ii), and the percentage of angles (between pairs of eigenvectors) that are less than 45° (iii). Every line is a function of the imaginary diameter. We plot four real distributions. Light green: a uniform distribution in which all real parts are distributed uniformly in the interval (−0.5, 0.5). Dark green: a uniform distribution in which all real parts are distributed uniformly in the interval (−9.5, 0.5). Light pink: a single valued real distribution in which all real parts are equal to zero. Dark pink: a single valued real distribution in which all real parts are equal to −0.5. In all cases the network size is N = 200 and the feedforward Frobenius norm is fixed at 75. dre indicates the diameter of the uniform distribution of the eigenspectrum’s real part. B, Same as panel A, but plotted as a function of the feedforward Frobenius norm. Different colours correspond to 5 different spectra; all spectra have fixed single-valued real distributions (equal to zero) and different imaginary diameters. dim indicates the diameter of the uniform distribution of the eigenspectrum’s imaginary part. C, Normalised inner product between vectors from a simplified 3-by-3 upper triangular matrix (Eq 4) as a function of the imaginary diameter (β in Eq 4) for three conditions: “strong feedforward” (ϕnorm = 30 and α − γ = −0.3); “weak feedforward” (ϕnorm = 3 and α − γ = −0.3); and “large dre” (ϕnorm = 3 and α − γ = −3). D, Same as panel C, but plotted as a function of the feedforward norm for three different conditions: “small dim” (β = 100 and α − γ = 0); “intermediate dim” (β = 1000 and α − γ = 0); and “large dim” (β = 10000 and α − γ = 0).
Fig 4.
Geometry of output trajectories.
A, The effective rank of the eigenvector matrix V of as a function of the imaginary diameter (left) and the feedforward norm (right). B, Amplified directions and effective rank of the matrix
(see text) in the linear and nonlinear cases as a function of the imaginary diameter (left) and the feedforward norm (right). The feedforward structure is random from a uniform distribution, and the real distribution is uniform on (−0.5, 0.5). In all cases the network size is N = 200. The feedforward Frobenius norm is fixed at 75 for the plots with varying imaginary diameter. The imaginary diameter is fixed at 20 for the plots with varying feedforward norm.
Fig 5.
A, Representative examples of non-normal amplification defined by the timescale of the transient response of the nonlinear network—period, Δt, for which ‖r(t)‖ ≥ 1: “weak” (Δt ≤ 500 ms); “short transient” (500 < Δt < 2000 ms); and “long transient” (Δt ≥ 2000 ms). Grey dotted line indicates ‖r‖ = 1. B, Timescale of the response in the nonlinear network (as in panel A), parametrised by the norms of the spectrum and feedforward structure. Yellow indicates timescale longer than 10 seconds. Boxes correspond to the values used for the plots in panel A (colour coded): (feedforward norm, spectrum norm) = (100, 700), (500, 500), and (700, 100) for weak, short transient, and long transient, respectively. C, Maximum norm of the dynamical response per initial condition for different percentages of the norm assigned to the spectrum, ranging from a matrix whose entire norm is assigned to the spectrum (yellow; 100% case, normal matrix) to a matrix whose entire norm is assigned to the feedforward part (dark red; 0% case, nilpotent matrix). The network size is N = 200 in all panels. Both eigenspectrum and feedforward structures are random uniform.
Fig 6.
A large negative outlier increases amplification in upper triangular matrices without self loops and is proportional to I/E ratio in networks satisfying Dale’s law.
A, Maximum response norm for the preferred initial condition as a function of the imaginary diameter using upper triangular connectivity matrices with the zero trace condition and different outliers (coloured coded). The network size is N = 200 and the feedforward Frobenius norm is set to 75 in all cases. B, Percentage of directions whose norm is amplified more than 50% as function of the imaginary diameter as in panel A. C, The percentage of angles, between pairs of eigenvectors, that are less than 45°, as a function of the imaginary diameter as in panel A. D, Position of the outlier as a function of the I/E ratio for a network with 100 excitatory and 100 inhibitory neurons sparsely connected with no self loops. An initially random network is optimised with the Stability-Optimised Circuit (SOC) algorithm [8] with I/E = 40 (see Methods). The additional outliers are calculated by linearly scaling all inhibitory weights to I/E = 3, 5, 10, 20, 40.
Fig 7.
The effect of inhibitory dominance in Dalean matrices.
A, Eigenspectra of connectivity matrices satisfying Dale’s law: 100 excitatory and 100 inhibitory neurons sparsely connected (probability of connection, p = 0.1) without self loops, constructed with spectrum of radius 10 and global inhibitory dominance of strength I/E (indicated on top of each panel). Outlier, outer radius (Router), and inner circle (Rinner) are highlighted. See Methods for details. B, Eigenspectra of connectivity matrices from panel A after optimising inhibitory weights with the SOC algorithm (see Methods for details). C, Value of the purely real outlier before (open circle) and after (closed circle) optimisation. Dashed line represents the analytical expression (Eq 9). Circles correspond to average over 1000 realisations. D, Imaginary diameter of the outer (left) and inner (right) circles. Open and closed circles represent average values before and after SOC optimisation algorithm, respectively, for 1000 random realisations. The outer and inner radii are calculated as the radius for which the density of imaginary elements drops below 0.005 and below half the maximum density, respectively. Dashed grey line (left) indicates dim = 20, and purple dashed line (right) represents the analytical expression (Eq 10). E, Maximum norm per initial condition for different I/E ratios. Grey dotted line corresponds to a response norm that is 50% larger than the norm of the initial condition. Pink dashed lines indicate the percentage of initial conditions that elicit transients with maximum norm larger than 50% for I/E = 3 (lower percentage) and I/E = 40 (higher percentage). We linearly scale all weights to keep the same Frobenius norm (equal to 100) for comparison. F, The spectrum and the feedforward norms for different values of I/E in the corresponding real Schur transformation. G, Percentage of amplified conditions and effective rank of the corresponding matrix (defined in the text) in the linear case.
Fig 8.
From short transient to non-amplifying with weakened inhibition.
A, Schematic of a network with strong excitatory and inhibitory connections and I/E ratio of 40. B, The network’s dynamics given the preferred initialisation. The resulting network is in the short transient regime; the preferred initialisation yields amplifying dynamics. C, Maximum response norm for all orthogonal conditions, in decreasing order. Grey dotted line corresponds to a response norm that is 50% larger than the norm of the initial condition. Pink dashed line indicates the percentage of initial conditions with maximum norm larger than 50% of the initial condition. D, Schematic of the same network from panel A, but the inhibitory weights are scaled down by a factor of 40 (yielding an I/E ratio of 1), which could be interpreted as the resulting effect of modulation of the inhibitory neurons (or synapses). E, The dynamical response given the preferred initialisation; inset depicts the same dynamics on a different scale. F, Maximum response norm per condition. The network is unable to amplify any inputs. The maximum norm of the dynamics is equal to the norm of the initial condition (set to be 1) for all initialisations. The network is composed by 100 excitatory and 100 inhibitory neurons sparsely connected and without self loops. The schematics (panels A and D) is adapted from ref. [8].
Fig 9.
The relationship between E-to-I and E-to-E connectivity strengths alters the imaginary distribution of eigenspectra of Dalean networks.
A, Schematics of the mean-field analysis of a network with a group of excitatory (E) and a group of inhibitory (I) neurons. The mean weight from E-to-E, E-to-I, I-to-E, and I-to-I are represented by WEE, WIE, WEI, and WII, respectively. B, Weight matrix of a simplified network from panel A [10]. Inhibitory connections are optimised by the SOC algorithm [8]. C, Imaginary diameter of a network with 100 excitatory and 100 inhibitory neurons as a function of the ratio E-to-I to E-to-E weights. D, Feedforward (orange) and spectrum (red) norm as a function of the ratio E-to-I to E-to-E weights for the same networks from panel C. E, Maximum norm per initial condition for different ratios E-to-I to E-to-E weights. Grey dotted line indicates response norm that is 50% larger than the norm of the initial condition. Pink dashed lines indicate the maximum percentage of orthogonal initial conditions that evoke response norm 50% larger than initial condition for WIE/WEE = 1.8 (lower percentage) and WIE/WEE = 0.4 (higher percentage).