Processing of Feature Selectivity in Cortical Networks with Specific Connectivity

Although non-specific at the onset of eye opening, networks in rodent visual cortex attain a non-random structure after eye opening, with a specific bias for connections between neurons of similar preferred orientations. As orientation selectivity is already present at eye opening, it remains unclear how this specificity in network wiring contributes to feature selectivity. Using large-scale inhibition-dominated spiking networks as a model, we show that feature-specific connectivity leads to a linear amplification of feedforward tuning, consistent with recent electrophysiological single-neuron recordings in rodent neocortex. Our results show that optimal amplification is achieved at an intermediate regime of specific connectivity. In this configuration a moderate increase of pairwise correlations is observed, consistent with recent experimental findings. Furthermore, we observed that feature-specific connectivity leads to the emergence of orientation-selective reverberating activity, and entails pattern completion in network responses. Our theoretical analysis provides a mechanistic understanding of subnetworks’ responses to visual stimuli, and casts light on the regime of operation of sensory cortices in the presence of specific connectivity.


Introduction
Cortical neurons in early sensory areas are driven by inputs from a combination of feedforward and recurrent sources that eventually determine the selectivity of neuronal responses. In primary visual cortex (V1) of mammals, for instance, initiated by the seminal observations of orientation selective (OS) neurons by Hubel and Wiesel [1,2], the contribution of feedforward and recurrent mechanisms has been thoroughly investigated [3][4][5]. Theoretical arguments were developed that the cortical recurrent network could be responsible for the emergence of orientation selectivity [6][7][8]. The results of inactivation experiments, on the other hand, have demonstrated that the thalamic input to cortical neurons are selective for orientation, even in absence of cortical interactions [9,10].
The contribution of cortical recurrent networks to orientation selectivity has recently been reconsidered, in light of new experimental findings in rodents [11][12][13]. Given the salt-and-The current, I i (t), represents the total input to the neuron, the integration of which is governed by the leak resistance, R, and the membrane time constant, τ = 20 ms. When the voltage reaches the threshold at V th = 20 mV, a spike is generated and transmitted to all postsynaptic neurons, and the membrane potential is reset to the resting potential at V 0 = 0 mV. It remains at this level for short absolute refractory period, t ref = 2 ms, during which all synaptic currents are shunted.
Network connectivity. Each neuron receives input from exc = 20% of the excitatory population and inh = 50% of the inhibitory population, sampled randomly. Inhibitory synapses are arranged to be g = 8 times more effective than the excitatory ones [31,32], thus our networks are highly inhibition-dominated. This feature is motivated by dense connectivity of inhibitory neurons observed in different cortices [16,33,34], and functional reports for dominance of inhibition in the cortex [35,36]. Postsynaptic currents are modeled as δ-functions, where the total current is delivered instantaneously to the postsynaptic neuron after each spike. Synaptic coupling is measured by the amplitude of the resulting postsynaptic potential (PSP), J = RI. The strength of connections made by excitatory neurons in the local network is J exc = 0.2 mV; as a result, J inh = −gJ exc = −1.6 mV. Both excitatory and inhibitory PSPs decay exponentially with the membrane time constant τ, once activated.
In addition to input from local networks, neurons receive a background input from nonlocal recurrent sources. Each neuron receives an input from 5 000 non-local excitatory neurons with spontaneous firing rates of 1 spike/s, modeled as a stationary Poisson process. Synapses from non-local inputs have the same strength as local excitatory synapses, i.e. J exc = 0.2 mV. Synaptic transmission delays are fixed at 1.5 ms throughout.
We implement feature specific (FS) connectivity in the network by changing the weights of already existing connections between neurons accordingly. The weight of a connection between the jth presynaptic neuron and the ith postsynaptic neuron, w ij , is modulated by a factor that depends on the cosine of the angular difference between the respective input preferred orientations of the two neurons Dw ij ¼ w ij ½1 þ m FS cos ð2ðy Ã i À y Ã j ÞÞ: The parameter μ FS describes the degree of FS modulation of the connectivity (FS Mod), with μ FS = 0 corresponding to no modulation (no FS connectivity) and μ FS = 1 corresponding to the case with the strongest possible modulation (zero weight for orthogonal POs). Network stimulation. When the visual stimulus, i.e. an oriented elongated bar, is presented, feedforward input from the lateral geniculate nucleus (LGN) drives cortical neurons. The feedforward (thalamic) input to each neuron comprises an input from N lgn = 50 LGN cells, each with a baseline firing rate of r lgn = 20 spikes/s, using a synapse of strength J lgn = 1 mV. This amounts to a baseline firing rate s b = N lgn × r lgn = 1 kHz of the input to each neuron. The total input depends on the orientation of the stimulus, θ, and the preferred orientation (PO) of the neuron, θ Ã , according to a cosine function sðy; y Ã Þ ¼ s b ½1 þ m cos ð2ðy À y Ã ÞÞ: The parameter μ is the modulation of the input tuning, which is set to μ exc = 20% for excitatory neurons and μ exc = 0% for inhibitory neurons in our simulations. Similar to the non-local recurrent input, the feedforward input is represented by a stationary Poisson process with rate s.
To measure the output tuning curves in numerical simulations, we stimulate the networks for 12 different stimulus orientations between 0°and 180°. For each stimulus orientation, the activity of the network is simulated and recorded for 5 trials, each lasting 1.5 s. The onset transient (150 ms) is removed from the analysis.
Orientation selectivity. To quantify orientation selectivity (as in e.g. Fig 1), we compute the Preferred Orientation and the Orientation Selectivity Index, OSI, of each neuron from its output tuning curve, r(θ), obtained in numerical simulations. We first compute the circular mean [37] of the firing rate measured at each orientation, The output preferred orientation is then extracted as the angle of the resultant, arg(R), and the length of it, jRj, yields a global measure of orientation selectivity, OSI [38].
To each output tuning curve, r(θ), we fit a von Mises function: using a nonlinear least squares method. From the best fitting function, the tuning width is extracted as For each output tuning curve, r(θ), we also compute the baseline (zeroth Fourier, F0) and the modulation (second Fourier, F2) component. The baseline is obtained from the mean of the tuning curve over all orientations, and the modulation is obtained from the second Fourier component of the tuning curve.
Pairwise correlations. Pairwise correlations (in Fig 2F and S4 Fig) are computed from single-neuron spike counts in small bins. For a pair of neurons with spike count vectors n i and n j (computed from 15 s total simulation time), the correlation is obtained as the correlation coefficient of the respective spike counts CC ij ¼ r n i ;n j ¼ covðn i ; n j Þ s n i s n j ¼ E½ðn i À m n i Þðn j À m n j Þ s n i s n j where μ and σ denote vector of the means and standard deviations of the spike counts. Pairwise correlations are also computed, in the same fashion, for longer bins (the dashed line in Fig 2F and panels D-F in S4 Fig). This is computed from spike counts of neurons for 200 trials of 500 ms duration each. All neurons with firing rates higher than 1 spikes/s are considered in this analysis. The correlation coefficient computed here is similar to computing noise correlations as the product of z-scores between a pair summed across all trials (see e.g. [24]).
Linear gains. The spectrum of eigenvalues of the weight matrix shown in S3 Fig is normalized according to the linearized input-output gain, z, for individual neurons: Each synaptic weight is replaced by an effective weight J eff = zJ. This gain is obtained by linearizing the mean response rate of a LIF neuron [39] ; where μ and σ are mean and standard deviation of the current fluctuation input to the neuron, respectively,Ṽ th ¼ ðV th À mÞ=s andṼ 0 ¼ ðV 0 À mÞ=s, and erf is the error function.
The linearization is performed about the baseline state, s = s b . Under this condition, and assuming homogeneous firing rates in the network (mean-field ansatz) and neglecting Fig 1. Orientation selectivity in networks with random or specific patterns of connectivity. (A) Random networks: connectivity between neurons is independent of the preferred orientation (PO; color coded) of the lumped feedforward input to each neuron. Triangles: excitatory neurons, circles: inhibitory neurons. (B) Networks with specific excitatory to excitatory (EE) connections: the weight of synapses between excitatory neurons is increased or decreased, if the two neurons have similar or dissimilar (close to orthogonal) input POs, respectively. (C) Tuning curve of a sample excitatory neuron with and without FS connectivity. For each orientation, the mean (circles) ± one standard deviation (shading) of the firing rate over 5 trials (1.5 s each) are plotted. The tuning width (TW) is extracted from a fit (see Materials and Methods) to the tuning curve (solid: random network, dashed: network with specific connectivity). The orientation selectivity index [38], OSI = 1−Circular Variance, is computed from the actual data points. (D) Same as (C) for a sample inhibitory neuron. (E) Mean (solid line) ± one standard deviation (shading) of excitatory output tuning curves, computed from the entire population, both for random and for specific networks. The green curve is the input tuning curve, normalized in amplitude to have the same mean as the average output tuning curve in the specific correlations, mean and variance of the input can be written as [40] Here, r b is the mean neuronal firing rate when all neurons are driven by the same baseline input s b without any modulation r b = f(μ b ,σ b ). We compute this rate numerically by solving the respective self-consistency equations [40].
To obtain the linear gain we consider a small perturbation the input, δs, and compute the resulting output perturbation, δr,  The linear gain is then given by J lgn z = δr/δs. In S3 Fig, z s was obtained from choosing δs of the same size as the input modulation, δ s = s m = μs b .
Stability criterion for networks with specific connectivity. We aim to determine when the eigenvalues responsible for the modulation component in the input (here, the eigenvalues with the most positive real parts, see S3B Fig) become unstable. The modulation eigenvalue, λ m , is a result of feature specific recurrent synaptic connectivity in the network. If neurons are sorted according to their preferred orientations, the weight matrix, W, can be approximated as a circulant matrix (for the network with all-to-all connectivity, W is a circulant; here, for our random networks, we approximate the matrix as a circulant matrix corrupted with noise). In the mean-field approximation, therefore, the operation of W on an input vector, s r ¼ Ws can be written in terms of a convolution where w is the first row of W. The convolution corresponds to a product in the Fourier domain r ¼ŵŝ: We are interested here in the second Fourier component, which is an eigenmode of the network, and therefore write The modulation of the output is now a product of two modulations: a modulation in the cosine input is multiplied by the modulation component of the cosine connectivity.
We can therefore compute λ m as the modulation component of the cosine tuning of each row of the weight matrix. The parameter μ FS determines the degree of modulation, and given the parameters of network, the resultant value can be computed as if only the excitatory connections are specific (S3B Fig). A dynamic instability sets in for the critical m Ã FS for which Re(λ m ) = 1, and we obtain This is the vertical dashed line indicated in Fig 2E. The output activity of the network in response to input stimuli is obtained by considering the full N-dimensional dynamics of neuronal firing rates. Assuming linear operation of the network, the linear rate equation in the stationary state leads to the following stimulus-response relation The network operator A = (1 − W) −1 is obtained by applying an analytic function to the weight matrix W, and hence the corresponding modulation eigenvalue of A, l 0 m , can be obtained by applying the same analytic function This is the factor with which the network amplifies any modulation in its input, as long as its operation is linear. Output modulation is therefore amplified by the network with FS connectivity, relative to its baseline value, r 0 m , in absence of FS connectivity, according to r m r 0 The dashed curve in Fig 2E is derived from this relation. To some extent, it explains the nonlinear dependency of the amplification gain on the connectivity parameter.

Intracortical amplification of feedforward tuning
Using large-scale simulations of inhibition-dominated cortical networks, we studied the dynamic consequences of adding specificity to recurrent connections. Concretely, we considered enhanced connectivity between neurons with similar preferred orientations (Fig 1A, 1B). We found that such specific recurrent synaptic interactions between excitatory neurons indeed generally amplify the tuned output responses (Fig 1C-1H). This is beyond the orientation selectivity that can already arise in random networks in absence of specific interactions [31,41,42] (S1 Fig). Such an enhancement of orientation selectivity has in fact been reported developmentally after eye opening in experimental studies [18,43] Consistent with recent experimental findings [21][22][23], the excitatory recurrent input to excitatory neurons from within the network has a similar tuning as the feedforward input, only with a larger amplitude (Fig 1J and S2 Fig). Such tuned recurrent inputs are conspicuously lacking in random networks ( Fig 1I). Also, recurrent excitatory input to inhibitory neurons, and recurrent inhibitory input to both excitatory and inhibitory neurons are on average untuned (Fig 1J), as we have not implemented specific connectivity in those connections. In both random and specific cases (Fig 1I and 1J, respectively), however, the untuned components of feedforward and recurrent contributions cancel each other, and only the tuned component can trigger output spikes.

Regimes of intracortical amplification
We next asked how the properties of orientation selectivity depend on FS connectivity. It might be expected that orientation selectivity is enhanced as FS connections are enhanced. We therefore systematically increased the strength of FS connectivity in our networks, while keeping all the other parameters including the structure of connectivity the same (Fig 2).
Sample spiking activity, population responses and network tuning curves for three networks (from random to highly specific) are shown in Fig 2A-2C, respectively. Network tuning curves extracted from all simulated networks covering a range of FS connectivity are shown in Fig 2D. The tuning of output tuning curves is quantified by their respective modulation (second Fourier, F2) components. This is shown in Fig 2E, separately for excitatory and inhibitory neurons, respectively.
Surprisingly, our results show a non-monotonic dependence of tuning amplification on FS connectivity, where intermediate levels of specificity lead to the strongest amplification (Fig 2D  and 2E). Further mathematical analysis revealed that the reason for this non-monotonic dependence is the emerging instability of the eigenmode associated with modulation (see Materials and Methods and S3 Fig; hereafter, we refer to this kind of instability as "spectral instability"). Indeed, at the edge of spectral stability ( Fig 2E) the network displays the highest amplification; beyond that point, increasing specificity cannot enhance the output responses anymore.
Note that neurons are never reaching their maximal firing rate, which is 500 spikes/s given the refractory period of 2 ms for the neuron model used here. In fact, we never observed runaway firing rates in our networks. Therefore, a saturation nonlinearity is not the reason for the apparent stability of the system. Instead, nonlinear recurrent interactions in the network are responsible for this stabilization.
An explanation of this network operation can be provided as follows. When the modulation components of output tuning curves are comparable to their baseline components on average, very limited rectification occurs in the network, and the input-output transformation of tuning curves is mainly linear (see [32] for details). When the modulation component becomes too large, a rectifying nonlinearity sets in for very low firing rates. As a result of this nonlinearity, our cosine input tuning curves are not transformed to cosine output tuning curves, because the linearity of network operation is violated. Now the orthogonality of baseline and modulation components, which holds in the linear regime of operation [31], is not effective any more. As a consequence, the modulation component cannot be amplified by the network beyond any bound: for very large output modulations, the negative part of the cosine tuning curve is rectified and hence it obtains a non-zero mean value. This now leads to a projection over the common-mode of the network, which is in turn suppressed in inhibition-dominated networks [31]; this means that the output rectified modulation becomes suppressed by the strong negative feedback which was only acting on the uniform mode (the baseline component of the input) before (for a similar observation, see [32]). This in effect yields a suppressive mechanism which controls the amplification of modulation for very large values and hence nonlinearly stabilizes the network. This is also the reason for the discrepancy of the linear prediction and the actual amplification close to the critical value of FS connectivity, already before entering the region with spectral instability.
In the regimes where the nonlinear mechanisms are not dominant, our theoretical analysis fully explains how feedforward and recurrent parameters interact in generating the output selectivity. According to this analysis, the output modulation in the absence of recurrent specificity (r 0 m ) is amplified by feature-specific connections by a factor of Here μ FS is the strength of feature-specific connectivity (ranging from 0 for random connectivity to 1 for the maximally specific network), N is the total number of neurons in the network and f is the fraction of excitatory neurons, J exc is the absolute weight of excitatory connections and z is the linearized gain of leaky integrate-and-fire neurons about the baseline operating point of the network. Already from here, several questions about the amplification of orientation selectivity in our networks can be answered: first, what is important for this amplification is the product of recurrent excitation (fN exc ) and its feature-specificity (μ FS ). That is, the same degree of featurespecific connectivity in a more weakly (strongly) connected network leads to a smaller (larger) amplification. Furthermore, the total amplification obviously depends on how tuned the feedforward input is, as the recurrent specificity is only multiplying the output tuning existing in the non-specific case, which is in turn directly affected by the tuning of the feedforward input (see [31,41,42]).
It can also be asked how this amplification scales with network size, especially for infinitely large networks. To answer this, we can evaluate the critical value of feature-specificity at which the network becomes spectrally unstable: m Ã FS ¼ 2 fN exc zJ exc . Let us first assume that the linear gain, z, is 1, which would be the case for perfect integrate-and-fire neurons [32]. For infinitely large networks, N ! 1, if the weights are not rescaled and the connection probability is the same, m Ã FS would be zero, that is the network becomes unstable with very small feature-specificity.
To avoid that, rescaling the connection probability or synaptic weights becomes necessary. Another solution is to balance the specific excitatory connectivity with specific inhibitory connections. However, there is no biological evidence so far for orientation-specific connectivity of inhibitory neurons, as they are reported to densely connect to their local neighborhood [16], which, in rodents, comprises neurons with heterogeneous preferred orientations. Another solution, which is consistent with unselective inhibitory connections, is that inhibitory neurons are not balancing the specific input to the neurons, but affecting the linear gains, z. This way, it is possible that a stronger inhibitory feedback, presumably as a result of synaptic plasticity of inhibitory-excitatory connections, decreases z, which in turn increases the m Ã FS and stabilizes the network in response to modulated inputs. For networks of leaky integrate-and-fire neurons, in particular, z can be computed [44]; it is therefore possible to compute the exact dependence of the m Ã FS given any combination of parameters for the network.

Pairwise correlations in the network
Amplification of orientation selectivity in FS networks is accompanied by an increase in pairwise correlations. Unlike the amplification itself, however, this increase has a monotonic dependence on FS connectivity, and correlations increase in the network as FS connectivity becomes stronger (Fig 2F and S4 Fig).
Notably, the largest correlations arise in the spectrally unstable regime (Fig 2F). For spectrally stable networks, average correlations are rather weak, consistent with an asynchronousirregular (AI) state of network activity [40]. Population responses deviate more from AI-type network activity, however, in the spectrally unstable regime, and they approach a highly synchronous state, characterized by very large temporal fluctuations of the population activity ( Fig  2C). Emergence of large correlations indeed starts when networks approach the edge of spectral stability, as has been demonstrated and analyzed before [45].
The regime of recurrent amplification can therefore be summarized in terms of three regimes of (a) low, (b) medium and (c) high FS connectivity: (a) shows very little tuning amplification and close-to-zero pairwise correlations; (b) exhibits maximum amplification and a mild degree of correlations (noise correlation close to 0.1); (c) entails a decreasing amplification, while at the same time pairwise correlations still increase.

Spontaneous and evoked patterns of activity
We then asked the question of how spontaneous patterns of activity are influenced by FS connectivity (Fig 3). Inspecting the raster plots of activity in the three regimes of low, medium and high FS connectivity (Fig 3A-3C) reveals that the most structured pattern of spontaneous activity is observed in the regime of high FS connectivity. Temporal patterns of spontaneous population activity show structured activity, reminiscent of the patterns evoked by an oriented stimulus.
To quantify the similarity between spontaneous and evoked activity patterns, we computed a similarity index (SI) in bins of width 20 ms (Fig 3A-3C, lower panels). The most similar responses are again observed in the spectrally unstable regime (Fig 3C). This spontaneous activation of evoked responses in absence of a specific stimulus exhibits slow transitions between different preferred orientations (i.e. different points on the line attractor of the network; see [46] for similar observations).
To obtain further insights on the patterns of spontaneous activity in each regime, we plotted the distribution of similarity index and the preferred orientation of population activity for all three networks (Fig 3D, 3E). Although the most similar spontaneous activity is obtained for the spectrally unstable network, the network with medium FS also shows some enhancement of SI as compared to the random network ( Fig 3D). However, in contrast to the spectrally stable networks with uniform distributions of preferred orientations, the spectrally unstable network does not represent all orientations uniformly in its spontaneous activity. Some orientation-selective responses are over-represented, which is a result of the specific realization of the random connectivity matrix, reflected by the eigenvalue spectrum of the weight matrix (see S3B Fig).
A bias for certain preferred orientations in spontaneous activity can be quantified by the selectivity index of the distribution of preferred orientations, shown in Fig 3E. A flat distribution with no bias whatsoever would return 0, whereas a distribution of spontaneous activity concentrated in a single preferred orientation would return 1. All networks in the spectrally stable regime have a uniform distribution of preferred orientations, while transition to instability renders the spontaneous activity of networks more biased towards specific evoked states as FS connectivity increases (Fig 3F, gray line). Thus, the increase in the SI of population activity for these networks (Fig 3F, black line) comes at the price of not visiting all states with the same frequency. In the intermediate regime, however, the SI of spontaneous activity is increasing, while the bias remains at low levels.

Emergence of reverberating activity and pattern completion
We focused so far on how FS connectivity can lead to different regimes of tuning amplification, activity correlation, and spontaneous activation of specific activity patterns. However, additional functional properties can result from FS connectivity. In particular, we observed the emergence of reverberating activity, and pattern completion (Fig 4).
First, we observed that upon turning off an oriented stimulus networks with FS connectivity can show persistent activation for a brief period of time (50-100 ms; Fig 4B). As a result of feature-specific connectivity, some reverberation of activity emerges between neurons with preferred orientations close to the orientation of the stimulus. Such network-based short-term memory is lacking in the random network, and the evoked response quickly vanishes as soon as the oriented stimulus is turned off (Fig 4A).
Second, the network with FS connectivity was capable of pattern completion. Specifically, if only a fraction of excitatory neurons was stimulated with an oriented stimulus, the network with specific connectivity was capable of activating the non-stimulated neurons, with a similar pattern compared to that obtained by direct stimulation (Fig 4D, 4F). In a random network, there are no such spontaneous patterns to begin with (Fig 4C, 4E).

Discussion
Using numerical simulation and computational analysis of large-scale spiking networks, our study demonstrated how the refinement of connectivity during development contributes to the processing of orientation selectivity in cortical networks. It provided a mechanistic link between FS connectivity and processing of orientation selectivity in a biologically realistic network model, fully consistent with recent experimental findings in the rodent visual cortex [21,22]. Our mean field and spectral radius analysis could in fact provide a computational account of orientation selectivity in realistic model cortical networks operating in different regimes of FS connectivity.
An important feature of our networks was the dominance of inhibition in recurrent connectivity. This yields a selective suppression of the common-mode, which leads to an enhancement of output selectivity and contrast invariance of output tuning curves [31]. Inhibition also determines the effective linear neuronal gains in the networks, as the operating point about which the system is linearized is determined by the fixed point of the network (resulted from balance of excitation and inhibition) in response to the homogeneous input [44]. The inhibition-dominance of connectivity, in turn, leads to the balanced state in terms of network dynamics, where the net excitatory input to a single neuron is more than its threshold and only a comparable negative feedback from the network keeps the membrane potential below threshold and allows the neuron to operate in a fluctuation-driven regime of activity (as it is shown in Fig 1 and S1  and S2 Figs).
In contrast to previous modeling studies which suggested the marginal regime of recurrent connectivity is best for orientation selectivity [6], our modeling study suggested that there is an optimal value for feature-specific connectivity of the excitatory population. Beyond that, FS is detrimental to tuning amplification. Our model thus casts light on the operating regimes of intracortical amplification in the presence of specific connectivity. We found that there is a linear regime of amplification, where the results of a mean-field analysis of the network dynamics in the asynchronous-irregular, AI, state holds [40,47], and where the network shows the highest amplification of the feedforward tuning. Beyond that optimal value of FS connectivity, the output selectivity decreases again, as a result of nonlinear interactions within the network. Notably, the same nonlinear mechanisms protect the network from global instability (involving runaway excitation). Thus, the optimal regime of tuning amplification is at the edge of  2) extracted from the spiking activity of stimulated and non-stimulated excitatory neurons during the evoked state, for random and specific networks, respectively. In the random network, the non-stimulated fraction shows non-selective activity. In the network with FS connectivity, however, both stimulated and non-stimulated population show orientation selective responses. Orientation Selectivity in Feature-Specific Cortical Networks instability of the AI state, just before the eigenmode corresponding to the modulation component would become linearly unstable (cf. [47]). In contrast to the conclusion of [47], therefore, our results suggest the stable AI state as a more suitable regime for sensory processing.
In this optimal regime, we observed a moderate increase of correlations (Fig 2F and S4 Fig), despite the network being spectrally stable. A comparable level of noise correlations has in fact been reported recently in mouse visual cortex [24]. This result is also consistent with larger pairwise correlations reported recently in supra-and infra-granular layers of monkey primary visual cortex [28,29], assuming that FS connectivity is emerging in those layers as a result of Hebbian learning. We therefore hypothesize that the emergence of pairwise correlations might be considered as a signature of FS connectivity to the degree to which it implies reaching the edge of spectral stability [45]. In light of this hypothesis, it will be interesting to see whether the close-to-zero correlations in the input layers of monkey V1 [28,29] reflect the lack of FS connectivity in those layers. Also, as FS connectivity is not present at eye opening in mice [18], we predict that only very low pairwise correlations should be observed at this stage, with moderate correlations appearing only later during development, coinciding with the emergence of FS connectivity [16][17][18][19].
Note that, although correlations are observed in the activity of our networks, they were not necessary, and not detrimental, for the emergence of tuning amplification in the linear regime. In fact, the mean-field theory employed here was developed under the assumption that neurons receive uncorrelated spike trains (see Methods and Materials). This theory was nevertheless generating an excellent approximation of the mean amplification of the network, and the transition to spectral instability (Fig 2E). The success of our model, therefore, shows that mechanisms underlying the amplification of feedforward tuning in the linear regime do not include correlations across neurons; tuning amplification and correlations both emerge as a result of network dynamics, but they do not seem to be linked together causally.
As opposed to the low or moderate degree of correlations in the regimes of stable dynamics, relatively large pairwise correlations were observed in the spectrally unstable regime (Fig 2F). That is, in presence of FS connectivity, the amplification gain of the network decreases, and pairwise correlations simultaneously increase. The former effect decreases the signal-to-noise ratio, and the latter effect compromises the decoding of sensory information [48]. In contrast, the regime with low FS connectivity shows only very small pairwise correlations. This enhances the stimulus encoding, but limits the network in amplifying the sensory signal. The intermediate regime appears to provide a useful trade-off between information content of the population code and signal-to-noise ratio.
A similar trade-off can be reported for the patterns of spontaneous activity (Fig 3F). Networks with very large FS showed the most selective spontaneous patterns, resembling the evoked responses of the network best. This was, however, compromised by a non-uniform representation of orientation patterns in the spontaneous activity (see [44] for a similar observation). The intermediate regime, in contrast, showed only a moderate enhancement of selectivity of spontaneous patterns, but without introducing a bias toward a specific preferred state (Fig 3F). Another advantage of the intermediate regime is the fast switching between different states, in contrast to the very slow time scale of transitions between network states in the spectrally unstable regime (see also [46]). Note that it would be possible to further enhance the similarity of the spontaneous and evoked patterns in the intermediate regime by including NMDA synapses with slower kinetics into the model, similar to a previous modeling study [49]. Implementing this mechanism in a biologically more detailed model would ensure that the neuronal ensembles which activate together during spontaneous activity become even more pronounced and remain more active. Whereas such ensembles would be random groups of neurons which are more connected together by chance in a random network, the spontaneously active subnetworks would become feature-specific in networks with FS connectivity. Now, a pronounced functional map of orientation selectivity determines co-active ensembles in spontaneous activity and makes them similar to the evoked ones.
A biologically more realistic version of our model should also account for two further features. First, we neglected heterogeneity in synaptic connections, as the number and strength of synapses were the same for all networks, and only the strength of synapses was modified based on FS of pre-and post-synaptic neurons. A more biologically realistic network with heterogeneous connectivity (including heterogeneity in the number and strength of synapses, heterogeneity in FS connectivity, and heterogeneity in the balance of excitation and inhibition) leads to more realistic responses. However, our main findings here also hold under such conditions, and do not change qualitatively (results not shown). Second, we only studied networks in which only the excitatory population underwent a change in synaptic connectivity. It is possible, however, that during development, excitatory to inhibitory or inhibitory to excitatory connections also change. Potentiation of inhibition as a result of such a plasticity may well ensure the stability of the modulation eigenmode, preventing the network from entering the spectrally unstable regime. As a result of potentiation of inhibitory connections the firing rate of excitatory neurons can also decrease. This might lead to moderate firing rates which seem to be more compatible with experimental results in rodents. It remains, therefore, an interesting topic for further studies how concomitant changes in the weights of excitatory and inhibitory connections affect network dynamics and neuronal tuning curves.
In our networks here with FS excitatory connections, we also found two further emergent properties. First, we observed pattern completion in our networks: when only half of the excitatory population was stimulated, networks with FS connectivity responded as if the whole stimulus pattern was present. We also observed an emergence of reverberating activity as a result of specific connectivity, where orientation selective population activity persisted for a few tens of millisecond after removal of the visual stimulus. Such a persistent activity has indeed been reported in cats [50], with time constants very similar to those reported here. It is therefore interesting to see whether a similar effect exists in mouse visual cortex. As feature-specific connectivity is absent at eye opening, the prediction is again that reverberating activity only emerges later during development, as soon as a significant degree of FS connectivity is established [18]. It is also tempting to speculate about the biological function of this reverberating activity. It might be crucial for creating association in time, and instrumental to solve the problem of temporal feature-binding. Using analytical and computational tools, we therefore provide a very-much needed framework to understand the role of FS connectivity in visual processing in particular, and sensory processing in general. . For each orientation, the trial average (circles) ± one standard deviation (shading) of the firing rate (during 1.5 s of stimulation) are plotted. The tuning width, TW, is extracted from a fit to the tuning curve (solid line; see Materials and Methods). The orientation selectivity index [38], OSI = 1−Circular Variance, is computed from the actual data points. (C) Same as in (B), for a sample inhibitory neuron. (D) Data from 800 excitatory (red) and 200 inhibitory (blue) sample tuning curves, each aligned to the preferred orientation (PO) of its input. The respective tuning curve of the input (same for all neurons) is plotted in green. It is normalized such that it has the same mean value as the output tuning curve. (E) Mean (black line) ± standard deviation (gray shading) of output tuning curves, computed from the entire neuron population. The green curve is again the normalized input tuning curve, and the dashed lines indicate the average firing rate for excitatory (red) and inhibitory (blue) populations. OSI and TW are computed from the fit (dotted black line). (F) Distribution of the zeroth (F0, baseline) and the second (F2, modulation) Fourier components of output tuning curves in the network. (G, H) Distribution of OSI and TW for the population of excitatory (red) and inhibitory (blue) neurons, respectively. The TW distribution is only plotted for neurons with less than 10% error of fit (see Materials and Methods). The distribution of this error is shown in the inset in (H). (I) Distribution of the difference between input and output preferred orientations, ΔPO = Output PO −Input PO, for excitatory (red) and inhibitory (blue) populations. (J) Excitatory (red) and inhibitory (blue) recurrent input to a sample neuron with input preferred orientation, θ Ã = 0°in response to a stimulus of orientation θ = 0°. The weight matrix is obtained by linearizing the network dynamics and computing neuronal gains (see Materials and Methods for further explanation). The large negative eigenvalue, corresponding to the non-selective mode, is not shown for graphical reasons. It is a result of the inhibition dominance in our networks, and it is responsible for selective attenuation of the common mode. The eigenvalues with second and the third largest magnitude, λ 1 and λ 2 , are marked by circles in (B) and (C). The black cross denotes the prediction based on our theory (see Materials and Methods for details). Lower panel: Eigenvectors corresponding to the three largest eigenvalues of the network are plotted versus the input preferred orientation of the corresponding neurons, separately for excitatory and inhibitory neurons (x-axis). The first eigenvector has uniform components and corresponds to the large negative eigenvalue λ 0 (not shown in the spectrum). Only the real part of all components of the eigenvectors are plotted. Distribution of pairwise correlations in the network with random connectivity. Pearson correlation coefficient, CC, of spike counts for different bin sizes (5, 10, 20 and 50 ms, respectively) are computed for randomly sampled neurons (100 excitatory and 100 inhibitory). CC is plotted separately for excitatory-excitatory (EE, red), inhibitory-inhibitory (II, blue), and excitatory-inhibitory (EI, purple) correlations. The mean CC is given for each distribution. (B) Same as (A) for networks with an intermediate degree of specific connectivity (50%; μ FS = 0.5). At this degree of specific connectivity, the modulation eigenmodes are still stable. (C) Same as (A) for networks with a very high degree of FS connectivity (100%; μ FS = 1). The modulation eigenmodes are unstable for this network. Note that the distributions show qualitatively the same behavior for different bin sizes. (D-F) Distribution of pairwise correlations for all three networks for a very large bin size. It is computed from 200 trials of 500 ms spiking activity, and all neurons with an average firing rate more than 1 spikes/s are included in this analysis. (TIFF)