Cellular Adaptation Facilitates Sparse and Reliable Coding in Sensory Pathways

Most neurons in peripheral sensory pathways initially respond vigorously when a preferred stimulus is presented, but adapt as stimulation continues. It is unclear how this phenomenon affects stimulus coding in the later stages of sensory processing. Here, we show that a temporally sparse and reliable stimulus representation develops naturally in sequential stages of a sensory network with adapting neurons. As a modeling framework we employ a mean-field approach together with an adaptive population density treatment, accompanied by numerical simulations of spiking neural networks. We find that cellular adaptation plays a critical role in the dynamic reduction of the trial-by-trial variability of cortical spike responses by transiently suppressing self-generated fast fluctuations in the cortical balanced network. This provides an explanation for a widespread cortical phenomenon by a simple mechanism. We further show that in the insect olfactory system cellular adaptation is sufficient to explain the emergence of the temporally sparse and reliable stimulus representation in the mushroom body. Our results reveal a generic, biophysically plausible mechanism that can explain the emergence of a temporally sparse and reliable stimulus representation within a sequential processing architecture.


Introduction
The phenomenon of spike-frequency adaptation (SFA) [1], which is also known as spike-rate adaptation, is a fundamental process in nervous systems that attenuates neuronal stimulus responses to a lower level following an initial high firing. This process can be mediated by different cell-intrinsic mechanisms that involve a spike-triggered self-inhibition, and which can operate in a wide range of time scales [2][3][4]. These mechanisms are probably related to the early evolution of the excitable membrane [5][6][7] and are common to vertebrate and invertebrate neurons, both in the peripheral and central nervous system [8]. Nonetheless, the functional consequences of SFA in peripheral stages of sensory processing on the stimuli representation in later network stages remain unclear. For instance, light adaptation in photoreceptors strongly shapes their responses [9,10] and affects stimulus information in second-order neurons [11]. In a seminal work by Hecht and colleagues [12], it was shown that during dark adaptation, 10 or less photon absorptions in the retina were sufficient to give a sensation of light within a millisecond of exposure and the response variability could be largely accounted for by quantum fluctuations. This is an interesting empirical result, and still it is theoretically puzzling that the intrinsic noise of the nervous system [13] has only little influence on the detection of such an extremely weak stimulus. A proposal by Barlow [14] suggested that successive processing in sensory neural pathways decrements the number of response spikes and therefore the informativeness of each spike increases while the level of noise decreases. However, it remains unclear how such temporally sparse spike responses can reliably encode information in the face of the immense cortical variability [15] and the sensitivity of cortical networks to small perturbations [16,17].
The widespread phenomenon of a dynamically suppressed trialby-trial response variability in sensory and motor cortices [18][19][20] along with a sparse representation [21,22] hints at an increased reliability of the underlying neuronal code and may facilitate the perception of weak stimuli. However, the prevailing cortical network models of randomly connected spiking neurons, where the balance of excitation and inhibition is quickly reinstated within milliseconds after the arrival of an excitatory afferent input, do not capture this dynamic [16,17,[23][24][25]. Recent numerical observations suggest that a clustered topology of the balanced network [26] or attractor networks with multi-stability [27] provide possible explanations for suppressing cortical variability during afferent stimulation.
In this study, we introduce an alternative and unified description in which a temporally sparse stimulus representation and the transient increase of response reliability emerge naturally. Our approach exploits the functional consequences of SFA in multi-stage network processing. Here, we show that the SFA mechanism introduces a dynamical non-linearity in the transfer function of neurons. Subsequently, the response onset becomes progressively sparser when transmitted across successive processing stages. We use a rigorous master equation description of neuronal ensembles [28][29][30] and numerical network simulations to arrive at the main result that the self-regulating effect of SFA causes a stimulus-triggered reduction of firing variability by modulating the average inhibition in the balanced cortical network. In this manner the temporally sparse representation is accompanied by an increased response reliability. We further utilize this theoretical framework to demonstrate the generality of this effect in a highly structured network model of insect olfactory sensory processing, where sequential neuronal adaptation readily explains the ubiquitously observed sparse and precisely timed stimulus response spikes at the level of the so-called Kenyon cells [31][32][33][34]. Our experimental results qualitatively supports this theoretical prediction.

Temporal sparseness emerges in successive adapting populations
To examine how successive adapting populations can achieve temporal sparseness, first we mathematically analyzed a sequence of neuronal ensembles ( Figure 1A), where each ensemble exhibits a generic model of mean firing rate adaptation by means of a slow negative self-feedback [2,28,35] (Materials and Methods). This sequence of neuronal ensembles should be viewed as a caricature for distinct stages in the pathway of sensory processing. For instance, in the mammalian olfactory system the sensory pathway involves several stages from the olfactory sensory neurons to the olfactory bulb, the piriform cortex, and then to higher cortical areas ( Figure 1A).
The mean firing rate in the steady-state of a single adaptive population can be obtained by solving the rate consistency equation, r r i~f ((g r r i{1 zm i ){t s q s r r i Ds 2 i ), where r r i is the equilibrium mean firing rate of the i{th population, g, m i and s 2 i are coupling strength, mean and variance of the total input into the population, respectively, f is the response function (input-output transfer function, or f {I curve) of the population mean activity, q s is the quantal conductance of the adaptation mechanism per unit of firing rate, and t s is the adaptation relaxation time constant [2,28,35]. The firing rate model assumes that individual neurons spike with Poisson statistics, and that the adaptation level only affects mean input into neurons, resulting in a change to the steady-state mean firing rate. It is known that any sufficiently slow modulation (1=r max vt s ) linearizes the steady-state solution, r r i , due to the self-inhibitory feedback being proportional to the firing rate (see Materials and Methods) [36].
Here, for simplicity, we studied the case where all populations in the network exhibit the same initial steady-state rate. This is achieved by adjustment of a constant background input to population i, given g (doted arrows, Figure 1A), resembling the stimulus irrelevant interactions in the network. All populations are coupled by the same strength g. First, we calculated the average firing rate dynamics of the populations' responses following a step increase in the mean input to the first layer (black arrow, Figure 1A). By solving the dynamics of the mean firing rate and adaptation level concurrently, we obtained the mean-field approximation of the populations' firing rates (Materials and Methods). As it is typical for adapting neurons, the responses of each population consisted of a fast transient following stimulus onset before it converges to the new steady-state (tonic response part) with a stable focus (Materials and Methods). The Figure 1B shows the mean firing rate of three consecutive populations. The phasic response to the step increase in the input is preserved across stages. However, the tonic response becomes increasingly suppressed in the later stages ( Figure 1B). This phenomenon is a general feature of successive adaptive neuronal populations with a non-linear transfer function f which, is linearized in steady-state due to adaptation negative feedback [36]. This result emerges as the change in the i{th population mean rate that can be determined by solving the rate consistency equation now for a step change in the input D r r i~f (gD r r i{1 {D r r i t s q s ). The necessary condition for the suppression of the steady-state responses is a sufficiently strong

Author Summary
Many lines of evidence suggest that few spikes carry the relevant stimulus information at later stages of sensory processing. Yet mechanisms for the emergence of a robust and temporally sparse sensory representation remain elusive. Here, we introduce an idea in which a temporal sparse and reliable stimulus representation develops naturally in spiking networks. It combines principles of signal propagation with the commonly observed mechanism of neuronal firing rate adaptation. Using a stringent numerical and mathematical approach, we show how a dense rate code at the periphery translates into a temporal sparse representation in the cortical network. At the same time, it dynamically suppresses trial-by-trial variability, matching experimental observations in sensory cortices. Computational modelling of the insects olfactory pathway suggests that the same principle underlies the prominent example of temporal sparse coding in the mushroom body. Our results reveal a computational principle that relates neuronal firing rate adaptation to temporal sparse coding and variability suppression in nervous systems.
adaptation (Materials and Methods). It is worth to note that the populations exhibit under-shoots after the offset of the stimulus ( Figure 1B). This is due to the adaptation level that accumulated during the evoked state.
The result in this sub-section ( Figure 1B-C) was established with a current based leaky integrated-and-fire response function. However, the analysis presented here extends to the majority of neuronal transfer functions since the stability and linearity of the adapted steady-states are granted for many biophysical transfer functions [35,36]. This simple effect leads to a progressively sparser representation across successive stage of a generic feedforward adaptive processing. We assess temporal sparseness by computing the time-dependent integral S i r (T)~Ð T 0 r i (t)dt, where r i (t) is the mean firing rate of population i and T is the increasing observation time window. Normalization of this measure by the spike count in the first population S 1 r (T~500) indicates that responses in the later stages of the adaptive network are temporally sparser ( Figure 1C). This is expressed in the sharp increase of the rate integral during the transient response, whereas the first population integral shows an almost constant increase in the number of spikes.
Does the suppression of the adapted response level impair the information about the presence of the stimulus? To explore this, we studied a secondary increase in stimulus strengths with an equal magnitude, after 1 second, when the network has relaxed to the evoked equilibrium ( Figure 1D). The secondary stimulus jump induced a secondary phasic response of comparable magnitude in the first population ( Figure 1D). However, in the later populations this jump evoked an increased peak rate in the phasic response ( Figure 1D). Notably, the coupling factor g between the populations shapes this phenomena. Here, we adjusted g to achieve an equal onset response magnitude across the populations for the first stimulus jump at t~0, and a slight increase in the population onset response in the first population is amplified in the later stages. This is due to the fact that the later stages accumulated less adaptation in their evoked steady-state (the level of adaptation is proportional to mean firing rate). Importantly, this result confirms that the sustained presence of the stimulus is indeed stored at the level of cellular adaptation [37], even though it is not reflected in the firing rate of the last population (Materials and Methods). Therefore, regardless of the absolute amplitude of responses, the relative relation between secondary and initial onset keeps increasing across layers. This type of secondary overshoot is also experimentally known as sensory sensitization or response amplification, where an additional increase in the stimulus strength significantly enhances the responsiveness of later stages after the network converged to an adapted steady-state [10,11,38,39].

Adaptation increases response reliability in the cortical network
The mean firing rate approach as above is insufficient to determine how reliable the observed response transients are across repeated simulations. In a spiking model of neo-cortex (the balanced network), self-generating recurrent fluctuations strongly dominate the dynamics of interactions and produce highly irregular and variable activity [24]. This prevailing cortical model suggests that balance of excitation and inhibition is quickly reinstated within milliseconds after the onset of an excitatory input and adjusts the network fluctuations level [23]. Therefore, it has been questioned whether a few temporarily meaningful action potentials could reliably encode the presence of a stimulus [16].
To investigate the reliability of adaptive mapping from a dense stimulus to a sparse cortical spike response across successive processing stages we employed the adaptive population density formalisms [28,29] (Materials and Methods) along with numerical network simulations. We embedded a two-layered sensory network with an afferent ensemble projecting to a cortical network ( Figure 2A). The afferent ensemble consisted of 4,000 adaptive neurons that included voltage dynamics, conductancebased synapses, and spike-induced adaptation [28]. It resembles the sub-cortical sensory processing and each neuron in the afferent ensemble projects randomly to 1% of the neurons in the cortical network. This is a large circuit of the balanced network ( Figure 2A) with 10,000 excitatory and 2,500 inhibitory neurons with a typical random diluted connectivity of 1%. The spiking neuron model in the cortical network again includes voltage dynamics, conductance-based synapses, and spike-induced adaptation [28]. All neurons are alike and parameters are given in Table 3 in Muller et al. [28]. With appropriate adjustment of the synaptic weights, the cortical network operates in a globally balanced manner, producing irregular, asynchronous activity [24,40,41]. The distribution of firing rates for the network approximates a power-law density [42] with an average firing rate of &3:0Hz ( Figure 2B) and the coefficients of variation (C v ) for the inter-spike intervals are centered at a value slightly greater than unity ( Figure 2C) indicating the globally balanced and irregular state of the network [41]. Noteworthy is that the activity of neurons in both stages is fairly incoherent and spiking in each sub-network is independent. Therefore, one can apply an adiabatic elimination of the fast variables and formulate a population density description where a detailed neuron model reduces to a stochastic point process [28,29] that provides an analytical approximation of the spiking dynamics and helps understanding the network simulation results in this section. This framework allows for a detailed study of a large and incoherent network without the need of numerical simulations (Materials and Methods).
The background input is modelled as a set of independent Poisson processes that drive both sub-networks (dashed arrows, Figure 2A). The stimulus dependent input is an increase in the intensity of the Poisson input into the afferent ensemble (solid arrow, Figure 2A). Before the stimulus became active at time t~0, a typical neuron showed an irregular spiking activity in both network stages ( Figure 2B-D). Whenever a sufficiently strong stimulus is applied all neurons in the afferent ensemble exhibited a transient response before the population mean firing rate converges back to a new level of steady-state ( Figure 2D,E). The population firing rate of neurons in the cortical network also exhibited a transient evoked response ( Figure 2D,E). However, in the balanced network individual neurons are heterogeneous in their responses ( Figure 2D), since the number of inputs from afferent and recurrent connectivity are random. In contrast to the rate model in the previous section where individual neurons were assumed to spike in a Poissonian manner, the adaptive neuron model in the neural network simulation operates far away for this assumption since the adaptation endows a long lasting memory effect on the spike times [28,29] that extends beyond the last spike. The time constant of this memory is determined by the time constant of adaptation (t s = 110 ms). This non-renewal statistics determines the shape of the transient component of the population response in Figure 2E. The spiking irregularity shows that the evoked state in the afferent ensemble is more regular than its background. The balanced network still exhibits a fairly irregular spiking and its average C 2 v stays approximately constant slightly above 1 ( Figure 2D). The population firing rate in the numerical simulations (solid line, Figure 2E) follow well the adaptive population density treatment (filled circles, Figure 2E).
To measure the effect of neuronal adaptation on the temporal sparseness, we again computed the number of spikes per neuron after the stimulus onset, S e (T). We compare our standard adaptive network with an adaptation time constant of t s = 110 ms (solid lines, Figure 2F) to a weakly-adaptive control network (t s = 30 ms; dashed lines, Figure 2F). Note, that the adaptation time constant in the weakly adaptive network is about equal to the membrane time constant and therefore plays a minor role for the network dynamics. It showed that both sub-networks generated sharp population level phasic response, which in the case of the cortical network evoked a single sharply timed spike within the first &10{20ms in a subset of neurons ( Figure 2B,F). In the control case, the response is non-sparse and response spikes are distributed throughout the stimulus period (dashed lines, Figure 2F). Overall, strong adaptation reduces the total number of stimulus-induced action potentials per neuron and concentrates their occurrence within an initial brief phasic response part following the fast change in the stimulus. This temporal sparseness is reflected in the cumulative number of spikes per neuron ( Figure 2F) which increases sharply. Thus, in accordance with the results of the rate- based model in the previous section, one can conclude that the sequence of adaptive processing accounts for the emergence of a temporally sparse stimulus representation in a cortical population.
We also estimated the fraction of neurons that significantly changed their number of spikes after stimulus onset. By construction, all cells in the afferent ensemble, both in adaptive and weakly-adaptive cases, produce a significant response. However, neurons in the cortical layer are far more selective. In the weakly-adaptive network 58% of all neurons responded significantly. In the adaptive network this number drops to 36%. This is calculated by comparing the count distribution across trials in 200 ms windows before and after the stimulus onset (Wilcoxon rank sum test, p-value = 0.01).
To reveal the effect of adaptation on the response variability, we employed the time-resolved Fano factor [20], F , which measures the spike-count variance divided by the mean spike count across 200 repeated simulations. Spikes were counted in a 50 ms time window and a sliding of 10 ms [18]. As before, we compared our standard adaptive network ( Figure 2G,I; t s = 110 ms) with the control network ( Figure 2H,J; t s = 30 ms). Since the Fano factor is known to be strongly dependent on the firing rate, we adjusted the stimulus level to the latter such that the averaged steady state firing rates in both networks were mean-matched [18]. The input Poisson spike trains (F~1) translated into slightly more regular spontaneous (tv0) activity in the afferent ensemble ( Figure 2G), as neuronal membrane filtering and refractoriness reduced the output variability. After the stimulus onset (t~0), due to the increase in the mean input rate, the average firing rate increased, however the variance of the number of events per trial did not increase proportionally. Therefore, we observed a reduction in the Fano factor ( Figure 2G). This phenomena is independent of the adaptation mechanism in the neuron model and a quantitatively similar reduction can be observed in the weakly adaptive afferent ensemble ( Figure 2H). A comparison between our standard adaptive and the control case reveals that the adaptive network is generally more regular in the background and in the evoked state ( Figure 2G,H). This is due to the previously known effect, where adaptation induces negative serial dependencies in the inter-spike intervals [29,43] and as a result reduces the Fano factor [29,44].
In the next stage of processing, the distribution of F across neurons during spontaneous activity is high due to the selfgenerated noise of the balanced circuits [23,24]. This closely follows a wide spread experimental finding where F w1 in the spontaneous activity of sensory and motor cortices [18][19][20] ( Figure 2G,H). This highly variable regime can be achieved in the balance network with strong recurrent couplings [23]. Whenever a sufficiently strong stimulus was applied, the internally generated fluctuations in the adaptive balanced network were transiently suppressed, and as a result the Fano factor dropped sharply ( Figure 2I). However, this reduction of the Fano factor is a temporary phenomenon and F converges back to slightly above the baseline variability ( Figure 2I). At the same time, the evoked steady-state firing remained in the irregular and asynchronous state ( Figure 2D). Indeed, this transient effect corresponds to a temporally mismatch in the balanced input conditions to the cortical neurons since the self-inhibitory and slower adaption effect prevents a rapid adjustment to the new input regime. This can be observed in the time course of variability suppression that closely reflects the time constant of adaptation ( Figure 2I). However, with stronger adaptive feedforward input we can prevent the return of the Fano factor to the base line, this phenomena is due to the regularizing effect of adaptation in the afferent ensemble. In this scenario, the afferent ensemble structured the input to the cortical ensemble, contributing to the magnitude of the observed variability reduction. Indeed, whenever the excitatory feedforward strength is considerably strong, relative to the recurrent connections, the cortical network moves away from the balanced condition. Thus, such strong input resets the internal spiking dynamics within the cortical network and as a result it regulates the spiking variability [45]. This mechanism evidently can be used to prevent the recovery of the high variability. However, we deliberately use a weak stimulation to focus on the transit suppression of cortical variability that is mediated by the slow self-regulation due to adaptation. For instance, under the control condition where a pure Poisson input (with similar synaptic strength) is provided to the cortical balanced network, the reduction in F is reduced but the time scale of recovery remains unaltered (crosses in Figure 2I). We contrast this adaptive behavior with the variability dynamics in the weakly adaptive balanced network ( Figure 2J). In this case there is no reduction in F , because for a short adaptation time constant the convergence to the balanced state is very rapid [24]. The small increase in the input noise strength leads to an increase of the self-generated randomness of the balanced network [17,23].
In the above comparison, we adjust the stimulus strength to achieve the same steady-state firing rate (tonic response) in the afferent and cortical ensemble. In a next step we studied the effect of adaptation on the detectability of a weak and transient peripheral signal, which might be impaired by the self-generated noise in the cortical network. To this end we employed the population density approach (Material and Methods) to study the mean and variability of the cortical network responses to a wide range of signal strengths. We change the stimulation protocol to a brief signal with a duration of 50ms over the spontaneous background. The stimulation magnitude is adjusted to elicit the same onset firing rate in the afferent ensemble network in both adaptive and weakly adaptive cases. By modification of the feedforward coupling between afferent and cortical network relative to the intracortical recurrent coupling we study the circuit responses ( Figure 3A). Evidently, the strength of the feed-forward coupling to the cortical ensemble modifies its spontaneous background, and therefore also the total adaptation level. The adaptive network proves more sensitive to brief and weak stimuli. It significantly magnifies the mean stimulus response in the adaptive network relative to the background. Even for a considerably weak stimulus the relative amplitude of the response to background firing is pronounced ( Figure 3A). This result resembles the amplification of a transient in the sequence of adaptive networks as it is observed in the previous section ( Figure 1A). How reliable are the responses across trials? To answer this question we calculated the Fano factor (Material and Methods) for the cortical ensemble response in the above scenario. This calculation indicates that the response variability in the adaptive network is significantly lower than in the weakly adaptive network over a large range of the feed-forward coupling strength ( Figure 3B). Interestingly, our results of the population density treatment quantitatively follow the former prediction based on a network simulation that in a balanced network without adaptation the variability initially increases with signal strength (dashed line in Figure 3B and Table 1 in [23]) and only after a critical level of the feed-forward strengths the recurrent noise is suppressed due to the stronger influence of excitatory inputs.
Adaptive networks generate sparse and reliable responses in the insect olfactory system As a case study to demonstrate how the sequential effect of the adaptation shape responses, we investigated its contribution to the emergence of the reliable and sparse temporal code in the insects olfactory system, which is analogous to the mammalian olfactory system. We simulated a reduced generic model of olfactory processing in insects using the phenomenologically adaptive neuron model [28]. The model network consisted of an input layer with 1,480 olfactory sensory neurons (OSNs), which project to the next layer representing the antennal lobe circuit with 24 projection neurons (PNs) and 96 inhibitory local inter-neurons (LNs) that form a local feed-forward inhibitory micro-circuit with the PNs. The third layer holds 1,000 Kenyon cells (KCs) receiving divergent-convergent input from PNs. The relative numbers for all neurons approximate the anatomical ratios found in the olfactory pathway of the honeybee [46] ( Figure 4A). We introduced heterogeneity among neurons by randomizing their synaptic time constants and the connectivity probabilities are chosen according to anatomical studies. Synaptic weights were adjusted to achieve spontaneous firing statistics that match the observed physiological regimes. The SFA parameters were identical throughout the network with t s~1 10ms (see Materials and Methods for details). Experimentally, the cellular mechanisms for SFA exist for neurons at all three network layers [47][48][49][50][51][52][53]. Notably, strong SFA mediating currents have been identified in the KCs of Periplaneta americana [52].
Using this model, we sought to understand how adaptation contributes to temporally sparse odor representations in the KC layer in a small sized network and under highly fluctuating input conditions. We simulated the input to each OSN by an independent Poisson process, which is thought to be reminiscent of the transduction process at the olfactory receptor level [53]. Stimulus activation was modelled by a step increase in the Poisson intensity with uniformly jittered onset across the OSN population (Materials and Methods). Following a transient onset response the OSNs adapted their firing to a new steady-state ( Figure 4B,C). The pronounced effect of adaptation becomes apparent when the adaptive population response is compared to the OSN responses in the control network without any adaptation (t s = 0; dashed line, Figure 4C). In the next layer, the PN population activity is reflected in a dominant phasic-tonic response profile (green line, Figure 4D), which closely matches the experimental observation [54]. This is due to the self-inhibitory effect of the SFA mechanism, and to the feedforward inhibition received from the LNs (magenta line, Figure 4D). Consequently, the KCs in the third layer produced only very few action potentials following the response onset with an almost silent background activity (red line, Figure 4E,G). The average number of emitted KC response spikes per neuron, S e , is small in the adaptive network (average v2) whereas KCs continue spiking throughout stimulus presentation in the non-adaptive network ( Figure 4G). This finding closely resembles experimental findings of temporal sparseness of KC responses in different insect species [31][32][33] and quantitatively matches the KC response statistics provided by Ito and colleagues [34]. The simulation results obtained here confirm the mathematically derived results in the first results section ( Figure 1B) and show that neuronal adaption can cause a temporal sparse representation even in a fairly small and highly structured layered network where the mathematical assumptions of infinite network size and fundamentally incoherent activity are not fulfilled (Materials and Methods). We further investigated the effect of adaptation on the fraction of responding neurons by counting the number of KCs that emit spikes during stimulation. In the adaptive circuit and in presence of local inhibition only 9% of KCs produce responses (23% in the adaptive network when inhibition is turned off). In contrast, in the non-adaptive network with local inhibition 60% of KCs responded. The low fraction of responding neurons in the adaptive network quantitatively match the experimental findings in the moth [34] and the fruit fly [55].
To test the effect of inhibition in the LN-PN micro-circuitry within the antennal lobe layer on the emergence of temporal sparseness in the KC layer, we deactivated all LN-PN feedforward connections and kept all other parameters fix. We found a profound increase in the amplitude of the KC population response, both in the adaptive (red line, Figure 4F) and the non-adaptive network (dashed red line, Figure 4F). This increase in response amplitude is carried by an increase in the number of responding KCs due to the increased excitatory input from the PNs, implying a strong reduction in the KC population sparseness. Importantly, removing local inhibition did not alter the temporal profile of the KC population response in the adaptive network (cf. red lines in Figure 4E,F), and thus temporal sparseness was independent of inhibition in our network model.
How reliable is the sparse spike response across trials in a single KC? To answer this question, we again measured the robustness of the stimulus representation by estimating the Fano factor across 200 simulation trials ( Figure 4H). The network with adaptive neurons and inhibitory micro-circuitry exhibited a low Fano factor (median &0:3) and a narrow distribution across all neurons. This follows the experimental finding that the few spikes emitted by KCs are highly reliably [34] (network 1, Figure 4H). Turning off the inhibitory micro-circuitry did not significantly change the response reliability (Wilcoxon rank sum test, p-value = 0.01; network 2, Figure 4H). However, both networks that lacked adaptation exhibited a significantly higher variability with a median Fano factor close to one (Wilcoxon rank sum test, pvalue = 0.01; networks 3 and 4, Figure 4H), independent of the presence or absence of inhibition micro-circuits.
To explore whether neuronal adaptation could contribute to temporal sparseness in the biological network, we performed a set of Calcium imaging experiments, monitoring Calcium responses in the KC population of the honeybee mushroom body [33] (Materials and Methods). Our computational model ( Figure 4) predicted that blocking of the inhibitory microcircuit would increase the population response amplitude but should not alter the temporal dynamics of the KC population response which is independent of the stimulus duration. In a set of experiments, we tested this hypothesis by comparison of the KCs' evoked activity in the presence and absence of GABAergic inhibition (Materials and Methods). First, we analyzed the normalized Calcium response signal within the mushroom body lip region in response to a 3 s, 2 s, 1 s and 0.5 s odor stimulus ( Figure 5A). We observed the same brief phasic response following stimulus onset in all four cases with a characteristic slope of Calcium response decay that has been reported previously to account for a temporally sparse spiking response [33]. These responses, unlike those of PNs in the previous processing stage of the insect olfactory system [56,57], are independent of the stimulation duration ( Figure 5A). Bath application of the GABA A antagonist picrotoxin (PTX) did not change the time course of the Calcium response dynamics ( Figure 5B-D). The effectiveness of the drug was verified by the increased population response amplitude in initial phase ( Figure 5C). Next, we tested the GABA B antagonist hydrochloride (CGP) using the same protocol and again found an increase in the response magnitude but no alteration of the response dynamics ( Figure 5E,F). This suggests the absence of inhibition does not change the temporal scale of KCs responses in line with the model prediction.

Discussion
We propose that a simple neuron-intrinsic mechanism of spiketriggered adaptation can account for a reliable and temporally sparse sensory stimulus representation across stages of sensory processing. The emergence of a sparse representation has been demonstrated in various sensory areas, for example in visual [58], auditory [59], somatosensory [60], and olfactory [61] cortices, and thus manifests a principle of sensory computation across sensory modalities and independent of the natural stimulus kinetic. Our results show that adaptation allows to reliably represent a stimulus with a temporally restricted response to stimulus onset and thus more stimuli can be represented in time which is the basis for a temporally sparse representations of a dynamically changing stimulus environment.
At the single neuron level, SFA is known to induce the functional property of a fractional differentiation with respect to the temporal profile of the input and thus offers the possibility of tuning the neuron's response properties to the relevant stimulus time-scales at the cellular level [2][3][4][62][63][64][65]. Our results indicate further that sensory processing in a feedforeward network with adaptive neurons focuses on the temporal changes of the sensory input in a precise and temporally sparse manner ( Figure 1B; Figure 2E and Figure 4E) and at the same time the constancy of the stimulus is memorized in the cellular level of adaptation [37] ( Figure 1D). The constancy of the environment is an important factor of state-dependent computations [66] that evidently should be tracked by the network. Such context-dependent modulations set the background and have been observed in different sensory systems where responses are strongly influenced by efferent contextual input [67][68][69]. In this paper, we show that information about the context of a given stimuli maybe stored in the adaptation level across processing stages while at the same time the network remains sensitive to changes. Thus, sequential adaptive populations adjust the circuit transfer function in a self-organizing manner to avoid response attenuation to secondary stimuli. These results add a further possibility of network level interactions to the previous suggestions that SFA optimizes the context depended responses and resolves ambiguity in the neuronal code [8,70] at the single neuron level. This allows a sensory system to detect extremely small changes in stimulus over a large background by means of an adaptive response without contextual information loss [39]. One prominent example is primate vision where, in the absence of the self-generated dynamics of retina input due to microsaccades, observers become functionally blind to stationary objects during fixations [71].
A sparse temporal representation of stimulus permits very few spikes to transmit high quantities of information about a behaviorally significant stimulus [72]. However, it has been repeatedly questioned whether a few informative spikes can survive in the cortical network, which is highly sensitive to small perturbations [16,73]. Our results show that a biologically realistic cellular mechanism implemented at successive network stages can transform a dense and highly variable Poisson input at the periphery into a temporally sparse and highly reliable ensemble representation in the cortical network. Therefore, it facilitates a transition from a rate code to a temporal code as required for the concerted spiking of cortical cell assemblies [74] (Figure 2D). These results reflect previous theoretical evidence that SFA has an extensive synchronizing-desynchronizing effect on population responses in a feedback coupled network [75,76].
A balance between excitation and inhibition leads to strong temporal fluctuations and produce spike trains with high variability in cortex [16,[23][24][25]77]. However, the adaption level adjusts with a dynamics that is slow compared to the dynamics of excitatory and inhibitory synaptic inputs. This circumstance allows for a transient mismatch of the balanced state in the cortical network and thus leads to a transient reduction of the selfgenerated (recurrent) noise ( Figure 2I). This, in turn, explains why the temporally sparse representation can be highly reliable, following the experimental observations [21,22]. Moreover, a recent and highly relevant in vivo data set hints toward our theoretical prediction, where adaptation may alter the balance between excitation and inhibition and increase the sensitivity of cortical neurons to sensory stimulation [78]. Here, our main result exploited the transient role of adaptation mechanisms on the cortical variability suppression, after which the variability recovers to the unstimulated values, even though the network remains stimulated ( Figure 2I). One can achieve a longer time scale of variability suppression by an increase in the effect of the afferent strength (as a network mechanism), due to the reduction of the input irregularity in the evoked state. This proposal can be supported by the experimental evidence that thalamic inputs strongly drive neurons in cortex [79] and fits the previous theoretical suggestion by [45]. Noteworthy, in our model the irregularity of inter-spike intervals, measured by C 2 v , in the balanced network does not change significantly in different conditions, which matches the experimentally reported evidence [80]. The recent theoretical studies [26,27] show that the slow time scales variability suppression can be also achieved within a clustered topology in the balanced network [26] or likewise in an attractor-based networks of cortical dynamics [27]. In these approaches, the reduced variability can be attributed to an increased regularity of the spike trains. This hints that further research to understand the role of interactions between the network and cellular mechanisms in the cortical variability and other network statistics are certainly needed. Additionally, the link between the temporal sparseness achieved here by cascaded network of adaptive neurons with spatial sparseness of responses [81,82] requires more elaborated research. The insect olfactory system is experimentally well investigated and exemplifies a pronounced sparse temporal coding scheme at the level of the mushroom body KCs. The olfactory system is analog in invertebrates and vertebrates and the sparse stimulus representation is likewise observed in the pyramidal cells of the piriform cortex [61], and the rapid responses in the mitral cells in the olfactory bulb [83] compare to those of projection neurons in antennal lobe [54,84]. Our adaptive network model, designed in coarse analogy to the insect olfactory system, produced increasingly phasic population responses as the stimulus-driven activity propagated through the network. Our model results closely match the repeated experimental observation of temporally sparse and reliable KC responses in extracellular recordings from the locust [31], fruit fly [85] and manduca [32,34], and in Calcium imaging in the honeybee [33]. Although Calcium responses are slow, it has been suggested that they closely correspond to the population activity dynamics [86]. In our experiments we could show that systemic blocking of GABAergic transmission did not affect the temporal sparseness of the KC population response in the honeybee ( Figure 5) signified by the transient Calcium response [33]. Therefore, the stable temporal activity in the mushroom body qualitatively matches with our theoretical predication of population rate dynamics ( Figure 1). This result might seem to contradict former studies that stressed the role of inhibitory feedforward [87] or feedback inhibition [88,89] for the emergence of KC sparseness. However, the suggested inhibitory mechanisms and the sequential effect in the adaptive network proposed here are not mutually exclusive and may act in concert to establish and maintain a temporal and spatial sparse code in a rich and dynamic natural olfactory scene. In this paper, we deliberately focus on the temporal aspect of the responses, since it seems that spatial sparseness is mediated by connectivity schemes [90][91][92].
The adaptive network model manifests a low trial-to-trial variability of the sparse KC responses that typically consist of only 1-2 spikes. In consequence, a sparsely activated KC ensemble is able to robustly encode stimulus information. The low variability at the single cell level ( Figure 4H) carries over to a low variability of the population response [29,30]. This benefits downstream processing in the mushroom body output neurons that integrate converging input from many KCs [46], and which were shown to reliably encode odor-reward associations in the honeybee [93].
Next to the cellular mechanism of adaptation studied here, short-term synaptic plasticity may produce similar effects. The activity-dependent nature of short term depression (STD) produc-es correlated presynaptic input spike trains [94]. Hence, it facilitates weak signal detection [94] similar to adaptation [95]. Moreover, STD can also generate a sharp transient in the stimulus response [96,97] that can propagate to higher layers of the network. Therefore it is plausible to utilize short-term synaptic plasticity to achieve similar results to the ones obtained here with SFA. However, STD may have some drawbacks in comparison to adaptation, namely a low signal-to-noise ratio, and a low-pass filtering of input that is more sensitive to high frequency synaptic noise [62,68]. Evidently, STD takes effect at the single synapse while SFA acts on a neuron's output. The combination of both mechanisms that are encountered side-by-side in cortical circuits [99,100] may provide a powerful means for efficient coding [98].
Our results here are of general importance for sensory coding theories. A mechanism of self-inhibition at the cellular level can facilitate a temporally sparse ensemble code but does not require well adjusted interplay between excitatory and inhibitory circuitry at the network level. This network effect is robust due to the distributed nature of the underlying mechanism, which acts independently in each single neuron. The regularizing effect of self-inhibition increases the signal-to-noise ratio not only of single neuron responses but also of the neuronal population activity [29,30,37] that is post-synaptically integrated in downstream neurons.

Rate model of a generic feedforward adaptive network
To address analytically the sequential effect of adaptation in a feedforward network, we consider a model in which populations are described by their firing rates. Although firing rate models typically provide a fairly accurate description of network behavior when the neurons are firing asynchronously [101], they do not capture all features of realistic networks. Therefore, we verify all of our predictions with a population density formalism [29] as well as a large-scale simulation of realistic spiking neurons. To determine the mean activity dynamics of a consecutive populations, we employed an standard mean firing rate model of population i as where f is the transfer input-output function, t s is the adaptation time-scale, g is the coupling factor between two populations and a is the adaptive negative feedback for the population i with q s strength and s i is the standard deviation of the input. In our rate model analysis, we use the transfer function of the leaky-integrate and fire neuron that can be written as  Figure 6A). It is important to note that whenever the conditions for stability are satisfied, the fix point is reached via a focus attractor (Figure 6 C,D) since the Jacobin of this system (under the physiological condition of t s q s vg) always has a complex eigenvalue with a negative real part. It is also known that the r r i is a linear function with respect to its input, given a sufficiently slow t s or strong adaptation q s and a non-linear shape of f [35,36,102]. It can also be shown that whenever the adaptation is ineffective (q s t s~0 ) we have . This derivative scales with m i =(h{V r ) ( Figure 6B). Now, we can plug back the adaptation into the steady state solution, which has a magnitude of r r i q s t s . In Figure 6 B, we numerically determine the condition for L r r i L r r i{1 v1, that it reads g 4 vq s t s , given the parameters C, h, t m , V r and s as they are stated in the caption. An increase in the population rate D r r i{1 leads to a reduced increase D r r i in the next population, and therefore the adapted level of responses satisfy D r r i{1 wD r r i . For realistic adaptation values this mapping closely follows the result of a previous study where it was shown that the effect of increasing the cells input conductance on its f-I curve is mainly subtractive [102]. Note that for very weak adaptation the steady-state is not affected considerably.
The magnitude of the transient response firing rate for an adaptive population lies between the adapted steady-state rate and the response rate without adaptation. Given the level of new input it can be calculated analytically [2]. Hence, the slow dynamics of adaptation and fast response f-I curve reflect two states of operation where the onset response very closely follows the properties of the non-adapted response curve and the adapted steady state produces a subtractive input-output relationship. Noteworthy, the assumption that all populations have a same background firing rate is not a necessary condition. One can achieve the same result by using heterogeneous couplings g i or stimulus independent private input m i that may induce more realistic variations in background rates as observed in different stages of sensory processing. The crucial point of the inherited dynamics due to adaption is the fundamental nonlinearity that (1) the transient response amplitude is hardly affected by adaptation, (2) the adapted steady-state is fall apart from it and can become subtractive.

Population density approach to the adaptive neuronal ensemble
In Muller et al. [28] it is shown that by an adiabatic elimination of fast variables a detailed neuron model including voltage dynamics, conductance-based synapses, and spike-induced adaptation, in the incoherent state (Asynchronous and Irregular state) reduces to a stochastic point process. Thus, we define an orderly point process with a hazard function argument with state variable x as where N½t,tzDt) is the number of events in Dt. We assume the dynamic of the adaptation variable is where t k is the time of kth spike in the ensemble. Thus, the state variable distribution at time t in the ensemble is governed by a master equation of the form We solve Eq. 7 with the help of the transformations t s~g (x) :~{t log(x=q) and y(t s )~g(g {1 (t s zq)) numerically [28]. The master equation here belongs to a non-renewal process [28,29] and its renewal correspondence can be seen in [28]. It turns out that indeed h x is the input-out transfer function of neurons in the network where its instantaneous parameters are give by the input statistics [28]. For instance, the transfer function of a conductance based leaky integrate and fire neuron can be written as where, t r and t Ã are refectory period and an input-dependent effective time constants, respectively. The m 0 and s 0 appearing in W CB are the average and variance of the free (i.e., spike-less) membrane voltage distribution [35] and C is the membrane capacitance. Here, we used the mean-field formalism developed by [23] and [103] to approximately determine the averaged input within a standard balanced network, as the parameters of the hazard function W CB suggested by [35], which uses the calculated average firing rate of inhibitory n I and excitatory neurons n E in the randomly connected network. The analytical results in this paper assumed the standard W CB as a from of the hazard function h x and the value of t Ã is estimated form simulations of the detailed neuron model with an step like input increase. Here, it is important to note that conductance-based model approximately follows the current based neuron model with a colored noise, where t Ã is shorter than the membrane time constant of the neuron, which now depends on the total conductance [103,104]. Therefore, given the h xDt estimate, we can approximate the numerical solution of the master equation (eq. 7) by applying the exponential Euler method for the death term, and reinserting the lost probability as it is fully described in [28]. The functional form of the solution in a compact form can be written as where t 0 s is the initial condition state of the system and k t is a constant defined by Ð Pr(t s Dt)dt s~1 [28,29]. Similarly, one can derive the distribution of t s just after the event, [28]. Then, the relationship between t s and the ordinary ISI distribution can be written as where V(t s zDDt)~e { Ð D ts h ts (tszuDt)du . Now the n th moment m n of the distribution and its coefficient of variation C v can be numerically determined. Note that the framework here is closely connected to the spike response model, also known as the generalized linear neuron model [30]. Alternatively, the same ISI distribution can be also derived form the discretization of the master equation as it is demonstrated in [37]. It can be shown that the firing rate and the consistency equation of the ensemble is Now to calculate the counting statistics, we applied the techniques are introduced by Farkhooi et al [29], and defined a joint probability density as where an n th event occurs at time t n wt and the state variable is t n x . We can write nz1 th event time and state of adaptation joint density recursively, jt n x ,t)r n (t n ,t n x jt 0 x ,t)dt n dt n x : To simplify the integral equations, we use Bra-Ket notation following a suggestion by [105], defined as PDf T~ð p(s,a'Da)f (a)da ð14Þ and SPDg~ð p(s,a'Da)g(a')da': Thereafter, we derive the Laplace transform (Lff (t)g~f f (s)) of the joint density in the eq. 13 bỹ r r nz1 (s,t nz1 x Dt 0 s ,t)~r r n (s,t n x Dt 0 x ,t)Dr r(s,t nz1 x Dt n x ,t)T ð16Þ where r 1 (t 1 ,t 1 s Dt 0 x ,t)~r(t 1 ,t 1 x Dt 0 x ,t). Next, we define the operator P n (sDt),r r n (sDt)~S1DP n (sDt)DPr(t s ,t)T~S1DP n (sDt)DPr(t x ,t)T: ð17Þ Now, by employingP P(1,sDt)~1=(m 1 s 2 )½r r 2 (sDt){2r r 1 (sDt)z1 as in [29], we derivẽ P P(n,sDt)~1=(m 1 s 2 )½r r nz1 (sDt){2r r n (sDt)zr r n{1 (sDt), ð18Þ whereP P(n,sDt) is the Laplace transform of the probability density of observing n events in a given time window. Now we derivẽ A A sDt~X kr r k (sDt)~S1DP(sDt)=(I{P(sDt))DPr(sDt)T ð19Þ where I is the identity operator. This equation represents the Laplace transform of the auto-correlation function. Using autocorrelation functionÃ A sDt , we can calculate the Fano factor that provides an index for the quantification of the count variability. It is defined as J T~s

2
T =m T , where s 2 T and m T are the variance and the mean of the number of events in a certain time window T. It follows from the additive property of the expectation that m T~Ð T 0 r(u)du and in the case of a constant firing rate, it is simply m T~req T. To calculate the second moment ofP P(n,s), we requireÃ A s~P kr r k (s) in eq. 19. Thus, the Fano factor is J J sDt~1 =m 1 s 2 {(1z2Ã A sDt ) and the inverse Laplace transform is where A(uDt)~L {1 ½Ã A sDt . In [29], we demonstrate in detail that the asymptotic property of F~lim T?? J T at equilibrium can be derived as where j k is the linear correlation coefficient between two k lagged intervals. Provided the limit exits, we find the familiar relationship of F~C 2 v ½1z2 P ? k~1 j k in the steady-state.

Computational model of insect olfactory network
Our model neuron is a general conductance-based integrateand-fire neuron with spike-frequency adaptation as it is proposed in Muller et al. [28]. The model phenomenologically captures a wide array of biophysical spike-frequency mechanisms such as Mtype current, afterhyperpolarization (AHP-current) and even slow recovery from inactivation of the fast sodium current [28]. The model neuron is also known to perform high-pass filtering of the input frequencies following the universal model of adaptation [2]. Neuron parameters used follow the Table 3 in Muller et al. [28]. The conductance model used for the static synapses between the neurons is alpha-shaped with gamma distributed time constants from C(2:5,4) and C(5,4) for excitatory and inhibitory synapses, respectively. All simulations were performed using the NEST simulator [106] version 2.0beta and the Pynest interface.
The network connectivity is straight forward: each PN and LN receives excitatory connections from 20% randomly chosen OSNs [57,107]. Additionally, every PN receives input from 50% randomly chosen inhibitory LNs [57,107]. In our model the PNs do not excite one another and each PN output diverges to 50% randomly chosen KCs [33,87,90].

Experimental methods
Experiments were performed following the methods published in Szyszka et al [33]. In summary, foraging honeybees (Apis mellifera) were caught at the entrance of the hive, immobilized by chilling on ice, and fixed in a plexiglas chamber before the head capsule was opened for dye injection. We retrogradely stained clawed Kenyon cells (KC) of the median calyx, using the calcium sensor FURA-2 dextran (Molecular Probes, Eugene, USA) with a dye loaded glass electrode, which was pricked into KC axons projecting to the ventral median part of the a-lobe [33]. After dye injection the head capsule was closed, bees feed and kept in a dark humid chamber for several hours.
The processing of imaging data was performed with custom written routines in IDL (RSI, Boulder, CO, USA). In summary, changes in the calcium concentration were measured as absolute changes of fluorescence: a ratio was calculated from the light intensities measured at 340 nm and 380 nm illumination and the background fluorescence before odor onset was subtracted leading to DF with F = F340/F380. Odor stimulation was preformed under a 206 objective of the microscope, the naturally occurring plant odor octanol (Sigma Aldrich, Germany), diluted 1:100 in paraffine oil (FLUKA, Buchs, Switzerland), was delivered to both antennae of the bee using a computer controlled, custom made olfactometer. To this, odor loaded air was injected into a permanent airstream resulting in a further 1:10 dilution. Stimulus duration was 3 seconds if not mentioned otherwise. The air was permanently exhausted.
For GABA blockage, a solution of 150 ml GABA receptor antagonist dissolved in ringer for final concentration (10 25 M picrotoxin (PTX, Sigma Aldrich, Germany) or 5610 24 M CGP54626 (CGP, Tocris Bioscience, USA)) was bath applied to the brain after pre-treatment measurements. Measurements started 10 min after drug application. The calcium signals are analyzed in Matlab (The MathWorks Inc., Natick, USA). The normalization of the responses are performed per animal and the plotted traces are the averaged values across subjects.