Presynaptic expression modelled as changes in stochastic release

Here, changes in the presynaptic weight P were explored in terms of their impact on stochastic release at connections onto a single-compartment point neuron (see Methods). In other words, the effects of changes in P on short-term dynamics are not reported here, as only the vesicle release probability p was affected by LTP (we thus set p = P); we revisit that aspect in the next section.

With the latency paradigm, STDP leads early inputs to potentiate and late inputs to depress. In this paradigm, a volley of stimuli arrives at the postsynaptic neuron with varying delays (Fig 2A), plasticity therefore resulted in the shortening of the time to respond—the latency—of the postsynaptic neuron, as well as a temporal sharpening of the response, with fewer spikes and shorter inter-spike intervals [24]. The average latency reduction (Fig 3A and 3B), as well as the overall distribution of synaptic weights, decrease of postsynaptic activity duration and increase of postsynaptic firing frequency (Fig 3C, 3D and 3E) did not differ appreciably with the locus of plasticity. In comparison to the purely postsynaptic case, simulations with presynaptic plasticity presented a smaller variance of the latency shift across simulations (Fig 3B, inset). Potentiation also developed faster with presynaptic expression (Fig 3F). This can be framed as a consequence of potentiation requiring glutamate release [48], so that in a more reliable synapse, with a high p value, there is a greater propensity for potentiation. Conversely, depression was slower with presynaptically expressed plasticity, again because lowered probability of release effectively also led to less plasticity (Fig 3F).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. With stochastic release, presynaptic plasticity typically promoted faster learning.

(A-F) Simulations in the latency paradigm (see Fig 2A). (A) Sample postsynaptic traces from trials before (grey) and after (black) plasticity. Initial response latency is marked by green dashed line. (B) STDP shortened the spike latency, as previously shown [24]. All graphs are colour-coded: only presynaptic plasticity (red), only postsynaptic plasticity (blue), or both pre- and postsynaptic plasticity (black) are implemented, with lines denoting the average of 10 independent realizations and the shading the standard error of the mean (SEM). Inset: presynaptic plasticity was faster than postsynaptic plasticity alone (t-test, p-value = 0.008). (C) Synaptic weight distribution after 150 trials, normalized and sorted relative to the fixed presynaptic delay. (D, E) Postsynaptic response duration (i.e, the interval between first and last spike in each trial) and the burst frequency did not differ for different expression loci. (F) Time evolution of average synaptic weight among early and late presynaptic inputs (i.e., input cells that spiked in the first or the second half of the stimulus) show how post-only expression (blue) was slower for the early group. Inset shows linear slope (x10⁻³ /trial) across the first 100 trials. (G-I) Simulations in the correlation paradigm (see Fig 2B). (G) Potentiation and depression of the average synaptic weight among correlated inputs was faster in the presynaptic case. Inset shows linear slope (x10⁻⁴ /s) across the first 50 seconds. (H) However, for highly correlated inputs (c>0.9), learning was faster with postsynaptic expression. This indicated that which form of plasticity led to faster learning depended on the details of the input firing pattern. Inset shows linear slope (x10⁻⁴ /s) across the first 50 seconds. (I) The map shows the difference of P and q at the end of simulations. All inputs were correlated, but half expressed plasticity presynaptically, and the other postsynaptically. We found that across the explored parameter space, the half with presynaptic expression (red) typically won out, although the half with postsynaptic expression (blue) was victorious for a smaller parameter space where input firing frequency was low and correlations quite high.

https://doi.org/10.1371/journal.pcbi.1009409.g003

Next, we explored the correlation paradigm, in which plasticity selectively potentiates correlated inputs (Fig 2B) [43]. Here, all plasticity implementations detected the input correlations. However, presynaptically expressed plasticity generally promoted faster learning, e.g. synaptic weights evolved more rapidly (Fig 3G), similar to what we found above for the latency paradigm. However, there were exceptions to this general observation—for strong correlations, postsynaptic plasticity was faster to potentiate for correlated and faster to depress for uncorrelated inputs at certain input frequencies (Fig 3H). Which form of plasticity led to faster learning thus depended on the details of the firing statistics.

To explore this exception in more detail, we ran simulations where all of inputs were correlated, but half of them expressed plasticity only presynaptically, and the other half only postsynaptically. We imposed a limit to the total sum of weights so these two input populations competed, so that one potentiated at the expense of the other, which depressed. With this approach, we systematically explored the correlation-frequency space in distinct simulations where all inputs had a specified correlation and firing rate. We found that postsynaptic expression won for very highly correlated inputs for sufficiently low input frequencies (Fig 3I).

Presynaptic expression modelled as changes in short-term plasticity

We next explored the effects of altering short-term dynamics (see Methods). This adds another aspect of presynaptically expressed plasticity, since short-term plasticity takes into account the history of presynaptic activity. In this scenario, presynaptic changes redistribute synaptic resources used over a limited time period, modulating vesicle release probability p according to recent activity [14, 39]. Since p is varying on a short timescale, the presynaptic weight in this case corresponds to the baseline value P^B around which p fluctuates (so P^B = P), so that p = p(P^B, t). Even if the amplitude of an individual EPSP were affected equally by pre- and by postsynaptically expressed plasticity, the total input from a burst would still differ dramatically depending on the site of expression (Fig 1B).

In the simulations with the timed input configuration, results differed considerably depending on the specific locus of plasticity in the latency configuration. Postsynaptic expression alone provided the largest latency reduction, and also achieved it faster than the other plasticity implementations (Fig 4A and 4B). With presynaptic expression, the change in latency was smaller compared to the mixed setting with both pre- and postsynaptic expression, for which results may vary between extremes according to the ratio of pre- and postsynaptic expression. Effects of postsynaptic plasticity over response duration and intraburst frequency (Fig 4C and 4D) were also more marked, as expected from a higher integrated input (Fig 1B). The simulations with both sides changing appeared closer to either the presynaptic case (duration, Fig 4C) or the postsynaptic case (frequency, Fig 4D). Here, changes in P had a relatively greater influence on response duration, while changes in q had greater impact on the response frequency. Nevertheless, synaptic efficacy was still potentiated faster and depressed slower in the presynaptic case (Fig 4E). This was similar to the above stochastic release implementation of presynaptically expressed plasticity, although it was less pronounced. This means that even if the rate of learning was effectively faster, presynaptic expression affected latency less rapidly than postsynaptic expression did (Fig 4F).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Altering short-term plasticity was less efficient at reducing postsynaptic latency.

(A-E) Simulations in the latency paradigm (see Fig 2A) are colour-coded: red denotes presynaptic plasticity alone, blue postsynaptic plasticity alone, and black combined pre- and postsynaptic plasticity. Lines denote the average across 10 realizations, and the shading the SEM. (A) Example traces of postsynaptic activity before (grey) and after plasticity (coloured). Initial response latency is illustrated by the vertical dashed line. (B) Latency reduction was both faster and more marked for postsynaptic (blue) than for presynaptic (red) or combined (black) plasticity. Inset: The slope of latency reduction was steeper when postsynaptic expression was involved (t-tests: between pre- and postsynaptic expression, p-value < 10⁻⁶; between presynaptic expression and both, p-value = 0.0008, between postsynaptic expression and both p-value = 0.003) (C) Combined and presynaptic plasticity reduced response duration more than with postsynaptic expression alone. (D) Burst frequency was similarly increased with all three forms of plasticity, although rate change was faster with postsynaptic plasticity. (E) Time course of average synaptic weights for early (left) and late (right) inputs. Inset shows linear slope (x10⁻³ /trial) across the first 50 trials. (F, G) Simulations in the correlation paradigm (see Fig 2B) (F) Time course of average synaptic weights for correlated (left, “corr”) and uncorrelated (right, “unc”) inputs were largely indistinguishable across plasticity loci. Inset shows linear slope (x10⁻⁵ /s) across the first 100 seconds. (G) As with Fig 3I, colour represents the difference between P and q. This map of competition between input populations with pre- or postsynaptically expressed plasticity indicated a less marked differentiation except for very high (0.9) or very low (0.1) correlation coefficients.

https://doi.org/10.1371/journal.pcbi.1009409.g004

On the other hand, under modulation of short-term plasticity, plasticity rates in the correlated inputs paradigm evolved differently compared to the above stochastic release implementation (Fig 4G). In the simulations where long-term plasticity affected short-term plasticity, the rate of change was slightly faster with postsynaptic than with presynaptic plasticity.

These findings show that the outcome in the latency paradigm was more affected by the locus of expression of plasticity than in the correlation paradigm. In conclusion, computational advantages could be tailored to optimally achieve a specific functional outcome by recruiting pre- or postsynaptic plasticity differentially.

Comparisons with a biologically tuned model

The above minimalist toy models had the advantage that they provided full control of several key parameters. However, the relevance of the findings for the intact brain was unclear. To address this shortcoming, we explored the biological plausibility in a model [17] (see Methods) that was fitted to long-term synaptic plasticity data obtained from connections between rodent visual cortex layer-5 pyramidal neurons [39, 49, 50]. We could thus to some extent verify whether the results obtained with the minimal models hold in a more complex, data-driven context. We want to clarify upfront that in this model, LTP is expressed both pre- and postsynaptically, whereas LTD is solely presynaptically expressed. This asymmetry may seem odd, but it is derived from experimental data, and we have previously found that this arrangement provides certain computational advantages [17].

We first explored the latency paradigm (Fig 2A). To avoid disrupting the parameter tuning, instead of normalising the total synaptic change on each side, we kept the data-derived ratios and blocked either pre- or postsynaptic changes. Even so, we found that both pre- and postsynaptic plasticity components independently led to the shortening of postsynaptic latency (Fig 5A–5C). As with the earlier toy models that were not biologically tuned, postsynaptic changes appeared to affect spike latency more. Thus, looking at the case with both pre- and postsynaptic plasticity, postsynaptic potentiation essentially helped to reduce the latency compared to presynaptic plasticity alone, but pre- and postsynaptic plasticity together were slower than postsynaptic plasticity alone (Fig 5B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. A biologically tuned model verified key findings obtained with minimalist models.

(A) Sample traces of postynaptic activity before (grey) and after only presynaptic (red), only postsynaptic (blue), or both pre- and postsynaptic learning (black). Lines indicate the average across 10 realizations, and the shading the SEM. The initial response latency is indicated by the green dashed line. (B) The postsynaptic response latency was shortened by learning, although both faster and more efficiently with postsynaptic learning. (C, D) Changes in duration and burst frequency of postsynaptic activity mirrored those obtained with the stochastic minimalist models (Fig 4C and 4D). (E) Distribution of pre- (P) and postsynaptic efficacies (q) after 200 learning trials. (F) Average synaptic weight of early (left) and late (right) presynaptic inputs evolved in distinct manners, however (compare e.g. Fig 4).

https://doi.org/10.1371/journal.pcbi.1009409.g005

In keeping with experimental results [39, 50]—which showed presynaptic LTP, presynaptic LTD, and postsynaptic LTP, but no postsynaptic LTD—the tuned model lacked the capacity for postsynaptically expressed depression. As a consequence, postsynaptically expressed potentiation led to inflated postsynaptic frequency and duration when implemented alone (Fig 5D and 5E). However, the presynaptic LTD was enough to produce a temporally sharpened response of shorter duration. With postsynaptic plasticity, the dynamics developed faster (Fig 5F), a result of a positive-feedback loop arising from increased postsynaptic firing rates (compare Fig 5A).

In the correlation paradigm (Fig 2A), groups of correlated and uncorrelated inputs clustered (Fig 6A) without the need for added competition through weight normalization [44, 51]. This only occurred when both pre- and postsynaptic plasticity components were implemented, and was not achieved through other models with physiologically compatible parameters [45].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. The biologically tuned model clustered inputs with correlated and uncorrelated activity.

(A) Normalized averages (across 10 independent realizations) for presynaptic (P), postsynaptic (q) and combined pre- and postsynaptic (W) plasticity of correlated (corr) and uncorrelated (unc) inputs show that meaningful learning and segregation of inputs only occurred when both pre- and postsynaptic learning mechanisms were engaged. Surprisingly, presynaptic (red) or postsynaptic expression alone (blue) could not cluster differentially correlated inputs (W, right). (B) The postsynaptic spiking frequency increased when postsynaptic plasticity was engaged (blue and black), but not with presynaptic-only learning (red). (C) Average fraction of correct classifications between correlated and uncorrelated inputs for pre- and postsynaptic expression combined was optimal for presynaptic frequencies in the range 50 and 80 Hz.

https://doi.org/10.1371/journal.pcbi.1009409.g006

To better understand the robustness of this property, we quantified the capacity of separation between correlated and uncorrelated populations with a linear separator. It was trained to classify inputs as correlated or uncorrelated according to the average and variance of W values (Fig 6C). The presynaptic frequency range for optimal separation was between 50 and 80 Hz (Fig 6C). At the lower end of the range, it was bounded by the STDP correlation time scale of τ = 20 ms (see Methods), meaning inter-spike intervals longer than 20 ms could not represent the minimal interval of correlation. At the upper end of the range, the high presynaptic frequency yielded overall potentiation that included uncorrelated inputs, limiting the separation from the more potentiated correlated population (S1 Appendix).

In the same way as in the latency paradigm (Fig 5D), postsynaptic potentiation increased postsynaptic firing rate (Fig 6B). However, presynaptic plasticity alone produced no such effect. In combination with postsynaptic plasticity, presynaptic plasticity helped to lower postsynaptic firing frequency as q saturated (Fig 6B), thus keeping postsynaptic firing rates within narrower bounds.

Pre- and postsynaptic expression favour different coding schemes

Our study showed that the locus of expression of plasticity determined affinity for different coding schemes. Presynaptic plasticity expressed as the regulation of release probability alone did not result in appreciable differences for steady-state postsynaptic activity compared to postsynaptic expression (Fig 3B). However, in the presence of short-term plasticity, presynaptic expression of long-term plasticity had a smaller impact on the spike latency than did postsynaptic expression (Fig 4B). This was because, as synaptic response amplitude grew, fewer inputs were needed to evoke a postsynaptic spike. With presynaptic expression, however, the spike still depended on the sum of a larger number of inputs. However, weight changes developed faster with presynaptic plasticity, thereby increasing the speed of learning. This effect, however, was not present in the correlation paradigm, where both pre- and postsynaptically expressed cases performed similarly.

Presynaptically expressed plasticity alone was not ideally suited for changing rate coding, because presynaptic short-term plasticity acts as a filter on the presynaptic firing rate. As a consequence, postsynaptic instantaneous firing frequency shows reduced changes (Figs 4D and 5D) or no changes (Fig 6B) when compared to postsynaptic plasticity. Presynaptic plasticity thus appeared to act as a limiter or a form of homeostasis for postsynaptic activity, in agreement with previously published interpretations [38]. The flip-side of this stabilizing feature of changes in short-term plasticity [56] is in other words the loss of ability to rate code well. An important cautionary take-home message from this observation is that the default implementation of plasticity as purely postsynaptic may thus lead to an erroneous overestimation of the impact on postsynaptic firing rates.

Frequently, the effect of unreliability of single synapses is considered to simply be one of noise or energy economy [57]. However, one can in fact consider this unreliability as a representation of uncertainty over a synaptic weight compared to its optimal value [58, 59]. It would then be plausible to consider presynaptic plasticity as an uncertainty tuning over the posterior distribution in a probabilistic inference framework [60].

A biologically tuned model corroborated the toy model predictions

The same basic properties were observed in the biologically tuned model with simultaneous pre- and postsynaptic plasticity. Learning was dramatically affected by postsynaptic plasticity, while the presynaptic side appeared to act more on the rate of learning and on weight dynamics. It is possible that these results could be modified according to the ratio of pre- versus postsynaptic forms of plasticity, to optimally achieve a specific computational outcome. It is noteworthy that the biologically tuned model was also capable of separating groups of correlated and uncorrelated inputs without the need for a hard competitive mechanism.

Experimental tests of model predictions

Since it is possible to specifically block pre- or postsynaptic STDP pharmacologically [39, 50], several of our findings related to the locus of expression of plasticity are possible to directly test experimentally. For example, at connections between neocortical layer-5 pyramidal cells, it is possible to block nitric oxide signalling to abolish pre- but not postsynaptic expression of LTP [50]. It is also possible to use GluN2B-specific blockers such as ifenprodil or Ro25–6581 to block presynaptic NMDA receptors necessary for presynaptically expressed LTD without affecting postsynaptic NMDA receptors that are needed for LTP [39, 61]. As a proxy for learning rate, one could explore in vitro how blockade of different forms of plasticity expression impacts the number of pairings required for plasticity, or alternatively how the magnitude of plasticity is affected for a given number of pairings [50, 53]. In vivo, the impact on cortical receptive fields could similarly be explored. For example, we predict that receptive field discriminability is poorer when presynaptic LTP is abolished by nitric oxide signalling blockade [17].

Conclusions

Here, we have challenged the standard assumption that modelling synaptic plasticity as a weight change is neutral and unbiased. To do so, we relied on two classic STDP studies [24, 43], extending them with stochastic release and with short-term plasticity, and subsequently revisited our findings with a more physiologically realistic model [17]. We found that even in a simple feed-forward scenario, the locus of expression may have a surprising and considerable impact on learning outcome—e.g., the biologically tuned model could not properly segregate differentially correlated inputs if either pre- or postsynaptically expressed STDP was lost. We expect that these effects will only be greater in recurrent networks, where presynaptic plasticity at loops and re-entrant pathways will exacerbate the effects of changes in synaptic dynamics due to alterations of the accumulated difference. This additional level of complexity may in particular complicate very large recurrent network models [62, 63].

As our collective understanding of the expression of long-term plasticity has improved, it has become clear that the long-held notion that plasticity is expressed predominantly postsynaptically is erroneous [7–9]. Since presynaptic expression is still relatively poorly studied, our understanding of long-term presynaptic plasticity in health and disease needs to be generally improved [64]. Specifically, our study highlights the need for more detailed modelling of the role of the site of expression. It is clear that it has implications for information coding, be it spike based, rate based, or probabilistic. Therefore, in modelling long-term plasticity, choosing the location of changes in weight is a matter of gravity.

Neuron model

All of the simulations consisted of one postsynaptic neuron receiving a number of presynaptic Poisson inputs. In the first section, we used a simple leaky integrate-and-fire model defined by (1) in which the membrane potential V decayed exponentially with a time constant of τ_V = 20ms to the resting value of E_v = −74 mV, and the threshold for an action potential was V_th = −54 mV. After each spike it was reset at V₀ = −60 mV with a refractory period of 1 ms.

Inputs were received with probability p_j and increased the conductance-based excitatory contribution (g), with reversal potential E_e = 0 mV. An impulse with amplitude q_j * q_max (q ∈ (0, 1] and q_max the maximal amplitude) was summed for each l^th input received at time from the presynaptic neuron j, and decayed exponentially with a time constant of τ_g = 5ms: (2) where Θ(x) is the Heaviside function. In the the last section, we used the adaptive exponential integrate-and-fire model [65] to reduce unrealistic bursting and to comply with the biological tuning [17]: (3) (4)

The corresponding parameters for a pyramidal neuron were C = 281 pF, g_L = 30 nS, E_L = −70.6 mV, Δ_T = 2mV, c_z = 4nS, τ_W = 144ms. Spiking threshold was V_T = −50.4 mV, and after each spike V was reset to the resting potential E_L while z increased by the quantity b = 0.0805 nA (as in [65]).

Stimulation paradigms

The postsynaptic neuron was in one of two stimulus paradigms. The first one was based on [24] and is referred to as the Latency Paradigm (Fig 2A). In every 375-ms-long trial, the postsynaptic cell received a volley of Poisson inputs that arrived with a specific delay, normally distributed around a time reference, for each specific presynaptic neuron. Each input lasted for 25 ms with a spiking frequency of 100 Hz. We measured the time to spike of the first postsynaptic spike in response to a bout of stimuli using the mean of the presynaptic delay distribution as a reference point. For clarity, in the Results, curves that represent latency shift, intra-burst frequency or burst duration were smoothed using a moving average filter with a window of three points.

The second paradigm was based on [43] and is referred to as the Correlation Paradigm (Fig 2B). This configuration consisted of continuous Poisson inputs with fixed frequency. However, half of the inputs had correlated fluctuations of activity, with a time window of τ_corr = 20 ms, while the other half was uncorrelated. Correlations were implemented using method described in [66]. An additional scenario with competition between pre- and postsynaptic plasticities, all inputs are correlated but half changes presynaptically and the other half postsynaptically. The total sum of weights was kept fixed so that competition was observable in a wide range of parameters.

Additive STDP model

For the majority of the simulations we opted to implement STDP with the simple additive model proposed by Song and Abbott [24]: (5)

Each increment to the synaptic weights W_ij (since there was only one postsynaptic cell, we consider W_j = W_ij throughout this paper) was computed after a pair of pre- and postsynaptic spikes, t_i and t_j, and the parameters were set to τ_STDP = 20ms, c_pot = 0.005, and c_dep = −0.00525. We separated the synaptic weight W_j as a product between pre- and postsynaptic counterparts, baseline probability of release P_j = (0, 1] and quantal amplitude q_j = (0, 1] respectively, so that W_j = q_jP_j. The probability of release was simulated in two different ways, one equivalent to regulating the probability of stochastic interactions and the other via short-term plasticity.

To enable comparison of convergence rates for different types of plasticity expression, we ensured that the weight change ΔW = W^t+1 − W^t at time step t was the same regardless of whether plasticity was expressed presynaptically, postsynaptically, or both. To achieve this, we normalised the weight changes so that if only q was changed: (6) and similarly, if only P was changed: (7)

The initial value of all simulations was the same for P and q, so that in these cases ΔW^P = ΔW^q ≡ d. When expression was both pre- and postsynaptic, the amount d was divided equally across P and q (so ΔP^Pq = Δq^Pq ≡ Δ) as follows: (8) Solving for Δ so that ΔW^Pq = d: (9)

We also kept the same range of total W change as equal throughout the simulations. Since both start at the same initial value (P⁰ = q⁰), the largest possible change for P or q separately was Δ_tot = P⁰(1 − q⁰) = q⁰(1 − P⁰). For changing P and q simultaneously, we limited the maximal values P and q so that ΔW_tot = P^TOPq^TOP − P⁰q⁰ is also the same. In this case, .

Biologically tuned STDP model

We compared the results of the straightforward additive model to a slightly more complex STDP model that acts separately over pre- and postsynaptic factors [17]. Parameters were fitted to experimental data from connections between pyramidal cells from layer 5 of V1 [39, 49, 50]. The equations for pre- and postsynaptic changes followed: (10) (11) where is increased at each spike from the presynaptic neuron j and at each spike from the postsynaptic neuron i. ϵ is to emphasise that ΔW was calculated before x_j+ and y₋ were updated, upon the arrival of a new spike. y₊ and y₋ are postsynaptic traces, (12) (13) with decay times and respectively, and x_j+ was a presynaptic trace with decay time : (14)

The parameter values were taken from [17]: d₋ = 0.1771, , d₊ = 0.15480, c₊ = 0.0618, and . To avoid manipulation of the fitting, weight changes were not normalised in this case. To avoid the postsynaptic side being forever potentiated, a small scaling was introduced postsynaptically in each step: , being < Δq > the average change over all postsynaptic side and α a scale factor (0.5).

In the last section, we used a linear least-squares classifier to infer whether presynaptic inputs were correlated or uncorrelated. A linear model was fitted to separate the values of synaptic weight averages and variances from half of the inputs (labelled correlated or uncorrelated), and then used to classify the other half of inputs.

Presynaptic factor

Presynaptic control of the probability of release per stimulus was implemented either as a Markovian process or as short-term plasticity, with presynaptic weight P_j. In the former case, probability (p_j) of stochastic neurotransmitter vesicle release followed a binomial distribution, and p_j = P_j. Based on the findings reported by [67], each presynaptic neuron had N = 5 release sites that functioned independently. In the second case, we considered a dynamic modulation of the EPSPs through short-term plasticity. The probability of transmission was decomposed into the instantaneous probability of release and availability of local resources r_j(t), so that . These two factors modulate transmission in a short term scale around a baseline value of release probability , which makes in this STP scenario. The dynamics of and r_j(t) followed the model proposed by Tsodyks and Markram [68]: (15) (16)

Depression and facilitation time constants, τ_D = 200 ms and τ_F = 50 ms respectively, were chosen as representative values for connections between pyramidal neurons [69]. The resulting short-term plasticity is mostly depressing, that is the resulting p_j is lower than except for very low values of and high input frequencies [70].