Fig 1.
A: The model network has a simple feed-forward structure. The top picture shows three pre- and one postsynaptic neurons, connected by synapses. Line width in this example corresponds to synaptic strength. Bottom picture shows the postsynaptic membrane potential in response to the input. B: Illustration of Anti-Hebbian Membrane Potential Dependent Plasticity (MPDP). A LIF neuron is presented twice with the same presynaptic input pattern. Excitation never exceeds Vthr. MPDP changes synapses to counteract hyperpolarization and depolarization occuring in the first presentation (blue trace), reducing (arrows) them on the second presentation (green trace). C: Homeostatic MPDP on inhibitory synapses is compatible with STDP as found in experiments. Weight change is tested for different temporal distances between pre- and postsynaptic spiking, with the presynaptic neuron being an inhibitory neuron. Δw here denotes the change of the increase in conductance in an inhibitory synapse upon a presynaptic spike. The resulting spike timing characteristic is in agreement with experimental data on STDP of inhibitory synapses [16]. Note that an increase of the weight leads to a suppressive effect on the membrane potential.
Fig 2.
Hebbian learning with homeostatic MPDP on inhibitory synapses.
A conductance based integrate-and-fire neuron is repeatedly presented with a fixed input pattern of activity in presynaptic inhibitory or excitatory neuron populations (top row—blue dots are excitatory, red dots are inhibitory spike times). The number of input neurons is Ni = 142 for the inhibitory population and Ne = 571 for the excitatory population. Second row shows the membrane potential before learning. The upper red line is the threshold for potentation of inhibitory synapses , lower red line is resting potential and threshold for depression
. The third row shows the voltage as before with added teacher input by an additional population of excitatory neurons; this input induces a spike at t = 100ms. The fourth row shows the voltage after 100 learning steps with MPDP on inhibitory synapses only, with teacher input, the next row shows the recall without teacher. The spike is almost at the same position in the recall case. The last row shows the voltage after 1000 recall trials during which the inhibitory synapses were allowed to change under the MPDP rule. Despite this, the output spike is still close to the desired time, which shows that the output is approximately stable.
Fig 3.
Hebbian learning with homeostatic MPDP.
A postsynaptic neuron is presented the same input pattern multiple times, alternating between teaching trials with teacher spike (blue trace) and recall trials (green trace) to test the output. Initially, all weights are zero (left). The green area between the voltage and threshold for potentiation ϑP signifies the total amount of potentiation, similarly the red area between voltage and ϑD for depression; the latter is only visible in the second to right panel. Learning is Hebbian initially until strong depolarization occurs (second to left). When the spike first appears during recall, it is still not at the exact location of the teacher spike (second to right). Continued learning moves it closer to the desired location. Also, the time windows of the voltage being above ϑD and below ϑP shrink and move closer in time (right). Synaptic plasticity almost stops. The number of learning trials before each state is 1, 16, 53, and 1600 from left to right.
Fig 4.
Capacity of networks with MPDP.
A: Fraction of pattern where the network generates an output spike within 2 ms distance of target time , and no spurious spikes. Network size is N = 1000. The desired spikes are learned within ≈ 600 steps. B: Average distance of output spikes to target for the same network size. Training continues even though the desired spikes are generated; however, they are pushed closer to the desired time. C: Average fraction of recalled spikes after 10000 learning blocks for all network sizes as a function of the load. Networks with N = 200 have a high probability to not be able to recall all spikes even for low loads. Otherwise, recall gets better with network size. The thin black line lies at fraction of recall equal to 90%. The critical load α90 is the point where the graph crosses this line. D: Average distance of recalled spikes as a function of the load. The lower the loads, the closer the output spike are to their desired location. E: Critical load as a function of network size for all four learning rules. MPDP reaches approximately half of the maximal capacity.
Fig 5.
Capacity of networks under input noise.
All network are of size N = 1000. A: Recall as a function of the load for different levels of noise during recall. Noise is imposed as an additional stochstastic external current. Networks were trained with MPDP. Up to a noise level σinput = 1mV during recall, there is almost no degradation of capacity. B: Same as A, but with stochastic input noise of width 0.5mV during network training. The capacity is slightly reduced, but resistance against noise is slightly better. C and D: Same as A and B, but the network was trained with FP-Learning. The capacity is doubled. However, the network trained without noise shows an immediate degradation of recall with noise. If the network is trained with noisy examples (D, σinput = 0.5mV), also recall with noise of the same magnitude is perfect. E: Comparison of capacity of networks trained with MPDP and FP-Learning depending on input noise during training and recall. Solid lines: MPDP, dashed lines: FP-Learning. Lines that are cut off indicate that the network failed to reach 90% recall for higher noise. x-axis is noise level during recall. Different colors indicate noise level during training. Curiously, although FP-Learning suffers more from higher noise during recall than during training, the capacity drops less than with MPDP. F: Comparison of weight statistics of MPDP (solid lines) and FP-Learning (dashed lines) after learning. Left plot is the mean, right plot is the standard deviation. With MPDP, the weigths stay within a bounded regime, the mean is independent of noise or load during training; the cyan line for α = 0.1 occludes the others. FP-Learning rescales the weights during training with noise: The mean becomes negative, and the standard deviation grows approximately linearly with noise level. This effectively scales down the noise by stochastic input.
Fig 6.
Fraction of recalled spikes under input noise.
This plot shows the fraction of recalled spikes after learning as a function of the input noise during recall for load α = 0.04 and network size N = 1000. The network trained with MPDP perfectly recalls up to σinput = 0.5mV, and with a slight drop for σinput = 1mV. With the network trained with FP-Learning, there is a drop of the fraction of recalled spikes even with slight noise.
Fig 7.
Recall and capacity with input jitter.
A: Recall of networks trained noise-free with MPDP if during recall the input patterns are jittered (N = 1000). The black line lies on top of the blue and red ones (same in B). Up to σjitter = 0.5ms, the recall is unhindered. A curious feature is a “slump” in the recall for strong input jitter and intermediate loads. This slump is even more visible for the larger network with N = 2000 (B). The slump strongly correlates with the variance of the weights as a function of network load (C for N = 1000, D for N = 2000). The mean of the weights stays almost constant. E: Critical load as a function of input jitter during recall. The networks are trained noise free with different learning rules. Solid lines show N = 2000, dashed lines N = 1000. Crosses show sampling points. If a line is discontinued, this means that for this input jitter the networks do not reach 90% recall anymore. Recall for MPDP stays almost constant until σjitter = 0.5, while for the other learning rules a considerable drop-off of recall is visible. F: Noise free recall of networks trained with noisy input. For MPFP, E-Learning and FP-Learning alike the capacity drops with increasing training noise. The exception is ReSuMe. Here, the capacity strongly increases if the noise is small.