Figure 1.
Schematic diagrams of network architecture of the single population, two population, and four population models.
A) Single-population firing rate model. In the single-population model the network receives input from an external current I(t) during the cue period of a simulated working memory task. The synaptic activity S(t), and consequently the firing rate increases from the input of the external current, resulting in a dynamic increase in its effective self connectivity C11 as a result of a concomitant dynamic change in synaptic facilitation W1. B) 2-population distributed model. In the distributed 2-population model, 2 “local” networks, 1 and 2 are connected recurrently by long-range projections C12 and C21 whose strength is weaker than the populations' self connectivity C11 and C22. Both populations receive input from an external current I(t) during the cue period of simulated working memory task. This results in changes in their respective population firing frequencies along with changes in their effective long-range projections and self couplings through dynamic facilitation. In the spiking unit version of the model, populations 1 and 2 consist of networks of 100 (or 1000) spiking units each with all-to-all connectivity. C) 4-population model. In the 4-population model, four local networks (1, 2, 3, and 4) all receive input from an external current I(t) during the cue period of the simulated working memory task. Each local population receives input from self connectivity, and weaker input from the long-range projections from the other populations. Long-range projections between the population pair 1 and 2, and between the pair 3 and 4 (i.e. C12, C21, C34, and C43) are stronger than the connection strength between populations 1 and 3 or 4, or between 2 and 3 or 4. In the spiking unit version of the model with 200 units, each population corresponds to a network of 50 spiking units with all to all connectivity. In the spiking unit version of the model with 2000 units, each population corresponds to a network of 500 units with all to all connectivity. The strength of the connections is scaled such that the total strength of connectivity to units is the same as the 200 unit network.
Figure 2.
Phase portraits and bifurcation diagrams of the single population reduced model obtained for different values of the self connectivity C11 and the maximum facilitation Wmax.
The reduced 2-dimensional model is obtained at steady state calcium concentration (dCa/dt = 0). Shown in the phaseplane are the nullclines for Facilitation (W) and Synaptic activity (S). A) Nullclines with self connectivity C11 = 4.8. The stable nodes of the system are where the nullclines cross. Here there is a single fixed point (solid black dot) corresponding to the baseline state. The trajectory of the system during a working memory task is indicated by the black line with an arrow. During the sample period, the applied current I(t) raises the synaptic activity, and firing rate, with a concomitant rise in the facilitation (approximately 33% increase). After a 300 ms cue period, I(t) becomes 0, and synaptic activity rapidly decreases towards the facilitation (W) nullcline, and the bottleneck. Because of the bottleneck, the trajectory then returns very slowly along the path of the facilitation nullcline towards the stable baseline state. In the present example, the synaptic activity and consequently the firing rate is still above the stable baseline rate at the end of the 10-second delay period, as indicated by the termination of the trajectory line. The system therefore maintains an increase from its baseline firing rate for the duration of the delay period. A continuum of elevated delay rates occur for different values of the parameters. B) W and S nullclines of the single population network with C11 = 5. Note there are now 3 stable points where the nullclines cross: 2 attracting (black dots) and one saddle node (white dot). C) Possible system trajectories of the network in (B) with C11 = 5. The saddle separatrix is indicated by the blue line. Two different possible trajectories of the system are shown which result from varying the magnitude of the external current I(t). For I(t) = 0.5 the trajectory does not cross the saddle separatrix and stays in the basin of attraction of the baseline state. In this case, given sufficient time the system returns to the baseline stable state. The resulting pattern of behavior is that of cue-coupled or decaying memory cells. For I(t) = 0.75 the trajectory cross the saddle separatrix, entering the baseline of attraction of the higher firing rate stable state. The system for this trajectory is shown to be in the higher firing rate state by the end of the delay period. The resulting pattern of behavior is that of response-coupled or ramping cells. In both cases the bottleneck can be adjusted such that the rate at which the system returns to the baseline state, or approaches the higher firing state can be arbitrarily slow, resulting in a continuum of different average firing frequencies during the delay, and apparent bistability at frequencies between the two stable fixed points. D) Phase portrait with C11 = 5.2. The system again possesses a single stable state at a higher rate than in (A). E) Bifurcation diagram of steady state synaptic activity as a function of maximum facilitation (Wmax). Curves shown are for Wmax equal to 0.925, 1, and 1.25 (producing synaptic facilitation in the 30–60% range). Solid lines indicate stable fixed points, hashed lines are unstable fixed points. Note that 3 fixed points are present over a wide range of the facilitation. F) Bifurcation diagram for the parameters of maximum facilitation Wmax and the threshold θ. The curves correspond to the limit points at different values of these parameters. The two branches of the limit points meet at a cusp point. For the region interior to the two curves there are three rest states and bistability, and outside them there is a single rest state and monostability.
Table 1.
Network parameters.
Table 2.
Mean connectivity values for the 2 population network.
Table 3.
Mean connectivity values for the 4 population network.
Figure 3.
PSTH histograms of the single population firing rate model during the simulated working memory task.
Different patterns correlated with working memory are obtained for different values of the facilitation (or depression) and the magnitude of the input current during the cue period. A) Decaying memory cell behavior. The baseline frequency is approximately 6 Hz, and at the beginning of the delay period the firing rate is approximately 9 Hz. The firing rate has returned to the baseline level by the end of the delay period. Average increase in firing rate from baseline to delay period is approximately 1.5 Hz. B) Apparent bistable memory cell behavior. The delay firing rate does not correspond to a stable state, but the decay towards the baseline state is sufficiently slow so that no significant decrease in firing rate occurs by the end of the period. The baseline firing rate is approximately 6 Hz, and the delay period firing rate is approximately 10 Hz. C) Nonresponsive pattern behavior. The population responds during the presentation of the cue, but immediately returns to and maintains the baseline firing rate during the delay. D) Ramping response-coupled cell behavior. The baseline frequency is approximately 6 Hz, and at the beginning of the delay period the firing rate is approximately 9 Hz. The firing rate adopts the rate of the higher fixed point attractor by the end of the delay period. Average increase in firing rate from baseline to delay period is approximately 6.5 Hz. E) Decaying inhibited pattern. In this example the value of maximum facilitation is less than the background facilitation, resulting in dynamic synaptic depression. The baseline frequency is approximately 6 Hz, and at the beginning of the delay period the firing rate has decreased to approximately 2.5 Hz. The firing rate has returned to the baseline rate by the end of the delay period, mirroring the decaying memory cell activity observed in (A). Average decrease in firing rate from baseline to delay period is approximately 1.75 Hz. F) Apparent stable inhibition. The delay firing rate does not correspond to a stable state, but the decay towards the baseline state is sufficiently slow so that no significant decrease in firing rate occurs by the end of the period. The baseline rate is approximately 4.5 Hz, and the delay period firing rate is approximately 1 Hz. Every pattern observed in the cortical database as reported in [5] is observed with the exception of delay inhibition that increases throughout the delay period.
Figure 4.
Phase Diagram of different patterns exhibited by the single population model as a function of the maximum facilitation parameter and self-connectivity.
Left: Patterns exhibited with dynamic synaptic facilitation. Actual facilitation is less than the maximum facilitation parameter and falls within a physiological reasonable range (10–60%). Right: Patterns exhibited with dynamic synaptic depression which takes place when values of Wmax are less than the baseline level (Wmin). Pattern activity was determined by examining the firing rate activity during 3 periods of the working memory task: the first 5 seconds of the delay period (D1) immediately following the cue, the second 5 seconds of the delay period (D2), and the last 5 seconds of the baseline period (B) immediately preceding the cue period. Significant differences were taken to be present if the absolute difference between any two periods was greater or equal to 0.5 Hz, which is approximately the lower limit of significant differences observed in the real cortical parietal and prefrontal cells. Fixed rate memory cell behavior consisted of activity in which the absolute difference in average firing rate between D1 and D2 was less the 0.5 Hz, while both those periods exhibited an average firing rate greater or equal to 0.5 Hz above the baseline rate. Ramping cue-coupled cell behavior consisted of activity in which D2 exhibited an average firing rate greater or equal to 0.5 Hz above that of D1, and the firing rate of D1 was greater or equal to that of B. Decaying memory cell behavior consisted of activity in which D1 exhibited an average firing rate greater or equal to 0.5 Hz above both B and D1. Nonresponsive cell behavior consisted of activity in which the difference between any of the periods (B, D1, and D2) was less than 0.5 Hz. Decaying inhibition cell behavior consisted of activity in which the average firing rate during D1 was at least 0.5 Hz less than that exhibited during B and D2.
Figure 5.
PSTH histograms of activity exhibited by the 2-population firing rate model at different values of the inter-area connectivity.
All general patterns in the cortical data as reported in [5] are exhibited by the network, with frequencies in the range of the real data. Firing pattern behaviors shown are A) decaying memory cell, B) Stable memory cell, C) Set cell D) Decaying inhibition cell, E) Stable inhibition cell, and F) ramping inhibition cell.
Figure 6.
Phase diagrams of the patterns exhibited by the 2- and 4-population firing rate models.
The phase diagrams were created using the same procedure carried out for the generation of the phase diagram of the single-population model. A) Phase diagrams of the activities of the 2-population model as a function of the inter-population connectivity. Left: Phase diagram of population 1. Right: Phase diagram of population 2. The self connectivity of population 2 (0.9) is slightly less than population 1 (1.0) so that the phase diagrams of the two populations are not identical. Excitatory-Inhibitory (E-I) and Inhibitory-Inhibitory (I-I) connectivity architectures between the populations are examined. The pattern category “other” corresponds to decaying or ramping cell behavior in which the average firing rate of one period of the delay–D1 or D2–is significantly less than the baseline (i.e. at least 0.5 Hz less than the firing rate during the period B), while the other is significantly greater (i.e. at least 0.5 Hz greater than the firing rate during the period B). Note that all of the pattern types (excitatory, inhibitory and nonresponsive) are obtained over a wide range of the parameters. B) The overlapping simultaneously occurring patterns in the 2 population network across the range of inter-population connectivity values. The figure shows the 1 and 2 populations' (left and right phase diagrams of panel A to the left) phase diagrams superimposed on each other. The 1-population states are shown as solid, and the corresponding 2-population states are shown in outline. Discrete regions of the overlapping phase diagrams reveal simultaneously occurring network behaviors observed in cortex during working memory. Included is simultaneous occurrence of decaying memory behavior in both populations (red-red overlap), and stable memory behavior in one population and ramping cell behavior in the other (blue-green overlap). Also common network behaviors observed are the simultaneous occurrence of the excitatory patterns in one population and the nonresponsive pattern in the other. Inhibitory patterns tend to occur in one population primarily with ramping cell behavior in the other. C) Phase diagrams of the activities of the 4-population model as a function of the inter-population connectivity. Upper left: Phase diagram of population 1. Upper right: Phase diagram of population 2. Lower Right: Phase diagram of population 3. Lower Right: Phase diagram of population 4. All the pattern types are exhibited by the network. However, the populations partition themselves such that they exhibit almost exclusively either excitatory or inhibitory memory patterns across the values of interpopulation connectivity. D) Simultaneously occurring patterns in populations 1 and 2, and in populations 3 and 4. E) Simultaneously occurring patterns in populations 1 and 3. The figure shows 1 and 3 (excitatory) populations' phase diagrams superimposed on each other. Over a significant range of the parameters, memory cell behavior and ramping cell behavior co-occur (red and green overlap). The existence of such simultaneous population behavior has been indicated in prefrontal cortex. Memory cell behavior is also seen to simultaneously occur in multiple populations across a wide continuous range of the parameter space (red-red and red-blue overlap).
Figure 7.
PSTH histogram of the single-population firing rate model (left) and the corresponding average spiking unit activity in a network of 50 units obtained from the spiking model (right).
Note the general pattern is the same in both networks.
Figure 8.
Average histograms from example units of the 2-population spiking model showing the excitatory and inhibitory working memory patterns, and average histograms from representative single neuron recordings from the database of prefrontal and parietal neurons exhibiting the same general patterns.
PSTH histograms of spiking model units are shown in the left columns and real neuron histograms in the right columns. The gap between baseline and delay periods corresponds to the cue period (firing rate not shown). Histograms of the model units were averaged over 10 simulated trials of the working memory task. Patterns exhibited are A) Stable fixed rate memory (i.e. bistable) cell behavior, B) ramping cell behavior, C) decaying memory cell behavior, D) fixed rate inhibition cell behavior, E) ramping inhibition, and F) decaying inhibition.
Figure 9.
Distributions of units exhibiting the different pattern types in the spiking models and variability in pattern expressed from trial to trial in the units of those models.
A) Distribution of pattern types exhibited by units in the 2- and 4-population spiking models (left and right respectively). Distributions correspond to the average number of units exhibiting each pattern type over 20 trials of the simulated working memory task. Pattern numbers correspond to the same patterns as labeled in table 4. Specifically 1) Ramping cells 2) fixed rate memory cells, 3) Decaying memory cells 4) ramping set cell with no frequency difference between D1 and baseline, 5) nonresponsive cell 6) ramping inhibited cell with no frequency difference between D1 and baseline, 7) decaying inhibition cell, 8) fixed rate delay inhibited cell, and 9) ramping delay inhibited cell B) Distribution of the percentage of parietal (left) and prefrontal (right) cells exhibiting each pattern type during the performance of unimodal and cross-modal working memory tasks. Note that the distribution of the real cells and the units in the spiking model are similar with nonresponsive cells being most common, excitatory memory cells (decaying, fixed rate, and ramping) being the next most common, and inhibitory patterns (decaying, fixed rate, and ramping) being the least common. Also patterns 4 and 6 (ramping excitatory and inhibitory cells with no frequency differences between D1 and baseline) occur infrequently in both the models and real cells. C) Consistency of pattern type expressed across trials for each of the 200 units in the 2-population spiking model (left) and the 4 population spiking model (right). Each unit was classified as displaying one of the pattern types in its average PSTH histogram (across 20 trials). The ordinate indicates the percentage of trials in which the dominant pattern (the pattern occurring most often across trials and is typically that which is observed in the averaged PSTH histogram) of the average was actually exhibited by the unit. In the 2-population graph, units 1–100 are all members of one population, while units 101–200 are part of the other population. In the 4-population graph, units 1–50, 51–100, 101–150, and 151–200 are the members of populations 1 through 4 respectively. Note that the overall average variability in pattern expression is the same in the 2- and 4-population models, with the classified pattern type of each unit being displayed in approximately 47% of the trials, and an approximate range of 30–70% of trials showing different patterns on any given trial. However, while the variability is distributed evenly across both populations in the 2-population model, in the 4-population model the variability is significantly greater in units in those populations exhibiting primarily inhibitory patterns (populations 2 and 4).
Table 4.
Average firing frequencies (Hz) during the baseline and delay periods, and the frequency difference between baseline and the average of the delay period (delta), for the units exhibiting each of the pattern types by the 2- and 4-population spiking models.
Figure 10.
Raster plots of 10 trials (left) for an excitatory (A) and an inhibitory (B) unit from the 4-population 2000 unit spiking network.
Note that different pattern types occur on different trials. Figures A and B (right) show an expanded version of 4 of the 10 trials from both units to highlight different patterns exhibited. From top to bottom of the expanded trials, the excitatory unit shows a decaying rate, nonresponsive, increasing rate, and persistent stable firing rate pattern, and the inhibitory unit shows decaying inhibition, stable inhibition, increasing rate excitation, and increasing inhibition. C) Rasters from 4 trials selected from a real neuron recorded from the prefrontal cortex during presentation of the same memorandum of the cross-modal task exhibiting stable persistent activation during the delay in the average PSTH. Although the cells exhibits stable persistent activation on average, on specific trials the cell exhibits the decaying rate activation pattern.
Figure 11.
Average PSTH histograms of 16 units randomly chosen from the 2000 unit 4-population spiking model.
Note that the entire range of pattern types continue to be exhibited (excitatory and inhibitory stable, decaying, and ramping delays and nonresponsive) with firing statistics similar to real neurons.
Figure 12.
Distributions of patterns exhibited across 20 trials of the simulated task.
A) top: Distribution of pattern occurrence for the 2-population, 2000-unit spiking network. The distribution is similar to that observed for the 200 unit networks and the parietal and prefrontal neuronal populations. Bottom: Distribution of patterns exhibited in the average (across 20 trials) PSTH histograms for the 2-population, 2000-unit spiking network. Note that the distribution of average PSTH exhibits the bistable excitatory and inhibitory patterns more prominently than actually exhibited from trial to trial by the units. B) top: Distribution of pattern occurrence for the 4-population, 2000-unit spiking network. The distribution shows similar relative occurrences of the persistent excitatory and inhibitory patterns (stable, ramping, decaying) as all previous networks and real neuronal data. Relative changes in specific pattern occurrences (i.e. decrease in the prevalence of the nonresponsive patter) result from each population's activity approaching that of its corresponding mean firing rate model single pattern with increasing size. Thus a larger number of populations would be needed (i.e. greater than 4) to maintain the relative occurrence of all patterns and thus to maintain an invariant distribution. Bottom: Distribution of patterns exhibited in the average (across 20 trials) PSTH histograms for the 4-population, 2000-unit spiking network. Note that the average PSTHs exhibited are essentially all the bistable excitatory and inhibitory patterns. Thus analysis of average patterns across trials might falsely indicate that these are the only patterns of relevance occurring.
Figure 13.
Reliability of units in the 2000 unit networks for exhibiting from trial to trial the pattern indicated by each unit's average PSTH histogram.
A) Reliability of each unit in the 2-population, 2000 unit network. The average reliability (percentage of trials exhibiting the pattern observed in the average PSTH histogram) ranges from approximately 40–100% of trials, with an average of approximately 47%. This is similar to that observed for the 200 unit 2- and 4- population spiking models. B) Reliability of each unit in the 4-population, 2000 unit network. The average reliability is significantly higher across all 2000 units (approximately 75%) although the range of variability of individual units is similar to that observed in each of the previous networks. These reliability values are similar to that observed in the real cortical data of parietal and prefrontal cells.
Figure 14.
Analysis of spiking variability.
A)Distributions of the coefficient of variation of the interspike intervals for units of the 2000 unit spiking model during the baseline and delay. Distributions correspond to cells exhibiting all patterned delay behaviors. High variability in ISI firing occurs during baseline and delay periods. Delay CVs show a bimodal distribution across all pattern types. B) Mean baseline and delay ISI CV for stable persistent excitation delay cells (left) and stable persistent inhibition delay cells (right). The coefficient of variation decreases from baseline to delay in excited delay cells and increases in inhibited delay cells in agreement with the behavior observed in the database of prefrontal and parietal cells.