Fig 1.
Comparative illustration of multiplexing theories.
A. Consider a simple scene comprising 3 discrete objects of different saliency. B. LJ-multiplexing. Multiple Items are processed across consecutive fast cycles in decreasing order of saliency, and this sequence is repeated over slow oscillatory cycles. Note that items of low saliency may not be processed at all. C. F-multiplexing. Within any given slow cycle, only a single item is represented, repeated across multiple fast cycles. Across slow cycles, the represented item may switch, due to changes in the stimulus or top-down attentional mechanisms altering relative stimulus saliency.
Fig 2.
A. The model comprises two areas: a lower area approximating primary visual cortex, composed of multiple retinotopically arranged ‘simple cell’ layers duplicated across orientation, spatial phase and spatial frequency tuning. For input, these cells received a visual stimulus (excitatory conductance driven via an appropriate Gabor filter), noise (independent across cells) and a common slow rhythmic inhibitory conductance to mimic an alpha/theta oscillation (9 Hz in all of our simulations). In turn, the lower area sent convergent connections to a simplistic higher area, comprised in this case of just 4 cells, each receiving input from one retinotopic quadrant of the lower area (i.e. pure spatial selectivity). B. Within each area, a number of local circuit mechanisms could be implemented: lateral excitation (see Methods for details); local inhibition with a 2-D Gaussian distribution of synaptic strength both to and from a layer of inhibitory interneurons; and global inhibition implemented via a single interneuron receiving input from and sending output to all the excitatory cells in the lower area. In the simulations reported, local excitation and inhibition were always included, whereas global inhibition was varied as an important parameter determining network behaviour. C. The power spectrum of activity generated in the lower area in response to a single stimulus object (light rectangle, 9 x 39 pixels, with a single-pixel dark outline) as a function of its contrast, calculated for the PSTH of all activated cells (10 repeats of 1 second of activity for each contrast level tested; note that no global inhibition was included in these simulations). The network readily generates gamma frequency activity, via a PING mechanism as illustrated diagrammatically below the plot (pyramidal cells and their times of firing shown in gray; inhibitory interneurons in orange; the orange rectangles in the background represent the time course of inhibitory feedback). The frequency of gamma activity is contrast dependent, matching experimental results.
Fig 3.
Network states as a function of global inhibition amplitude in the lower and higher area.
A. The colour map shows the network state dependence on parameter values (other parameters set to default values except as stated, 5 repeats of 2 seconds of simulated activity for each condition), with green reflecting poor spike separation, red selective representation of a single item (F-multiplexing), and blue alternation between different items (LJ-multiplexing); see Methods for details of the indices used to measure spike separation and item representation. B–E. Polar plots of higher level activity, spike probability (across simulation repeats) as a function of slow oscillation phase. Colours (red, blue and green) indicate spikes driven by the high, mid and low contrast objects respectively. The different plots correspond to the different network states, as marked in panel A. B. Without global inhibition in either area, all items were represented but there was poor spike separation (green area in A). C. With global feedback inhibition only in the higher area, the network displayed F-multiplexing, with a single item repeatedly represented at gamma frequency within each slow cycle (red area in A). D. The inclusion of global feedback inhibition in the lower area instead yielded LJ-multiplexing, with alternation between different items (blue area in A). E. With global inhibition in both areas, the network tended to generate fewer spikes, skipping some of the available gamma cycles. Multiplexing tended to be LJ-style.
Fig 4.
F- and LJ-style multiplexing: simulation results and diagrammatic representation.
This figure shows the results of a simulation (100 repeats) in which key parameters were changed over the course of the simulation in order to illustrate and explain the different network states highlighted in Fig 3. Simulation results (top of the Fig) show post-stimulus time histograms (PSTHs) for cells in the upper and lower areas, averaged over 100 repeats of the same simulation parameters. The stimulus image was that shown in Fig 2, comprising three discrete objects of different contrast values (see Methods). Parameters were fixed at default values (Fig 3B), except that at successive fixed time points, global inhibition was turned on, first in the higher area (Fig 3C) and then in the lower area as well (Fig 3E). Red traces are for the cells responding to the highest contrast object (averaged over all the active cells in the relevant quadrant of the lower area; and showing the probability of firing at a given time point for the single cell in the higher area responding to that object), and similarly blue for the mid-contrast object and green for the low-contrast. The purple background in the lower area shows the amplitude of the slow oscillatory inhibitory conductance. The same colours are preserved in the diagrammatic representation of activity. As in Fig 2, inhibitory interneurons, their time of firing and the time course and spatial extent of this inhibitory feedback are indicated in orange. A. With no global inhibition in either lower or higher area, activity driven by the three stimulus objects generates independent gamma oscillations in the lower area, and the higher area simply follows this activity. However, the output is not well temporally segregated. B. Global inhibition is added to the higher area. On each slow cycle, the first-arriving burst of inputs to the higher area drive a cell which in turn competitively inhibits the other higher area cells. That inhibition decays just in time for the same cell to be driven by its next burst of inputs from the lower level (i.e. a PING gamma is set up in the higher area, matching the dynamics of the gamma in the lower area, and selective communication is achieved, or F-multiplexing). C. Global inhibition is added to the lower area. On each slow cycle, when the first ensemble fires competitive inhibition disrupts the activity of the other ensembles. The second ensemble recovers, yielding a second cycle of inhibition. By that point, the first ensemble has recovered and fires again, and the third ensemble never has the chance to fire. Effectively, the independent gamma activities in the lower area are disrupted and replaced by a single global gamma, with activity of just a single ensemble on each cycle.
Fig 5.
F-multiplexing–the slow oscillation.
Results from several series of simulations (100 repeats each), summarizing activity in the higher area (red for high contrast object, blue for mid, green for low). A. Without the slow oscillation to rhythmically reset the system, selection via the CTC mechanism fails over time, as the independent gamma oscillations in the lower area shift in and out of synchrony (because of their slightly different frequencies). Parameters as for Fig 3C (i.e. strong global inhibition in the higher area), except that no slow oscillation was included. Following 200 ms of blank screen, the stimulus (3 objects, see text) was turned on. i) shows a PSTH (spike probability across simulation repeats), ii) is a cross-correlogram (calculated for spikes from all 3 cells), showing the poor temporal separation of activity (the large central peak shows that different higher area cells often spike near-synchronously), iii) shows the fraction of spikes driven by each object over the whole simulation period. B. Compared to A, selection is even worse when the network starts from a properly randomised state at stimulus onset. As for A, except that the blank screen at the start was replaced by pixelated white noise. C. Including the slow oscillation (with amplitude high enough to suppress lower area firing) yields good selection, well maintained over time, and with a predictably structured temporal output even in the face of variable stimulus onset times. As for B, except that the slow oscillation (9 Hz) was included. Also, stimulus onset was random (equal probability) from 150 to 250 ms. iii) shows a polar phase plot of firing probability relative to the slow oscillation. D. As for C, except that the amplitude of the slow oscillation was halved, such that some lower area cells were never fully suppressed. As a result, lower area synchrony was disrupted, and object representation/segregation was poor. E. The system is also able to switch selected object over time, following a change in stimulus, or, e.g., top down attention. As for C, except that objects of equal contrast (value 1) were used, and input gain varied to mimic top-down spatial attention. Initially, gain was set to match the previously used contrast values (1, 0.95 and 0.9 for the different objects), but after 645 ms, it was switched to prioritise the second object (i.e. 0.95, 1 and 0.9). ii) and iii) show polar phase plots for before and after the gain change respectively.
Fig 6.
LJ-multiplexing–representation of multiple items.
A. The system is capable of representing more objects (more closely matching the proposals of the LJ-multiplexing theory), provided that ensembles auto-inhibit for a longer period, and therefore fire only once per slow cycle. Results from a series of simulations (100 repeats), summarizing activity in the higher area (red for high contrast object, blue for mid, green for low). Parameters as for Fig 3D (i.e. strong global inhibition in the lower area), except that the synaptic decay time constant for local inhibition in the lower area was increased to 20 ms. Following 200 ±50 ms of blank screen, the stimulus (3 objects, see text) was turned on. i) shows a PSTH (spike probability across simulation repeats), ii) is a cross-correlogram (calculated for spikes from all 3 cells), iii) shows a polar phase plot of firing probability relative to the slow oscillation. B. As for A, except that the amplitude of the slow oscillation was halved, such that some lower area cells were never fully suppressed. Again (c.f. F-multiplexing, Fig 5D), lower area synchrony was disrupted and object representation was impaired.
Fig 7.
The feature-pair conjunction model.
A. The lower area of the model is similar to the first model (Fig 2), with retinotopically arranged layers of excitatory and inhibitory cells, but with the addition of layers of excitatory cells selective for stimulus colour (red, green and blue), again duplicated across spatial scales, but without orientation selectivity. The higher area is more significantly modified: in this case, there is no spatial selectivity, with all cells receiving inputs from the full spatial extent of the lower area. Instead, due to restricted connectivity, these cells are selective for particular colour-orientation conjunctions, as indicated diagrammatically. B. Summary of network activity in response to a red-vertical stimulus patch, and a slightly lower contrast blue-horizontal patch, across the same three network states as described earlier (default, F- and LJ-multiplexing), showing the activity across one slow oscillatory cycle for each state (average of 1 s simulated activity, 100 simulation repeats). Plots of mean firing rate vs phase of slow oscillation are shown for activity in the lower area. For the higher area, the probability of firing at a given time point is shown, for each of the correctly activated conjunction cells (red-vertical and blue-horizontal) as well as for either of the false conjunctions (red-horizontal or blue-vertical).
Fig 8.
Robustness of network states to parameter changes.
Colour maps show the network state as a function of pairs of parameters (other parameters set to default values except as stated, 5 repeats of 2 seconds of simulated activity for each condition), with green reflecting poor performance (many false conjunctions), red selective representation of a single item (F-multiplexing), and blue alternation between different items (LJ-multiplexing); see Methods for details. A. With other parameters set to default values, behaviour of the network as the amplitude of global inhibition in the lower and higher areas is varied is similar to that described previously for the first network (Fig 3). Polar phase plots to the right (i–iv) show the actual activity patterns for the indicated parameter values (red, high contrast item; blue, mid-contrast item; black, false conjunctions). B. With less feedforward excitation from lower to higher area (120 pS peak synapse amplitude, reduced from 135), the pattern of network states shifts down and to the left. Polar plots below the colour map show activity patterns: the F-multiplexing state is well maintained, but LJ-multiplexing performs poorly, with a tendency to skip gamma cycles. C. With the same feedforward excitation as in B, but with the time constants of synaptic decay for global inhibition in the lower and higher areas reduced (to 3.0 and 2.4 ms respectively), something like the original pattern of results in A is restored. D. With these parameters fixed, and the amplitude of global inhibition set to 0 nS (i.e. taking a vertical slice through point i in panel C), the F-multiplexing state is seen to be maintained over a broad range of values as the amplitude and decay time constant of global inhibition in the higher area are covaried, that is, the activity is not tightly dependent on precise parameter values. E. Similarly, with higher area global inhibition set to zero (taking a horizontal slice through point ii in panel C), the amplitude and decay time constant of global inhibition in the lower area can be covaried and the LJ-multiplexing state maintained.
Table 1.
Cellular parameters for excitatory and inhibitory (including global) cells.
Table 2.
These are indexed according to the connected cells (excitatory E, inhibitory I and global G), e.g. E-I indicates the synapses from excitatory to inhibitory cells. Where relevant, the amplitude indicates the maximum synaptic amplitude between nearest neighbours, e.g. before applying the 2-D Gaussian weighting for distance.