Figure 1.
Experimental paradigm and storage process.
(A) The experimental paradigm. First, an array of items was shown for 1000ms, followed by a 1000ms blank screen. Next, a probe with the colour of one of the items in the array was presented, but at a random orientation. Subjects had to adjust the orientation of the probe item to match that of the relevant item in the original array. (B) Graphical model of the storage process. Several items i, each composed of two features (here, orientation ϕi and colour ψi), eliciting individual responses xi in a neuronal population code, are combined together additively to form a final memory state yN.
Figure 2.
Distribution of errors in human subjects.
Top: Probability of errors between recalled and target colour (this particular experiment cued the location and required colour to be recalled), for 1, 2, 4 and 6 items (shown simultaneously). One can see that the tail of the distribution grows when an increasing number of items is stored. Bottom: Errors relative to non-target values presented in each array. Any bias towards 0 indicates misbinding. Error bars show one standard error of the mean, for 8 subjects. A resampling-based estimation of the probability of misbinding error was performed, and the p-value for a non-zero non-target response component is shown for N = 2, 4 and 6. Misbinding errors are significantly present in all conditions. See Methods for a description of the resampling analysis. The magenta lines (and outline showing standard error of the mean) show histograms obtained from randomly sampling from mixture models derived from the resampling analysis, removing inter-item correlations. Recalculated based on [7].
Figure 3.
Top: Receptive fields of units (one standard deviation), shown for the three different types of population codes: conjunctive, feature and mixed. Bottom: Activity profile over the entire stimulus space for the two shaded units on the left.
Figure 4.
Recall model and posterior for different population codes.
(A) Example memory states for the different population codes, when three items are stored. Coloured circles indicate the veridical feature values. Left: Conjunctive population code, involving little interaction even between nearby items. Middle: Feature population code. Right: Mixed population code — a few conjunctive units provide just enough binding information to recall the features associated with the appropriate items. (B): Cued posterior probabilities, given the veridical colour to be recalled (the three curves correspond to cueing the three possible colours; vertical bars indicate the true stored orientations). (C) Graphical model representation of the process of recall. The final memory state and colour are observed; the orientation must be inferred.
Figure 5.
Distribution of errors of the model.
The model is capable of recreating error distributions seen in the literature, such as those shown in Fig. 2. (Top row) Distribution of errors around the target angle. The central bump is at 0o, showing that recall is normally accurate. The distribution has a non-zero baseline which combines all sources of error. (Bottom row) Distribution of errors relative to all non-target angles. A central tendency in those plots has been interpreted as supporting evidence for the presence of misbinding errors in the responses. Histogram computed on 5000 samples of the model (no standard deviation is shown as all samples are equally probable) The p-values for a resampling analysis of the non-target mixture proportion are shown in each panel. The null hypothesis of no misbinding error can be significantly rejected for all item numbers. The magenta curves represent the resampling-based histograms assuming no misbinding error. Mixed population, M = 200, σx = 0.25.
Figure 6.
Mixed population code. This shows a qualitative fit of the model (green; the shaded area represents one standard deviation) to the human experimental data (blue; data from [7]). M = 144, conjunctivity ratio = 0.85, σx = 0.1). Inset: similar data fits, for [1] (M = 200, ratio = 0.85, σx = 0.4). Observe the different decrease in memory fidelity for an increasing number of items.
Figure 7.
Fisher information fit for one object.
Comparison between similar metrics: the memory fidelity (fitted κ) of single samples collected for different memory states associated with a single memory state (double the value is shown in dashed blue to take account of the doubly stochastic nature of single sampling); the theoretical Fisher information derived in (8); the large M limit for the Fisher information (35); the average inverse variance of samples from the posterior distribution; and the average curvature of the log-posterior at its maximum. This refers to a Conjunctive population code with M = 200, τ = 4, σx = 0.1, σy = 10−5 and 500 samples.
Figure 8.
Misbinding errors when varying the proportion of conjunctive units.
These plots are based on a mixed population code recalling the orientation of one of two stored items (the correct value is indicated by the red vertical bar). There is a fixed total number of 200 units; the ratio of feature to conjunctive units increases for the graphs going from top to bottom. Left: Average (and standard deviation, shown by the penumbra) of the log-posterior distributions over orientation, given the stored memory states averaged over 1000 instantiations of the noise. If the population code only consists of feature units, the posterior comprises two equal modes the incorrect mode disappears as the fraction of conjunctive units increases. However, feature units improve the localization; as their number decreases, the widths of the posterior modes increases. Right: Distribution of 1000 sampled responses, showing how misbinding errors tend to disappear when sufficient conjunctive information is available. The red (respectively green) vertical lines indicate the target (respectively non-target) item orientation. The red Gaussian curve shows the probabilit distribution of a Gaussian distribution centred at the correct target value and with a standard deviation derived from the Fisher information of the associated population code.
Figure 9.
Memory fidelity and mixture proportions as a function of the ratio of conjunctive units.
(A) Standard deviation of the Von Mises component (in blue) from the mixture model fitted to samples of the model shown in Fig. 8 as a function of the fraction of conjunctive units. The (theoretically-calculated) Fisher information is shown in green for the associated population codes. (B) Mixture proportions of the mixture model fitted on the model samples. This metric is less sensitive to random fluctuations of the samples, and shows that if 50% of the units are conjunctive, then 75% of responses will be correctly associated with the appropriate target angle. (C) P-value for a resampling-based estimation of the probability of the non-target mixture proportion to be different than zero. We see that the null hypothesis of the non-target mixture proportion being zero can be rejected from 70% of conjunctive units and less.
Figure 10.
The hierarchical code comprises two layers: the lower layer receives the input, and is randomly connected to the upper one, which provides (possibly additional) binding information. Bottom: layer one consisting of either a feature population code or a conjunctive population code. Receptive fields of units of a feature population code are shown (one standard deviation). Top: effective receptive fields of three layer two units are shown. Layer two units randomly sample a subset of the activity of layer one units, and pass a weighted sum of their inputs through a nonlinearity.
Figure 11.
Memory fidelity and misbinding errors as function of conjunctivity in hierarchical population code.
Left: Memory fidelity based on model samples, while varying the ratio of lower to upper layer units in a hierarchical population code with a constant number of 200 units. The number of (randomly placed) items increases from top to bottom. The memory fidelity decays with increasing item number and conjunctivity. Right: Mixture proportions based on model samples. For a single item, the correct target angle is always retrieved (blue curve). The drop for high ratio of upper to lower layer is expected, as few units are left in the lower layer to represent the item appropriately. For increasing numbers of items, nontarget responses are prevalent (green curve), but including a suitable proportion of upper layer units does allow the appropriate angle to be retrieved. Random responses are marginal with the parameters used here. M = 200, σx = 0.2.
Figure 12.
Memory curve fit for hierarchical population code.
Model fit (green; the penumbra represents one standard deviation) to the human experimental data (blue; data from [7]). These qualitative fits are similar to those obtained for a mixed population code (see Fig. 6), despite the significantly different implementation. (M = 200, ratio = 0.9, σx = 0.3). Inset: fit for [1]. Notice the difference in performance for large number of items. (M = 200, ratio = 0.9, σx = 0.55)
Figure 13.
Error types for different population codes.
The graphs quantify different sorts of error in terms of the weights in a mixture model capturing local variability around an item, misbinding errors and random choices [7]. Human experimental curves are shown on the bottom right. This shows how misbinding errors are crucial components to fit human performance. Conjunctive population code: M = 225 units, σx = 0.3, Feature population code: M = 100 units, σx = 0.08, Mixed population code: M = 144 units, conjunctivity ratio = 0.85, σx = 0.1
Figure 14.
Stimulus pattern to induce misbinding.
Feature-space representation of three stimuli used to study misbinding errors and characteristics of the population codes. Three items are separated by a distance Δx. This set of items will generate interference patterns as shown by the dotted lines. The circles represent one standard deviation of the receptive field response levels. The green circles represent a population code in which the three stimuli are well separated. The blue circle represents a code for which all the stimuli lie inside a single receptive field and would generate misbinding errors. The target is randomly chosen on each trial as one of the three items.
Figure 15.
Recall of stimuli shown in Fig. 14.
100 individual samples from the model are generated for specific parameters (M = 200, σx = 0.25), mixed (left) or hierarchical (right) population codes and inter-stimulus distances Δx = {0.22, 1} rad. Shaded regions correspond to one standard deviation around the mean over 10 repetitions. Top row: Fitted mixture proportions from a mixture model (with one Von Mises component per target/non-target and a random uniform component, similar to [7]). For small Δx, no amount of conjunctivity can improve the results, indicating intra-receptive field misbinding. For large Δx, there is a change from non-target to target responses as the proportion of conjunctive units increases. The target is randomly chosen for each trial.. Second row: Width of the Von Mises component of the mixture model (represented as the standard deviation corresponding to the fitted concentration κ). The dotted black line corresponds to the distance Δx between items in the stimuli pattern.
Figure 16.
Patterns of errors as a function of stimulus separation for different proportions of conjunctive units.
This shows data as in Fig. 15, but as a function of the varying distances in radians between stimuli in the diagonal pattern, for two mixed populations with 50% and 98% conjunctivity. We compute the ratio between the target mixture proportion and the sum of the target and non-target mixture proportions (in blue). We do the same for a non-target mixture proportion (in green). The black vertical bars show half the size of a conjunctive receptive for each population. We see that for separations smaller than the size of a receptive field, misbinding errors are prevalent. This changes as soon as the pattern of stimuli covers more than one receptive field. The vertical red dashed bar shows twice the size of a receptive field. In this situation, each stimulus occupies one receptive field, and misbinding should rarely occur.