Distinguishing two sources of neural variability in distributional codes

A key challenge for Bayesian theories of sensory processing is linking observable quantities, such as stimuli, neural activity, and behavior, to hypothesized probabilistic computations which occur at a more abstract, computational level (Fig 1A and 1B). Following previous work, we assume that the brain has learned a generative model of its sensory inputs [3, 64, 73–75], and that populations of sensory neurons encode posterior beliefs over latent variables in the model conditioned on sensory observations: a hypothesis we refer to as “posterior coding.” The responses of such neurons necessarily depend both on information from the sensory periphery, and on relevant information in the rest of the brain forming prior expectations. In a hierarchical model, likelihoods are computed based on feedforward signals from the periphery, and contextual expectations are relayed by feedback from other areas [64] (Fig 1B).

In our notation, E is the variable observed by the brain—the sensory input or evidence—and x is the (typically high-dimensional) set of latent variables. I is a high-dimensional vector representing all other internal variables in the brain that are probabilistically related to, and hence determine “expectations” for x. Note that the term “prior” is often overloaded, referring sometimes to stationary statistics learned over long time scales, and sometimes to dynamic changes in the posterior due to higher-level inferences or internal states. Therefore, we refer to the dynamic effect of internal states on x as “expectations”. The brain’s internal model of how these variables are related, p_b(x, E, I), gives rise to a posterior belief, p_b(x|E, I), that we assume to be represented by the recorded neural population under consideration. In a typical experiment, stimuli are parameterized by a scalar, s. For instance, when considering the responses of a population of neurons in primary visual cortex (V1) to a grating, E(s) is the grating image on the retina with orientation s, and x has been hypothesized to represent the presence or absence of Gabor-like features at particular retinotopic locations [76] or the intensity of such features [72, 77] (Fig 1B). We will return to this example throughout, but our results derived below are largely independent of the exact nature of x. In higher visual areas, for instance, x could be related to the features or identity of objects and faces [74, 75]. In these examples, I represents all other internal information that is relevant to x, such as task-relevant beliefs, knowledge about the context or visual surround, etc.

The rules of Bayesian inference allow us to derive expressions for structure in posterior distributions as the result of learning and inference. Importantly, the rules of probability apply to the relationships between abstract computational variables such as E, x, I, and their distributions, and not generally those between neural responses implementing those computations; it is a conceptually distinct step to link variability in posteriors to variability in neural responses encoding those posteriors. We use ‘’ to denote the encoding from distributions over internal variables x into neural responses (Fig 1D) [45]. For reasonable encoding schemes , the chain rule from calculus applies: small changes in the encoded posterior result in small changes in the expected statistics of neural responses (Fig 1E, Methods). Using f_i to denote the mean firing rate of neuron i, we can express its sensitivity to a change in stimulus, s, as (1) where 〈⋅, ⋅〉 is an inner product in the space of distributions over x. In general, this expression should average over variability in the posterior due to sources other than s. For now, we are suppressing this extra “noise” for the sake of exposition, but will return to it later in Results. The second term in brackets is the change in the posterior as s changes, and the first term relates those changes in the posterior to changes in the neuron’s firing rate. Notice that Eq (1) bridges two levels of abstraction, from Bayesian inference (what happens to the posterior over x as s is changed) to neural activity (how these changes to the posterior manifest as observable changes to neural activity).

It follows that, in the Bayesian inference framework, there are two distinct sources of neural variability acting at different levels of abstraction: variability in the encoding of a given posterior (Fig 2A–2C), and variability in the posterior itself (Fig 2D–2F) [78].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Two distinct sources of neural co-variability in the Bayesian inference framework: Stochastic encoding of a fixed posterior (a-c) and variability in the posterior itself (d-f).

a) Consider the case where there is no variability in I or E and inference is exact (indicated by straight arrows from E and I to p_b), but posteriors are noisily realized in neural responses r (indicated by a zigzag arrow from p_b to r). b) Exact inference always produces the same posterior for x for fixed E and I. c) The neural encoding of a given distribution may be stochastic, so a single posterior (b) becomes a distribution over neural responses r. The shape of this distribution may or may not relate to the shape of the posterior in (b), depending on the encoding (e.g. there is a correspondence in sampling, but not in parametric codes). d) Both sensory noise and variable expectations induce trial-to-trial variability in the posterior itself (indicated by zigzag arrows from E and I to p_b). This variability in the encoded posterior adds to variability in the encoding, as in (a-c), and can be understood as affecting the average or expected neural responses (indicated by a straight arrow from p_b to r). e) Variability in the posterior can be thought of as a distribution of possible posteriors. f) Each individual posterior in (e) is a point in the space of expected statistics of r, such as expected spike counts. Variability in the underlying posterior may appear as correlated variability in spike counts.

https://doi.org/10.1371/journal.pcbi.1009557.g002

Distributional coding schemes [3, 5, 45, 47] typically assume that a given posterior may be realized in a distribution of possible neural responses, which we refer to as variability in the encoding (Fig 2A–2C). For instance, it has been hypothesized that neural activity encodes samples stochastically drawn from the posterior [34, 48–55, 69, 79]. Alternatively, neural activity may noisily encode fixed parameters of an approximate posterior [7, 58, 60–62, 80, 81]. Such distributional encoding schemes are reviewed in [3, 5, 6, 46, 47]. Previous work has linked (co)variability in neural responses to sampling-based encoding of the posterior [4, 34, 48, 52, 55, 67–69]. Our results are complementary to these; here we study trial-by-trial changes in the posterior itself, and how these changes affect the expected statistics of neural responses such as mean spike count and noise correlations of neural responses. Importantly, our results apply to a wide class of distributional codes including all of the above (Methods).

Trial-by-trial variability in the encoded posterior is an additional source of neural variability above and beyond variable encoding of a fixed posterior discussed above (Fig 2D–2F). There are two principal causes for variability in the posterior itself. First, there is variability in the observation E, or in neural signals relaying forward information about E, which induce variability in the likelihood [82–85]. Second, there is variability in internal states that may influence sensory expectations [37, 86]. Our initial results focus on the variability in the posterior due to variability in task-relevant beliefs or expectations [34, 86] since our primary goal is to understand task-specific influences on neural responses. Such variable expectations may reflect a stochastic approximate inference algorithm [48] or model mismatch, for example if the brain picks up on spurious dependencies in the environment as part of its model [78, 87–89]. Later, we will describe the effect of task-independent sources of variability in the posterior (“noise”), and how it is shaped by learning a task. In the remainder of this paper, we make these ideas explicit for the case of two-choice discrimination tasks for which much empirical data exists.

Inference and discrimination with arbitrary sensory variables

In two-choice discrimination tasks, stimuli are classically parameterized along a single dimension, s, and subjects learn to make categorical judgments according to an experimenter-defined boundary which we assume to be at s = 0. We will use C ∈ {1, 2} to denote the two categories, corresponding to s < 0 and s > 0. While our analytical results hold for discrimination tasks using other stimuli, for concreteness throughout this paper, our running example will be of orientation discrimination, in which case s is the orientation of a grating with s = 0 corresponding to horizontal, and C refers to clockwise or counter-clockwise tilts (Fig 3A). While our derivations make few assumptions (see Methods) about the nature of the brain’s latent variables, x, our illustrations will use the example of oriented Gabor-like features in a generative model of images (Figs 1B and 3A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3.

a) In a discrimination task, the brain performs inference over its latent variables (p_b(x|…)) and trial category ((p_b(C|…))) conditioned on the sensory observation (E). We will first focus on the subject’s graded belief about the binary category, written as π ≡ p_b(C = 1|…), and ignore the influence of other internal states that are part of I (or assume they are fixed), before returning to them later. Implicitly, all inferences are with respect to an internal model p_b (black arrows). A Bayesian observer learns a joint distribution between x and C, implying bi-directional influences during inference: x → π is analogous to “decoding,” while π → x conveys task-relevant expectations. b) Visualization of how prior expectations (top row) and likelihood (bottom row) contribute to the posterior (middle row), with x as a one-dimensional variable. Changes to s change the likelihood (middle column). Changes in expectation, π, imply changes in expectations (right column). Crucially, changes in the posterior in both cases (middle row) are approximately equal (gray arrow) as explained in the text.

https://doi.org/10.1371/journal.pcbi.1009557.g003

Whereas much previous work on perceptual inference assumes that the brain explicitly infers relevant quantities defined by the experiment [2, 58, 60, 90], we emphasize the distinction between the external stimulus quantity being categorized, s, and the latent variables in the subject’s internal model of the world, x. For the example of orientation discrimination, a grating image E(s) is rendered to the screen with orientation s, from which V1 infers an explanation of the image as a combination of Gabor-like basis elements, x. The task of downstream areas of the brain—which have no direct access to E nor s—is to estimate the stimulus category based on a probabilistic representation of x (Fig 3A) [34, 91]. Crucially it is the posterior over x, rather than over s, which we hypothesize is represented by sensory neurons.

Task-specific expectations and self-consistent inference

Probabilistic relations are inherently bi-directional: any variable that is predictive of another variable will, in turn, be at least partially predicted by that other variable. In the context of perceptual decision-making, this means that sensory variables, x, that inform the subjects’ internal belief about the category, C, will be reciprocally influenced by the subject’s belief about the category (Fig 3A). Inference thus gives a normative account for feedback from “belief states” to sensory areas: changing beliefs about the trial category entail changing expectations about the sensory variables whenever those sensory variables are part of the process of forming those categorical beliefs in the first place [34, 64, 86, 92].

A well-known self-consistency identity for probabilistic models is that their prior is equal to their average inferred posterior, assuming that the model has learned the data distribution ([3, 4, 93]; Methods). We can use this identity to write an expression for the optimal prior over x upon learning the statistics of a task (Methods): (2)

Eq (2) states that, given knowledge of an upcoming stimulus’ category, C = c, the optimal prior on x is the average posterior from earlier trials in the same category [94].

Throughout, we use the subscript ‘b’ to refer to the brain’s internal model, and the subscript ‘e’ to refer to the experimenter-defined model (Methods). To use the orientation discrimination example, knowing that the stimulus category is “clockwise” increases the expectation that clockwise-tilted Gabor features will best explain the image, since they were inferred to be present in earlier clockwise trials. Importantly, Eq (2) is true regardless of the nature of x or s. It is a self-consistency rule between prior expectations and posterior inferences that is the result of the brain having fully learned the statistics of its sensory inputs in the task, i.e. p_b(E) = p_e(E) ([3, 4, 93]; Methods). This self-consistency rule allows us to relate neural responses due to the stimulus (s) to neural responses due to internal beliefs (π) without specific assumptions about x.

In binary discrimination tasks, the subject’s belief about the correct category is a scalar quantity, which we denote by π = p_b(C = 1|…). Given π, the optimal expectations for x are a correspondingly graded mixture of the per-category priors: (3)

The posterior over x for a single trial depends on both the stimulus and belief for that trial: (4)

We assume beliefs about C (i.e. π) and x are represented by separate neural populations, for instance if π is represented in a putative decision-making area of the brain while x is in sensory cortex; however, their relation to each other via anatomically “feedforward” or “feedback” pathways is not crucial for the main result except in the context of designing specific experimental interventions. For simplicity, we will say task-related expectations “feed back” to x, as this is the common and familiar case in which x stands for whatever is represented by sensory cortex. We will next derive the specific pattern of neural correlated variability when π varies.

Variability in the posterior due to changing task-related expectations

Even when the stimulus is fixed, subjects’ beliefs and decisions are known to vary [8]. Small changes in a Bayesian observer’s categorical belief (Δπ) result in small changes in their posterior distribution over x, which can be expressed as the derivative of the posterior with respect to π: where s = 0 and indicate that the derivative is taken at the category boundary where an unbiased observer’s belief is ambivalent. Note that a 50/50 prior at s = 0 is not required, and our results would still hold after replacing all instances of with whatever the observer’s belief is at s = 0.

Our first result is that this derivative is approximately proportional to the derivative of the posterior with respect to the stimulus. Mathematically, the result is as follows: (5) where the symbol should be read as “approximately proportional to” (see Methods for proof). This result is visualized in one dimension in Fig 3B: small changes in categorical expectation (±Δπ) and small changes in the stimulus (±Δs) result in strikingly similar changes to the posterior over x.

Eq (5) states that, for a Bayesian observer, small variations in the stimulus around the category boundary have the same effect on the inferred posterior over x as small variations in their categorical beliefs. The proof makes four assumptions: first, the subject must have fully learned the task statistics, as specified by equations (2) and (3). Second, the two stimulus categories must be close together, i.e. the task must be near or below psychometric thresholds. Third, variations of stimuli within each category must be small.

We further discuss these conditions and possible relaxations in S1 Text, and visualize them in S1 and S2 Figs. Finally, we have assumed that there are no additional noise sources causing the posterior to vary; we consider the case of noise in the section “Effects of task-independent noise” below.

Feedback of variable beliefs implies differential correlations

“Information-limiting” or “differential” correlations refer to neural (co)variability that is indistinguishable from changes to the stimulus itself, i.e. variability in the f′ ≡ df/ds direction for a fixed s. For a pair of neurons i and j, differential covariance is proportional to [16]. Covariance is transformed to correlation by dividing by the square root of the product of both neurons’ variances, σ_i σ_j, which gives an expression for differential correlation proportional to , where is the normalized “d-prime” neurometric sensitivity of neuron i [95].

Applying the “chain rule” in Eq (1) to Eq (5), it directly follows that (6) implying that the effect of small changes in the subject’s categorical beliefs (π) is approximately proportional to the effect of small changes in the stimulus on the responses of sensory neurons that encode the posterior. Both induce changes to the mean rate in the f′ ≡ df/ds direction.

Importantly, when a subject is trained on multiple tasks and given a cue about the task context, the influence of their categorical expectations on posterior-coding neurons should then depend on the cue, since the cue informs their expectations of possible stimuli. Below, this task-dependence of f′ will enable strong experimental tests of this theory using a cued task-switching paradigm.

A direct consequence of Eq (5) is that variability in π adds to neural covariability in the f′−direction above and beyond whatever intrinsic covariability was present before learning. We obtain, to a first approximation, the following expression for the noise covariance between neurons i and j: (7) where Σ^intrinsic captures “intrinsic” noise such as Poisson noise in the encoding. It follows from (6) that (8)

Interestingly, this is exactly the form of so-called “information-limiting” or “differential” covariability [16]. In the feedforward framework, differential covariability arises due to afferent sensory or neural noise and limits the information about s in the population [16, 30, 33]. This source of differential correlations is captured by Σ^intrinsic and is assumed to be always present, independent of task context. Here, we highlight an additional source of differential correlations due to feedback of variable beliefs about the stimulus category. Unless these beliefs are true, or unless downstream areas have access to and can compensate for π, the differential covariability induced by π limits information like its bottom-up counterpart ([27, 30, 37]; also see Discussion). Importantly, and unlike the feedforward component of differential covariability, the feedback differential covariability predicted here arises as the result of task-learning and is predicted to increase while behavioral performance in the task improves, and to change depending on the cued task context in a task-switching paradigm.

Feedback of variable beliefs implies choice probabilities aligned with stimulus sensitivity

A direct prediction of the feedback of beliefs π to sensory areas, according to (6), is that the average neural response preceding choice 2 will be biased in the + f′ direction, and the average neural response preceding choice 1 will be biased in the −f′ direction, since the subject’s behavioral responses will be based on their belief, π. Excess variability in π, e.g. due to spurious serial dependencies, or simply approximate inference, will therefore introduce additional correlations between neural responses and choices above and beyond those predicted by a purely feedforward “readout” of the sensory neural responses [8, 19, 20, 31, 32, 34, 96]. Our results predict that this top-down component of choice probability should be proportional to neural sensitivity: (9) where is the “d-prime” sensitivity measure of neuron i from signal detection theory [95] (Methods). Interestingly, the classic feedforward framework makes the same prediction for the relation between neural sensitivity and choice probability assuming an optimal linear decoder [19], raising the possibility that, contrary to previous conclusions, the empirically observed relationship between CPs and neural sensitivity that emerges over learning [97] is not just due to changes in the feedforward read-out as commonly assumed [8, 98] but is instead the result of changes in feedback indicating variable beliefs.

Effects of task-independent noise

The above results assumed no measurement noise nor variability in other internal states besides the relevant belief π. Just as in the previous sections, we will distinguish variability in the encoded posterior, to which we can apply the self-consistency constraint of Bayesian inference, from neural variability due to the neural representation of a fixed posterior through . In the presence of noise, the posterior itself changes from trial to trial even for a fixed stimulus s and fixed beliefs π [83]. To study the consequences of this added variability, we introduce a variable, ∊, that encompasses all sources of task-independent noise on each trial, and condition the posterior on its value: p(x|E(s), π; ∊) (Methods). Following our decomposition of variance in Fig 2D, this noise, like π, acts on the encoded posterior rather than on the neural responses directly. Unlike π, ϵ is by definition task-independent (Methods). This impacts our initial results in two principal ways, laid out in the following two sections: first, although ideal learning and self-consistent inference still implies that the average posterior equals the prior (Eq (2)), the average must now be taken over both s and the distribution of noise p(∊). This reduces the generality of our main result on the proportionality of and (Eq (6)), but leads us to consider a new classification of linear distributional codes (LDCs) with interesting robustness properties to noise. Second, task-independent noise can be amplified or suppressed by a task-dependent prior, which we also find to have a task-dependent effect on neural covariability that sometimes, but not always, increases differential correlations.

Interactions between task-independent noise and task-dependent priors.

Although we assumed that noise ϵ arises from task-independent mechanisms, it is nonetheless shaped by task learning: task-independent noise in the likelihood interacts with a task-specific prior to shape variability in the posterior (Fig 4). Below, we show that this mechanism can change task-dependent differential correlations in neural responses even if a subject’s beliefs (π) do not vary. This idea is reminiscent of recent mechanistic models of neural covariability in which a static internal state, such as attention to a location in space, influences the dynamics of a recurrent circuit, allowing variation in some directions while suppressing others [100, 101]. Here, we investigate what happens when the brain employs a static prior over stimuli for a given task, finding that noise is selectively suppressed in certain directions by the prior.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Numerical simulation of how variable likelihoods both determine and interact with the shape of the prior. Top two rows: mean of observations E depends on s as shown in panel (a). Bottom two rows (gray background): covariance of observations E depends on s as shown in panel (l). a) In this setup, observations are noisy, reflecting both internal and external noise. We set E to be two-dimensional, drawn from p_e(E|s), a Gaussian whose mean depends on s (black curve). Ellipses indicate one standard deviation of p_e(E|s). b) Now visualizing the space of latent variables, x, which we also set to be two-dimensional, such that the brain’s internal model assumes p_b(E|x) is Gaussian, centered on x, with the shown variance. Each red circle shows a single contour of the likelihood, as a function of x, for a different E drawn from the p_e(E|s = 0) set of observations. c) After learning, the prior is extended along the curve that parameterizes the mean of x, such that the brain’s distribution on observations, p_b(E) marginalized over x, approximates the true distribution on observations, p_e(E) marginalized over s (Methods). d) Posteriors in the zero-signal or s = 0 case, given by the product of the likelihoods in (b) with the prior in (c). e) The direction in distribution-space corresponding to differential covariance in neural-space is the dp_b/ds-direction, averaged over instances of noise. f) The fraction of variance in distribution-space (d) along the dp_b/ds-direction. After learning, a larger fraction of the total variance is in the dp_b/ds-direction, corresponding to increased differential correlations in neural space. g-k) Identical to (b) through (f), except that the brain’s internal model is more precise relative to the variations in observations E, modeled as smaller variance for p_b(E|x). This has the effect of reducing the effect of learning a new prior in panel (k). l-v) Identical to (a) through (k), except that here, s parameterizes the covariance of observations E rather than their mean. This is analogous to, for example, an orientation-discrimination task with randomized phases, such that the average stimulus in pixel-space is identical for both categories. While panel (v) (precise model) shows a slight increase in variance in the dp_b(x|…)/ds direction, consistent with (f) and (k), panel (q) (imprecise model) shows a slight decrease.

https://doi.org/10.1371/journal.pcbi.1009557.g004

We again study the trial-by-trial variability in the posterior itself as opposed to the shape or moments of the posterior on any given trial. Formally, we study the covariance due to noise (ϵ) in the posterior density at all pairs of points x_i, x_j, i.e. Σ_ij ≡ cov(p_b(x_i|…), p_b(x_j|…)). We have shown (Methods) that, to a first approximation, the posterior covariance is given by a product of the covariance of the task-independent noise in the likelihood, Σ^LH(x_i, x_j), and the brain’s prior over x_i and x_j: (10)

The effect of learning a task-dependent prior in Eq (10) can be understood as “filtering” the noise, suppressing or promoting certain directions of variability in the space of posterior distributions.

Differential correlations emerge from this process if variability in the dp_b(x|…)/ds-direction is less suppressed than in other directions. Whether this is the case, and to what extent, depends on the interaction of s and x, an analytic treatment of which we leave for future work. Here, we present the results from two sets of numerical simulations: one in which the mean of x depends on s (Fig 4A–4K) and one in which the covariance of x depends on s (Fig 4L–4V).

In these simulations, we assume both x and E to be two-dimensional, with isotropic Gaussian likelihoods over E given x in the brain’s internal model. To connect back to our earlier examples of visual discrimination tasks, this two-dimensional stimulus space can be thought of as a simplified two-pixel image. We fixed the brain’s internal likelihood p_b(E|x) over learning, and modeled the learning process as gradient descent of a loss function that encourages the brain’s model of observations to approach their true distribution, i.e. changing the prior p_b(x) so that the marginal likelihood p_b(E) moves towards the true distribution of stimuli in each experiment p_e(E) (Methods). Importantly, learning such a correspondence between p_e(E) and p_b(E) in the stimulus space is a sufficient condition for self-consistent beliefs about one’s internal latent variables, x [4]. In these simulations, the brain’s posterior p_b(x|…) varies for a fixed s because different values of E are provided for the same s; this variability in E given s is the result of both external and internal noise sources. In the first set of simulations, the mean of the observations depended on s (Fig 4A). Small variations in s around the boundary s = 0 primarily translated the posterior, resulting in a two-lobed dp_b(x|…)/ds structure (Fig 4E and 4J). After learning, the prior sculpted the noise such that trial-by-trial variance in posterior densities was dominated by translations in the dp_b(x|…)/ds-direction (Fig 4F and 4K).

The intuition behind this first set of results is as follows. During learning, both uninformative s = 0 and informative s < 0 or s > 0 stimuli are shown. As a result, the learned prior (equal to the average posterior) becomes elongated along the curve that defines the mean of the likelihood (Fig 4C and 4H), which is also the direction that defines dp_b(x|…)/ds (Fig 4E and 4J). After learning, if noise in E happens to shift the likelihood along this curve, then the resulting posterior will remain close to that likelihood because the prior remains relatively flat along that direction. In contrast, noise that changes the likelihood in an orthogonal direction will be “pulled” back towards the prior (Fig 4B–4D). Thus, multiplication with the prior preferentially suppresses noise orthogonal to dp_b(x|…)/ds. Applying the chain rule from Eq (1), this directly translates to privileged variance in the differential or f′f′^⊤ direction in neural space. As shown in Fig 4G–4K, the magnitude of this effect further depends on the width of the brain’s likelihood relative to the width of the prior. We find that decreasing the variance (increasing the precision) of p_b(E|x) dramatically attenuates the change (Fig 4K). Intuitively, this is because any effect of changing the prior is less apparent when the likelihood is narrow relative to the prior. Importantly, both scenarios occur in common perceptual decision-making tasks [102].

While these first results are intuitive, they rely, in part, on the assumption that the primary effect of s is to translate the posterior on x. In a second set of simulations, we investigated what happens when s determines the covariance of E rather than its mean (Fig 4L–V). This is analogous to an orientation discrimination task with randomized phases, since the mean image is identical in both categories, so the category is determined by coordinated changes in “pixels.” Otherwise, the brain’s internal model, the learning procedure, and noise were identical to the first set of simulations. In the case of relatively narrow likelihoods, we again found that the fraction of variance in the dp_b(x|…)/ds-direction slightly increased after learning (Fig 4V), consistent with the first set of results. Surprisingly, we found the reverse effect—a slight decrease in the fraction of variance in the dp_b(x|…)/ds-direction, when the brain’s likelihood was relatively wide (Fig 4Q). While the first set of results—where s primarily translates the posterior on x—appear robust, this second set of results indicates that the interaction between Bayesian learning and noise is subtle, and whether it results in an increase or decrease in the (relative) variability along the f′ direction in neural space depends on the particular relationship between s, E, and x. We leave a further exploration of this interaction to future work.

Note that whereas our results on variability due to the feedback of variable beliefs implied an increase in neural covariance along the f′f′^⊤-direction over learning, the effect of “filtering” the noise does not necessarily increase nor decrease variance, even in cases where the fraction of variance along dp_b(x|…)/ds increases. Adapting a prior to the task may affect both total and relative amounts of variance in the task-relevant direction, depending on the brain’s prior at the initial stage of learning.

Connections with empirical literature

To summarize, we have identified three signatures of Bayesian learning and inference that should appear in neurons that encode the posterior over sensory variables: (i) a feedback component of choice probabilities proportional to normalized neural sensitivity (Eq (9)), (ii) a feedback component of noise covariance in the “differential” or f′f′^⊤ direction (Eq (8)), and (iii) additional structure in noise correlations due to the “filtering” of task-independent noise by a task-dependent prior. We emphasize that our results only describe how learning a task changes these quantities, and makes no predictions about their structure before learning. In this section, we highlight previous literature that has isolated changes in choice probabilities and noise correlations associated with a particular task, whether over learning or due to task-switching.

Eq (9) predicts that the top-down component of choice probability should be proportional to the vector of normalized neural sensitivities to the stimulus.

Indeed, such a relationship between CP and d′ was found by many studies (reviewed in [96, 103, 104]). For instance, Law and Gold (2008) demonstrated a relationship between neurometric sensitivity (quantified by neurometric threshold, proportional to 1/d′) and choice probability that only emerged after extensive learning. However, this finding can also be explained in a purely feedforward framework assuming an optimal linear decoder [19, 98]. Thus, the empirical finding that choice probabilities are often higher for neurons with higher neurometric sensitivity supports both the theories of feedforward optimal linear decoding and our theory based on feedback of a categorical belief, and in general CP will reflect a mix of both feedforward and feedback mechanisms [35]. The relative importance of these two mechanisms in producing CPs remains an empirical question. For much stronger support for our theory, we will therefore focus on empirical data on noise correlations, whose task-dependence is much harder to explain using feedforward mechanisms, and for which no feedforward explanation has been proposed.

Two existing studies have isolated task-dependent component of noise covariance by holding the stimulus constant while switching between two comparable tasks that a subject is performing, thus altering their task-specific expectations while minimally affecting the feedforward drive. The difference in neural response statistics to a stimulus that is shared by both tasks isolates the task-dependent component to which our results apply. Bondy et al [27] recorded from neural populations in macaque V1 while the monkeys switched between different coarse orientation tasks. After removing the shared component of noise correlations between tasks, the task-specific component was found to align with d′d′^⊤ structure as predicted by Eq (8) (Fig 5A) (note that proportionality between covariance and f′f′ is equivalent to a proportionality between correlation and d′d′ after dividing both sides by σ_i σ_j). While this is encouraging for our theory, feedforward explanations of the results of [27] cannot be entirely ruled out because of the long retraining time between tasks. Cohen and Newsome [22] recorded from pairs of neurons in area MT while two monkeys switched discrimination tasks on a trial-by-trial basis and found that correlations also changed in a manner that is consistent with but not definitive of f′f′^⊤ structure (see Box 2 in [37]). By aligning the task to the recorded neurons, the results of [22] only measure a narrow slice of the full set of possible covariances (roughly and in the two tasks). Building on these studies, a stronger test for task-dependent f′f′^⊤−covariance structure would be to interleave tasks on a trial-by-trial basis (as done by [22]) while recording from a large and random set of neurons (as done by [27]). A strong test of the dependence of structured noise covariance on feedback would be to causally manipulate cortico-cortical feedback during such a task, e.g. by cooling.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Examples of predicted effects in existing empirical literature.

Corr denotes correlation, and is the normalized sensitivity of neuron i defined as . a) Data replotted from Fig 5c of [Bondy et al. (2018)], who isolated the task-dependent component of noise correlations in macaque V1 and found a strong relation between elements of this correlation matrix and neural sensitivities (r = 0.61, p < 0.001, from original paper). This relationship between Corr_ij and follows from the relationship between Σ_ij and as in Eq (8). b) Data replotted from Fig 2d of Rabinowitz et al. (2015) [26], who found that the strength of top-down ‘modulator’ weights is linearly related to d′. c) Illustration of the task-dependent and arbitrary nature of f′: each row of grating stimuli define a different discrimination task, with each task defining a different f′ vector in neural space. For instance, relative to fine discrimination around vertical (green), one may modify the spatial frequency (blue), the phase (purple), the reference orientation to horizontal (red), or consider a coarse discrimination between vertical and horizontal (yellow). The span of possible f′ directions is large. Hence, the probability is small of finding that the specific f′ direction corresponding to a given task is aligned with a leading principal component of measured noise covariance, unless this alignment is the result of learning and/or performing the task.

https://doi.org/10.1371/journal.pcbi.1009557.g005

Another approach to differentiate between feedforward and feedback contributions to neural variability is to statistically isolate them within a single task using a sufficiently powerful regression model. [Rabinowitz et al.] [26] used this type of approach to infer the primary top-down modulators of V4 responses in a change-detection task. They found that the two most important short-term modulators were indeed aligned with the vector corresponding to neurometric sensitivity in the task (data replotted in Fig 5B). This is consistent with our predictions if the latent ‘modulator’ reflects variations in the subject’s belief state, which in this case would be a graded belief in the two categories of “change” versus “no change.” Indeed, Rabinowitz et al (2015) further reported that the modulator state correlates with subjects’ decisions.

A final line of evidence comes from simultaneous recordings of large populations of neurons, and analyzing the magnitude of neural variability in the task-specific f′ direction relative to other directions. While it is well established that there is often low-rank structure to noise covariance (hypothesized to be at least partly due to recurrent dynamics [100, 101]), and while it is expected that at least some variance is in the f′−direction due to feedforward noise [33], these in principle need not be aligned. Even in orientation discrimination tasks, the f′ direction depends on arbitrary experimental choices such as the particular discriminanda (e.g. fine discrimination at vertical or horizontal, or coarse discrimination of cardinal or oblique targets) and on other arbitrary features of the stimulus such as contrast, phase, spatial frequency, etc (Fig 5C). This makes the f′ direction largely arbitrary, and implies that finding high variance in the f′ direction for a particular task, out of the many possible f′ directions for other possible tasks, is highly unlikely under the assumption of a fixed, task-independent covariance structure. In light of this, the results of Rabinowitz et al [26], as well as other recent findings of alignment between f′ and the low-rank modes of covariance [27–29], suggests that task information directly enters into the noise covariance structure, e.g. due to feedback of beliefs or the “filtering” of noise by a task-specific prior as we have highlighted here.

Inferring variable internal beliefs from sensory responses

We have shown that internal beliefs about the stimulus induce corresponding structure in the correlated variability of sensory neurons’ responses (Fig 6A). Conversely, this means that the statistical structure in sensory responses can be used to infer properties of those beliefs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Inferring the structure of internal beliefs about a task.

a) Trial-to-trial fluctuations in the posterior beliefs about x imply trial-to-trial variability in the mean responses representing that posterior. Each such ‘belief’ yields increased correlations in a different direction in r. The model in (b-d) has uncertainty in each trial about whether the current task is a vertical-horizontal orientation discrimination or an oblique discrimination. b) Correlation structure of simulated sensory responses during discrimination task. Neurons are sorted by their preferred orientation (based on [34]). c) Eigenvectors of correlation matrix (principal components) plotted as a function of neurons’ preferred orientation. The blue vector corresponds to fluctuations in the belief that either a vertical or horizontal grating is present, and the yellow corresponds to fluctuations in the belief that an obliquely-oriented grating is present. See Methods for other colors. d) Corresponding eigenvalues color-coded as in (c). Our results on variable beliefs (π) predict an increase over learning in the eigenvalue corresponding to fluctuations in belief for the correct task, while our results on filtering noise predict a relative increase in the task-relevant eigenvalue compared with variance in other tasks’ directions (e.g. if both blue and yellow decrease, but yellow more so).

https://doi.org/10.1371/journal.pcbi.1009557.g006

In order to demonstrate the usefulness of this approach, we used it to infer the structure of a ground-truth model [34] from synthetic data. The model discriminated either between a vertical and a horizontal grating (cardinal context), or between a −45deg and +45deg grating (oblique context). The model was given an unreliable (80/20) cue as to the correct context before each trial, and thus had uncertainty about the exact context. We simulated the responses of a population of primary visual cortex neurons with oriented receptive fields that perform sampling-based inference over image features. The resulting noise correlation matrix—computed for zero-signal trials—has a characteristic structure in qualitative agreement with empirical observations (Fig 6B) [27].

Interestingly, the resulting simulated neural responses have five significant principal components (PCs) (Fig 6C and 6D). Knowing the preferred orientation of each neuron allows us to interpret the PCs as directions of variation in the model’s belief about the current orientation. For instance, the elements of the first PC (blue in Fig 6C) are largest for neurons preferring vertical and negative for those preferring horizontal orientation, indicating that there is trial-to-trial variability in the model’s internal belief about whether “there is a vertical grating and not a horizontal grating”—or vice versa—in the stimulus, corresponding to the f′−axis of the cardinal task corresponding to the most likely task in our simulations. Analogously, one can interpret the third PC (yellow in Fig 6C and 6D) as corresponding to the belief that a +45° grating is being presented, but not a −45° grating, or vice versa. This is the f′-axis for the wrong (oblique) task context, reflecting the fact that the model maintained some uncertainty about which was the correct task in a given trial. The remaining PCs in Fig 6C and 6D correspond to task-independent variability (see S3 Fig). This example demonstrates that at least some of a subject’s beliefs about the structure of the task can be read off of the principal directions of variation in neural space, possibly revealing cases of model mismatch, such as finding significant variability corresponding to the “other” task in the third principal component here. An interesting direction for future work will be to compare such an analysis to behavioral data, which can also reveal deviations between the experimenter-defined task and a subject’s model of the task, for instance through the use of psychophysical kernels.

Maintaining uncertainty about the task itself is the optimal strategy from the subject’s perspective given their imperfect knowledge of the world. However, when compared to perfect knowledge of context, it decreases behavioral performance which is optimal when the internal model learned by the subject matches the experimenter-defined one—an ideal which Bayesian learners approach over the course of learning. An empirical prediction, therefore, is that eigenvalues corresponding to the correct task-defined stimulus dimension will increase with learning, while eigenvalues representing other tasks should decrease. Furthermore, the shape of the task-relevant eigenvectors should be predictive of psychophysical task-strategy. Importantly, they constitute a richer, higher-dimensional, characterization of a subject’s decision strategy than psychophysical kernels or CPs [105], and can leverage simultaneous recordings from neuronal populations of increasing size.

Feedback, correlations, and population information

We began by distinguishing two principal ways in which correlated neural variability arises in a Bayesian inference framework. The first is neural variability in the encoding of a fixed posterior, which has previously been studied primarily in the context of neural sampling codes [48, 52, 55, 67, 68]. The second, which we study here, is variability in the posterior itself, which is shaped by two further mechanisms. The first source of variability in the posterior itself that we study comes from variability in task-relevant categorical belief (π), projected back to sensory populations during each trial. We showed that variable categorical beliefs induce commensurate choice probabilities and neural covariability in approximately the f′−direction, assuming the subject learns optimal statistical dependencies. This holds for general distributional codes if noise is negligible and stimuli are narrowly concentrated in the sub-threshold regime. Note that this latter requirement of a narrow stimulus distribution is partly an artifact of our mathematical approach, which was to show perfect alignment between feedback and f′ in a limit, while making as few assumptions as possible about the brain’s internal model or how distributions are encoded in neural activity. We expect our results to degrade gracefully, e.g. near-perfect alignment between decision-related feedback and f′ may be predicted given only mild restrictions on distributional codes and more permissive assumptions about the experimental distribution of stimuli (see S1 Text for further discussion). When noise is non-negligible, the same alignment result additionally holds for a newly-identified class of Linear Distributional Codes (LDCs). A second source of variation in the posterior itself is task-independent noise that interacts with a task-dependent prior. Although not solved analytically, we found in simulation that the task-dependent component of this variability increased differential correlations after learning in three out of four simulated conditions, through the mechanism of suppressing variability that is inconsistent with the task-specific prior. The first set of these simulations highlight the fact that empirically detecting effects of a changing prior requires using stimuli that are “open to interpretation,” i.e. stimuli for which p_b(E|x) is relatively wide [102]. The second set of these simulations showed that an increase in differential correlations is not guaranteed by suppressing noise that is inconsistent with the prior, but depends on the particular relationships between stimuli, noise, and the brain’s internal model. While this limits the power of this noise-suppression mechanism to predict differential correlations that increase with learning, such a mechanism may nonetheless play an important role in explaining existing data. We hope that future work on this noise-suppression mechanism will lead to new empirically-decidable signatures of particular kinds of internal models.

Both of these two sources of task-dependent posterior variability depend on feedback: in one case there is dynamic feedback of a particular belief π each trial, and in the other case there is task-dependent (but belief-independent) feedback that sets a static prior each trial, then interacts with noise in the likelihood, analogous to models of “state-dependent” recurrent dynamics [100, 101, 106].

Our results directly address several debates in the field on the nature of feedback to sensory populations. First, they provide a function for the apparent ‘contamination’ of sensory responses by choice-related signals [17, 26, 27, 31, 32, 44]: top-down signals communicate task-relevant expectations, not reflecting a choice per se but integrating information in a statistically optimal fashion as previously proposed [64, 86]. Second, if this feedback is variable across trials, reflecting the subject’s variable beliefs, it induces choice probabilities that are the result of both feedforward and feedback components [31, 32, 34, 107].

Our results suggest that at least some of measured “differential” covariance may be due to optimal feedback from internal belief states, or as the interaction between task-independent noise and a task-specific prior. In neither case is information necessarily more limited as the result of learning, despite an increase in so-called “information limiting” correlations. In the first case, while feedback of belief (π) biases the sensory population, that bias might be accounted for by downstream areas [21, 30]. In principle, these variable belief states could add information to the sensory representation if they are true [37] which is not the case in psychophysical tasks with uncorrelated trials that deviate from the temporal statistics of the natural world [87]. In the second case, the noise in the f′−direction does limit information, but to the same extent as before learning; there is not necessarily further reduction of information by “shaping” the noise with a task-specific prior. For a fixed population size, it is covariance in the f′−direction, not correlation, that ultimately affects information. Both of these possibilities call for caution when interpreting studies that estimate the information content of sensory populations by estimating the amount of variance (or correlation) in the f′− (or d′−) direction. These insights must be taken into account when interpreting recent experimental evidence for strong differential correlations [28, 29]. Take the recent results of [29], for example, who analyzed V1 calcium imaging data in mouse V1 and found that the f′ direction was among the top 5 principal components among thousands of neurons. We emphasize that these results are highly statistically surprising from the feedforward perspective when considering the arbitrariness of the f′ direction, which corresponds to the specific task in a given experiment (Fig 5C).

Interpreting low-dimensional structure in population responses

Much research has gone into inferring latent variables that contribute to low-dimensional structure in the responses of neural populations [108–110]. Our results suggest that at least some of these latent variables can usefully be characterized as internal beliefs about sensory variables. We showed in simulation that the influence of each latent variable on recorded sensory neurons can be interpreted in the stimulus space using knowledge of the stimulus-dependence of each neuron’s tuning function (Fig 6). Our results are complementary to behavioral methods to infer the shape of a subject’s prior [111], but have the advantage that the amount of information that can be collected in neurophysiology experiments far exceeds that in psychophysical studies allowing for richer characterization of the subject’s internal model [112].

A fresh look at distributional coding

We introduced a general notation for distributional codes, , that encompasses nearly all previously proposed distributional codes. Thinking of distributional codes in this way—as a map from an implicit space p_b(x) to observable neural responses p(r)—is reminiscent of early work on distributional codes [45], and emphasizes the convenience of computation, manipulation, and decoding of p_b(x|…) from r rather than its spatial or temporal allocation of information per se [3, 5, 47]. Our results leverage this generality and show that there are properties of Bayesian computations that are identifiable in neural responses without strong commitments to their algorithmic implementation. Rather than assuming an approximate inference algorithm (e.g. sampling) then deriving predictions for neural data, future work might productively work in the reverse direction, asking what class of generative models (x) and encodings () are consistent with some data. As an example of this approach, we observe that the results of Berkes et al. (2011) are consistent with any LDC, since LDCs have the property that the average of encoded distributions equals the encoding of the average distribution, as their empirical results suggest [4].

Distinguishing between linear and nonlinear distributional codes is complementary to the much-debated distinction between parametric and sampling-based codes. LDCs include both sampling codes where samples are linearly related to firing rate [34, 48–51, 91], parametric codes where firing rates are proportional to expected statistics of the distribution [7, 45, 56, 99]. Examples of distributional codes that are not LDCs include sampling codes with nonlinear embeddings of the samples in r [52, 53, 55, 69], parametric codes in which the natural parameters of an exponential family are encoded [58, 60, 62, 80], as well as “expectile codes” recently proposed based on ideas from reinforcement learning [113, 114].

Our focus on firing rates and spike count covariance is motivated by connections to rate-based encoding and decoding theory. We do not assume that they are the sole carrier of information about the underlying posterior p_b(x|…), but simply statistics of a larger spatio-temporal space of neural activity, r [85, 93]. For many distributional codes, firing rates are only a summary statistic, but they nonetheless provide a window into the underlying distributional representation.

Posterior versus likelihood distributional coding

Probabilistic Population Codes (PPCs) have been instrumental for the field’s understanding of the neural basis of inference in perceptual decision-making. However, they are typically studied in a purely feedforward setting assuming a representation of the likelihood, not posterior [58, 60]. In contrast, Tajima et al. (2016) modeled a PPC encoding the posterior and found that categorical priors bias neural responses in the f′−direction, consistent with our results [70]. This clarifies and formalizes a connection between Tajima et al (2016) and Haefner et al (2016) (who simulated sampling-based hierarchical inference and found excess variance in the f′−direction): the crucial ingredient in both is the feedback of categorical beliefs rather than the choice of sampling or parametric representation per se.

The assumption that sensory responses represent posterior beliefs through a general encoding scheme agrees with empirical findings about the top-down influence of experience and beliefs on sensory responses [64, 107, 115]. It also relates to a large literature on association learning and visual imagery (reviewed in [116]). In particular, the idea of “perceptual equivalence” [117] reflects our starting point that changes in the posterior belief (and hence changes in the percept) can be the result of either changes in sensory inputs or changes in prior expectations. When prior expectations vary, they manifest as correlated neural variability which can be understood in terms of equivalent changes in sensory inputs (Fig 6). Through learning, expectations come to align with past variations in stimuli (Fig 3) leading to testable predictions for the induced structure in neural covariability (Eq (8)).

Attention and learning signals

Our results imply that the increase in alignment with neural sensitivity, f′, for both differential correlations and choice probabilities, depends on the extent to which inference in the brain is approximate. Attention, which is classically characterized as allocating limited resources [118], can be expected to improve the approximation. This could happen either by reducing excess variability in expectations about the stimulus category, or by reducing noise in the likelihood. This would be compatible with empirical data which show that both noise correlations [40–42] and choice probabilities [31] are reduced in high-attention conditions.

Also complementary to our work is a mechanistic network model by [119] in which choice-related feedback was shown to be helpful for the learning of a categorical representation in areas intermediate to sensory areas and decision-making areas. As in our work, correlated variability and decision-related signals emerge as the result of task-specific changes in feedback connections.

[44] provided an alternative rationale for the brain to introduce additional differential correlations by showing that they make it easier for a decision-making area to learn the linear decoding weights to extract task-relevant information from the sensory responses.

Limitations and deviations from assumptions

Our derivations implicitly assumed that the feedforward encoding of sensory information, i.e. the likelihood p_b(E|x), remains unchanged between the compared conditions. This is well-justified for lower sensory areas in adult subjects [120], or when task contexts are switched on a trial-by-trial basis [22]. However, it is not necessarily true for higher cortices [121], especially when the conditions being compared are separated by extended periods of task (re)training [27]. In those cases, changing sensory statistics may lead to changes in the feedforward encoding, and hence the nature of the represented variable x [122, 123].

In the context of our theory, there are three possible deviations from our assumptions that can account for empirical results of a less-than-perfect alignment with f′ [42, 124]—each of them empirically testable. First, it is plausible that only a subset of sensory neurons represent the posterior, while others represent information about necessary ingredients (likelihood, prior), or carry out other auxiliary functions [50, 53]. Our predictions are most likely to hold among layer 2/3 pyramidal cells, which are generally thought to encode the output of cortical computation in a given area, i.e. the posterior in our framework [125]. Second, subjects may not learn the task exactly implying a difference between the experimenter-defined task and the subject’s “subjective” f′−direction for which our predictions apply. This explanation could be verified using psychophysical reverse correlation to identify the task the subject has effectively learned from behavioral data, in which case we would predict alignment between decision-related feedback and the subject’s internal notion of the discriminanda. However, not all deviations between the subject’s internal model and the experimenter model result in deviations from our predictions. For instance, the subject’s categorical decision may be mediated by other variables between x and C, including an internal estimate of s itself. In fact, one can treat a scalar internal estimate of s much like a large set of fine-grained internal categorical distinctions beyond the two categories imposed by the experiment, such as distinguishing “large positive s” from “small positive s.” Trial-by-trial variations in such fine-grained categorical expectations should, unsurprisingly, feed back and modulate sensory neurons in the f′ direction, even if the categories defined by the experimenter are broad (discussed further in S1 Text). Third, some misalignment between f′ and decision-related feedback may be indicative of significant task-independent noise in the presence of a nonlinear distributional code, which could be tested by manipulating the amount of external noise in the stimulus. If decision-related feedback is found to align with f′ under a range of noise levels and tasks, this would be evidence in favor of a Linear Distributional Code. Given the generality of our derivations, and subject to the caveats just discussed, empirically finding misalignment between f′ and top-down decision-related signals may be interpreted as evidence that neurons do not encode posterior distributions, that the encoding is instead nonlinear (i.e. not an LDC), or that task-learning is suboptimal. In the latter two cases, we would nonetheless predict low-rank task-dependent changes in neural covariability, albeit not aligned with f′.

Self-consistency is implied by learning the data distribution

Following previous work [3, 64, 72, 74], we assume that the brain has learned an implicit hierarchical generative model of its sensory inputs, p_b(E|x), in which perception corresponds to inference of latent variables, x, conditioned on those inputs. The subscripted distributions p_b(·) and p_e(·) refer to the brain’s internal model and the experimenter’s “ground truth” model, respectively.

Here we provide a brief proof that, once a probabilistic model learns the data distribution, its average posterior is equal to its prior [3, 4, 93]. This proof is not novel, and a similar argument can be found in the supplement of Berkes et al (2011). Assume that p_b(E) = p_e(E). Then, (11) where the top line (the prior) follows from the definition of marginalization, and the bottom line (the average posterior) follows from the assumption. In other words, the self-consistency rule we use—that the prior equals the average posterior—is implied by the brain’s internal model learning the statistics of the task. Further discussion and an alternative approach to this proof is given in S1 Text.

Optimal self-consistent expectations over x

In the classic two-alternative forced-choice paradigm, the experimenter parameterizes the stimulus with a scalar variable s and defines category boundary which we will arbitrarily denote s = 0. If there is no external noise, the scalar s is mapped to stimuli by some function E(s), for instance by rendering grating images at a particular orientation. In the case of noise, below, we consider more general stimulus distributions p_e(E|s).

We assume that the brain does not have an explicit representation of s but must form an internal estimate of the category each trial, C, based on the variables represented by sensory areas, x [91]. From the “ground truth” model perspective, stimuli directly elicit perceptual inferences—this is why we include p_e(x|E) as part of the experimenter’s model. In the brain’s internal model, on the other hand, the stimulus is assumed to have been generated by causes x, which are, in turn, jointly related to C. These models imply the following conditional independence relations:

We assume the brain learns the joint distribution p_b(x, C) that maximizes reward, or equivalently that best matches the ground-truth distribution p_e(C, x) in expectation. This entails a conditional distribution “decoding” C from x of the form (12)

We next derive the reciprocal influence of C on x (Eq (2) in the main text) by applying Bayes’ rule to Eq (12):

The substitution of p_b for p_e in the third line follows from the fact that, even from the perspective of an external observer, p_e(x|s) is the inference made by the brain about x induced by the stimulus E(s). Hence, p_e(x|s) is equivalent to p_b(x|E(s)). The fractions p_e(C)/p_b(C) and p_b(x)/p_e(x) become one, assuming that the subject learns the correct categorical prior on C and a consistent internal model. We note that this distribution can be learned even if s is not directly observable by the brain, since its model has access to the true category labels if subjects are informed of the correct answer each trial, as well as to each individual posterior p_b(x|s), as this is what we assume is represented by the sensory area.

As described in the main text we marginalize over the subject’s belief in the category, π = p_b(C = 1), to get an expression for expectations on x given the belief (Eq (3)). Unlike C, π is not a random variable in the generative model but the parameter defining the subject’s belief about the binary variable C. The resulting posterior on x, abbreviated in Eq (4), is given by (13)

We assume that the category boundary s = 0 is itself equally likely to occur conditioned on each category (usually true by definition), but note that this is not a requirement that the categories are a priori equally likely. This simplifies Eq (13) when conditioning on s = 0: (14)

Encoding the posterior in neural responses

Our above derivations considered perturbations of an approximate Bayesian observer’s posterior over their internal variables, p_b(x|E(s), π). We next link these computational-level changes in the posterior to predictions for observable changes in neural firing rate. “Posterior coding” hypothesizes that the (possibly high-dimensional) posterior p_b(x|E(s), π) is encoded in the spiking pattern of a population of neurons over some time window. We do not restrict the space of neural responses r to total spike counts or average spike rates, but instead consider r on a single trial to live in a high-dimensional “spatiotemporal” space, i.e. an N × B array of spike counts for all N neurons in a population resolved into B fine-timescale bins [93]. That is, , where r_ib is the spike count of neuron i at time b. This definition subsumes both “spatial” and “temporal” codes, a distinction that lies at the center of some debates over the neural representation of distributions [3, 5, 47].

We define distributional codes of the posterior as any encoding scheme where the posterior distribution on x is sufficient to determine the neural response distribution over the range of possible stimuli. (Note that this excludes the possibility of separately encoding the likelihood and the prior.) Formally, we say (20) where is a higher-order function that maps from distributions over x to distributions over r. In the notation of Zemel et al (1998), the distribution on r is conditioned directly on the encoded distribution on x, or p(r|p_b(x|…)). Our only restrictions on x and are that p_b(x|…) must change sufficiently smoothly with s, and must be sufficiently smooth over the relevant range of stimulus values, so that the derivatives and linear approximations throughout are valid. A second restriction on x and is that the dominant effect of s on r must be in the mean firing rates rather than their higher-order moments of r. While this is a theoretically complex condition to meet involving interactions between s, x, and , it is easily verified empirically in a given experimental context: if changes to s primarily influence the mean spike count, it is irrelevant whether these changes coded for the mean, variance, or higher-order moments of p_b(x|…). If the space of r is the full “spatiotemporal” space of neural activity patterns, this definition encompasses all previously proposed parametric [7, 62, 70, 80], and sampling-based [34, 48, 49, 51–53] encoding schemes as special cases, among others. However, it excludes sub-populations of neurons in which only the likelihood or prior, but not the posterior, is encoded [58, 60, 126].

Tuning curves as statistics of encoded distributions

The total spike count of neuron i in terms of r is a function of r that sums responses over time bins:

In an encoding model defined as in Eq (20), each neuron’s tuning curve is thus defined by the expectation of S_i at each value of the stimulus s: (21)

The slope of this tuning curve, , is given by the chain rule: where the inner product is taken between two functions, since derivatives were taken with respect to the distribution p_b(x|E(s), π). Eq (1) shows how we use smoothness and linearization assumptions to decouple our analysis of changes in posteriors (e.g. dp_b/ds) from their effect on mean firing rates under arbitrary distributional encodings (e.g. df_i/dp_b). The proportionality between dp_b/ds due to changing stimuli and dp_b/dπ due to feedback of beliefs (Eq (5)) implies an analogous proportionality in neural responses:

Derivation of (9): Feedback of categorical belief implies choice probability proportional to d′

We assume the subject’s choice is based on their posterior belief in the stimulus category, i.e. value of π. Conditioning neural responses on choice is then equivalent to conditioning on the sign of (if there is an additional stage of randomness between belief π and behavioral choice, what follows will remain true up to a proportionality, [21]).

Let be the “choice triggered average” of neuron i, defined as the difference in mean response to choice 1 and choice 2. To isolate top-down effects, consider the noiseless case where neural responses depend exclusively on s (which is fixed) and π (which is varying). We then write as the difference in expected neural response between the and cases:

For small variability in π, this can be approximated linearly:

Substituting in the proportionality (6), it follows that . Dividing both sides of this proportionality by the standard deviation of the neuron’s response, σ_i, and incorporatig the fact that [19, 20], we arrive at the following equation for the top-down component of choice probability after learning: where d′ is the “d-prime” sensitivity measure from signal detection theory [95].

Derivation of (8): Feedback of categorical belief implies differential covariance

Consider any scalar variable a that linearly shifts neural responses in an arbitrary direction u, above and beyond all of the other factors influencing the population (denoted “…”):

When a varies from trial to trial (independently of other factors in “…”), it adds a rank-1 component to the covariance matrix: where Σ^intrinsic is the covariance due to all other factors, i.e. due to neural noise and variability in any of the terms in “…” [37].

It follows that variability in the posterior along dp_b/ds manifest as covariability among neurons in the f′f′^⊤ direction. The noise covariance structure due to var(π) is predicted to be (22)Σ^intrinsic may be thought of as neural noise above and beyond variability in belief. is the rank-one component in the f′f′^⊤ direction due to feedback of variable beliefs, and α is the proportionality constant from (5).

Mathematical details for task-independent noise

We consider three potential sources of task-independent noise in posteriors: first, there are additional “high level” variables in I that may be probabilistically related to x but are not task-relevant. Just as variability in π induces variability in p_b(x|E(s), π), variability in these other internal states may induce variability in the posterior. Second, there may be measurement noise in the observation of E or noise in the neurons afferent to those representing x, resulting in a variable likelihood function even for constant stimuli [83]. Third, the stimulus itself may be stochastic by design, drawn according to some p_e(E|s), which we reparameterize below as a deterministic function of both s and the noise, E(s, ϵ_E). In our notation, we partition the complete noise in the posterior, ϵ, into {ϵ_I, ϵ_L, ϵ_E} corresponding to “internal state” noise, “likelihood” noise, and stimulus noise respectively.

We assume that the all noise sources are unaffected by task learning or task context and are independent of both s and π.

By approximating the joint effect of π and ϵ_I on the density of x as multiplicative, the full posterior decomposes as follows:

The first term (i) is the “noisy likelihood” conditioned on the noisy stimulus E(s, ϵ_E). The second term (ii) is the task-dependent component of the prior studied above. The third term (iii) captures the influence due to other internal variables besides π.

The two noise terms, (i) and (iii), may be combined into a single term. With some slight abuse of notation, we can replace p_b(E(s, ∊_E)|x; ∊_L) with p_b(s|x; ∊_L, ∊_E) so that the ϵ terms appear together. Combining terms, one can thus interpret both (iii) and (i) as noise in the likelihood, despite one arising from feedback and the other being feed-forward:

This motivates our discussion only of “noisy likelihoods” in the main text—it implicitly includes stimulus noise, feedforward noise, and noise due to variable internal states besides π.

Numerical details for Fig 4: Filtering noise by a static self-consistent prior

Fig 4 was created by simulating a discretized 2D space with both x and y coordinates ranging in [−5, 5] for both E and x. We set the distribution of observations, p_e(E|s), to be a Gaussian whose mean or covariance depended on s. One value of E was sampled from this Gaussian on each “trial” and provided to the learning model as an observation. In Fig 4A–4K, the mean of p_e(E|s) was parameterized by a smooth (cubic) function of s, and the covariance of p_e(E|s) was set to a constant . For Fig 4L–4V, the mean of p_e(E|s) was set to zero while the covariance of p_e(E|s) was parameterized by s as follows: where , , and . In all cases, p_e(s) was set to a uniform distribution in [−3, + 3].

In addition to the two different distributions on E just defined, we considered two cases for the brain’s internal model—one in which the generative model p_b(E|x) is assumed to be precise, and one in which it is assumed to be imprecise. In both cases, we assumed , i.e. that E is Gaussian-distributed, centered at the corresponding value of x. In the precise case, we set , and in the imprecise case we set . These parameters were chosen so that p_b(E|x) was always narrower than the marginal distribution of evidence p_e(E), which ensured that a prior p_b(x) could be found such that the marginal distribution on E is correct, or p_b(E) = ∫_x p_b(x)p_b(E|x) = p_e(E).

We are interested in adapting the brain’s prior, p_b(x), to the marginal statistics of observations defined by the task, or p_e(E) = ∫_s p_e(s)p_e(E|_s)ds. To achieve this, we discretized x space so that p_b(x) is a high-dimensional vector, and optimized the following objective by gradient descent: (28) where is the entropy of the prior, included for regularization, and λ sets its importance; we used λ = 0.001 throughout.

Gradient descent in this space is both more stable and simpler to implement using the log prior rather than the prior. Changing variables to the log-prior simply requires multiplying the Loss gradient by the prior itself, since . The gradient of the KL term in (28), with respect to the prior density p_b(x), is given by which after the log-transform simplifies to (29)

Remarkably, (29) means that updating the prior in the direction that maximally improves the match to the marginal distribution of observations simply requires taking a step with the log prior in the direction of the average posterior. Next, the gradient of with respect to the log prior is (30) giving a combined gradient of (31)

Starting with a uniform prior over the discretized 2D space of x values, learning consisted of drawing a large number of random likelihoods (randomizing both s and E) to estimate the average posterior, then the log prior was updated by taking a gradient descent step along (31) while ensuring the constraint that ∫_x p_b(x)dx = 1 after each iteration. We found that a large initial stepsize of 100 was required for learning, which we then halved every hundred iterations for a total of 300 iterations.

To measure the change in covariance of the posterior density itself along dp_b(x|…)/ds, we compared the first and last iteration, which have the same set of variable likelihoods (i.e. identical p_e(E|s)) but different priors. For each, we drew a large set of E ∼ p_e(E|s = 0), and numerically computed the covariance of the posterior mass on x in the discretized space, Σ_p. Because we had discretized x into 85 bins in each dimension, this makes Σ_p a 7225 × 7225 matrix. We likewise computed dp_b(x|…)/ds separately before and after learning (Fig 4E, 4J, 4P and 4U show dp_b(x|…)/ds after learning) by drawing a large number of random posteriors and taking the difference of their average at s = +.01 and s = −.01. We plotted the change in relative variance along dp_b(x|…)/ds in Fig 4F, 4K, 4Q and 4V, defined as where u is the unit vector pointing in the dp_b(x|…)/ds-direction.

This entire process—including learning and estimating the fraction of variance along the dp_b(x|…)/ds direction—was repeated 4 times for each experiment configuration. We found that these results were highly repeatable; error bars in Fig 4F, 4K, 4Q and 4V indicate standard error of the mean across the 4 runs of each condition.

Numerical details for Fig 6: Inferring the internal model

Complex tasks (e.g. those switching between different contexts), or incomplete learning (e.g. uncertainty about fixed task parameters), will often induce variability in multiple internal beliefs about the stimulus. Assuming that this variability is independent between the beliefs, we can write the observed covariance as . Here, each vector u^(k) corresponds to the change in the population response corresponding to a change in internal belief k. The coefficients λ^(k) are proportional to the variance of the trial-to-trial variability in belief k, as in var(π) above, and Σ⁰ represents all task-independent covariance.

The model in our proof-of-concept simulations has been described previously [34]. In brief, it performs inference by neural sampling in a linear sparse-coding model [3, 48, 72]. The prior is derived from an orientation discrimination task with two contexts—oblique orientations and cardinal orientations—that is modeled on an analog direction discrimination task [22]. We simulated the responses of 1024 V1 neurons whose receptive fields uniformly tiled the orientation space. Each neuron’s response corresponds a set of samples from the posterior distribution over the intensity of its receptive field in the input image. We simulated zero-signal trials by presenting white noise images to the model. The eigenvectors not described in the main text correspond to stimulus-driven covariability, plotted in S3 Fig for comparison.