Choice behavior indicates attribute-level comparisons

Two rhesus monkeys performed a multi-attribute decision-making task (Fig 1a). Choice options varied in sweetness (sucrose concentration) of a fluid reward and probability of delivery. The sweetness and probability of an option were each represented by a colored bar, whose height varied monotonically with attribute magnitude. Bar color indicated whether the mapping between height and value was direct (i.e., bigger is better) or indirect (bigger is worse). Sweeter and more probable outcomes were indicated by larger blue and yellow bars (direct mapping) or smaller magenta and green bars (indirect mapping) (Fig 1b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Multi-attribute choice task.

(a) Left to right: Monkeys initiated a trial by holding a touch-sensitive bar and fixating a central point. Choice options were represented by a pair of colored bars of variable height. The left bar indicated the sweetness of an option and the right bar indicated the probability of receiving it. Monkeys freely viewed the options while holding the touch bar, and reported a choice by releasing the bar while directing gaze to one option. (b) Sweetness and probability attributes were represented by colored bars, whose height indicated the value of the attribute. Values varied monotonically in 5 increments. Blue and yellow bar heights directly mapped to the value of the attribute (− = low, + = high). Magenta and green bar heights indirectly mapped to the value of the attribute. (c) The probability of choosing option A (arbitrarily designated) increased with the difference between option A and B probabilities (ProbA − ProbB) and sweetnesses (SwtA − SwtB). Differences were computed from ordinal levels of each attribute. (d) Schematic of three trial types. Top row: Example decisions between two pairs of bars. White column: all attributes have the same mapping, either all direct (shown) or all indirect (not shown). All putative choice computations would take place between bars of the same mapping. Gray: mappings differ among attributes within an option, but like attributes are the same, so that attribute comparisons take place between bars with the same mapping, but integration takes place across different mappings (blue dashed circle). Orange: mappings differ within like attributes but are the same within an option, so that attribute comparisons take place between bars with different mappings (blue dashed circle), but value integration takes place between bars with the same mapping. (e) Accuracy on trials with an objectively better option is consistently lower when like attributes differ (orange). Bars show averages, points show session-wise accuracy in each trial type. Post-hoc comparisons *p ≤ 0.05, **p ≤ 0.01. (f) Weights for each attribute were estimated by beta coefficients from logistic regressions predicting choice probabilities separately from groups of trials in which attribute mapping differed within option (gray) or within attribute (orange) as in d. Since two coefficients of opposite sign were estimated for each attribute (for option A and B), plots show the mean absolute value of these. Histograms show bootstrapped sets of trials of each type. Lower coefficients indicate more choice variability. **Wilcoxon rank-sum test p < 0.001.

https://doi.org/10.1371/journal.pbio.3003281.g001

Monkeys D and C performed 43 and 30 sessions, respectively, and averaged 1,085 and 1,203 trials per session. Choice probabilities indicated that they weighted both attributes to select sweeter, more probable options (Fig 1c). Foundational models of economic choice, such as expected utility theory [36] and prospect theory [37] assume that probability is multiplicatively combined with other attributes to calculate option values (but see [38,39]). In contrast, we previously found that behavior in this task is better fit by models that combine attributes linearly [31]. Formal model comparisons found that choice behavior in this data set was also better fit by an additive rather than multiplicative model (S1a Fig). Since these models are highly similar, this was confirmed with model recovery, showing that we can reliably select the correct generative model from simulated choices (S1b Fig). Therefore, choice behavior of both monkeys indicated that they computed choices by weighing and linearly combining sweetness and probability attributes, and we used this model for further analyses (S1c–S1j Fig).

Next, we separated trials with different attribute mappings to assess inefficiencies in the decision process [31]. We considered two operations that could be involved in computing decisions: integrating values within an option, and directly comparing like attributes of the different options (i.e., sweetness versus sweetness and probability versus probability). Integration or comparison should be easier when the relevant bars have the same mapping, since combining the area of two bars or deciding which of two bars is taller could be performed with simple perceptual judgements. However, when the relevant bars have different mappings (direct versus indirect), the monkeys must translate across mappings by referencing an internal representation of the bar’s meaning – either its position on a value scale or its specific attribute magnitude (e.g., 50% probability) – in order to integrate or compare the information. This process can introduce subtle but measurable inefficiencies into the choice process [31]. To measure this, we separately analyzed behavior during three trial types. First were trials in which all four bars had a consistent mapping (all direct or all indirect, Fig 1d, white), so that integration of attributes within an option or comparison of like attributes across options would always rely on bars with the same mapping. We compared these to trials in which two bars were direct and two were indirect, in different arrangements. When mappings differed within option but like attributes had the same mapping (i.e., sweetness matched sweetness and probability matched probability), then attribute-level comparisons would take place between bars with the same mapping, but value integration would involve bars with different mappings (Fig 1d, gray). On the other hand, if the two attributes within each option had the same mapping but differed from the two attributes of the other option, then combining sweetness and probability into integrated values would involve bars of the same mapping, whereas comparing like attributes without integration would occur across different mappings (Fig 1d, orange). Therefore, separating trials according to whether mappings are mismatched within option or within attribute allowed us to test whether the need to translate between mappings impeded specific component processes that might underlie choice computation.

We first tested this without relying on behavior models by subselecting trials in which one option was superior to the other in both sweetness and probability, and therefore was objectively better (n = 12,038 and 8,888 trials across all sessions, Monkey D and C, respectively). Focusing on these trials allowed us to compute choice accuracies without needing to account for subjective weighting of sweetness versus probability. In both subjects, accuracy on choices with an objectively better option was lower when mappings differed within like attributes, but was unaffected when mappings differed within an option (ANOVA Monkey D: F_2,126 = 9.32, p = 1.68 × 10⁻⁴, Tukey-corrected post-hoc comparisons: all consistent versus differ within option p = 0.86, all consistent versus differ within attribute p = 0.002, differ within option versus differ within attribute p = 2.13 × 10⁻⁴; Monkey C: F_2,87 = 4.01, p = 0.02, Tukey-corrected post-hoc comparisons: all consistent versus differ within option p = 0.99, all consistent versus differ within attribute p = 0.050, differ within option versus differ within attribute p = 0.036) (Fig 1e). Consistent with our previous findings [31], this pattern suggests that choices in this task involve, at least in part, comparisons between like attributes.

Next, we used the same attribute arrangements to test for attribute comparisons across all choices, not just those with an objectively better option. To do this, we separately modeled trials in which bar mappings differed within an option (Fig 1d, gray) or within like attributes (Fig 1d, orange) with logistic regressions (Eqs. 1, 3). If different bar mappings impeded within-attribute comparisons, we would expect less consistent choices on trials where mappings differed within like attributes (orange), quantified as shallower fitted slopes. Indeed, estimated beta coefficients were consistently lower on these trials, indicating more choice variability (Fig 1f). Taken together, patterns of choice behavior provide consistent evidence that attribute comparisons contribute to decisions in this task. Given this, our goal was to assess whether and how such comparisons are made by OFC neurons.

OFC neurons encode attributes of multi-attribute options

We recorded a total of 322 well-isolated single neurons from OFC as monkeys performed the multi-attribute choice task (163 Monkey D, 159 Monkey C). To understand how neurons encode information about sweetness and probability, we analyzed each neuron’s activity in sliding windows with a regression model predicting firing rate from the value of each attribute of the chosen and unchosen options, as well as the mappings (direct or indirect) of each attribute (Eq. 5). Neurons that encode the integrated value of a choice should have non-zero regression coefficients for both attributes of the chosen option, simultaneously with the same sign. Some neurons fit these criteria (Figs 2a and S2a), but this encoding pattern was relatively infrequent and accounted for approximately 10% of neurons at any time (Figs 2b and S2e). If the value of the chosen option was compared to the unchosen option, we would expect to see the two attributes of the unchosen option also jointly represented, but almost no neurons did this (Figs 2b and S2e). Instead, many neurons encoded the value of only one attribute, most frequently an attribute of the chosen option (Figs 2a and S2b). These neurons showed no tendency for firing rates to be modulated by the value of the paired attribute in the same option (S2f Fig and S2g Fig). Such single attribute responses were found in 20–30% of neurons in any time bin during the choice epoch, making them the most common type of response (Figs 2b and S2e).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Attributes encoded by OFC neurons.

(a) Types of attribute encoding in six example neurons. Lines show signed coefficient estimates for chosen or unchosen sweetness (Swt) or probability (Prob) over time. Thick segments denote regions of significance (p ≤ 0.01 × 3 consecutive time bins). Top row left to right: A neuron encoding both sweetness and probability of the chosen option. A neuron whose firing rate varies with all four attributes. Middle row: Neurons encoding the chosen sweetness only (left) or chosen probability only (right). Bottom row: Neurons whose firing rates vary with both chosen and unchosen sweetnesses (left) or probabilities (right). (b) Proportion of neurons with different encoding patterns. (c) Schematic showing relative value of an attribute that is discordant with the chosen options. The choice is between a low (level 2) sweetness and high (level 5) probability vs. a high (level 4) sweetness and low (level 2) probability, and the first option is selected. Only direct mappings are shown for simplicity. The chosen/unchosen model would assign attribute values for chosen sweetness, chosen probability, unchosen sweetness, unchosen probability as [2 5 4 2], but the better/worse model for the same trial would assign better sweetness, better probability, worse sweetness, worse probability as [4 5 2 2]. (d) Proportion of neurons assigned to each model. (e) Neurons assigned to the best/worst (purple) or chosen/unchosen (green) model over time, sorted by the time of better/worse encoding. Many neurons shifted from encoding better/worse attributes early and chosen/unchosen attributes later. (f–g) There were no correlations between beta coefficients for chosen and unchosen attributes or better and worse attributes.

https://doi.org/10.1371/journal.pbio.3003281.g002

We also found neurons that encoded the same attribute of both chosen and unchosen options, with firing rates modulated in the same direction (e.g., higher firing with higher value of either option’s sweetness) (Figs 2a and S2c). This type of response is inconsistent with a comparison signal, but was found in approximately 15% of neurons, soon after the options appeared (Figs 2b and S2e). We reasoned that, if these neurons encode unintegrated attributes before a choice is determined, categorizing attributes according to which option was chosen or not chosen would be the wrong reference frame for quantifying these responses. Instead, the neurons might distinguish attributes according to whether they are better or worse than the other like attribute on that trial, regardless of which one belongs to the option that is ultimately chosen (Fig 2c). Therefore, we tested whether neurons encoded the values of the better or worse sweetness or probability (Eq. 6). Since better attributes tend to be chosen, this model is correlated with the previous one, so we used model comparisons to distinguish them.

We found that up to 20% of OFC neurons were best fit by the better/worse model, indicating that they encoded like attributes relative to one another, not relative to the choice (Fig 2d). This encoding was most frequent early in the choice, and about 500 ms later gave way to a predominant encoding of attributes according to whether they were chosen or not. This pattern suggests that OFC initially processes unintegrated attributes in a reference frame that does not depend on, and therefore can precede, a choice. We considered models in which attributes of an option are integrated in different ways, but the most common encoding in both subjects was of independent attributes, either as the better or worse sweetness/probability early in the trial, or as part of the chosen or unchosen option later in the trial (S2i Fig). There was a tendency for the same neurons to shift from coding attributes relative to one another (better/worse reference frame) to encoding them relative to the choice (chosen/unchosen reference frame) (binomial test, p = 5.15 × 10⁻⁴), although there were also many neurons that encoded chosen/unchosen attributes without previously encoding better/worse attributes (Fig 2e). This pattern was found in each monkey separately, but only significant in one (Monkey D, C: p = 0.002, p = 0.056).

So far, we found that many OFC neurons encode attributes separately. Early in the choice, they are encoded in relation to the same attribute in the other option (i.e., better/worse reference frame) and later in relation to the decision (i.e., chosen/unchosen reference frame). This could suggest a comparison process between better and worse attributes that produces a chosen option. Therefore, we tested for antagonistic value coding consistent with a comparison signal, by looking for inverse relationships among the regression coefficients. However, in both the better/worse reference frame and chosen/unchosen reference frame, we found no significant relationships (Fig 2f–2g). Therefore, although encoding patterns are suggestive of a transition from encoding attributes in a pre-choice to post-choice frame of reference, this analysis did not find evidence for within-attribute comparisons in OFC.

Gaze behavior relates to attributes and attribute values

Since our task represented attributes with separate cues, we were able to use eye tracking to assess visual exploration of attributes and whether directing gaze to an attribute affects neural responses in OFC. To do this, we defined pre-choice fixations as those directed to an attribute bar between the time of option presentation and choice (Fig 3a). Since monkeys had to fixate an option to report their choice, we excluded this final fixation that coincided with the bar release. Both monkeys visually explored the choice options, but often did not foveate all four attributes on the screen (Fig 3b). Since both attributes contributed to their choices, this suggests that subjects used both directed gaze and peripheral vision to evaluate the options. Consistent with this, the first fixation of both monkeys was more likely to be on the higher value option on objective trials, where both attributes of one option were more valuable than both attributes of the other (Fig 3c). Monkey D and C fixated the higher value option on 76.5% and 68.5% of these trials, suggesting that they rapidly evaluated peripheral targets before shifting their gaze, as monkeys do on single attribute choices [40]. However, our subjects were less proficient at this covert evaluation when attributes needed to be weighed. Across all trials, including those in which the higher value sweetness and higher value probability belong to different options, their tendencies to look first at the chosen option dropped to 62.2% and 52.7% (Monkey D, C). If the subjects evaluated unintegrated attributes, however, we might expect their first fixations to be directed to better attributes, regardless of which option was chosen. Indeed, both subjects’ first fixations were directed to the better sweetness or probability on each trial more consistently than to the chosen option (Fig 3d). This tendency was stronger for sweetnesses, but also present for probability (Fig 3e). Taken together, patterns of gaze behavior indicate that subjects used peripheral vision to evaluate unintegrated attributes, and they were biased to look at valuable attributes first as they made their choices.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Fixation patterns during multi-attribute choices.

(a) Gaze paths (black) on two example trials. Outlines around the colored bars show inclusion regions assigned to each attribute (not visible to the monkey). Red circles indicate fixations included in analyses. Fixations on the central fixation point at trial start (gray circle) and on the selected option at bar release (yellow) were excluded. (b) Number of pre-choice fixations made by each monkey. Bars = average across monkeys and sessions, circles = session averages. Yellow bars = trials included in the gaze-aligned neural analyses. (c) Proportion of trials in which both attributes of one option were higher value than both attributes of the other, and the monkey looked at the higher value option first. Bars = average across sessions, circles = average in each session. (d) Proportion of trials with one or more pre-choice fixation in which the monkey first looked at an attribute in the option he would ultimately choose (green) or at an attribute that was higher value than the like attribute of the other option, regardless of choice (purple). Bars = average across monkeys and sessions, lines = session averages. Black = Monkey D: Gray = Monkey C. Proportions were compared with sign-rank test across sessions. **Monkeys D: p = 1.59 × 10⁻⁸, C: p = 1.73 × 10⁻⁶. (e) Proportion of trials in which the monkey first looked at a sweetness bar (Swt) and it was the better or worse sweetness (left), or first looked at a probability bar (Prob) and it was the better or worse probability (right). Bars = average across sessions and monkeys, circles = session proportions. (f) Schematic of three types of gaze transitions in an example where the first fixation is on the sweetness bar of the left option (left), and proportions of each type of gaze transition (right). Navy: to the paired attribute; Dark teal: to the like attribute of the other option; light teal: to the unlike attribute of the other option. Bar plots = average across sessions and monkeys, lines = proportions within sessions. One-way ANOVA of session-wise measures: Monkeys D: F_2,126 = 71.4, p = 1.86 × 10⁻²¹, C: F_2,87 = 65.5, p = 4.50 × 10⁻¹⁸. **Bonferroni-corrected post-hoc comparisons p < 0.001.

https://doi.org/10.1371/journal.pbio.3003281.g003

If the monkeys made choices by comparing like attributes, they might also sequentially evaluate the same attribute of each option. However, if their decision strategy relied on integrating the values of two attributes within an option, they might sequentially evaluate the sweetness and probability bar of a single option. To assess these possibilities, we categorized the first gaze transition on trials with at least 2 pre-choice fixations as shifting either to the paired attribute within the same option (e.g., sweetness A to probability A), to the like attribute of the other option (e.g., sweetness A to sweetness B), or to the nonmatching attribute in the other option (e.g., sweetness A to probability B) (Fig 3f). We found that gaze more frequently shifted between like attributes than between unalike attributes of either option. Therefore, patterns of gaze transitions were consistent with choice probabilities and accuracies that indicate monkeys compared unintegrated attributes.

OFC neurons encode attributes relative to the focus of gaze

Having quantified gaze behavior, we next asked whether OFC neurons encode the value of attended attributes or options. To do this, we aligned neural activity to the onset of pre-choice fixations, and used multiple linear regressions to determine how firing rates varied with the value of attributes relative to the focus of the subjects’ gaze. Each attribute of the choice was labeled as either the fixated attribute (e.g., sweetness A), the fixated attribute’s pair in the same option (e.g., probability A), the attribute in the unfixated option that is like the fixated one (e.g., sweetness B), or the other attribute in the unfixated option (e.g., probability B) (Fig 4a). If OFC neurons encode attended attributes only, then we expected neuron activity to be explained by the value of the fixated attribute alone, but if they encoded the integrated value of the attended option, then firing rates should be modulated by both the fixated attribute and its pair (green attributes in Fig 4a). On the other hand, if OFC neurons compare like attributes, we expected firing rates to encode the fixated attribute and the like attribute of the unfixated option (dark green and dark orange in Fig 4a). Since evidence suggests that gaze shifts early in a decision are primarily used for evaluation and later ones are confirmatory once a latent choice is identified [41–43], we included only the first and second fixations on trials with 2 or more pre-choice fixations (Fig 3b). Further, to focus on neural activity that shifts with the targets of gaze, we first regressed out activity related to static chosen and unchosen attributes (“Materials and methods”).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. OFC neurons encode attributes relative to the focus of gaze.

(a) Schematic showing attribute labels, depending on the monkeys’ targets of gaze for two example fixations. The first fixation is on the sweetness bar of the left option, and the second fixation is on the sweetness bar of the right option. (b) Four example neurons encoding attributes in an attention-based reference frame. Plots show beta coefficients from multiple regressions, with firing rates aligned to the onset of a fixation. Thick segments denote regions of significance (p ≤ 0.05 × 3 consecutive time bins). Colors as in a. (c) Percent of neurons in intact (left) and shuffled data (right) that significantly encode each attribute. (d) CPDs for each attribute across all recorded neurons. Shading shows mean CPD ± sem. CPDs were compared to shuffled controls with rank-sum tests. Bars: p ≤ 0.05 × 3 consecutive time bins. (e) Same as d, except separately for first and second fixations. Arrows indicate significant encoding before the second fixation. See also S3c–S3f Fig.

https://doi.org/10.1371/journal.pbio.3003281.g004

Soon after directing gaze to an attribute, many OFC neurons encoded attribute values relative to the focus of the monkey’s gaze. In particular, neurons tended to encode the fixated attribute, often with a positive beta coefficient, and the like attribute of the unfixated option, often with a negative beta, as shown in example neurons in Fig 4b. This pattern, equivalent to a weighted subtraction, is consistent with competitive inhibition between an attended attribute and the matching attribute in the other option, and suggests that OFC neurons do carry out attribute-level comparisons. The comparison signal, however, appears to depend on directed attention, since the same signatures were not found in our previous analyses that did not take gaze into account. We considered an alternative model, which hypothesizes that neurons are selective for the sweetness or probability in a gazed-at option, but insensitive to which attribute of the option the fixation is directed at. However, we found that this model fit the vast majority of neurons worse than the present one, indicating that OFC neurons preferentially encode attributes relative to gaze, regardless of the identity of the attribute (S3 Fig). In the following sections, we quantify gaze-dependent attribute encoding across our recorded population of OFC neurons.

Overall, there was a clear increase in neurons encoding values of both the fixated attribute and the like attribute of the other option soon after the start of a fixation (Fig 4c). At the same time, we did not find a change in the proportion of neurons encoding the other attributes on the screen, including the paired attribute in the same option that should be encoded if the neurons integrated option values. However, the proportion of neurons that reached our threshold for significance at any time was modest (≤10%), so we also calculated the unique variance accounted for by each of the attributes across all neurons in our data set, using the coefficient of partial determination (CPD). Similar to the thresholded results, there were peaks in variance explained by the fixated attribute and the like attribute of the other option soon after the start of a fixation (Fig 4d). Therefore, at the population level, there was increased encoding of like attribute values, relative to the focus of the monkeys’ gaze.

Before analyzing this effect further, we ensured that our results were related specifically to directed gaze, and were not artifacts of the analyses. First, we ran a shuffling procedure in which the same attribute values were included in each trial, but their relationship to the monkey’s gaze was shuffled. In other words, we used the same residualized data, but attribute values were randomly designated the fixated attribute, paired attribute, like attribute of the other option, and the other attribute of the other option. As expected, shuffling removed the encoding that followed fixation onset in both thresholded data and CPDs (Fig 4c and 4d). Further, if only unfixated attributes were shuffled, then only encoding of the fixated attribute was found (S4g Fig).

Next, since monkeys tend to look at better attributes and chosen options first, we ensured that encoding patterns were not spurious effects of correlations between the monkeys’ gaze and the value of the choice attributes. To do this, we simulated neuron responses that encoded task variables but were agnostic to the fixation patterns the monkey generated, then tested whether the analyses we used produced spurious encoding of fixated attributes. We simulated neuron responses of six types, corresponding to the most common encoding patterns found in our data set: neurons encoding chosen sweetness only, chosen probability only, better sweetness only, better probability only, chosen value (i.e., both chosen sweetness and chosen probability), and non-selective neurons (S5a Fig). We simulated firing rates that reflected only the variable being tested, plus random noise, using actual trial variables (e.g., chosen sweetness/probability on each trial) from each session, and then tested the simulated neurons on actual patterns of fixations produced by the monkey on the same trials. Using this approach, we found <1.5% of simulated neurons of each type encoded attributes relative to the monkeys’ fixations (S5b–S5g Fig), consistent with the expected false alarm rate of the significance criterion we used (p ≤ 0.01). Therefore, there was no evidence that fixation-dependent encoding arose from confounds in the behavior or analyses.

Finally, encoding of fixated and like attributes occurred approximately 200 ms after the start of a fixation. Since each fixation lasted approximately 200 ms (S4a and S4b Fig), and monkeys tended to shift their gaze between like attributes, we assessed whether encoding assigned to the like attribute of the other option was actually a consequence of the subject moving their eyes to that attribute on the second fixation. To check this, we analyzed the first and second fixations separately. Following each fixation, we obtained similar results in both proportion of neurons encoding these variables and CPDs, each occurring approximately 200 ms after the respective fixation (Figs 4e and S4c–S4f). Interestingly, on the second fixation only, there was also encoding of the two relevant attributes before the fixation was initiated. This was separate in time from post-fixation encoding, and can be attributed to the prior (first) fixation (Fig 4e). Together, these analyses confirm that OFC neurons did encode both the fixated attribute and the like attribute in the other option at the same time.

OFC neurons jointly encode attributes

Next, we assessed whether different attributes tend to be jointly encoded by single OFC neurons, as in the example neurons in Fig 4b, or if the population effects resulted from different pools of neurons encoding a single attribute at a time. We considered four possible combinations of two attributes that might be encoded together: the fixated attribute and its pair, as expected if the neuron encoded the attended option’s value (Fig 5a, pink); the fixated attribute and the like attribute of the other option, as expected if like attributes are compared (Fig 5a, blue); the two like attributes that were not fixated (Fig 5a, brown); and the two attributes of the unfixated option (Fig 5a, purple). We found a small but significant proportion of neurons jointly encoding the fixated attribute and the like attribute of the other option, consistent with the possibility of within-attribute comparisons (Figs 5b and S6b). Interestingly, we also found a small number of neurons that jointly encoded the two like attributes that are not fixated, which may be related to the monkeys’ tendency to peripherally evaluate attributes. On the other hand, there was no tendency to jointly encode the two attributes of either the fixated or unfixated option, again arguing against integrated encoding of option values in this gaze-dependent reference frame. The same patterns were found when data were separated into first and second fixations or by monkey (S6c–S6d Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Neural signatures of attribute comparison in OFC.

(a) Schematic of pairs of attributes that might be grouped in an attention-dependent reference frame. Each panel shows groupings for an example where the sweetness bar of the left option is fixated. (b) Percent of neurons jointly encoding each pair of attributes defined in a. Shuffled data are in S4b Fig. Bars = incidence of joint coding is greater than expected if the two attributes of a group were coded independently. To correct for each attribute belonging to two defined groups, significant joint encoding was defined by binomial test p ≤ 0.005 × 3 consecutive time bins. Shaded region = time windows analyzed in the scatter plots in c–f. (c–f) Scatter plots of mean beta coefficients of neurons with significance for one or both of the attributes in a group, color coded as in a. Statistics are from Pearson correlations, with p-values Bonferroni corrected for two comparisons each. Black circles = Monkey D, gray circles = Monkey C.

https://doi.org/10.1371/journal.pbio.3003281.g005

Attribute comparison takes place in a gaze-based reference frame

Together, these results are consistent with the possibility that OFC neurons compute comparisons among like attributes during multi-attribute choice. In a final analysis, we tested for neurophysiological signatures of value comparison in this gaze-dependent reference frame. Our earlier analyses found no evidence for value comparisons, either among attributes or options (Fig 2f–2g), but this could be because OFC computes comparisons in an attention-dependent reference frame and those analyses did not take gaze into account. Therefore, we tested whether values of the unfixated attribute or option antagonistically weight value encoding of the fixated attribute/option. To do this, we computed the average beta coefficient in a time window following an attribute fixation, and included neurons that reached significance for one or both regressors of interest at any time in that window.

Consistent with comparisons of like attribute values, we found an inverse relationship between the beta coefficients of the fixated attribute and the like attribute of the other option (Fig 5d). These effects held when fixations on sweetness and probability were separated in the model (sweetness fixations: r = −0.47, p = 3.8 × 10⁻⁵, probability fixations: r = −0.86, p = 1.0 × 10⁻¹⁴), and also when the comparison included all neurons without thresholding for significance (S6e–S6h Fig). In the same time window, there was no relationship between the two attributes of either the fixated or unfixated option (Fig 5c or 5f, respectively), indicating no evidence that OFC neurons encode integrated values in a gaze-dependent manner. Interestingly, there was a negative relationship among coefficients of the two like attributes that were not fixated (Fig 5e), again likely arising from the subjects’ tendency to peripherally evaluate information. Since there was no relationship among attributes within the same option, the negative relationships among like attributes that were both fixated and not fixated could only arise if separate populations of neurons were modulated by each (fixated and not fixated). In addition, there was a weaker tendency for fixated attributes to be negatively related to the other attribute of the other option, raising the possibility of a small degree of comparison between the fixated attribute’s value and the value of the unmatching attribute in the other option (S6i–S6j Fig). In sum, our results demonstrate a consistent neural signature of value comparisons that are primarily among like attributes and depend on the monkeys’ current focus of gaze.

Overall, there was a bias among OFC neurons to encode the fixated attribute positively, meaning that firing rates increased with increasing value (Fig 5d). The like attribute of the other option, on the other hand, tended to be encoded negatively. Since the first gaze transition tended to shift between these two attributes (Fig 3f), the first fixated attribute had a high probability of subsequently becoming the like attribute of the unfixated option (Fig 6a). Therefore, we expected the beta coefficients to flip from encoding the first fixated attribute positively to negatively at the start of the second fixation, and vice versa for the like attribute of the other option. To test this, we ran a multiple regression on the firing rates aligned to the first and second fixations separately, but used regressors defined by the first fixation throughout. As predicted, approximately 200 ms after the first fixation, average coefficients across the recorded population were positive for the fixated attribute and negative for the like attribute of the unfixated option. Approximately 200 ms later, however, the signs flipped (Fig 6b). Aligning the same data to the second fixation revealed that the inverted encoding pattern peaked approximately 200 ms after the second fixation, and the same effect was apparent in each monkeys’ data individually (S7 Fig). Taken together, these patterns confirm predictions of a model in which OFC neurons inversely encode like attributes of multi-attribute options, in a reference frame defined by the monkey’s focus of gaze.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Attribute encoding is modulated by gaze shifts.

(a) Schematic illustrating that a gaze transition between like attributes reverses the attribute designations. If the like attribute of the other option from the first fixation (orange) is fixated second, then it becomes the fixated attribute and the first fixated attribute (green) becomes the like attribute of the other option. (b) Average beta coefficients from trials with two or more pre-choice fixations. Firing rates are aligned to the first fixation (left) or the second fixation (right), with the same regression model in each. Shading shows the mean coefficient estimate across all neurons ±sem.

https://doi.org/10.1371/journal.pbio.3003281.g006

Subjects

Two adult male rhesus macaques (D and C) participated in the experiment. At the start of recording, they were 9 and 7 years old and weighed 10.0 kg and 8.2 kg, respectively. Each was surgically implanted a titanium headpost and cranial recording chamber made of biocompatible plastic (Monkey D – Ultem, Jerry-Rig USA. Monkey C – PEEK, Crist Instruments). The chambers were centered on stereotaxic coordinates calculated from 3T MR images obtained of each subject’s brain. In Monkey C, the chamber was placed over the left hemisphere; in Monkey D, the right. All procedures were in accord with the National Institutes of Health Office of Laboratory Animal Welfare (OLAW) guidelines and were approved by the Icahn School of Medicine at Mount Sinai Animal Care and Use Committee (Animal Welfare Assurance reference number D16-00069 (A3111-01)), under protocol #2017-0084. Data and analysis code are available online [61].

Behavior models

Choices in each session were fit with two potential models that varied only in how attribute values were combined. In the additive model (Eq. 1), sweetness and probability were independently parameterized and linearly combined.

(1)

In the multiplicative model (Eq 2), attributes were independently parameterized and multiplicatively combined.

(2)

In both models, options were arbitrarily designated A and B. ProbA or B and SwtA or B are the ordinal magnitudes of probability and sweetness available in each option. are fitted weights of probability/sweetness attributes. (Prob/Swt)Dir(A/B) is the mapping for each attribute bar, coded as −1 or 1, with the corresponding coefficient (). XPos(A/B) is the x-position of option A or B on the task screen, and YPos(A/B) is its y-position. Since attributes within options were always proximal to each other and therefore correlated, only x and y coordinates of the center point of the overall option was used. and are the corresponding coefficients. is a constant to capture any bias toward A or B.

For both models, the probability of choosing an arbitrary option A was fit by generalized linear models with the logit link function. This is equivalent to a softmax decision rule [64] that probabilistically selects the option with the higher value (Eq. 3).

(3)

Behavior model recovery

We simulated 100 sessions of 1,000 choice trials each. As in choices presented to the monkeys, sweetness and probability magnitudes and mappings were randomly selected on each simulated trial, and each simulated option was assigned an X, Y position following the same constraints in the main task. Then, for each session, we used the parameters from the GLMs fit to the monkeys’ actual choices to predict a probability of selecting option A (Matlab function predict.m). This was done separately for additive and multiplicative models to obtain two vectors of trial-wise predictions, one for each model. For each set of predicted probabilities, we simulated choice outcomes on each trial by selecting option A with the probability p, and option B with probability 1 − p, where p is the probability predicted by the model on that trial. Each simulated session was then fit by the same additive and multiplicative GLMs as the original behavior to determine whether the model comparison would reliably select the appropriate additive and multiplicative model when the generative model was known.

Objective choice accuracies and condition-wise choice models

Among trials in which both attributes of one option were superior to both attributes of the other, accuracies were calculated as the proportion of trials per session in which the sweeter and more probable reward was selected, and one-way ANOVAs compared the average accuracies per session across the subsets of trials (as in Fig 1d). To model all choices in a condition (not just those with an objectively better option), trials with different mappings within options or within attributes were aggregated across all sessions. From each condition, we created 100 bootstrapped samples of 400 trials and fit each set of trials with a simplified version of the additive model (Eq 4).

(4)

Beta coefficients for the same attribute of option A and B were approximately equal and oppositely signed, so we took the average absolute value as the estimated weight for each attribute. Weights were compared across mapping conditions with Wilcoxon rank-sum tests.

Neural recording

Neurophysiological methods were similar to those reported previously [65]. Data were collected with tungsten microelectrodes (FHC) and 16-contact linear arrays (Plexon) lowered through chronic craniotomies, and advanced manually using custom-built microdrives. Between 4 and 14 electrodes or 1–4 16-channel probes were lowered per session. Target regions were identified on previously obtained 3T MR images, as the cortex between medial and lateral orbital sulci. Our recording areas range from 35/42.7 mm to 28/32.7 mm AP for Monkeys D/C, relative to the interaural line. This included anterior regions putatively identified as Area 11 and posterior regions putatively identified as Area 13. After recording, electrode placements were reconstructed from MRIs, and any neurons recorded outside the target regions were removed from further analysis.

Neural signals were collected, digitized, and saved (Ripple Neural Systems Grapevine Processer). Putative spikes were captured at 30 K and sorted into single and multi-units offline (Plexon OfflineSorter) and spike times were saved at 1 kHz resolution. Only single units were analyzed in the present study, and any unit with an overall firing rate <1 Hz was excluded from further analysis due to difficulty statistically characterizing their responses.

Trial-aligned encoding models and model comparison

Prior to analysis, spike times were smoothed with a 150 ms boxcar. Smoothed firing was then aligned to the appearance of choice options and analyzed in 200 ms windows, stepped forward by 20 ms. Full time windows analyzed included 1 s before until 1s after option onset, resulting in 90 overlapping time bins. In each time bin, (Eq 5) was fit to the average firing rate. ProbCh and SwtCh are the ordinal magnitudes of the chosen probability and sweetness. ProbUCh and SwtUch are the ordinal magnitudes of the unchosen probability and sweetness. ProbDirCh and SwtDirCh are the mapping (direct or indirect) of the probability and sweetness attribute of the chosen option, and ProbDirUch and SwtDirUch are the mappings of the unchosen attributes. Each regression term was centered on zero before fitting the model.

(5)

Significant encoding of a model term was defined as a non-zero beta coefficient. To compare encoding patterns consisting of conjunctions of attributes (e.g., both chosen attributes) to single attribute encoding, we defined significance thresholds based on a total false discovery rate of p ≤ 0.01 in a given time bin, then required significance for at least 3 consecutive time bins. We selected this criterion because it produced false discovery rates during the fixation epoch of <5%. Single attribute encoding included any one of four model terms reaching threshold, so the significance criterion was p ≤ 0.0025 for 3 time bins. For chosen option encoding, chosen sweetness and chosen probability must both be significant (likewise for unchosen option encoding), meaning the effective alpha is alpha_sweetness * alpha_probability. To achieve an effective alpha of 0.01, the threshold for each attribute was p ≤ 0.1 for three time bins. For like attribute encoding (i.e., both sweetnesses or both probabilities), the alphas of two conjunctive patterns are added, such that the effective alpha is (alpha_sweetnessA * alpha_sweetnessB) + (alpha_probabilityA * alpha_probabilityB). To achieve an effective alpha of 0.01, the significance criterion for each term was p ≤ 0.0705 for three time bins. Equating the significance criteria for each model term, rather than the effective alphas, produced qualitatively similar results, except there were unequal false discovery rates prior to options onset.

The same neuron firing rates in the same sliding windows were also fit with additional models. In each model, regressors were mean-centered before fitting. In the Better/Worse model (Eq. 6), ProbBet and SwtBet are the ordinal magnitudes of the higher value (better) probability and sweetness, respectively and ProbWrs/SwtWrs are the ordinal magnitudes of the lower value (worse) probability and sweetness, as in Fig 2b. If an attribute had equal values in each option, then the same number was entered into each regressor. ProbDirBet and SwtDirBet are the mapping (direct or indirect) of the better probability and sweetness, and ProbDirWrs and SwtDirWrs are the mappings of the worse probability and sweetness.

(6)

In the additive chosen value model (Eq. 7), attributes were first weighted by their relative importance in the additive behavior model for that session (Eq. 1). Since options A and B were arbitrarily designated in the behavior model, their weights were approximately equal and opposite. Therefore, we took the mean absolute value of the behavior coefficients for probability attributes ( and multiplied this by trial-wise probabilities, the mean absolute value of behavior coefficients for sweetness attributes ( multiplied by trial-wise sweetnesses, then added these to obtain a weighted additive value for chosen and unchosen options. The model also included attribute mappings as in (Eq. 5).

(7)

In the multiplicative chosen value model (Eq. 8), attributes were weighted by their relative importance in the multiplicative behavior model for that session (Eq. 2). This behavior model estimated one coefficient for the log ratio of probabilities (, and another for the log ratio of sweetnesses (. These were used as weights for each attribute, and attributes of chosen and unchosen options were multiplied to obtain integrated values. The model also included attribute mappings as in (Eq. 5).

(8)

Finally, we considered the possibility that neurons encoded risk, or uncertainty, rather than probability [66]. The risk model (Eq. 9) was identical to (Eq. 5), except the ordinal levels of probability were replaced with ordinal levels of uncertainty, such that probabilities 1, 2, 3, 4, 5 were replaced with 1, 2, 3, 2, 1. RiskCh and RiskUch are chosen and unchosen risk attributes, respectively.

(9)

Each of the encoding models above were used to explain trial-wise variance in firing rates in 200 ms sliding windows. In each window, the total model significance was computed, as well as the Akaike information criterion (AIC). If the same model had a significance of p ≤ 0.01 for three consecutive time bins and had the lowest AIC of the models under consideration (Eqs 5–7) in the main text, (Eqs 5–9) in S2i Fig), then it was included as a neuron best fit by that model in the relevant time bins. Note that this means that the same neuron could be assigned to different models in different time bins. To determine whether single neurons initially coding better/worse attributes were more likely to code chosen/unchosen attributes later in the trial, we found the proportion of neurons assigned to the better/worse model in any time bin in the first 350 ms after the choice appeared, and the proportion assigned to the chosen/unchosen model in any time bin between 350 and 700 ms and tested whether the probability of a neuron significantly encoding both of these was higher than expected if the two models were encoded independently (i.e., the joint probability of the two types of encoding). The rate of actual joint encoding was compared to this chance level with binomial tests. To investigate relationships between beta coefficients for different encoded attributes, we averaged coefficients from the sliding regressions in time windows with a leading edge 100–400 ms after the appearance of the choice options, and counted neurons with significance in any of the included bins as encoding neurons. This time range was chosen to encompass the majority of pre-choice data (the faster monkey had a median choice time of 520 ms), while allowing for visual processing delays after the options appeared and motor processing times before a choice was registered.

Gaze analyses

The EyeMMV toolbox [67] was used to define fixations from continuous eye movements. The minimum fixation duration was 50 ms to capture short fixations [68]. The tolerance to include a gaze position in a fixation cluster (t1) and to include it in the calculation of the fixation cluster mean (t2) were t1 = 2 and t2 = 1. Only fixations that fell within defined regions around each attribute bar were included in the analyses. To capture fixations that fell on the bar edges, these regions included the full bar height plus 1.0° of visual angle around all sides except the side adjacent to the other attribute, which was 1.0° away. Here, an additional 0.5° buffer was used, so there was no unassigned space between the bars. The eye tracker was calibrated at the start of each session, and gaze data were further aligned in post-processing by centering eye traces on the mean x-y coordinates during the initial fixation window on each trial. Analyses excluded the fixation that coincided with release of the touch bar (i.e., the choice report). For each pre-choice fixation, EyeMMV was used to quantify fixation position (in x, y coordinates), start time, end time, and duration. Fixation start times were used to align neural data to fixations. Except where noted, the first and second fixations of trials with 2 or more pre-choice fixations were used.

Neural responses aligned to fixations and analyzed similarly to trial-aligned data. Firing rates were smoothed with 150 ms boxcar, averaged in sliding windows of 200 ms stepped forward by 20 ms, and analyzed with GLMs. To remove fixation-unaligned effects, we first fit each time window for each neuron with (Eq 5), then fit the residuals with (Eq 10), where FixVal and FixDir are the ordinal value and mapping direction of the fixated attribute (e.g., sweetness A), PairVal and PairDir are the ordinal value and mapping of the attribute that is the other component of the fixated option (e.g., probability A), LikeAttOtherOptVal and LikeAttOtherOptDir are the value and mapping of the like attribute of the unfixated option (e.g., sweetness B), and OtherAttOtherOptVal and OtherAttOtherOptDir are the value and mapping of the other attribute of the unfixated option (e.g., probability B). FixAtt indicates whether the currently fixated attribute is sweetness or probability, and XPosFix and YPosFix are the X and Y position coordinates of the fixation. Significant encoding was defined as p ≤ 0.05 for three consecutive time bins. This criterion was selected because it resulted in <5% false discovery rates in shuffled data. Results were qualitatively similar if a stricter criterion was used (p ≤ 0.01 for three consecutive time bins).

(10)

CPD quantifies the increase in unexplained variance obtained by removing one predictor from a multiple regression model, and therefore measures the unique variance explained by that predictor. CPDs were calculated for all neurons in the same sliding windows as above.

Joint encoding was defined as significant encoding (also p ≤ 0.05 for three consecutive time bins) of two regressors in the same time bin. If the two regressors were encoded independently in a population, more coincident encoding would occur by chance as the prevalence of encoding either increased. This rate of chance joint encoding was calculated as the product of the proportions of neurons encoding either regressor, and the actual rate of coincident encoding was compared to chance with binomial tests. To quantify the tendency to code two regressors similarly or inversely, we averaged the regression coefficients for pairs of regressors from (Eq 10) in time windows where the leading edge was >0 ms and <300 ms from the start of fixation. We included any neuron that reached significance for encoding either of the regressors in one or more time windows in that range. Note that fixated and non-fixated attributes of both types had the same ordinal range in this analysis (1–5, mean centered to −2 to +2), so unequal range effects cannot explain beta anticorrelations in this task [21].

To test whether our main effects held when fixations on sweetness and probability were separated, we tested a model that removed the identity regressor and separated fixations on sweetness and probability into two predictors (Eq. 11). In this model, FixSwt and FixProb are the value of the fixated sweetness or probability, respectively, with 0 entered when a fixation was not directed at the relevant attribute.

(11)

To test the alternative possibility that single neurons were selective for the value of sweetness or probability in the option that was fixated regardless of which attribute in that option the monkey directed its eyes to, we fit neurons to a second model (Eq. 12). In this model, FixOptSwt and FixOptProb are the magnitude of the sweetness and probability bar, respectively in the fixated option; UnfixOptSwt and UnfixOptProb are the sweetness and probability magnitudes in the unfixated option; FixOptSwtDir, FixOptProbDir, UnfixOptSwtDir, and UnfixOptProbDir are the mappings of each of these attributes using the same convention. The remaining regressors are as in (Eq. 10).

(12)

Since (Eqs 10 and 12) have the same number of parameters, we compared them by computing the whole model R² for each model fit to each neuron, averaging the sliding analysis in the same time window as shown in Fig 5c–5f. Since nearly all neurons had higher R² for (Eq 10), we did not consider (Eq. 12) further.

Neuron simulations

Simulations were used to tested whether neurons that varied only with trial-wise variables could produce spurious encoding of fixated attributes, given the behavior patterns of our monkeys in this task. For simulated neurons, we used actual trial variables from each behavioral session, including sweetness and probability of the choice options, the monkey’s selection, and the first fixated attribute. We then simulated 5 neurons of each type (see Fig 5) per session, resulting in 215 and 150 simulated neurons of each type for Monkey D and C, respectively. Each neuron was simulated as a single vector of trial-wise firing rates that was seeded with an integer response (1–5) that depended on the task variable being tested. Non-selective neurons were randomly assigned an integer 1–5. Chosen value responses were determined by binning the additive combination of chosen sweetness and probability into 5 bins. The integer indicated which of 5 partially overlapping, uniformly distributed ranges a new seed, representing an average firing rate, was drawn from. These ranges were chosen to match typical firing rates of OFC neurons, with the lowest value set to 0 and the highest 6.5. The average value of each range stepped up from 1 to 5, such that random draws from each range would produce average responses that were ordered 1 < 2 < 3 < 4 < 5. These responses were inverted (to 5 < 4 < 3 < 2 < 1) with a probability of 0.5 to simulate the tendency of OFC neurons to encode value both positively and negatively. We then added two sources of gaussian noise to each trial: one that was unique to each simulated neuron and one, on average 10% as large, that was shared across simulated neurons in the same session. Depending on the simulated variable, (Eq. 5) or (Eq. 6) was used to confirm that the simulated neurons had the desired encoding properties. We then tested whether the same analyses used to show fixation-related encoding (described above) would produce spurious results.