The (Neuro)Economic Model of Choice

A central axiom of economic models of choice is that there exists a single, common scale of value that the decisionmaker uses to compare options, and that this common currency can be revealed through the actions of the animal (see Box 1 for a glossary of terms used in this preview). This hypothesis matches our intuitive notion that when we make decisions, we weigh them against each other. Mathematically, this principle is necessary for internally consistent (i.e., rational) choices. Decisions arise from computations occurring in neural systems. Under the assumption of this central axiom of economic choice, one would expect to find abstract representations of value somewhere in the nervous system. Therefore, it generated a great deal of excitement when neuroscientists studying decision making started finding correlates of value within the brain [1].

A number of studies that recorded activity from single neurons in the monkey orbitofrontal cortex (OFC) found neurons that changed their firing rate according to the subjective value of the reward being offered, showing a common currency for value, independent of (or “abstracted from”) the sensory and motor variables of the task [2]. In agreement with this model, previous experiments by Strait and colleagues have found evidence for common currency signaling in a nearby cortical structure, the ventromedial prefrontal cortex (vmPFC) [3].

In contrast, neuroeconomic experiments in rodents have largely failed to find these representations of abstract value signaling in the orbitofrontal cortex, instead finding more state-based representations of expectations of reward [4–9]. Moreover, the rodent studies have identified a prominent role for value coding in the ventral striatum (VS) [9–13], calling into question a unique role for the OFC in signaling value.

While it is certainly possible that this discrepancy could be chalked up to species-specific differences in brain architecture [14], and/or differences in task design, the study reported in this issue of PLOS Biology by Strait and colleagues [15] argues against these interpretations. Strait et al. find overlapping signatures of value coding in monkey vmPFC and VS, including evidence for abstract value coding, as required by the common currency model of decision making.

This result highlights a fundamental open question in the field of neuroeconomics: If multiple brain regions contain value signals, are these brain regions performing similar or distinct functions during the decision-making process? How do we account for the multiplicity of value signals found throughout the brain?

Computation

The fundamental question to ask about neural signals is: what computation are these signals providing to the behavior? Neural systems are computational processes evolved to select appropriate actions from information taken from one’s past (memory), one’s needs and goals (motivation), and one’s information about the world (sensory cues) [16].

The Question of Timing

Comparing the timecourse of representations in the vmPFC and VS, Strait et al. found that neural signals related to the value of the chosen action emerged earlier in the VS [15]. Interestingly, neural activity in the VS was able to predict the animal’s decision ahead of the choice period, but the neural activity in the vmPFC did not.

These results closely match a recent study in rodents where neural signals were recorded simultaneously from the OFC and VS in rats performing a neuroeconomic decision-making task [9]. Fig 1 shows a side-by-side comparison of the recording study in monkeys by Strait et al. [15] featured in this issue with a recording study in rodents by Stott and Redish [9]. In the study by Stott and Redish, reward expectation coding occurred earlier in the VS than in the OFC during trials in which the rat deliberated at the choice point (Fig 1D–1F). Strait et al. find a similar result of an earlier appearance of value-related signals in the VS than the vmPFC in the monkey (Fig 1A–1C). Both studies show that the VS contains value-related information before the OFC/vmPFC, hinting at potentially different roles for these structures, despite their similarity along other dimensions of value signaling [9,15]. In particular, the precise timing of activity in these two brain regions suggests that the VS activity arises earlier in the chain of events leading to action selection than the OFC or vmPFC.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic illustration of parallel neuroeconomic tasks in monkey and rat.

The top panel illustrates the task (A), recording locations (B), and one of the significant results (C) from the study highlighted in this issue of PLOS Biology by Strait and colleagues[15]. The bottom panel illustrates the task (D), recording locations (E), and one of the significant results (F) from the study by Stott and Redish, 2014[9]. Image credit: Karin Odell.

https://doi.org/10.1371/journal.pbio.1002174.g001

A Multiplicity of Value Signals in the Brain

Neuroeconomic experiments have identified a number of brain regions that exhibit neural responses related to reward value [9,17–22]. The data presented by Strait et al. [15] in this issue of PLOS Biology adds to this body of work, showing similar representations of value in the VS and vmPFC on the same task, albeit with differences in the timing of activity. Still, the broad overlap in value signaling between different structures invites the question of what these multiple brain regions are contributing to choice. There are three hypotheses that could explain the presence of value-related neural firing in multiple brain structures.

First, trivially, multiple structures could be contributing the same valuation computations to the system, at which point one would expect to see multiple copies of value-related signals. This hypothesis is unlikely to be correct, because studies have consistently found subtle differences between the representations of value, and because value-related signals have been found across the brain, including in structures well-known to be involved in other functions (such as sensorimotor integration [17]) and even in primary sensory areas such as the visual cortex [23].

Second, value representations could serve as a modulator of other signals. Value is important in a number of domains, including sensory, motor, and cognitive activities. In perceptual discrimination tasks, increasing the value of a target has been shown to increase neural firing in cells that detect sensory stimuli [23], in cells that are tuned to specific motor actions [24], and in cells that are more aptly described as “sensorimotor,” integrating sensory and motor information [17]. Therefore, value signals are important in focusing attention toward salient stimuli and in biasing motor commands toward more valuable actions [25]. These forms of secondary modulation (wherein a cell has a primary response to a stimulus or concept but shows firing rate modulation of that primary tuning) occur throughout the brain, such as attentional modulation of visual stimuli in the primary visual cortex [23] or task-related modulation of place representations in the hippocampus [26].

Third, when interpreting the role of different brain regions involved in choice, it is important to keep in mind that the brain may accomplish a given task using a variety of different solutions. A broad body of work now supports the view that the brain contains several functionally and anatomically distinct decision-making systems, each of which selects actions (makes decisions) using a different underlying computational algorithm [16,27,28]. These different decision-making systems could also be considered as different valuation systems [16,29], under the assumption that decisions reflect value. This interpretation could explain the existence of overlapping value signals seen in the vmPFC and VS. Both areas may routinely generate representations of value, as seen by Strait et al., but these brain areas may be differentially recruited depending on which action selection system is being engaged.

Each of these hypotheses call into question the centralized common currency hypothesis of value and opens new unanswered questions.

The Ignorance Questions

In his intriguing book Ignorance [30], Stuart Firestein suggests that the purpose of science is less to identify new answers as much as it is to identify new questions: what are the questions that you didn’t know were questions until you made this discovery? These new observations that value representations occur across the brain open up several new questions and call into question even the fundamental concept of value in a common currency.

The Concept of Value

As intriguing and logical as the common currency concept is, it has recently come under scrutiny [35]. Strait et al. [15] and others [1] have found multiple representations of value in the brain, but we do not yet know if these representations must agree with each other or not. If they do not, then we come to an interesting question of arbitration and mediation and what drives behavior in these conflict situations. The relationship between these valuation signals and the decisions being made by the animal is a very intriguing open and interesting question that will require additional experiments.