https://orcid.org/0000-0001-5051-0664
Figures
Abstract
Decisions as to whether to continue with an ongoing activity or to switch to an alternative are a constant in an animal’s natural world, and in particular underlie foraging behavior and performance in food preference tests. Stimuli experienced by the animal both impact the choice and are themselves impacted by the choice, in a dynamic back and forth. Here, we present model neural circuits, based on spiking neurons, in which the choice to switch away from ongoing behavior instantiates this back and forth, arising as a state transition in neural activity. We analyze two classes of circuit, which differ in whether state transitions result from a loss of hedonic input from the stimulus (an “entice to stay” model) or from aversive stimulus-input (a “repel to leave” model). In both classes of model, we find that the mean time spent sampling a stimulus decreases with increasing value of the alternative stimulus, a fact that we linked to the inclusion of depressing synapses in our model. The competitive interaction is much greater in “entice to stay” model networks, which has qualitative features of the marginal value theorem, and thereby provides a framework for optimal foraging behavior. We offer suggestions as to how our models could be discriminatively tested through the analysis of electrophysiological and behavioral data.
Author summary
Many decisions are of the ilk of whether to continue sampling a stimulus or to switch to an alternative, a key feature of foraging behavior. We produce two classes of model for such stay-switch decisions, which differ in how decisions to switch stimuli can arise. In an “entice-to-stay” model, a reduction in the necessary positive stimulus input causes switching decisions. In a “repel-to-leave” model, a rise in aversive stimulus input produces a switch decision. We find that in tasks where the sampling of one stimulus follows another, adaptive biological processes arising from a highly hedonic stimulus can reduce the time spent at the following stimulus, by up to ten-fold in the “entice-to-stay” models. Along with potentially observable behavioral differences that could distinguish the classes of networks, we also found signatures in neural activity, such as oscillation of neural firing rates and a rapid change in rates preceding the time of choice to leave a stimulus. In summary, our model findings lead to testable predictions and suggest a neural circuit-based framework for explaining foraging choices.
Citation: Ksander J, Katz DB, Miller P (2021) A model of naturalistic decision making in preference tests. PLoS Comput Biol 17(9): e1009012. https://doi.org/10.1371/journal.pcbi.1009012
Editor: Alireza Soltani, Dartmouth College, UNITED STATES
Received: April 19, 2021; Accepted: September 10, 2021; Published: September 23, 2021
Copyright: © 2021 Ksander et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available at https://github.com/johnksander/naturalistic-decision-making.
Funding: PM and DBK received funding and salary coverage from NIH and the BRAIN Initiative via R01 NS104818. JK received salary coverage from the Swartz Foundation #2020-13. Computational resources were provided by the Brandeis HPCC which is partially supported by the Brandeis Center for Bioinspired Soft Materials, and NSF MRSEC, DMR-2011846. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Decisions as to whether to stay in a current situation or switch to a different one face all animals continuously. Such “should I stay, or should I go now” decision-making has been studied most in the context of foraging [1–4], in which an animal decides whether to continue to stay at a current source of food or to seek an alternative source. In foraging studies, the current source of food necessarily yields diminishing returns, such that at some point in time it is optimal for the animal to seek a higher-quality option. Furthermore, this tendency appears “baked in” to animal behavior: while food sources in laboratory food preference tests remain in constant supply, animals nonetheless typically switch back and forth between two or more alternatives.
Here, we simulate a model of such switching back and forth in terms of transitions between quasi-stable attractor states of neural activity. Quasi-stable attractor states [5] are patterns of neural activity that are essentially self-sustaining but limited by adaptation processes or fluctuations that eventually lead to a loss of stability and a transition to a new pattern of activity [6]. Evidence for such attractor states is most abundant in neural activity arising from perception [7,8], with quasi-stability most apparent in the switching between bistable percepts. More recently, experimental evidence for transitions between attractor states has been found [9,10] and modeled [11–13] in perceptual decision-making tasks. For those tasks, the attractor states have been proposed to represent one of two possible percepts for a given stimulus [14], or the absence of a decision (i.e., an undecided state)[11,13]. In the more naturalistic decision-making model we present here, two activity states represent the ongoing choice to either stay with a current stimulus or switch to a new stimulus [4]. It is revealing that, according to behavioral data [15], the distribution of bout durations, i. e., when an animal stays at a stimulus (corresponding to the durations of the “stay” state in our model) is approximately exponential, which is a hallmark of noise-induced transitions between discrete states [16].
Our simulations differ from those aimed to model the circuitry underlying perceptual decision making in a second way. In preference tests the stimuli are separated in time, rather than via distinct neural pathways. For example, two tastes being compared could simply be different concentrations of sodium chloride concentration, which yields an inverted-U palatability response [17]. The two stimuli excite identical neurons during successive sampling bouts, with neurons indicating high palatability during one bout being the same as those indicating high palatability during the next bout. In this manner, a preference task is more akin to decision making tasks requiring a sequential comparison of a single parametric quantity [18–20], although in this paper the quantity being compared is palatability and the decision is an ongoing one of how long to stay at the stimulus.
Our work compares performance between two model classes, distinguished by their parameter settings. In one class, the system is inherently “fickle” in that the “stay” state is unstable; an animal operating by this model would stay for only a very short duration of time unless neural activity is stabilized by input indicative of a highly positive hedonic stimulus. One could say in these networks that a delicious stimulus entices the animal to stay, but otherwise the animal leaves. In the other class of model, the system is “committed” in the sense that the “stay” state is highly stable, such that an animal operating by this model would stay for relatively long durations, until input suggesting stimulus aversiveness causes a transition away from that state. One could say in these networks that the stimulus causes the animal to leave, but that otherwise the animal stays.
We produce multiple versions of each class of model, in order to assess how they could be reliably distinguished in behavioral or electrophysiological data [4,21,22]. Our simulations suggest that any observation of strongly reduced sampling-bout durations at one stimulus following the sampling of a highly hedonic stimulus would indicate animals operating with the “entice to stay” class of network. Also, in the “entice to stay” class of network, neural activity appears to change gradually over more than 100ms preceding a choice to leave a stimulus, whereas in the “repel to leave” network, the activity change is more sudden, but preceded by oscillations in the beta range. Therefore, we suggest that choice-aligned averaging of neural spike trains, combined with appropriate analysis of animal behavior in preference tasks [23] can distinguish the two classes of models.
Results
Network characteristics
Our initial goal was to set up a network with two distinct activity states, one representing an animal continuing to sample a stimulus, or “staying”, the other representing the choice to “switch” elsewhere. Fig 1 shows the behavior of such a network in the absence of stimulus (i.e. when the network receives only noisy background input). The raster plot in Fig 1B illustrates how the network’s activity abruptly shifts back-and-forth between two states of activity, which are quasi-stable attractor states. When the excitatory neurons in the “stay” population become highly active, those in the “switch” population become silent, and vice versa. Fig 1C shows these same data in terms of each population’s averaging spike rates. Noisy fluctuations in the spiking activity drive these transitions between stay and switch states. While Fig 1 indicates the quasi-stable nature of the circuit in the absence of any stimulus, when we include a stimulus in the circuit simulation, activation of the “switch” population represents the choice to end a bout of sampling.
Example data from a fast (“entice to stay”) network in the absence of stimulus. (A) Circuit diagram, indicating cross-inhibition between excitatory populations of cells labeled E, inhibited by their corresponding inhibitory population labeled I. (B) Spike rasters from model neurons arranged in rows grouped by their corresponding populations and color-coded as in (A). Each dot represents one spike from the model neuron. (C) Mean firing rate of each population as a function of time. (D) Dynamics of the fast-depression variable. (E) Dynamics of the slow-depression variable. (C-E) Line color indicates which population’s average variable is plotted with the color code as in (A).
Our model incorporates synaptic depression, whereby a rapid series of neural spikes depletes the supply of synaptic vesicles and therefore reduces synaptic efficacy. A docking process, which renders available vesicles release-ready is relatively fast (a few hundred ms), so we denote the fraction of docked vesicles as Dfast. Recycling and filling of vesicles to regenerate the activity-depleted pool is slower (many seconds), however, so we denote the fraction available for docking as Dslow. Fig 1D–1E show how the population-averages of these depression variables change as the model switches between activity states shown in Fig 1B–1C. Specifically, Fig 1D illustrates how docking sites release their vesicles during sustained spiking, and then quickly become replenished from the reserve pool after activity ceases. Fig 1E illustrates how activity empties reserve vesicle pools, as the vesicles stored there replenish empty docking sites. The reserve vesicle pools then slowly recoup their losses after the cells stop firing and vesicle regeneration outpaces the losses to docking sites. Note that reserve vesicle pools sometimes fail to refill fully before the next state transition. A long active state will deplete the reserve vesicle pool to a degree that those reserves cannot fully recover from without a lengthy inactive period.
Parameter sweep and example networks
In order to ensure that any results of our study are robust to parameter variations, we generated a number of distinct “entice to stay” and “repel to leave” networks before assessing qualitative differences between the two classes’ behavior. To this end, we first measured the mean durations of “stay” states in multiple networks generated across a five-fold range of excitatory-to-inhibitory (E-to-I) and inhibitory-to-excitatory (I-to-E) connection strengths.
Fig 2 indicates the range of parameters for which we found state-switching behavior, with mean duration of the “stay” state indicated in color in Fig 2. We chose five networks of each of the two classes, with “entice to stay” networks having the shortest intrinsic “stay” state durations of under two seconds (solid symbols in Fig 2) and “repel to leave” networks having the longest intrinsic “stay” state durations of over 100 seconds (open symbols in Fig 2). We typically paired networks (same symbol shape in Fig 2) with either the same E-to-I connection strength or the same I-to-E connection strength. Across the range of parameters, firing rates of excitatory and inhibitory neurons were at levels compatible with those of cortical neurons in an active state (near 10 Hz for excitatory cells and up to 60 Hz for inhibitory cells, S1 Fig). These results demonstrate that our circuit produces quasi-bistable state-switching behavior with realistic spike-rates and that this behavior is robust to variation in synaptic connection strengths (see also S2 Fig).
Average duration of the “stay” state in distinct networks with different synaptic connection strengths, without stimulus. Symbols indicate parameters of the five example networks, with solid symbols representing fast-switching “entice-to-stay” networks and open symbols representing slow-switching “repel to leave” networks. Identical shapes indicate the pairs used for comparison.
We next performed simulations with all ten example networks (symbols in Fig 2). Our results generalized well across these parameterizations, so for simplicity we first describe data arising from one pair of networks from each class–specifically, from the networks with square markers in Fig 2. In the supplementary materials (S3 and S4 Figs) we detail the simulation results from all ten networks in cases where they are not all included in the main text, for completeness.
Since, by definition, sampling bouts are in the presence of a stimulus, and their durations are a key behavioral measurement, our first goal was to find “baseline” stimulus values that reproduced similar mean durations of the “stay state” for each network. We therefore defined a neutral stimulus for each network, as one which produces mean “stay state” durations between eight and nine seconds, in the middle of the observed ranges of typical bout durations for a rat licking taste stimuli from a spout [15,24].
Fig 3 illustrates how application of the neutral stimulus to intrinsically fast-switching networks via excitation of neurons driving the “stay” state (upper left) realizes the “entice to stay” operation of a stimulus. The stimulus slows the network’s transitions from stay to leave from its stimulus-free duration of under two seconds such that the network achieves the desired mean bout duration (lower left). Conversely, application of the neutral stimulus to intrinsically slow-switching networks via excitation of neurons driving the “switch” state (upper right) realizes the “repel to leave” operation of a stimulus. In these networks, the stimulus quickens the network transition such that the network’s stimulus-free mean durations in the “stay” state of a few minutes are shortened to the desired mean bout duration (lower right). In these distinct manners, the same mean behavior to a given, neutral, external stimulus can arise from either class of network.
Left: Representing a neutral taste stimulus via excitatory input to “stay”-promoting neurons in a fast (entice-to-stay) network slows state transitions. Right: Whereas representing a neutral taste stimulus via excitatory input to “switch”-promoting neurons in a slow (repel-to-leave) network quickens state transitions to produce the same switching dynamics.
We do not simulate the behavior of the animal between bouts of tasting, but once the activity state indicates a switch of stimuli, we remove all stimulus input in the model to indicate movement away from the food source. We subsequently reduce input to the excitatory “switch”-producing cells to indicate completion of the movement and ensure commencement of the next sampling bout with the “stay” state. In the following sections the stimulus during the subsequent bout corresponds to the alternative tastant, which in many cases has a different palatability so produces a spike train of a different rate, as indicated in Fig 4.
Animals alternate between a less palatable stimulus A and a more palatable stimulus B. In the intrinsically fast-switching “entice-to-stay” network, the increased palatability arises from greater excitatory input to the excitatory “stay” pool of neurons, whereas in the intrinsically slow-switching “repel-to-leave” network, the increase in palatability is produced by a reduction of excitatory input to the excitatory “switch” pool of neurons.
Taste preference behavior
In taste preference tests, consuming one stimulus comes at the expense of another, primarily because the animal has limited desire and time for consuming food or liquid. Therefore, as in any time-limited task, there is an inherent competition in that more time spent doing A means less time spent doing B. However, a key unanswered question is whether the nature of one taste stimulus impacts the neural and behavioral responses to a subsequent, second taste stimulus. Association with a more palatable stimulus via temporal proximity could, in principle, enhance the perceived palatability of an alternative stimulus. However, the qualitative results of foraging studies as encoded in the Marginal Value Theorem, suggest the reverse is true. That is, during foraging, animals spend less time at a food source if there are better sources easily reached elsewhere. So, in a food preference task it is possible that higher stimulus palatability of one food source leads to a shortening of bout durations at the alternative food source. Therefore, we assessed whether such bout-to-bout competition between stimuli would arise in our networks.
Fig 4 illustrates how we simulate the taste preference task, such that the model switches between stimuli of different palatability (see Taste preference task simulations in the Methods section for full details). A key distinction between the two classes of models is apparent even before examining their performance, in the different representations of stimulus palatability. When testing the intrinsically fast-switching “entice-to-stay” networks, we simulate a more palatable tastant by increasing the excitatory input to excitatory neurons representing the “stay” activity state; this increases “stay” state durations. When testing the intrinsically slow-switching “repel-to-leave” networks, meanwhile, we simulate that same tastant by reducing the excitatory input to excitatory neurons representing the “switch” activity state; this also increases “stay” state durations. We identify the time spent in the “stay” state as the duration of active bouts of food sampling, and therefore proportional to the amount of food consumed (the basic behavioral measure of palatability). In these distinct manners, both classes of network represent taste stimuli of varying palatability via varying afferent firing rates.
To assess the impact of one stimulus on the other, we measured the mean bout durations (our measure of stimulus palatability) of stimulus A as a function of the mean bout duration of stimulus B. For these simulations, we fixed one of a pair of stimuli (stimulus A, baseline) and varied the palatability of the second stimulus (stimulus B, varied).
The results of these analyses, which are shown in Fig 5, differentiate network type. For both classes of network, competition between bout durations arises—with increased palatability and bout duration of the varied stimulus B, the bout durations of the fixed stimulus A decrease. For the intrinsically slow switching “repel-to-leave” networks (red points in Fig 5), the competition can produce a factor of two reduction in the sampling bout durations at the unchanged stimulus A. However, the competition has a much larger impact on the intrinsically fast switching “entice-to-stay” networks (blue points in Fig 5), such that sampling bout durations at the fixed stimulus A can decrease by up to ten-fold over the same range of B variation.
(Top) Behavior of intrinsically fast-switching (“entice-to-stay”) networks is shown as blue points, and intrinsically slow-switching “repel-to-leave” networks as red points. In both networks the greater the palatability of stimulus B, the lower the perceived palatability of the unchanging stimulus A, but the bout-to-bout competition is much stronger in the “entice-to-stay” network when stimulus B is highly palatable. (Bottom) Mean durations of the “stay” state are plotted as a function of the ratio of the inputs corresponding to the two stimuli, A and B. Each pair of a values for a given ratio provides one data point in the top panel. The input for stimulus A is held fixed while the input for stimulus B is varied as a parameter to produce distinct values along the x-axis. The two classes of network are distinguished in whether the input excites the “stay” pool (entice-to-stay network, lower left) or the “leave” pool (repel-to-leave network, lower right) and whether, respectively, high input or low input for stimulus B reflects the most hedonic stimulus B. Competition is revealed as the duration of responses to stimulus A (solid line) changes in the absence of any change in inputs for stimulus A (most evident in the left panel).
To explain this more strongly competitive interaction between successive stimuli in the “entice-to-stay” network (Fig 5, lower panels, solid curves), it is worth considering differences in how the relative inputs arising from hedonic stimuli impact these networks. In the “entice-to-stay” network, a positively hedonic stimulus B is simulated in terms of large amounts of input to the “stay” neurons in the network. These neurons receive strong input and fire at high rates as they represent an animal sampling the positively hedonic stimulus for long durations. A long duration of relatively high firing rates, meanwhile, is exactly the network activity that is expected to deplete the reserve vesicle-pool and maximize adaptation at the circuit-level. Note that the converse does not apply–that with a highly aversive stimulus B, and lower firing rates of the “stay” neurons, the sampling durations in response to stimulus A do not increase significantly.
In the “repel-to-leave” network, meanwhile, the coincidence of extended duration higher than baseline firing rate never arises. The increased hedonic value of B is instead achieved via reduced input to the “leave” neurons in the network, such that long sampling of a positively hedonic stimulus causes a reduction of synaptic depression in the circuit. One might expect that synaptic depression therefore induces competition in the opposite case—when stimulus B is highly unpalatable, the input to the “leave” neurons, and therefore their firing rates, are high—but in such a scenario the bout durations are too short for the reserve vesicle-pool to be depleted significantly.
In conclusion, Fig 5 demonstrates differences between the behavior of the intrinsically fast-switching “entice-to-stay” network and that of the intrinsically slow-switching “repel-to-leave” network, with strong competition between bout durations only possible in the “entice-to-stay” class of networks. Moreover, our modeling reveals an asymmetry in the nature of competition, whereby a highly hedonic alternative stimulus reduces the duration of sampling bouts, but a highly aversive alternative stimulus produces bout durations that are little different from those with a neutral alternative stimulus, or no alternative at all.
Network transition dynamics
One of our modeling goals was to assess whether these two classes of network show systematic differences in neural activity, in addition to the behaviorally observable quantitative differences in the degrees of competition shown in Fig 5. One structural difference has to do with activity in the circuit causes a transition away from stimulus sampling. In the case of the intrinsically fast-switching “entice-to-stay” networks, this transition is caused by a dip of input to the stay population, whereas in intrinsically slow-switching “repel-to-leave” networks the cause is a rise of input to the leave population. Given that neurons in any circuit are strongly interconnected in dynamic systems such as those described here, it is not clear that any change in one set of neurons could be measured independently of changes in other sets of neurons. However, we hypothesized that by aligning neural activity to the state-transition points we might make progress in uncovering distinctive signatures in the neural dynamics of our models that would allow us to distinguish the two classes of network using electrophysiological data.
Fig 6 presents such transition-aligned activity in examples of intrinsically fast-switching (entice-to-stay) networks (left) and intrinsically slow-switching (repel-to-leave) networks (right). While many differences between these examples (e.g. net firing rates), proved inconsistent across broader swaths of the two classes of models, two distinguishing features of the pre-transition activity are reliable: First, whereas in “entice-to-stay” networks (left), activity in the excitatory cells that cause the transition (E-switch, red) increases gradually during the 200ms before leave decisions, in “repel-to-leave” networks (right) E-switch activity remains low until only a few tens of milliseconds before the decision (at which point there is a sudden and rapid increase); second, the neurons inhibiting the transition (I-switch cells, purple) begin oscillating in the pre-transition period for “repel-to-leave” networks, but the same does not occur in “entice-to-stay” networks. These observations suggest measured neural activity (such as that acquired from gustatory cortex) could enable us to determine which class of network is active in a rodent’s brain, if aligned to the time-point ending a bout of sampling.
A transition is detected when synaptic output of the E-switch neurons significantly exceeds that of the E-stay neurons, so the transition time (0 ms) is inevitably marked by a spike in activity of the E-switch neurons. A decline in activity of neurons that promote the “stay” (E-stay, blue) or inhibit the switch (I-switch, purple) combined with a rise in activity of neurons that promote the switch (E-switch, red) or inhibit the stay (I-stay, yellow) precedes the detected transition. Left: the inherently fast-switching, “entice-to-stay” networks. Right: the inherently slow-switching “repel-to-leave” networks.
Discussion
It is worth comparing and contrasting the model we present here of stay-versus-switch decision making, based on a naturalistic task, with other models of decision making in systems neuroscience [11,13,14,25–31]. In particular, over the past few decades, much investment has been spent in studying winner-takes-all models of decision making that underlie the behavior of two-alternative forced choice tasks [25,29,32], usually in the context of perceptual decisions [33,34]. The framework of such behavioral tasks is trial-based, where the experimenter chooses the stimuli, and the subject makes a single, final choice in each trial. The relative preference for one alternative can be obtained after accumulating many discrete trials. Conversely, in naturalistic tasks such as food preference tests or foraging [1,35], the subject selects the stimulus, but the selection is not final. The choice is inextricably linked to the stimulus and is ongoing and dynamic.
Moreover, models of the two types of decision-making tasks must be distinct in two ways. First, any competition between stimuli must occur across time in a naturalistic task, since only one stimulus is present at any given time. Second, and importantly, the property of each of the two stimuli being evaluated is likely to provide input to the decision-making circuitry via the same neural pathway. Whereas in the most common perceptual decision-making paradigms, and therefore in models thereof, the two competing inputs correspond to opposing stimuli (such as directions of motion) and so excite different groups of neurons. In such perceptual tasks, the competition between stimuli can arise via a direct inhibitory interaction between the simultaneously active distinct groups of neurons. By contrast, in food preference tests the hedonic, appetitive neural response to one stimulus must compete with a similar hedonic, appetitive neural response to a temporally separated stimulus. Therefore, these naturalistic decisions have some similarity to parametric working memory tasks [19,20,36,37] where subjects compare two different levels of input via the same afferent pathway.
Our model is state-based, behaving similarly to models of bistable percepts [6,38] in that the neural activity can transition at a stimulus-related but non-deterministic time from one state to another, even if the stimulus does not change. Evidence for such noise-induced transitions between otherwise stable states has been obtained across a number of neural systems [5,39–44], even where trial-averaged activity may suggest a slower ramping akin to evidence accumulation [10]. Indeed, a similar state transition may underlie the more commonly studied perceptual decision-making tasks [9].
While our model does not designate the location of the simulated circuit, we assume all neurons in our model are located in the same region and are likely to be found in anterior cingulate cortex, where neurons represent value in self-paced decisions [45], or a subregion of the ventromedial prefrontal cortex, such as the infralimbic cortex, whose activity is needed for an animal to engage in feeding behavior [46].
The nature of competition
Decision making inevitably involves competition in that choices have opportunity cost. The more often one alternative is chosen, the less often another alternative can be chosen. In winner-take-all networks the competition is inherent in the network’s structure and a binary decision arises on a trial-by-trial basis [14,25]. That is, at the level of neural activity, increased spiking of one group of cells inhibits the spiking of other groups [11,14]. At the level of behavior one choice terminates the trial and prevents any other choice.
By contrast, in a food preference test implicit competition can arise in the absence of any correlation between the neural activity in response to one stimulus versus the other. This is simply due to the finite amount of time available, such that if one stimulus is extremely hedonic the animal will rarely leave it, so inevitably consume less of any alternative stimulus. That is, even if the bout duration of an alternative stimulus is unchanged, an animal would leave a more hedonic stimulus less frequently to visit that alternative, so consume less of the alternative than if that alternative were paired with a less hedonic stimulus.
Theoretically, it is possible for a highly hedonic stimulus to boost the hedonic value of an alternative stimulus while still maintaining such implicit competition between the total amounts of the two foods being consumed. Such behavior could arise if, for example, the neural activity were dominated by a slow synaptic facilitation such that the impact of a highly hedonic response lingered so as to boost responses to later stimuli. However, to align with the qualitative nature of the marginal value theorem [2,47]—one location is more easily left if the alternatives are better—we incorporated synaptic depression as the dominant effect of the network on a timescale of many seconds.
Synaptic depression—an inevitable consequence of synaptic transmission as vesicles are depleted—causes a reduction in the hedonic response to any stimulus that follows a highly hedonic stimulus. In this manner, synaptic depression accentuates competition between two taste stimuli in a food preference test, because the mean bout duration when sampling the less hedonic stimulus is shortened in the presence of a highly hedonic alternative stimulus. Such competition based on a trace-memory of prior stimuli is in addition to the implicit competition already present in a food preference task. We find that the reduction in bout durations in the presence of a highly hedonic alternative is most pronounced, with a ten-fold reduction possible, in the “entice to stay” networks we simulated here.
Our goal here is to assess what behavioral phenomena arise from a very simple decision-making circuit when common neuronal features are incorporated. The short-term synaptic depression we include is one of many biological processes that sculpt network activity over multiple timescales. Short-term depression allows our network to be reactive to the recent past, so could account for anhedonia in a short period following a strongly rewarding stimulus [48]. However, it is insufficient to produce the longer-term learning needed for an animal to anticipate the future based on past experience, as in the anticipatory contrast effect [49]. It is intriguing however, that the basic phenomenon of the devaluing of a rewarding stimulus (such as saccharine) in circumstances when a more highly rewarding or addictive stimulus is available or anticipated (such as cocaine or sucrose)[50–52], arises in our relatively simple circuit.
Our networks reproduce many effects underlying the marginal value theorem [2,47], which provides a framework for behavior observed in the foraging literature [35]. According to the marginal value theorem, an animal stays at a patch of food until its rate of return diminishes to the average rate of return obtained by moving from site to site across the environment. While we do not simulate an experiment with reduced stimulus over time during a bout, our model does reproduce some of the qualitative features essential in any neural circuit model of such foraging dynamics. First, the more hedonic/palatable the current stimulus, the greater the bout duration at that stimulus. Second, the more hedonic/palatable the alternative stimulus, the shorter the bout duration at the first stimulus. Third, the greater the time in the switching period between stimuli, the less the impact of a more hedonic alternative. Importantly, the third effect mitigates the second effect; longer times spent between stimuli will mitigate shorter sampling bouts in the presence of superior alternatives. Such reluctance to switch to more favorable alternatives arises from the cost of switching in value-based models, because animals do not accumulate reward while traveling between samples. Here we show that such behaviors, desirable from the point of view of optimality, arise qualitatively in a simple circuit with depressing synapses.
Methods
Properties of model neurons
Individual neurons were simulated with an exponential leaky integrate-and-fire model [53] following the equation:
where Vm is the membrane potential, Cm is the total membrane capacitance, El is the leak potential, Rm is the total membrane resistance, Δth is the spiking range, Vth is the spiking threshold, S is the synaptic input variable, Gsyn and Erev are the maximal conductance and reversal potential for synaptic connections, Gref is the dynamic refractory conductance, EK is the potassium reversal potential, and Gext is the input conductance. The “E” and “I” subscripts denote the variables specific to excitatory and inhibitory channels, respectively (e.g. SE and
are the synaptic input and reversal variables for excitatory channels; SI and
are the corresponding inhibitory variables). This equation simulates the neuron’s membrane potential until Vm>Vspike, at which point the neuron spikes.
When a neuron spikes, Vm is set to the Vreset value. Additionally, the neuron’s refractory conductance, synaptic output, s, and synaptic depression (noted as D) are updated according to the equations:
where ΔGref is the increase in refractory conductance, and pR is the vesicle release probability following a spike.
In the timestep immediately following a spike, the neuron’s membrane potential continues to follow the exponential leaky integrate-and-fire model equation. In this equation the separate excitatory (SE,i) and inhibitory (SI,i) synaptic inputs for cell i are obtained from the sum of all presynaptic outputs multiplied by the corresponding connection strengths, Wij, from neurons j (see Network architecture and connections):
each of which decay with the appropriate (excitatory or inhibitory) synaptic gating time constant τS according to:
Likewise, refractory conductance decays with the time constant τref according to:
The Gext input conductance serves as both noisy-background and stimulus inputs in the same manner. Inputs were modeled as Poisson spike trains with rates rnoise and rstimulus, which produce input spikes (from all sources) at timepoints {tsp}. Please note, the noisy-background includes both excitatory and inhibitory spiking input (included in
and
, respectively); the rnoise parameter specifies the rate for both excitatory and inhibitory background noise. The input conductance values for a given timepoint, t, are updated as:
where the conductance increases by ΔGext at the time of each input spike. The input conductance otherwise decays with the time constant τext according to:
The cellular parameters with values specific to excitatory neurons (e.g. that differ from inhibitory values) are: τs = 50 ms, and τext = 3.5 ms. The complementary values for inhibitory neurons are:
τs = 10 ms, and τext = 2 ms. The remaining parameters applicable to both excitatory and inhibitory neurons are: Gsyn = 10 nS, pR = .1, τfast = 300 ms, τslow = 7 s, pslow = .5, El = −70 mV, EK = −80 mV, Vreset = −80 mV, Rm = 100 MΩ, Cm = 100 pF, Vspike = 20 mV, ΔGext = 1 nS, Vth = −50 mV, Δth = 2 mV, τref = 25 ms, and ΔGref = 12.5 nS. The Poisson spike-train parameters rnoise and rstimulus are described in the next section. Neurons were simulated with a simulation timestep dt = .1 ms.
Synaptic depression
We modeled synaptic depression using two separate timescales, noted in the previous spike-update equations as Dslow and Dfast. These two variables reflect, respectively, the fraction of the maximum number of vesicles available in the reserve pool and the release-ready pool. Following a spike, the variables recover to a value of one with different timescales, because vesicles regenerate and are replenished slowly in the reserve pool, but may dock and become release-ready much more quickly once available in the reserve pool (Fig 7).
Vesicles are either in the reserve pool (a) or at docking sites (b). Available vesicles dock quickly at empty docking sites (c), but take much longer to regenerate once their internal neurotransmitter is released (d).
Specifically, Dslow represents the ratio of currently available reserve-pool vesicles out of the maximum possible, that is (Fig 7A). These dock quickly at empty docking sites on the timescale τfast (Fig 7C), but are replaced slowly on the timescale τslow. Dfast represents the ratio of docked vesicles out of total docking sites, that is
(Fig 7B). We also incorporate the constant parameter, fD = 0.05, which is equal to the ratio of the number of docking sites to the maximum size of the reserve pool of vesicles,
. Only docked vesicles can be released immediately following a spike, such that upon each spike we update Dfast↦Dfast(1−pR) where pR is the vesicle release probability.
During sustained spiking, the fast-docking can maintain a firing-rate dependent supply of docked vesicles until the reserve pool (Fig 7B) depletes. Vesicles dock at empty sites (Fig 7C) according to:
Reserve-pool vesicles fill the empty docking sites on the fast timescale τfast. On the other hand, the reserve-pool regenerates much more slowly according to:
The first term represents the reserve-pool vesicle regeneration on timescale τslow. The second term
accounts for the vesicles lost due to docking (Fig 7C).
Our model reflects the empirical evidence showing the effects of synaptic-depression at short timescales on the order of milliseconds, and longer timescales on the order of seconds [54,55]; depression timescales on the order of minutes have even reported in non-mammalian animals [56]. Additional, recent evidence [57] directly supports our fast-depression mechanism where available vesicles quickly refill empty docking sites. Our model provides a coherent mechanism for both fast-acting and long-lasting synaptic depression effects.
Network architecture and connections
Each network consists of 250 individual neurons, split into two populations of 100 excitatory cells (i. e., “stay” and “switch” populations, Estay and Eswitch) and two populations of 25 inhibitory cells (Istay and Iswitch). Fig 1A depicts the basic architecture of all networks simulated in this paper, with connections within and between populations indicated. For each pair of connected populations (or for self-connected excitatory populations) pairs of cells were connected probabilistically with a probability, P(connection) = .5. The strength of connections were symmetric across “stay” and “switch” populations, but depended on whether presynaptic or postsynaptic cells were excitatory or inhibitory, as indicated in Table 1.
Code availability
The code used to simulate our model is freely available online at https://github.com/johnksander/naturalistic-decision-making
Network states and stimuli
A network’s active state was evaluated by comparing the mean values of synaptic output, sE, averaged across all excitatory cells in each of the two excitatory populations. Specifically, when the difference between the mean output of the previously less active excitatory population exceeded that of the previously more active excitatory population by a threshold of 0.02 consistently for 50ms, we recorded a state change. The threshold was chosen to be .02 as a value which robustly captured the intended quasi-bistable behavior across multiple networks. If a simulation produced one second of consecutive timepoints without an active pool (i. e., because the difference in activity of the two excitatory populations was too small to produce above-threshold differences in synaptic output) the simulation was terminated and the corresponding parameter set was not used for further analysis. We only analyzed simulations of networks exhibiting quasi-bistability
In our simulations of preference tests, we did not simulate the animal’s behavior in between bouts of sampling a stimulus. Once the excitatory neurons in the “switch” population (E-switch cells) were recorded as more active than those in the “stay” population, using the threshold mentioned above, we removed the stimulus input to the network. 100 ms later, we induced a subsequent transition back to the “stay” state to represent the animal initiating a new bout of stimulus sampling. The transition back to sampling was accomplished by halving the noisy background input to E-switch cells until the network transitioned again to the “stay” state. At all other times in simulations, the noisy background input remained constant. Once a transition to the “stay” state was recorded (by excitatory cells in the “stay” population being more active than those in the “switch” population) input stimulus was applied to indicate the next bout of sampling.
Parameter sweep and example networks
We obtained a set of examplar networks by varying I-to-E and E-to-I connection strengths and recording the average active-state durations. Networks were simulated without stimulus (but with noisy-background input) for 1500 seconds. We then chose five pairings of intrinsically fast-transitioning and intrinsically slow-transitioning networks from this parameter space. The fast and slow networks represent the “entice-to-stay” and “repel-to-leave” decision-making accounts, respectively. Networks were paired across the two classes by selecting examples with connection strengths as closely matched as possible, while average state-durations were either end of the wide range recorded. Typically, fast networks transitioned in a few seconds, whereas slow networks transitioned in a few minutes. We performed all subsequent simulations with 10 exemplar networks (five pairs) as represented by the symbols in Fig 2.
For each of the 10 exemplar networks, we determined the level of stimulus necessary to equate the baseline sampling behavior, producing an average duration of 7.5 seconds for the “stay” state in each case. Increasing the strength of hedonic stimuli (increasing E-stay spiking input) slowed the fast network transitions, while increasing the strength of aversive stimuli (increasing E-leave spiking input) quickened slow network transitions. The specific intensities were found with Matlab’s fminbnd() optimizer function. We will refer to these stimulus values as the networks’ “baseline stimuli”, as we used these values as reference points for subsequent simulations.
Taste preference task simulations
Individual taste preference task simulations lasted 1500 seconds total. Each simulation compared sampling bout durations in response to two stimuli (A and B) each with a fixed value across the session. The input representing stimulus A was equal to the network’s baseline stimulus value, and constant across sessions. The input representing stimulus B was systematically varied across sessions, such that in each session it was fixed at a value between zero and sixty times the input for stimulus A. For a given network the stimulus inputs targeted the same population for all sessions. The stimulus value (A or B) alternated after each stay-to-leave transition. Simulations always began with stimulus A during the first stay-state, followed by stimulus B during the second stay-state, etc. This represents the animal alternating between the available stimuli after “leave” decisions.
Supporting information
S1 Fig. Firing rates of active cells in the “stay” state as a function of network connection strengths.
(Left) Firing rate of excitatory cells is in the vicinity of 10 Hz for all parameters leading to two quasi-stable network states. (Right) Firing rate of inhibitory cells varies strongly as a function of parameters, but note that the range of rates is similar for entice-to-stay networks (left edge) and repel-to-leave networks (right edge) so firing rate is not a clear indicator of type of network.
https://doi.org/10.1371/journal.pcbi.1009012.s001
(TIFF)
S2 Fig. State durations in an alternative network to indicate robustness of the system to incorporation of additional connections.
The results are produced for a circuit depicted in A) with additional connections (excitatory between excitatory pools and inhibitory both within and between inhibitory pools) all at half the strength of the excitatory within-pool connections of the standard network. B) Results indicate the same network responses as in the original network are possible given small compensatory shifts in the inhibitory-to-excitatory and excitatory-to-inhibitory connections (compare main manuscript, Fig 2).
https://doi.org/10.1371/journal.pcbi.1009012.s002
(EPS)
S3 Fig. Competitive interaction in taste preference tests.
All network results are shown, with marker symbols identifying the network parameters as depicted in Fig 2 (main text Fig). Results for all of the intrinsically fast-switching (entice-to-stay) networks exhibited a stronger impact on the duration of bouts at the stimulus of fixed input, A, as the strength of input from the alternative stimulus, B, was adjusted to reflect more hedonic input.
https://doi.org/10.1371/journal.pcbi.1009012.s003
(TIFF)
S4 Fig. Impact of varying one stimulus input on sampling durations.
In all panels, input corresponding to stimulus B is adjusted across sessions, while input corresponding to stimulus A is held fixed. Mean bout durations in response to stimulus B are shown as dashed lines and increase with increasing input in entice-to-stay networks (left panels), while they decrease with increasing input in repel-to-leave networks (right panels). Mean bout durations in response to stimulus A (which never changes across sessions) are shown as solid lines. Competition arises as durations for A (solid lines) decrease when durations for B (dotted lines) increase, with a much stronger effect in all of the entice-to-stay networks (left panels).
https://doi.org/10.1371/journal.pcbi.1009012.s004
(TIFF)
References
- 1. Pearson JM, Watson KK, Platt ML. Decision making: the neuroethological turn. Neuron. 2014;82(5):950–65. pmid:24908481; PubMed Central PMCID: PMC4065420.
- 2. Charnov EL. Optimal Foraging, the Marginal Value Theorem. Journal of Theoretical Population Biology. 1976;9(2):129–34. pmid:1273796
- 3. Cowie RJ. Optimal foraging in great tits (Parsus major). Nature. 1977;268:137–9.
- 4. Hayden BY, Pearson JM, Platt ML. Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci. 2011;14(7):933–9. pmid:21642973; PubMed Central PMCID: PMC3553855.
- 5. Miller P. Itinerancy between attractor states in neural systems. Curr Opin Neurobiol. 2016;40:14–22. pmid:27318972; PubMed Central PMCID: PMC5056802.
- 6. Moreno-Bote R, Rinzel J, Rubin N. Noise-induced alternations in an attractor network model of perceptual bistability. J Neurophysiol. 2007;98(3):1125–39. pmid:17615138; PubMed Central PMCID: PMC2702529.
- 7. Kanai R, Moradi F, Shimojo S, Verstraten FA. Perceptual alternation induced by visual transients. Perception. 2005;34(7):803–22. Epub 2005/08/30. pmid:16124267.
- 8. Jones LM, Fontanini A, Sadacca BF, Miller P, Katz DB. Natural stimuli evoke dynamic sequences of states in sensory cortical ensembles. Proc Natl Acad Sci U S A. 2007;104(47):18772–7. pmid:18000059; PubMed Central PMCID: PMC2141852.
- 9. Latimer KW, Yates JL, Meister ML, Huk AC, Pillow JW. NEURONAL MODELING. Single-trial spike trains in parietal cortex reveal discrete steps during decision-making. Science. 2015;349(6244):184–7. pmid:26160947.
- 10. Sadacca BF, Mukherjee N, Vladusich T, Li JX, Katz DB, Miller P. The Behavioral Relevance of Cortical Neural Ensemble Responses Emerges Suddenly. J Neurosci. 2016;36(3):655–69. pmid:26791199.
- 11. Miller P, Katz DB. Stochastic transitions between neural states in taste processing and decision-making. J Neurosci. 2010;30(7):2559–70. Epub 2010/02/19. 30/7/2559 [pii] pmid:20164341.
- 12.
Miller P, Katz DB. Stochastic Transitions between States of Neural Activity. In: Ding M, Glanzman DL, editors. The Dynamic Brain: An Exploration of Neuronal Variability and Its Functional Significance. New York, NY: Oxford University Press; 2011. p. 29–46.
- 13. Miller P, Katz DB. Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits. Journal of Computational Neuroscience. 2013;35(3):261–94. Epub 2013/04/24. pmid:23608921.
- 14. Wang XJ. Probabilistic decision making by slow reverberation in cortical circuits. Neuron. 2002;36:955–68. pmid:12467598
- 15. Davis JD. Deterministic and probabilistic control of the behavior of rats ingesting liquid diets. Am J Physiol. 1996;270(4 Pt 2):R793–800. Epub 1996/04/01. pmid:8967409.
- 16. Kramers HA. Brownian motion in a field of force and the diffusion model of chemical reactions. Physica. 1940;7:284–304.
- 17. Sadacca BF, Rothwax JT, Katz DB. Sodium concentration coding gives way to evaluative coding in cortex and amygdala. Journal of Neuroscience. 2012;32:9999–10011. PubMed Central PMCID: PMC3432403. pmid:22815514
- 18. Brody CD, Hernandez A, Zainos A, Lemus L, Romo R. Analysing neuronal correlates of the comparison of two sequentially presented sensory stimuli. Philosophical transactions of the Royal Society of London Series B, Biological sciences. 2002;357(1428):1843–50. Epub 2003/03/11. pmid:12626017; PubMed Central PMCID: PMC1693076.
- 19. Machens CK, Romo R, Brody CD. Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science. 2005;307(5712):1121–4. Epub 2005/02/19. pmid:15718474.
- 20. Miller P, Wang XJ. Inhibitory control by an integral feedback signal in prefrontal cortex: a model of discrimination between sequential stimuli. Proc Natl Acad Sci U S A. 2006;103(1):201–6. pmid:16371469; PubMed Central PMCID: PMC1324991.
- 21. Pearson JM, Hayden BY, Raghavachari S, Platt ML. Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr Biol. 2009;19(18):1532–7. pmid:19733074; PubMed Central PMCID: PMC3515083.
- 22. Hirokawa J, Vaughan A, Masset P, Ott T, Kepecs A. Frontal cortex neuron types categorically encode single decision variables. Nature. 2019;576(7787):446–51. Epub 2019/12/06. pmid:31801999.
- 23. Wikenheiser AM, Stephens DW, Redish AD. Subjective costs drive overly patient foraging strategies in rats on an intertemporal foraging task. Proc Natl Acad Sci U S A. 2013;110(20):8308–13. pmid:23630289; PubMed Central PMCID: PMC3657802.
- 24. Monk KJ, Rubin BD, Keene JC, Katz DB. Licking microstructure reveals rapid attenuation of neophobia. Chem Senses. 2014;39(3):203–13. Epub 2013/12/24. pmid:24363269; PubMed Central PMCID: PMC3921893.
- 25. Wang XJ. Decision making in recurrent neuronal circuits. Neuron. 2008;60(2):215–34. Epub 2008/10/30. pmid:18957215; PubMed Central PMCID: PMC2710297.
- 26. Bogacz R. Optimal decision-making theories: linking neurobiology with behaviour. Trends Cogn Sci. 2007;11(3):118–25. Epub 2007/02/06. S1364-6613(07)00029-0 [pii] pmid:17276130.
- 27. Wong KF, Huk AC. Temporal Dynamics Underlying Perceptual Decision Making: Insights from the Interplay between an Attractor Model and Parietal Neurophysiology. Frontiers in neuroscience. 2008;2(2):245–54. Epub 2009/02/20. pmid:19225598; PubMed Central PMCID: PMC2622760.
- 28. Wong KF, Wang XJ. A recurrent network mechanism of time integration in perceptual decisions. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2006;26(4):1314–28. Epub 2006/01/27. pmid:16436619.
- 29. Ratcliff R, Smith PL, Brown SD, McKoon G. Diffusion Decision Model: Current Issues and History. Trends Cogn Sci. 2016;20(4):260–81. Epub 2016/03/10. pmid:26952739; PubMed Central PMCID: PMC4928591.
- 30. Ratcliff R. A theory of memory retrieval. Psychological Review. 1978;85:59–108.
- 31. Usher M, McClelland JL. The time course of perceptual choice: the leaky, competing accumulator model. Psychol Rev. 2001;108(3):550–92. Epub 2001/08/08. pmid:11488378.
- 32. Liu YS, Holmes P, Cohen JD. A neural network model of the Eriksen task: reduction, analysis, and data fitting. Neural Computation. 2008;20(2):345–73. Epub 2007/11/30. pmid:18045022; PubMed Central PMCID: PMC2749974.
- 33. Shadlen MN, Newsome WT. Motion perception: seeing and deciding. Proceedings of the National Academy of Sciences of the United States of America. 1996;93(2):628–33. Epub 1996/01/23. pmid:8570606; PubMed Central PMCID: PMC40102.
- 34. Shadlen MN, Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology. 2001;86(4):1916–36. Epub 2001/10/16. pmid:11600651.
- 35. Adams GK, Watson KK, Pearson J, Platt ML. Neuroethology of decision-making. Curr Opin Neurobiol. 2012;22(6):982–9. pmid:22902613; PubMed Central PMCID: PMC3510321.
- 36. Romo R, Brody CD, Hernandez A, Lemus L. Neuronal correlates of parametric working memory in the prefrontal cortex. Nature. 1999;399(6735):470–3. pmid:10365959.
- 37. Jun JK, Miller P, Hernandez A, Zainos A, Lemus L, Brody CD, et al. Heterogenous population coding of a short-term memory and decision task. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2010;30(3):916–29. Epub 2010/01/22. pmid:20089900; PubMed Central PMCID: PMC2941889.
- 38. Rankin J, Sussman E, Rinzel J. Neuromechanistic Model of Auditory Bistability. PLoS Comput Biol. 2015;11(11):e1004555. Epub 2015/11/13. 10.1371/journal.pcbi.1004555. pmid:26562507; PubMed Central PMCID: PMC4642990.
- 39. Daelli V, Treves A. Neural attractor dynamics in object recognition. Exp Brain Res. 2010;203(2):241–8. pmid:20437171.
- 40. Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature. 2002;415(6869):318–20. pmid:11797008
- 41. Leutgeb JK, Leutgeb S, Treves A, Meyer R, Barnes CA, McNaughton BL, et al. Progressive transformation of hippocampal neuronal representations in "morphed" environments. Neuron. 2005;48(2):345–58. Epub 2005/10/26. S0896-6273(05)00773-7 [pii] pmid:16242413.
- 42. Abeles M, Bergman H, Gat I, Meilijson I, Seidemann E, Tishby N, et al. Cortical activity flips among quasi-stationary states. Proc Natl Acad Sci U S A. 1995;92(19):8616–20. pmid:7567985
- 43. Seidemann E, Meilijson I, Abeles M, Bergman H, Vaadia E. Simultaneously recorded single units in the frontal cortex go through sequences of discrete and stable states in monkeys performing a delayed localization task. J Neurosci. 1996;16(2):752–68. pmid:8551358.
- 44. Rainer G, Miller EK. Neural ensemble states in prefrontal cortex identified using a hidden Markov model with a modified EM algorithm. Neurocomputing. 2000;32:961–6. WOS:000087897800127.
- 45. Wang S, Shi Y, Li BM. Neural representation of cost-benefit selections in rat anterior cingulate cortex in self-paced decision making. Neurobiol Learn Mem. 2017;139:1–10. Epub 2016/12/07. pmid:27919831.
- 46. Riveros ME, Forray MI, Torrealba F, Valdes JL. Effort Displayed During Appetitive Phase of Feeding Behavior Requires Infralimbic Cortex Activity and Histamine H1 Receptor Signaling. Front Neurosci. 2019;13:577. Epub 2019/07/19. pmid:31316329; PubMed Central PMCID: PMC6611215.
- 47. Nonacs P. State dependent behavior and the Marginal Value Theorem. Behavioral Ecology. 2001;12(1):71–83.
- 48. Grigson PS, Twining RC. Cocaine-induced suppression of saccharin intake: a model of drug-induced devaluation of natural rewards. Behav Neurosci. 2002;116(2):321–33. Epub 2002/05/09. pmid:11996317.
- 49. Schroy PL, Wheeler RA, Davidson C, Scalera G, Twining RC, Grigson PS. Role of gustatory thalamus in anticipation and comparison of rewards over time in rats. Am J Physiol Regul Integr Comp Physiol. 2005;288(4):R966–80. Epub 2004/12/14. pmid:15591157.
- 50. Grigson PS. Reward Comparison: The Achilles’ heel and hope for addiction. Drug Discov Today Dis Models. 2008;5(4):227–33. Epub 2008/01/01. pmid:20016772; PubMed Central PMCID: PMC2794208.
- 51. Grigson PS, Freet CS. The suppressive effects of sucrose and cocaine, but not lithium chloride, are greater in Lewis than in Fischer rats: evidence for the reward comparison hypothesis. Behav Neurosci. 2000;114(2):353–63. pmid:10832796.
- 52. Grigson PS. Conditioned taste aversions and drugs of abuse: a reinterpretation. Behavioral Neuroscience. 1997;111(1):129–36. pmid:9109631
- 53. Fourcaud-Trocme N, Hansel D, van Vreeswijk C, Brunel N. How spike generation mechanisms determine the neuronal response to fluctuating inputs. J Neurosci. 2003;23(37):11628–40. Epub 2003/12/20. 23/37/11628 [pii]. pmid:14684865.
- 54. Abbott LF, Varela J, Sen K, Nelson S. Synaptic depression and cortical gain control. Science. 1997;275(5297):221–4. pmid:8985017
- 55. Varela JA, Sen K, Gibson J, Fost J, Abbott L, Nelson SB. A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex. Journal of Neuroscience. 1997;17(20):7926–40. pmid:9315911
- 56. Tabak J, Senn W, O’Donovan MJ, Rinzel J. Modeling of spontaneous activity in developing spinal cord using activity-dependent depression in an excitatory network. Journal of Neuroscience. 2000;20(8):3041–56. pmid:10751456
- 57. Kusick GF, Chin M, Raychaudhuri S, Lippmann K, Adula KP, Hujber EJ, et al. Synaptic vesicles transiently dock to refill release sites. Nat Neurosci. 2020;23(11):1329–38. Epub 2020/09/30. pmid:32989294.