Figure 1.
Hyperbolic and Exponential Reward Discounting Models
(A) Hyperbolic versus exponential reward discounting models as a function of the delay to the reward for two different sets of steepness parameters. The hyperbolic model has an initial steep decay followed by a flatter “tail”; thus, delayed rewards are less discounted with hyperbolic models than with exponential models.
(B) Preference reversal, which is commonly observed in humans and animals, is predicted by the hyperbolic model and is due to a decrease in the decay rate as the delay increases. Initially (at time 0), the large reward has a higher value than the small reward, and is therefore preferred. As the small reward draws near, the preference shifts to the small reward. The exponential model, which has a constant decay rate, does not predict preference reversal.
Figure 2.
At each trial the subject must select either a white or a yellow mosaic after the fixation cross turns red (“Go” signal). Each button press (green disk) adds a number of colored patches to the selected mosaic. In the example shown here, if the white mosaic is selected, the subject receives 5 yen in two steps of 1.5 s each. If the yellow mosaic is selected, the subject receives 20 yen in four steps. The position of the squares (left or right) was changed randomly at each step. For each trial, the initial numbers of black patches for both mosaics were randomly drawn from uniform distributions, and indicated different delays. The ITI, which corresponds to the reward display, was fixed (one time step). Thus, just after the reward display, a new trial began. The subjects had a total of 700 time steps to maximize their total gain.
Figure 3.
Reward Choice as a Function of Delays
(A) Example of a subject's reward choice as a function of delays. At each trial, the subject had the choice between a large reward RL and a small reward RS. The indifference line (solid line) was obtained with a logistic regression model (see Materials and Methods).
(B) Comparison of average indifference lines derived from the experiment with the theoretical indifference line that maximizes total gain in the experiment. Black solid line: average indifference line for all subjects obtained with the logistic regression model. Dotted blue line: average indifference line for all subjects obtained by fitting an exponential discounting model (the slope of the indifference line is 1). Dash-dotted red line: average indifference line for all subjects obtained by fitting a hyperbolic model (the slope of the indifference line is 4). Dashed green line: theoretical indifference line that maximizes the total gain in the experiment (the slope of the indifference line is 1).
Figure 4.
Coefficients of the Exponential Basis Functions Normalized to Unity for Each of the 20 Subjects (S1 to S20)
Note the sparseness of the coefficient distribution: all subjects exhibit a single peak for decay rates in the range 0.125 and 0.35 sec−1.
Table 1.
Parameter Sensitivity Analysis