Fig 1.
(A) Four reward schedules were employed. All had the same mean reward over the whole session, but in the learnable sessions (changing/learnable up; changing/learnable down) the average reward reversed halfway. RREs, which had the same scalar value, 2 drops, as the mean reward, only had a low probability of occurrence, P(RRE) = 0.1, except in the equiprobable condition where they were just as likely to occur as 1- and 3-drop outcomes (top panel). In changing/learnable sessions, monkeys have to learn that the average reward is higher/lower than expected at the beginning of a session and reverses halfway through the session (bottom panels). (B) On each trial, a visual cue (blue rectangle) appeared on either the left or the right of the screen. This instructed the monkey to make a hand response towards a touch sensor next to the corresponding side of the screen. If they responded correctly, then they received a juice reward of 1, 2, or 3 drops in size. (C) Illustrations of the 3 surprising effects of interest. sRPEs (left) occur when the obtained reward is better (i.e., 3 drops) or worse (i.e., 1 drop) than the expected average (2 drops). RREs (middle) occur when an infrequent reward is obtained (i.e., 2 drops). VS (right) occurs when the stimulus switches sides. (D) An example (stable/unchanging) session illustrating how sRPEs, RREs, and VSs occur on different trials. Trials on which each of the 3 types of surprise occur are marked. (E) Correlation matrix between the 3 main effects of interest showing that our task design allowed for separately examining the effects of sRPEs, RREs, and VSs. ITI, intertrial interval; RT, response time; RRE, rare reward event; sRPE, scalar reward prediction error; VS, visuospatial surprise.
Fig 2.
Behavioral effect of surprising events.
GLMEs predicting errors or outliers as a function of 3 types of surprising event: sRE, VS, and RRE. The 2 GLMEs only differ in that, in A, sRE is indexed by a single regressor while in B, sRE is not a single regressor but instead its component parts are made explicit in terms of the reward outcome experienced on the last 5 trials (t-1 to t-5). Dots represent the beta weight associated with each regressor determined by a GLME applied to each monkey separately. Black dots indicate group mean effects according to the full GLMEs, and vertical bars indicate the SEMs. The Position effect indicates that animals were more likely to make errors or outlier responses depending on the side of the screen they had to respond to but that different animals had different side biases. Errors and outliers became more likely as each session progressed and TrialNo increased. Data and code to reproduce the figure can be found at https://doi.org/10.5281/zenodo.3993116. a.u., arbitrary units; GLME, generalized linear mixed-effect model; RRE, rare reward event; SEM, standard error of the mean; sRE, scalar reward expectation; TrialNo, trial number; VS, visuospatial surprise.
Fig 3.
sRPEs in the striatum and VTA/SN.
(A) Prominent sRPE effects were observed in the right and left orofacial somatosensory and motor cortex and ventrolateral striatum extending into the nucleus basalis of Meynert. (B) We used a priori ROIs in the right and left VTA/SN to extract z-statistics of the regressors from the whole-brain analysis for each session. (C) The z-statistics of each session from a spherical ROI placed at the peak activity of the cluster encompassing the right and left ventrolateral striatum. Different colors indicate data from different animals, with the grey bar showing the grand mean. A red asterisk indicates significance according to the whole-brain analysis, and a black asterisk indicates significance according to a test on the extracted z-statistics. The ROI-based analyses illustrated the presence of the sRPE effects in the left and right striatum but revealed no evidence for VS or RRE signals at the same locations. (D) The z-statistics for the ROI in the right and left VTA/SN revealed an overall significant effect of sRPE but no effects of VS or RRE. When testing the right and left VTA/SN separately, the z-statistics for sRPE in the left VTA/SN were significant, with no effects of VS or RRE. The right VTA/SN revealed a similarly signed sRPE effect although it was, on average, smaller in size and there was more variation across individuals and sessions. Once again VS and RRE effects were not significant in right VTA/SN. (E) BOLD time courses extracted from 3 ROIs in the ventrolateral striatum, the SN/VTA, and the orofacial cortex. The time courses are averaged over both hemispheres. For the location of the orofacial cortex ROI see S1 Fig. Time courses are illustrated for each level of juice reward the monkey received (1, 2, or 3 drops). The orofacial area activity reflects reward amount, and thus all three time courses exhibit an initial positive peak. In contrast, ventrolateral striatum and SN/VTA process sRPEs, and thus, the time course for receiving 1 drop of juice—which is associated with a negative sRPE—results in supressed activity (a negative activity change). Data and code to reproduce the figure can be found at https://doi.org/10.5281/zenodo.3993116. a.u., arbitrary units; BOLD, blood-oxygen-level-dependent imaging; n.s., not significant; ROI, region of interest; RRE, rare reward event; SN/VTA, ventral tegmental area and substantia nigra; sRPE, scalar reward prediction error; VS, visuospatial surprise.
Fig 4.
Neural activity related to RREs.
(A) RRE effects in anterior lateral striatum during the changing/learnable sessions. (B) RRE effects in plOFC during changing/learnable sessions. The z-statistics of each session from a spherical ROI placed in (C) the striatum and (D) the plOFC. The ROI-based analysis illustrates the effects of RRE during changing/learnable sessions. Additionally, we found a significant positive effect of VS during changing/learnable sessions in the anterior lateral striatum. There was also a significant negative effect of sRPE during stable/unlearnable sessions in the plOFC although we are cautious about overinterpreting this result as it would not survive correction for multiple comparisons. A negative sRPE indicates a stronger response when reward outcomes are worse than expected. Data and code to reproduce the figure can be found at https://doi.org/10.5281/zenodo.3993116. a.u., arbitrary units; n.s., not significant; plOFC, posterior lateral orbitofrontal cortex; ROI, region of interest; RRE, rare reward event; sRPE, scalar reward prediction error; VS, visuospatial surprise.
Fig 5.
Neural activity related to visual surprise.
(A) VS effects found in lPFC. (B) The z-statistics of each session from a spherical ROI placed at the peak activity of the cluster in lPFC. Labelling conventions are the same as in Fig 3. The ROI-based analyses confirmed the presence of the VS effects. There was no effect of sRPE. There was a significant negative effect of RRE, which is indicative of encoding of surprise about the scalar reward value (it corresponds to an unsigned reward PE). Data and code to reproduce the figure can be found at https://doi.org/10.5281/zenodo.3993116. a.u., arbitrary units; lPFC, lateral prefrontal cortex; n.s., not significant; PE, prediction error; ROI, region of interest; RRE, rare reward event; sRPE, scalar reward prediction error; VS, visuospatial surprise.