Fig 1.
Stimuli and experimental procedures.
a) Reward learning phase (conditioning). Participants learned the reward association of the visual or auditory cues by performing a localization task and observing their monetary rewards contingent on the color (visual) or pitch (auditory) of the cues. In this task, after an initial fixation period (700–1400 ms), a visual or an auditory cue was presented to the left or right side of the fixation point and participants had to localize them by pressing either left or right arrow buttons of a keyboard (maximum response time 2 s). Here, stimuli in two example trials are shown, one with an auditory cue presented to the left and the other with a visual cue presented to the right. The correct performance led to either a high or a low monetary reward dependent on the identity of the cue (color of visual cues and pitch of auditory cues). b) Orientation discrimination task employed during the pre- and post-conditioning phases. To probe the effects of reward value on visual sensitivity, an orientation discrimination task was employed. A trial of this task started with a fixation period (700–1400 ms), followed by the presentation of a peripheral Gabor stimulus (9° eccentricity). Participants were instructed to discriminate the orientation of the Gabor stimulus (clockwise or counterclockwise tilt) by pressing down or up arrow buttons on a keyboard (maximum response duration = 2 s). Concurrent with the Gabor, a visual or auditory cue was also presented (intra- or cross-modal cues, respectively). Intra- and cross-modal cues were irrelevant to the orientation discrimination task and did not predict reward delivery. c) The schematic illustration of the different stages of an experimental session. Note that after a long conditioning block where participants learned the stimulus-reward associations, smaller blocks of conditioning were interleaved with the blocks of post-conditioning (16 repetitions) to prevent the extinction of reward effects. d) Design matrix of stimulus conditions used during the pre- and post-conditioning phases to assess the effect of reward (high or low) and sensory modality (intra- or cross-modal). A neutral condition was also included during the pre- and post-conditioning phases, which was never associated with any reward value and served to assess the responses evoked by the visual target. Note that the reward assignments of visual and auditory cues were counterbalanced across participants.
Fig 2.
ERP responses of the posterior ROI to the visual and auditory reward cues during the conditioning phase.
a) ERPs of visual reward cues with high (red traces) and low (blue traces) values measured in a posterior ROI (O1, O2, PO7 and PO8). The shaded grey areas correspond to 30 ms windows around the peak of P1 (70–170 ms) and N1 (180–250 ms) used to estimate the amplitude of these components for each condition (here the window is averaged across high and low value conditions) and the window used to estimate P3 responses (300–600 ms). The topographic distribution of P1, N1 and P3 components are shown for each reward value condition. b) Same as a for auditory reward cues. c-d) analysis of the amplitude and latency of the ERP components revealed significant value-driven modulations for the amplitude of P3 shown in c and the latency of the N1 component shown in d. Visual High Value: VH; visual Low Value: VL; Auditory High Value: AH; Auditory Low Value: AL.
Table 1.
Amplitude and latencies of visual ERP components evoked by visual high value (VH), visual low value (VL), auditory high value (AH), and auditory low value (LA) cues during the conditioning phase, measured in the posterior ROI.
Fig 3.
Behavioral performance in the orientation discrimination task.
a) Visual sensitivity (d-prime: d’) during the pre-conditioning phase for different conditions (Neutral: Neut; Intra-modal High Value: IH; Intra-Modal Low Value: IL; Cross-modal High Value: CH; and Cross-Modal Low Value: CL). b) Same as a, for reaction times (RT). c) Visual sensitivity during the post-conditioning phase, i.e. after the associative reward value of different cues were learned. d) Same as c for RTs. e) Effect size for d’ modulations, measured as the difference in d′ between high- and low-value conditions corrected for their difference during pre-conditioning, in intra- and cross-modal cue types (grey and white bars, respectively). Each dot represents the effect size for one individual subject. f) Same as e for reaction times (RT). * marks the significant effects (p < 0.05). Error bars are s.e.m.
Table 2.
Summary of behavioral and ERP data during the pre- and post-conditioning phases (the gray and white cells, respectively) for intra-modal (IH and IL, high and low values), cross-modal (CH and CI, high and low value), and neutral (Neut) conditions.
Significant pairwise comparisons are shown in bold fonts (p < 0.05).
Fig 4.
ERP responses of the posterior ROI (PO7, PO8, O1, O2) during the visual orientation discrimination task.
a) ERPs elicited by the neutral condition during pre- and post-conditioning (illustrated on the left and right, respectively). The topographic distributions of PA (90–120 ms), P1 (most positive peak 70–170 ms), N1 (most negative peak 180–250 ms), and P300 (300–600 ms) components, measured in respective grey shaded areas of the ERP time courses are also shown. b) ERPs in the presence of task-irrelevant, intra-modal reward cues, with high (red traces) and low (blue traces) values. The corresponding topographic distribution of each component is shown for each reward value condition. To test the significance of value-driven modulations, the difference between high- and low-value conditions was corrected for their pre-conditioning difference. The topographic distributions of these corrected modulations are shown for each component (the lowermost topographic maps). See also Fig 5C and S4 Fig. c) Same as b for ERPs in the presence of task-irrelevant, cross-modal reward cues.
Fig 5.
Mean amplitude of different ERP components of the posterior ROI (PO7, PO8, O1, O2) during the visual orientation discrimination task.
a) mean amplitude of PA, P1, N1 and P3 responses evoked by neutral (N), intra-modal high value (IH), intra-modal low value (IL), cross-modal high value (CH) and cross-modal low value (CL) conditions, during pre-conditioning. b) Same as a, for ERPs during post-condoning. c) Corrected effect sizes, measured as the difference between high- and low-value conditions corrected for their difference during pre-conditioning, for intra- and cross-modal cue types (grey and white bars, respectively). * mark significant differences (p< 0.05) and ** mark significant differences (p< 0.01). Each dot represents the effect size for one individual subject. Error bars are s.e.m.