https://orcid.org/0000-0002-4639-1444
Figures
Abstract
How does confidence relate to oculomotor dynamics during decision-making? Do oculomotor dynamics reflect deliberation and the buildup of confidence in the absence of visual stimuli? Here we examine the hypothesis that working memory, deliberation, and confidence warp oculomotor dynamics, both in the presence and absence of visual stimuli. We analyzed oculomotor dynamics in a decision-making task in which participants were provided with an abstract context in which to make the decision, and two similar option images which became eventually invisible. We show that fixations between the empty locations in which the images were formerly shown continued after the options disappeared, consistently with a sustained deliberative process facilitated by oculomotor dynamics. Both, oculomotor dynamics and decision patterns remained unchanged regardless of whether the stimuli were visible. Furthermore, our analyses show that the number of alternating fixations between stimuli correlated negatively with the confidence reported after each decision, while the observation time of the selected target correlated positively. Given that decisions in our experimental paradigm are reported in the absence of the stimuli, this suggests a relationship between evidence retrieval from working memory, confidence gathering and oculomotor dynamics. Finally, we performed a model comparison based on predictions from drift-diffusion models to assess the relationship between sequential fixations between images, deliberation and confidence gathering, and the ensuing choice. These analyses supported confidence as a contributing cognitive process encompassing serial evidence-gathering and parallel option consideration during decision-making.
Citation: Cos I, Senel G, Maldonado PE, Moreno-Bote R (2025) The relationship between confidence and gaze-at-nothing oculomotor dynamics during decision-making. PLoS One 20(7): e0310231. https://doi.org/10.1371/journal.pone.0310231
Editor: Nick Fogt,, The Ohio State University, UNITED STATES OF AMERICA
Received: September 16, 2024; Accepted: May 22, 2025; Published: July 7, 2025
Copyright: © 2025 Cos et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The images and data may be found at the following kaggle repository: https://www.kaggle.com/datasets/novecentous/lanimages/data
Funding: IC was supported by the European Union's Horizon 2020 research and innovation programme/Human Brain Project (Marie Sklodowska-Curie Research Grant Scheme IF-656262), by the Spanish Project 2022-140938NB-I00 (MICINN), and by the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreements No. 945539 (Human Brain Project SGA3). RM was supported by the Howard Hughes Medical Institute (HHMI; ref 55008742), The Bial Foundation (106/2022), Ministerio de Ciencia e Innovación (Ref: PID2020-114196GB-I00/AEI) and ICREA Academia. the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Decision-making entails evidence gathering and commitment to a particular option when the evidence is deemed sufficient [1–3]. As evidence is collected over time, individuals develop a subjective sense of confidence, which they may use to determine when to terminate the gathering of evidence and commit to an option [4,5]. Confidence estimation has been described as a cognitive process monitoring decision-making [5–9], providing an internal metric of the reliability of the evidence and of the trustworthiness of the choices [4,8,10]. It participates in all aspects of decision-making, from perception to the execution of motor responses [11], and has been used as a proximal measure for cognitive performance [12] and choice accuracy [5,13,14]. While much research has exclusively focused on the perceptual aspect of decision-making, it have been long acknowledged that motor aspects of the choices also play a significant role in option selection [15–18]. Nonetheless, the specifics of how sensorimotor aspects of decision-making contribute to the estimation of confidence remains unsettled [19].
In visually-guided decision-making, evidence is gathered via sequential sampling/oculomotor fixations over the details of each image stimulus [20], to be transferred from sensory to working memory representations in a process modulated by evidence coherence/complexity [21]. Consistent with this, observation times and the number of saccades typically increase with more difficult perceptual discriminations. This strongly suggests that option analysis and deliberation [5], along with its underlying subjective sense of confidence, participate of this process. In line with this, oculomotor processes have been shown to be predictive of the likely choice to follow, typically the stimuli fixated longer and/or more frequently [22], as has been also been shown during moral decisions [23–25]. However, the question remains whether deliberation and oculomotor control are segregated processes – as suggested by sequential sampling models [1,3,12] and previous studies in visual decision-making [26] –, or whether they are intrinsically related during decision-making. If the latter is the case, we should expect that oculomotor dynamics contribute to option assessment and working memory deliberation even when there is no sensory evidence gathered.
To address these questions, and by contrast to prior experimental paradigms using perceptual decisions, we needed a task in which the participants had to make complex judgements in response to abstract goals, which did not directly depend on the mere sensory properties of the stimuli presented. Thus, we designed a context-dependent task in which image options offer similar utility, at least a priori, and eventually disappear during the course of the trial, by extending from the look-at-nothing effect paradigm [27]. This is a visuo-motor effect consisting of looking at empty locations where images were formerly presented to aid with memory retrieval, which has been extensively studied by previous studies focusing on memory [28] and language comprehension tasks [29]. In particular, look-at-nothing fixations were early suggested to improve memory performance [27], as confirmed by a later study [30]. Based on this previous evidence, we deemed this experimental design appropriate for three reasons: first, eye movements were not constrained to fixation points; second, the absence of stimuli during specific phases of the trial ensured that eye movements could be decoupled from the gathering of visual evidence from external stimuli and related with internal deliberative processes in working memory; third, to promote deliberation, the contexts in which the decisions were made were unusual and the images presented offered options to have the similar value in those contexts (see list of items and contexts in Supplemental).
In brief, in our novel task participants were presented with a text describing the context in which the decision of that trial would unfold (Fig 1; e.g., What would you rather have a nightmare with?). This was followed by the presentation of two images (e.g., an evil clown vs. a witch) representing the two options within the referred context. Participants were instructed to choose one of both options based on the option’s subjective appropriateness for the given context. Importantly, both images disappeared after a Stimulus-On presentation period—indicating the GO signal, leaving behind two frames to indicate their former locations, and starting the Stimulus-Off interval during which the choice could be reported by pressing the right/left mouse button. Oculomotor movements were recorded both during stimuli presentation (look-at-something) and when stimuli were absent (looking-at-nothing).
A. Time-course of a typical trial of the LAN experiment. Each trial starts with a white screen with a fixation crosswire in its center. After 1s the crosswire disappears, and a brief text describing of the context of the decision is shown. 4s later the context disappears, and a second fixation crosswire is shown in the center of the screen. 1s after that, the fixation crosswire disappears, and both images representing the options to select upon are shown on the right and left of the screen, initiating the Stimulus-On interval. The GO signal for the participant to report his/her choice was given 4s later, when both images disappear while leaving their frames behind (in green). This starts the Stimulus-Off interval. These frames disappear 4s later, initiating a sequence of four questions about the choice, lasting 6s each: how complex was the decision, how interesting was the choice, how rewarding was the choice, and how confident were you about your decision. The trial ends with an empty white screen lasting 1s, acting as an inter-trial interval (ITI). B. Schematic of the sequence for a typical trial. We first provided the context in which both options were to be gauged. Both images represent the options to select upon. The four questions listed below characterize four aspects of each decision participants are asked about: complexity, interest, reward and confidence.
Results
Our results show that oculomotor movements between empty locations continued after the stimuli disappeared, consistent with a sustained deliberative process facilitated by visuo-motor dynamics, which alternate focus between working memory representations. We found that the longer the observation of a specific empty location, the higher the probability of choosing the image formerly shown in that area. Furthermore, the higher the frequency of visual alternations between stimuli or their empty locations, the lower the confidence reported after the choice. Along with a model comparison based on predictions from parallel or sequential drift-diffusion models, our results are strongly consistent with working memory, deliberation and confidence to be processes intrinsically related to oculomotor dynamics, regardless of the presence/absence of visual stimuli.
In the main experimental task, which we called Look-At-Nothing (LAN), participants were presented with an abstract context in which to make a choice, e.g., what to take to a deserted island, what would you rather have a nightmare with. The context was followed by two images (Stimulus-On interval), on the right and left sides of the screen (See Methods; Fig 1), showing two options to select upon (see Supplemental Material for the full list of contexts and stimuli images). Options were chosen so at to represent, at least a priori, options with similar utility. In the main tasks, the images disappeared after 4s, indicating the GO signal and starting the Stimulus-Off interval, in which two green empty frames replaced the images at their former location. From that time onwards, the participants could report their choice of option/image they deemed most appropriate given the context by pressing the right/left mouse buttons (see Methods). To purposely promote deliberation, we used unfamiliar contexts and options of comparable utility (see Suppl. Material and repository: https://www.kaggle.com/datasets/novecentous/lanimages/data). After reporting their choice, the participants were required to quantify cognitive factors related to their decision with a number ranging from a minimum of zero to a maximum of ten: the confidence in their choice, the complexity, interest, and reward provided by the options presented (Fig 1B). The participant’s gazing trajectories were captured throughout each trial by means of an EyeTribe (EyeTribe Inc, DK) oculometer (see Methods). To complement the primary LAN experiment, we performed three additional controls: a baseline Look-At-Something (LAS) experiment, in which both options were always visible to control for the influence of absent stimuli; a Motor Reversal (rLAN) experiment, in which the reporting action (Right|Left) and the side of the option presented were reversed, to control for the potential influence of the motor response on the choice; and a Short Time Observation (sLAN) experiment, with a reduced Images-On interval (from 4 to 2s), to control for the possibility that a commitment could have been reached before the GO signal (despite task difficulty and the fact that options exhibit a similar utility therein).
As metrics to characterize the unfolding of the decision-making process at each trial, we computed the time spent looking at each image during the Images-On interval or at the empty frames during the Images-Off interval. We also counted the number of back-and-forth saccades between images and/or their empty locations. We first anticipated that having complex, unfamiliar contexts and options with a-priori similar value would induce significant deliberation and, consequently, prolonged decisions. This was confirmed by the long decision times (DT; from the GO signal and after a 4s Stimulus-On interval for the LAN, LAS and rLAN experiments, and after a 2s Stimulus-On interval in the sLAN experiment). Fig 2A shows the distributions of DTs for the LAN: Avg. 1404.94ms, SE 271.46ms; LAS: Avg. 1372.60ms, SE 216.45ms; rLAN: Avg. 1637.84ms, SE 477.95m; sLAN: Avg. 1395.40ms, SE 617.67ms. We found no statistical differences between the distributions of decision times of the LAN vs. LAS experiments (Fig 2B; t-test, t = 0.70, CI= [−0.10, 0.22], df = 29, P = 0.48; Bayes Factor (null hypothesis)=5.55). Overall, this suggests that, all other factors being equal, deliberation in the LAN and LAS conditions extends over a similar time duration, thus suggesting a negligible contribution of the stimuli persistence to the decision during the decision phase. Likewise, we found no statistical differences between the LAN and sLAN conditions (t-test, t = 1.533, CI= [−0.07 0.53], df = 32, P = 0.13; BF(null hypothesis)=2.46). By contrast, we found a marginally non-significant statistical difference when we reversed the laterality of motor responses (LAN vs. rLAN; t-test, t = −1.73, CI= [−0.36 0.029], df = 32, P = 0.09; BF(null hypothesis)=1.86), suggesting that reversing the laterality of the response required some inhibition.
A. Histogram of Decision Times (DTs) for experiments LAN (red), LAS (green), rLAN (blue) and sLAN (turquoise). B. Mean and standard deviations of DTs for each participant and experiment LAN (green), LAS (red), rLAN (blue), sLAN (turquoise). All distributions were statistically undistinguishable except for the rLAN, which exhibited a further delay than the others (KS-test, LAN vs. rLAN, P = 0.024).
In accord with previous literature [22], we next hypothesized that the longer an option is observed, the more likely becomes its selection, with the novelty that here the image-stimuli were absent from the screen during the Stimulus-Off interval. We first tested during the Stimulus-On by calculating the probability of choosing the image on the right side of the screen (PR) as a function of the logarithmic ratio of times spent looking at the right over the left option for the LAN, LAS, rLAN, and sLAN experiments. In the LAN, LAS and sLAN, the PR exhibits a steep increase as the proportion of observation time tilts in favor of the image on the right during Stimulus-On (LAN; Fig 3A & 3C; t-test across the participants’ slope β1 parameters, t = 5.13, CI= [0.51, 1.22], df = 21, P = 4.41E-5; LAS; Fig 3F & 3H; t = 4.69, CI= [0.54, 1.59], df = 8, P = 0.0016; sLAN; Suppl. Fig 1F & 1H, t = 4.41, CI= [0.73 2.24], df = 10, P = 0.0013). The steep increase of PR during the rLAN experiment is marginally non-significant, although consistent with the motor reversal between observation and reporting sides, in favor of the image on the left (Suppl. Fig 1A & 1C, t = −2.30, CI= [−1.68, 0.0017], df = 8, P = 0.050). Although this was expected while the option images were visible (Stimulus-On) [31,32], we extended our analysis to the Stimulus-Off interval, during which the images were absent. Our analyses show that the effect persisted as well in the LAN experiment (Fig 3B & 3D; t-test across slope β1 parameters, t = 3.60, CI= [2.20, 8.22], df = 21, P = 0.0017), in the sLAN experiment (Suppl. Fig 1G & 1I; t = 5.03, CI= [0.59, 1.53], df = 10, P = 5.17E-4), in the rLAN experiment (Suppl. Fig 1B & 1D; t = −3.60, CI= [−1.29, −0.28], df = 8, P = 0.0069), and during the Stimulus-On interval after the GO signal for the LAS experiment (Fig 3G & 3I; t = 2.79, CI= [0.81, 8.43], P = 0.023). Most importantly, being all other conditions equal, we found no statistical difference between the LAN and LAS slope distributions neither during the Stimulus-On Observation Interval (t = 0.64, CI= [−0.42,0.82], df = 29, P = 0.52; Bayes Factor(null hypothesis)=0.42), nor after the GO signal (Stimulus-Off LAN vs. LAS experiment (t = −0.0131, CI= [−0.25, 0.24], df = 16, P = 0.98; Bayes Factor (null hypothesis)=0.41). This is consistent with the contribution of sensory stimuli to deliberation during the decision phase is negligible.
A-B. LAN Experiment Analyses. Estimate of the probability of selecting the Right image as a function of the ratio of times spent looking Right over looking Left images during the Stimulus-On (A) and during Stimulus-Off (B) intervals. Sigmoidal regression fit for each participant and grand average (in black). C-D. Participants’ sigmoid slope parameter (β1) per participant during the Stimulus-On (C) and Stimulus-Off (D) intervals of the LAN experiment. Statistical significance was reported (* P < 0.05). E. Comparison of the distribution of the participants’ sigmoid slope (β1) and intercept (β0) parameters between the Stimulus-On (SOn) and Stimlus-Off (SOff) intervals. F-J. Same as A-E, but for the LAS experiment. Note that in the LAN task there was no Stimulus-Off interval after the GO signal, but rather a second Stimulus-On interval, as both images were retained on the screen while the decision was reported.
We also confirmed that the β1 slope parameters obtained from the PR fits during the Stimulus-Off interval were significantly steeper than those during the Stimulus-On interval in the LAN experiment (Fig 3E; t-test, t = −2.97, CI= [−7.27,-1.39], df = 42, P = 0.0048), and, marginally, in the LAS Experiment (Fig 3J, Stimulus-On vs. Stimulus-On after GO; t-test, t = −2.13, CI= [−7.08, −0.014], df = 16, P = 0.049; Fig 3J). This is consistent with a continuous decision-making process forming over time, starting during the Stimulus-On interval and extending onto the Stimulus-Off/Stimulus-On interval after GO. Importantly, this also consistent with choice commitment being formed after the GO signal. The increasingly steeper slopes with time are also visible in the case of the rLAN task (Suppl. Fig 1A-D; t-test, t = −0.12, CI= [−0.95, 0.85], df = 16, P = 0.91; Suppl. Fig 1E) and sLAN task (Suppl. Fig 1F-I; t-test, t = 1.08, CI= [−0.39, 1.26], df = 20, P = 0.29; Suppl. Fig 1J), although these effects were not statistically significant.
In so far, we have shown that oculomotor dynamics are consistent with a long deliberation that extends into the decision-reporting phase, both in tasks in which the visual stimuli to decide upon are present (LAS) or absent (LAN). However, a relevant question is whether oculomotor dynamics are also related with confidence during deliberation. If that were the case, we first expected that longer decision times would accompany more difficult and thus less confident decisions. More importantly, we predicted that looking at an image (or its empty frame) relative to the other image for a long time, would relate to a higher confidence in the decision, while more balanced observation times or more frequent alternations between images would be indicative of a lower confidence (see Methods).
Consistent with the first prediction, we found a strong negative correlation between the reported confidence index and the decision time in three out of four experiments (LAN, Fig 4C, 4G, Avg. Correlation Index = −0.26 (±0.032 SE), t-test, t-stat = −8.58, CI = [−0.33, −0.20], df = 23, P = 1.26E-8; LAS, Fig 4F-G; Avg. Correlation Index = −0.10 (±0.062 SE), t-test, t-stat = −3.40, CI = [−0.35, 0.0090], df = 7, P = 0.0059; rLAN, Suppl. Fig 1K & Fig 4G, Avg. Correlation Index: −0.27 (±0.042 SE), t-test, t-stat = −6.87, CI = [−0.37, −0.19], df = 10, P = 4.32E-5; sLAN, Suppl. Fig 1L & Fig 4G, Avg. Correlation Index: −0.32 (±0.045 SE), t-test, t-stat = −7.28, CI = [−0.43, −0.23], df = 10, P = 2.65E-5).
A. Group average and standard error β−Regression Coefficients of a GLM of the observation time of the Selected Option during the SOn, SOff-UD and SOff-PD intervals vs. the indices of Confidence (C), Reward (R), Interest (I), Complexity (Cm), plus interactions: RxI, RxCm, IxCm for the LAN experiment. Metrics are z-scored. Confidence is the only predictive regressor of the time devoted at looking at either image. B. Scatter plot of the observation time of the selected image until decision time vs. the reported confidence (Correlation Index r = 0.24, P < 0.0001). C. Pearson correlation index per participant of the observation time until decision time with the reported confidence index (avg. 0.16; P < 0.0001). D-F. Same as A-C for the LAS experiment. G. Average and std. error correlation index of the observation time until decision time vs. the reported confidence for the four tasks.
Also, to assess the growing influence of confidence during the decision process, we used a GLM to regress the proportion of observation time devoted to the image ultimately selected as a function of the cognitive indices reported by each participant (confidence, reward, complexity, interest; See Methods; Equation 2) during the Stimulus-On (SOn), Stimulus-Off Until Decision (SOff-UD) and Stimulus-Off Post-Decision (SOff-PD) intervals. Group significance tests across participants showed that confidence was the only cognitive factor marginally predicting the choice during the early SOn interval of the LAN (t-test; t-stat = 1.98, CI = [−0.0065, 0.29], df = 21, P = 0.06), of the LAS experiment (t-test; t-stat = 2.39, CI= [0.0033, 0.19], df = 8, P = 0.044), and of the rLAN experiment (t-test; t-stat = −1.98, CI = [−0.16, 0.01], df = 10, P = 0.076). However, this influence became strongly significant during the following SOff-UD interval of the LAN (t-test; t-stat = 3.024, CI = [0.040, 0.21], df = 21, P = 0.0062; Fig 4A), LAS experiment (t-test; t-stat = 3.02, CI = [0.11, 0.30], df = 8, P = 0.0010; Fig 4D), and rLAN experiment (t-test; t-stat = −2.27, CI = [−0.41, −0.0041], df = 10, P = 0.043; Suppl. Fig 2A). Also, the relationship between observation time of the selected image and confidence during this last interval is reinforced by a complementary (Pearson) correlation analysis, yielding a r = 0.24 (P < 0.0001; LAN Experiment; Fig 4B), R = 0.16 (R < 0.0001; LAS Experiment; Fig 4E), and R = −0.45 (rLAN Experiment; Suppl. Fig 2B). The sLAN exhibited no significant effects. The remaining cognitive factors, although known to influence decision-making in other situations, did not yield any significant effect in this task.
To further characterize the coupling between gazing dynamics and confidence, we assessed the number of visual alternations between images (Number of Change-of-Target fixations; #CoT), as well as their frequency (fCoT: #CoT normalized by the duration of the interval of interest). Our prior was that these metrics should increase with the difficulty to determine the appropriate choice, implying more difficulty to gain confidence. We calculated both metrics during three specific phases of the decision-making process. First, during the initial stimulus presentation (Stimulus-On); second, during the subsequent phase until the decision is reported (SOff-UD for the LAN task and SOn-UD after the GO for the LAS task and until the decision is reported); third, during the post-decision interval (SOff-PD). A first analysis of the #CoT and fCoT for the LAN and LAS tasks shows that they are higher during Stimulus-On (LAN: Group Avg. #CoT ~ 4, fCoT ~ 1.3 Hz, Fig 5A–B; LAS: #CoT ~ 6, fCoT ~ 1.4 Hz; rLAN: #CoT ~ 4, fCoT ~ 1 Hz; Suppl. Fig 3A; sLAN: #CoT ~ 2, fCoT ~ 0.5 Hz; Suppl. Fig 3B). These metrics drop down to a much slower pace when they were removed and until a decision was reported (LAN: SOff-UD, Group Avg. #CoT ~ 1.5, fCoT ~ 0.25 Hz, Fig 5A–B; LAS: On-UD: Group Avg. #CoT ~ 2, fCoT ~ 0.25 Hz, Fig 5D–E; rLAN: On-UD: Group Avg. #CoT ~ 0.1, fCoT ~ 0.1 Hz; Suppl. Fig 3A; sLAN: On-UD: Group Avg. #CoT ~ 0.5, fCoT ~ 0.25 Hz; Suppl. Fig 3B). Finally, #CoT rises back up during the post-decision interval (LAN: SOff-PD: Group Avg. #CoT ~ 3, Fig A-B; fCoT ~ 1.25 Hz, Fig 5D–E; LAS: SOn-DP: Group Avg. #CoT ~ 3, Fig 5D–E; fCoT ~ 1.25 Hz, Fig 5B; rLAN: SOn-DP: Group Avg. #CoT ~ 3.5, fCoT ~ 6.5 Hz, Suppl. Fig 3A; sLAN: SOn-DP: Group Avg. #CoT ~ 4, fCoT ~ 0.75 Hz, Suppl. Fig 3B).
A. Average and standard error of the number of alternating fixations between the visual options (#CoT) for two typical participants (S1 and S2) and group average during the LAN Experiment, calculated during three intervals: Stimulus-On (SOn), Stimulus-Off Until Decision (SOff-UD), and Stimulus-Off Past Decision (SOff-PD). B. Same as A, but for the frequency of alternating fixations (fCoT) metric. C. Group average regression coefficients of two independent GLMs, between confidence and #CoT, and between confidence and fCoT, left and right, for the same three intervals SOn, SOff-UD and SOff-PD, during the LAN experiment. D-F. Same as A-C, but for the LAS Experiment during its equivalent time intervals. Note that #CoT and fCoT, covary negatively with confidence, both when the stimuli are visible (LAS) and invisible (LAN) prior to the decision, and positively after the decision.
We considered two compatible explanations for the processes underlying these modulations. First, the initial higher fCoT during the Stimulus-On period was consistent with the low initial evidence/confidence, while the lower fCoT until afterwards during the Stimulus-Off period could suggest a slower waging process caused by the deliberative process operating in working memory; as if retrieving details from absent images required a stronger introspection, which in the absence of stimuli, is facilitated by the dynamics of visual saccades. Second, during the reporting period, the participant has already gathered some evidence and is more confident about which option to select, hence the steeper sigmoid slopes during Stimulus-Off (Fig 3B) and the lesser need for alternating visual fixations between image stimuli (Fig 5). To further assess the influence of the presence/absence of the stimuli on the dynamics of gazing during decision-making, we explicitly assessed the difference between LAN and LAS group distributions of #CoT and fCoT metrics. The absence/presence of the stimuli yielded no significant difference during all three intervals of interest: for #CoT (SOn-UD interval, t-test: t-stat = −0.37, CI= [−2.15, 1.49], df = 28, p = 0.72; SOff-UD vs SOn-UD intervals, t-stat = 0.08, CI= [−2.41, 2.63], df = 29, p = 0.93; SOff-PD vs SOn-PD: t-stat = 0.70, CI= [−2.84, 5.82], df = 29, p = 0.49) and for fCoT (SOn-UD interval, t-test: t-stat = 0.21, CI = 1E-3* [−0.61, 0.76], df = 28, p = 0.83; SOff-UD vs SOn-UD intervals, t-stat = 0.09, CI = 1E-3* [−0.60, 0.66], df = 29, p = 0.93; SOff-PD vs SOn-PD: t-stat = 0.70, CI = 1E-3* [−0.47, 0.97], df = 29, p = 0.49). Finally, we regressed the confidence index as a function of the #CoT and fCoT metrics with a GLM (Eqs. 3 & 4), calculated independently for each participant, metric and interval of interest, and assessed grouped significance with a t-test across the resulting regression coefficients, for each metric and interval. This yielded a strong statistical dependence for both oculomotor dynamical metrics on confidence, on the LAN experiment during the SOff intervals, but not during SOn (LAN: #CoT; SOn t-test, t-stat = −1.41, CI= [−0.087, 0.016], df = 21, p = 0.17, SOff-UD t-test, t-stat = −9.07, CI= [−0.24, −0.15], df = 21, p = 1E-8, SOff-PD t-test, t-stat = 3.91, CI= [0.04, 0.014], df = 21, P = 8.13E-4; Fig 5C. Note that the statistics are very similar for #CoT and fCoT). Importantly, this analysis yielded that the relationship between the reported confidence and the #CoT and fCoT metrics was negative during SOff-UD prior to the decision, and positive after that (SOff-PD interval—Fig 5C, F). To further test whether the presence of the stimuli exerted an influence on the relationship between gazing dynamics and confidence, we performed the same analyses on the data from the LAS experiment, yielding a trend consistent with the LAN effects for each interval (Fig 5F), although marginally non-significant (LAS: SOn t-test, t-stat = −1.20, CI= [−0.14, 0.044], df = 7, p = 0.26, SOn-UD, t-stat = −2.10, CI= [−0.22, 0.017], df = 7, p = 0.073, SOn-PD, t-test, t-stat = 2.01, CI= [−0.01, 0.13], df = 7, PD = 0.083; Fig 5F). Along with the observation that the influence of confidence on gazing dynamics remained during the rLAN and sLAN controls (Suppl. Fig 4).
Altogether, these results strongly suggest a cognitive option-assessment process that depends on oculomotor dynamics, which gather evidence sequentially. This is supported by the increased alternation of saccades between stimuli when the options to decide upon require significant assessment. However, how sequential visual alternations are parsimonious with the parallel option representation for decisions between simpler motor actions remains to be clarified [33]. Although a complete answer to this question escapes the reach of this study, we can nonetheless qualitatively probe this by formulating and testing distinct predictions from three normative models of choice relating ocular dynamics and confidence within the context of a drift diffusion formulation of decision-making [3], each grounded in one of three hypotheses: parallel, sequential or a hybrid consideration of options (see Suppl. Material for the full account of this formulation). We tested their predictions using the data from the four experiments described. In brief, we developed three theoretical derivations encompassing deliberation, evidence gathering and confidence, consistently with the three hypotheses previously described: in the case of parallel option consideration, the level of confidence depends on the sum of times devoted to observing each stimulus; this dependence was the difference of their inverses for the case of sequential consideration (FIG 6E). We also included a hybrid model that combined the two predictions to entertain the possibility of decision-making as a combined sequential and parallel operation. We used the confidence index reported after each decision by the participants and the observation times of each trial to test the predictions of the three models, within each of the four experiments separately. Please note that the intended purpose of using this framework is not to establish any new theory, but rather to allow a comparison between hypotheses. Ultimately, we performed a Bayesian model comparison to determine a winner model within each experiment, by inverting each of the three models via variational Bayes [34] and obtaining a goodness of fit per participant and experiment [35]. Then, we applied Bayesian model selection to assess the model frequency and exceedance probabilities obtained from each model inversion [36] and experiment to ascertain the hypothesis receiving the strongest support in each experiment. Fig 6A-D shows that the hybrid model is the most frequent winner in all four experiments (Model Frequency = 1.0; Exceedance Probability = 1.0, in all four experiments). As further validation, FIG 6F shows the average confidence index provided the participants (in quartiles) matched with the corresponding group average decision times on those trials, for each experiment (FIG 6F), along with the grand average model predictions for each experiment. Consistent with the data gathered by the previous experimental analyses, these results strongly favor the hypothesis that the dynamics of evidence accumulation in this visual task extend beyond the typical parallel dynamics observed during action selection, and incorporate elements of sequential dynamics, biased with the dynamics of visual fixations, even when there are no visible stimuli.
A model of mixed parallel and sequential processing of option-stimuli explains the relationship between observation time and confidence the best. A. Bayesian Model Comparison metrics (Model Frequency & Exceedance Probability) obtained from a Random-Effect Analysis on the three hypotheses relating confidence and observation time (sequential-parallel, sequential, parallel) for the participants of the LAN Experiment 1. B. Same as A, but for the LAS Experiment. C. Same as A, but for the rLAN Experiment. D. Same as A, but for the sLAN Experiment. E. Prediction of Confidence for all three Models on the LAN Experiment. E. Average Prediction of Confidence as a function of the time devoted to look at the selected vs non-selected option, based on the posteriors obtained from inverting the model for all participants of LAN experiment. F. Confidence across participants (quartiles) as a function of decision-time (mean and std. error; thin blue line) for all four experiment, along with the average prediction for all three models (same color coding of A-D). G. Sample traces of evidence gathering for each model considered and predicted confidence index for each of them. From top to bottom: hybrid model (M1), parallel model (M2), sequential model (M3). Each left inset shows the temporal unfolding of evidence over a few trials until commitment. The right insets show the model predicted confidence index at decision for each of the trials. Note that, in the case of the parallel model (M2), evidence is driven continuously by a single drift parameter (the difference of drifts associated to each option). This is in accord with eq. 5, and, most importantly, it implies that evidence gathering operates independently from the alternating oculomotor dynamics between stimuli images. By contrast, evidence in the sequential model is guided by each oculomotor fixation, and according to its associated drift, thus the visible alternations of the evidence trace throughout each trial. The hybrid model (M1) combines M2 and M3 in a single formulation.
Discussion
Here we studied how oculomotor dynamics relates to reported confidence, and influences decision-making in a series of tasks in which the participants had to choose the preferred of two options, presented by images on a screen. Importantly, the choices were to be made in a specific and unusual context, introduced by a phrase shown at the beginning of each trial. In three of the four tasks, participants had to report their choices when the images were non-visible. To characterize the influence of the presence of the options on the decision itself, we first assessed differences of decision time between LAN and LAS tasks, finding no statistically significant differences. Next, we assessed the differences in terms of frequency and duration of alternating fixations between images during a first interval in which the option images were shown (Stimulus-On) and the second reporting interval during which the images were absent (Stimulus-Off), finding no statistical differences in either of both intervals. Finally, we assessed the relationship between their reported confidence and oculomotor metrics; the fraction of time devoted to looking at either image or its related empty frame, and the number of visuo-motor alternating fixations at either image option. This yielded a significant positive correlation between observation time of the image selected and the reported confidence, and a negative one with the number of alternating fixations between option images. Finally, our computational analyses of the relationship between confidence and image observation times yield the prediction that the dynamics of decision-making combine elements of sequential sampling and parallel option deliberation. This combines the simultaneous consideration of motor options [37] and the sequential sampling typical of complex, value-based decision-making [3]. Furthermore, our results also suggest an internal deliberative process during option assessment, which operates in working memory and is partially independent from sensory evidence availability. In this context, oculomotor dynamics play a role similar to a memory index that facilitates the processing options in a sequential fashion, as a function of the image being currently scrutinized. Moreover, the control of oculomotor dynamics is cognitively modulated as a function of the level of confidence during deliberation.
Confidence and Oculomotor dynamics during look at nothing. In line with previous research, our analyses yielded a negative correlation between the reported confidence and the number and frequency of saccades (#CoT, fCoT metrics) during SOff-UD interval prior to the decision [12,38,39], but also positive after that. In other words, the modulation of #CoT and fCoT prior to decision reporting is consistent with the participant’s confidence, increasing/decreasing the number of saccades between stimuli when the confidence is low/high. However, the post-choice reversal suggests a mechanism of double-checking, for which confidence becomes a positive covariate. Importantly, our results show that the presence or absence of stimuli exerted no significant difference in the relationship between confidence and gazing dynamics. Altogether, this is consistent with a cognitive process driving oculomotor movements to facilitate evidence gathering from working memory (LAN), and from the stimuli in the environment—whenever available (LAS). In brief, these results support that oculomotor dynamics were strongly modulated by confidence until commitment for a specific option. Likewise, oculomotor dynamics for the case of this particular task are independent of the presence or absence of the stimuli.
Parallel vs. Sequential Brain Dynamics for Decision-Making. Motor decision-making is typically described by models that assume options to be encoded and evaluated in parallel [1,40], and a selection mechanism for choice-making that can be implemented via mutual inhibition between neurons encoding different options [41,42]. Other models have nonetheless proposed that at least some aspects of decision-making occur sequentially [3,10,31,43–45]. Here we probed the dynamics of the look-at-nothing decision-making process by means of a comparison across three different hypotheses and subsequent models, each assuming a specific relationship between confidence and oculomotor dynamics [7]. First, a pure parallel account suggests that confidence depends exclusively on the total time devoted to looking at either option, and, implicitly, that evidence gathering and oculomotor dynamics are independent from each other. By contrast, a sequential model predicts that confidence depends on the difference of the time devoted to looking at each option, and therefore on the alternating oculomotor dynamics between both options—visible or not. Our analyses show that our data does not precisely match either of these predictions, thus supporting a decision-making process incorporating parallel and sequential option consideration, and computation sub-processes integrating the decision-making process. According to this model, confidence during decision-making depends both on the difference of observation time devoted to each option and on the total decision time as well.
Importantly, we have introduced intervals during which the options to decide upon were non-visible (LAN intervals), creating an effective decoupling between sensory processing and internal deliberation. The fact that oculomotor patterns during LAN intervals closely resemble those during the actual physical presentation of the option images strongly suggests that the evaluation of the options is being performed in a quasi-sequential fashion in working memory alone. While only neural data can resolve the question of what processes happen and when they happen during deliberation, our analysis predicts that the encoding ensues in parallel, with evaluation is most likely to be guided by an internal attentional mechanism, consistently with the mind’s eye hypothesis [46]. This view complements previous theoretical studies showing that flexible models, whenever presented with a dual-task stimuli, adopt a parallel processing at peripheral sensory levels [37,47]. However, these architectures necessarily require a buffering or working memory structure capable of keeping hold of the result of sensory processing for each option, which implies a bottleneck, and, therefore, serial processing [48,49].
Seminal studies on value-based decision-making showed that ventro-medial prefrontal/orbitofrontal cortical areas [3,50] and the dopaminergic system [51–54] are the main brain areas concerned with reward option assessment and decision-making. The results of this manuscript align with that, and with the notion that sensorimotor areas such as the frontal-eye-field, in concert with prefrontal brain areas [55], are involved in working-memory deliberation rather than with information gathering alone. Furthermore, our results have shown that the metrics of oculomotor exploration co-vary with the participant’s sense of confidence prior and during the decision-making process, consistently with a cognitive process monitoring the control of sensory reliability and performance [14,21,45,56]. While our results support the hybrid parallel and sequential option assessment solution to the parallel vs. sequential option assessment question, the dynamics of interaction encompassing the frontal eye field ld [57–59], the peri-genual anterior cingulate cortex [60], ventromedial prefrontal cortex and the ventral striatum [49] will only be ascertained with future simultaneous neural recordings across these areas.
Previous work has eloquently shown that gazing patterns correlate with choices when stimuli are visible [3,31]. These observations have been recently extended to memory tasks where stimuli become invisible after a presentation period. In one study where several options and their locations need to be memorized before the reporting choice stage [61], it has been shown that oculomotor patterns are different between the visible and non-visible option conditions. It is possible that in that study, the large number of options to be remembered (six in total) overloads working memory, thus slowing down decision-making during the LAN interval due to the impeded memory retrieval. This result contrasts with those of our binary choice task: even though contexts are abstract and uncommon, and the options complex, participants behave no differently in the LAN than in the LAS tasks, possibly because holding two options in working memory is within reach. Our study also included confidence reports and how they relate with the oculomotor patterns. Although some work has shown that the option observation time and number of alternating fixations between targets correlates with confidence during the presentation of visual options [10,62,63]. To our knowledge, ours is the first showing that a similar pattern holds when stimuli are non-visible. Again, this critical observation suggests that deliberation is in part a sequential process facilitated by gaze patterns in a situation where options are associated to locations in the visual field –the look-at-nothing effect [27]. In conclusion, our results speak in favor of a series of sequential operations to describe decision-making in the context of this task.
Materials and methods
Participants
A total of forty-two participants participated in four versions of a decision-making task: Look at nothing (LAN) experiment; 24, 12F + 12M, 18–35yold; Look at Something (LAS); 12, 6M + 6F; 18–35yold; Motor reversal (rLAN); 12 6M + 6F; 18–35yold; Short decision-time (sLAN); 10 5M + 5F;18–35 yold). Their contact information was obtained through the CBC database. Participants had no known neurological damage and normal vision. Due to limitations of the eye tracker device (Eyetribe, Inc, DK), which could not maintain stable tracking of light coloured eyes, or of participants wearing glasses, these had to be excluded from our study. Furthermore, since the main task was instructed in English, we required participants to hold at least a B2 level of English to participate in our study. Participants were also instructed not to wear make-up during the sessions to avoid light reflexes. Maximum session duration extended for 1.5h, receiving 5€ every 30 min as monetary compensation. All participants signed an informed consent form prior to initiating the experimental session and all methods were carried out in accordance with relevant guidelines and regulations. The experiments started on February 6th, 2019 and concluded on February 6th, 2020.
Ethics statement
Ethics regulations for this project were approved by the local ethics committee with Ref #0100. All participants were provided with an informed consent form explaining the experimental procedures, rights of the participants and data protection. This information was also verbally explained to each participant prior to the experiment. Their written consent was requested before the beginning of the experiment. All the participants were debriefed after the completion of the experiment. The information of the experimenter was provided for further questions.
Apparatus and stimuli
Experiments were performed in a soundproof room at our laboratory. The task apparatus consisted of a comfortable chair, placed in front of a table with a Samsung 20-inch LED monitor with 1600x900 resolution (first 12 participants of the 1st experiment only) or—thanks to lab renovation, an HP Omen 25-inch LED monitor with 1920x1080 resolution (last 12 participants of the LAN experiment and all participants of experiments LAS, rLAN, sLAN experiments). The screen was placed across the table 45cm from the participant, and was used to show the visual stimuli to the participant. An EyeTribe (EyeTribe, Inc, DK) tracking device was used to track and record eye movements and pupil size data from participants. The oculometer was placed on the table right under the screen, aligned with it and perpendicular to the participant’s line of sight. The tracker’s server version was 0.9.56, and we used a 12-screen data point routine to calibrate the system prior to starting the task. The participant was instructed to lean his/her head on a chinrest to maintain a fixed distance from the table/oculometer, to minimize their head movements during the experiment and to rest their eyes and hands during the breaks, and to hold her right hand index and middle fingers on two consecutive keys of the computer to report their choices. During the calibration process, participants were instructed not to move their heads after the calibration process, including the break times between the blocks, as to preserve the oculometer calibration profile throughout the experiment. Task flow control was performed with a Python custom-built script, implementing the Tkinter Graphical User Interface (GUI). The data from each session was transferred to a MySQL database (Oracle, CA), and analyzed with custom-built scripts in MATLAB (The Mathworks, Natick, MA), licensed to the Pompeu Fabra University.
Procedure and tasks
This study included four similar experimental tasks. All four were decision-making tasks in which participants were instructed to make decisions between two images shown on the screen. Each task addressed a specific aspect of the decision-making process, or control for a specific confound. The four tasks labelled as follows: 1) look-at-nothing (LAN): to specifically address the dynamics of decision-making when visual stimuli were absence, 2) Look-at-something, LAS: same as LAN but retaining the stimuli on the screen during the decision interval; 3) motor reversal (rLAN): same as LAN but inverting choice and motor response side; 4) short observation (sLAN): same as LAN but with a shorter (2s) image observation interval. Preparation for the four tasks was identical and is described next. After the task was explained to the participant and she/he was ready to start, we proceeded to calibrate the oculometer. To this end, the participant was instructed to approach the table and place the chin onto the chinrest to constrain head movements. We used the 12-point calibration profile provided by the EyeTribe software. The calibration was repeated until it reached a minimum score of 3 (out of 5). Participants not reaching a score of 3 were excluded from the experiment. Each experimental session consisted of eighty trials, equally distributed into four blocks. Each trial introduced a different context or situation in which two options were presented by two images, right and left in the screen, to decide upon. To introduce the participants to the task and familiarise themselves with the task dynamics, they performed two trials guided by the experimenter, and trained during ten additional trials.
The timeline of a typical trial of the LAN task is described next (see Fig 1). Each trial began by showing a 2x2cm black cross in the centre of the screen. After 1s, the crosswire disappeared, and a sentence describing the context of the decision was shown in the centre of the screen. After 4s, the text disappeared and a second crosswire was presented in the centre of the screen. After 1s, the Stimulus-On (SOn) interval started, showing two images related to the context were shown on the screen, left and right, equidistant from the centre of the screen, and from the top and bottom of the screen. After 4s, the Stimulus-Off (SOff) interval started with a GO, signalled by removing the images and leaving the frames behind. The SOff interval extended for 4s and was divided into two sub-intervals: from the GO Until the Decision is reported (SOff-UD), and until the end of the 4s (Post-Decision; SOff-PD). At the GO signal, the images were replaced by two empty green frames of the same size. Participants reported their choice by clicking the left/right mouse button if they select the left/right image, and with a maximum duration of 4s. After the decision was reported or after 4s, the participants were asked to quantify four cognitive aspects of their decision: complexity, interest and reward associated to the selected image, and the confidence in their decision, by rating them with a number [0–9]. Each question-answer lasted 6s. At each question, participants were given the option of jumping to the next by pressing the keyboard enter key, after reporting each rating (Fig 1). If the trial ended before a fixed trial duration of 28s, participants were required to wait until that trial duration had elapsed. We used this method to encourage participants to think carefully about the questions asked and to simultaneously provide them with a sense of control of the trial flow. The time slot for each block was 12m 42s. When participants completed all 80 trials, they were shown a message on the screen to inform them that the experiment had ended. At that point, they were free to disengage from the chinrest and the experimenter entered the room for debriefing and payment.
The differences in the timeline of the four Tasks are described next. The difference between LAS and LAN tasks, is that the images were shown during the initial image presentation interval, and after the GO signal during the decision reporting as well. The rLAN task was different from the LAN in so far; the image screen side and the motor response were reversed. In other words, if the participant wished to choose the right image, he/she had to press the left mouse button and vice-versa. Finally, the difference between sLAN and LAN tasks is the shorter SOn interval (2s).
Context and images
We designed a total of 90 contexts, based on situations in which we would not respond automatically in our daily lives to construct imaginary scenarios where to make decisions. At the beginning of each session, the participants trained to make decisions for 10 contexts, the responses to which were excluded from further analysis. The contexts were introduced by a single sentence printed on the screen, and binary options were presented by pictures showing objects or elements fitting each context. The image resolution was 265x256 pixels. Most of the pictures were obtained from two free datasets: BossStimuli, BradyLab. A few were collected via Google image search from the websites which did not require attribution and were available for non-commercial use. The list of contexts and images are available at this repository: https://www.kaggle.com/datasets/novecentous/lanimages/data. The order of presentation of the contexts in each session was one of three different randomized orders.
Behavioural analyses
Analysis of the processes underlying participants’ decisions was carried out through careful examination of gaze trajectories over the screen stimuli, and keypress responses with custom written MATLAB (The Mathworks, Inc; Natick, MA) code. We used mouse button-press to detect and quantify reactions times, and carefully analysed gaze trajectories to characterize the decision-making processes. Trials were discarded if the response time was negative or longer than 4s. On a single trial basis, we calculated the decision time (DT) and, from the gaze trajectory, we extracted the amount of time spent gazing over each image presented and over their locations once these were removed from the scene. From gazing trajectories, we also detected the amount of times participants change their focus from one image to the other. We performed pair-wise comparisons between the DT distributions obtained by means of t-tests between experiments. To assess effect-size we also calculated the Bayes factor associated to these tests.
Observation time vs choice likelihood
To analyse the cognitive processes of option analysis and deliberation during decision-making, we resolved to analysing eye movements and quantifying the time durations devoted to look at either stimulus, or at the frames marking their former locations. It was expected that participants will choose more often the image (and the location where the image had been formerly presented) they watch the longest. To test this for each participant, we quantified the logarithm of the ratio of the time duration devoted to look at the right over the left image for each trial, distributing them into quartiles, and calculated the number of times the participant chose the right over the left image for each quartile. We performed this calculation separately for the observation and for the reporting intervals. Second, we fitted a psychometric logistic curve to the data [64],
where β0,β1 are the regression coefficients and log(TR/TL) the ratio between right/left observation time. This curve was fitted on a single participant basis, obtaining a coefficient of determination, and a p-value quantifying the goodness-of-fit and confidence intervals. To assess significance per participant, we used a permutation test. In brief, we recorded the β1slope parameter obtained from the original fit. We then compared this value to the distribution of βvalues obtained from 10,000 shuffle data sets, in which the right choice preference values obtained for the four quartiles were randomly shuffled. If the original β1 was larger than 95% percentile of the distribution of β1obtained from the fits of the shuffled data, the result was considered significant at P < 0.05. To determine group significance, we performed a t-test on the set of β1 logistic regression coefficients (the sigmoid slope parameter) obtained from the original data, as a test that the distribution of slopes is significantly different from zero, establishing significance at P < 0.05.
Cognitive factors and observation time
The fixation patterns indicate which stimuli are informative or interesting about each option [65]. Furthermore, eye movements can also be informative about the location of the previously encoded stimuli in the scene and the decision strategies used during memory retrieval [66–68]. Here we recorded scales of the four cognitive factors listed next, which each participant was asked to rank with a number [0–10] after reporting each decision: 1) how Confident (C) are you of your choice; 2) how Rewarding (R) was the choice; 3) how Interesting (I) was the choice; and 4) how Complex (CM) was the choice, as potential factors participating of the decision-making process. As part of our gazing strategy analysis, we fitted a Generalized Linear Model (GLM) to the time devoted to observing the selected option and the values reported for the different cognitive factors: Confidence (C), Reward (R), Interest (I), Complexity (CM), and the interactions RxI, RxCM, IxCM. The fitting was performed separately for three intervals: Stimulus-On (SOn), Stimulus-Off (SOff), divided into two intervals: from the GO signal Until the Decision is reported (SOff-UD), and Past the Decision (SOff-PD) and until the 4s SOff interval is over. In some cases, we also used aggregates of these intervals for specific analyses.
T stands for the proportion of time devoted to look at the selected option over the total interval duration (SOn, SOff-UD, SOff-PD). Group significance was assessed with a t-test over the group coefficients obtained for each factor across participants, for each individual experiment, and established at a threshold of P < 0.05.
Deliberative saccadic movements and cognitive factors
Decisions may often favour the preferred option. However, when the difference between options is minimal and/or hard to assess, visual decisions based on a single saccade per image are bound to be uninformed. Furthermore, in our case, decisions should be based on the appropriateness of each option concerning the context, which demands some extra effort of future projection. Under these circumstances, we hypothesized that participants would use a strategy of alternately gazing to each image to gradually gather the evidence associated to each option, and to ultimately choose the most befitting option to each scenario [31]. To assess this, we first defined two metrics: the number of times participants fixated from target to target (#Changes of Target - #CoT, which is an integer number), and the Frequency of the Changes of Target (fCoT), defined as the #CoT divided by the duration of the interval of study. We calculated these two metrics within three time intervals: Stimulus-On (SOn), Stimulus-Off Until Decision (SOff-UD) and Stimulus-Off Past-Decision (SOff-PD). Second, to quantify the relationship between these metrics and the cognitive factors presented previously, we calculated two GLMs between the z-scored metrics and the z cognitive factors reported (see EQ3–4)
These two GLMs were fitted on the data of each participant. Group significance was established by running a t-test on the group coefficients obtained for each factor across participants, for each individual experiment.
Confidence and deliberation with parallel and sequential diffusion-models
In addition to the data-based analyses described until now, we also inquired about two hypotheses by which evidence could be accumulated by a drift-diffusion model in a binary decision-making task: by maintaining simultaneous consideration of both options (parallel hypothesis), or by alternating focus between both options (sequential hypothesis). To assess which hypothesis got the most of support, we first formalized both hypotheses as two diffusion models, in which choices occur as the accumulation of evidence reaches one of two bounds [1,40], and then performed a Bayesian model selection between them (see section next). Importantly, these two hypotheses have significantly different implications as to how we relate evidence accumulation to the confidence with which we commit to a decision, which we formalize for the case of here studied. The goal of this exercise was to obtain defining features for each model that relate observation times and confidence, and then perform a model comparison of the extracted features, as described next.
We based our analyses on the participants reported confidence after each decision they made. The common dynamics of both models may be described as follows. We assume that each option has an associated value and
, and that these are unknown to the participant. The preferred option is determined by the estimated difference between those values, which is determined by gathering noisy information associated to each option for some time, to ultimately choose the option with the highest value. Furthermore, although the participant may get estimates for either option (1 or 2), their associated noise renders the choices necessarily uncertain, and the probability of choosing the most valuable choice smaller than one, which has been formally assimilated to a quantification of the notion of confidence [45,69].
In the parallel diffusion model, the dynamics of accumulated evidence at time
obey to
where is the drift,
is the variance, and
is a zero-mean, unit-variance white noise process. The drift represents the average instantaneous value difference between the two options, which is corrupted by noise, turning the drift estimation into an inference problem. The noisy evidence in the right-hand side of Eq. (5) is accumulated over time as
, starting from zero. In this model, the accumulation of evidence occurs in parallel for both options—in the form of the relative difference between them. The accumulation of evidence stops when
hits for the first time one of the decision boundaries
or
, which determines both the choice and the decision time. The option 1 is chosen if the upper bound is hit first;
, as then the probability that
is larger than one half. The option 2 is chosen if the lower bound is hit first;
.
By assuming a flat normal prior over value, the (Bayesian) decision confidence at decision time
in such a model may be characterized as
where is the cumulative density function of the standard normal [69]. In this context, the decision confidence equals the probability of the participant being correct in the decision given that the accumulated evidence is
at decision time
[4,69]. Remarkably, the previous expression is exact and applies even if the bounds depend on time
, in any arbitrary fashion. In other words, decision confidence depends exclusively on decision time, and not on the unknown drifts, assuming all other parameters of the model being fixed. Therefore, longer decision times will involve a lower decision confidence regardless of the individual fractions of time looking at each option, confirming the intuition that if long times are required to make a decision, it must have been because the choice was difficult and the drift was close to zero.
By contrast to the parallel hypothesis and model just described, in the sequential diffusion model a single option is attended at any given time [3]. This we can model by means of a diffusion model, where the accumulated evidence
obeys to equation 5, with the difference that the drift
is now time-dependent. This drift reflects the average instantaneous evidence for the option that is currently attended, and equals
when that is option 1, and
when that is option 2. Thus, in this model, the accumulation of evidence occurs sequentially, alternating between options 1 and 2, according to the time devoted to observing each of them. As in the parallel model, we assume that there are two time-independent decision boundaries, and that the choice and the decision time is determined by a time at which one of the boundaries is hit as a result of the accumulated evidence.
Under specific assumptions, we may also derive an expression for decision confidence for the sequential model as well, which corresponds to the probability of choosing correctly. To that end, we assume that the options are of the same absolute value but of opposite sign (an offset can be added without modification of the equations), but the participant does not know whether or
, that is, it is not known which of the two options is better. Therefore, the goal of the participant is to determine whether
or
. It can do so by alternatively attending to each of the options during specific time durations
where
is the time devoted to observing option
during an epoch
. The sufficient statistics for this problem are simply the accumulated evidence
and
during the total times
and
that options 1 and 2 were paid attention to, respectively [69], and then the probability that
is
This probability is larger than one half only if , that is, when the evidence while observing option 1 rises above the evidence while observing option 2. This case supports the hypothesis that the first option is better than the second one. Finally, assuming the values are small, we have that
and
. After inserting them into Eq. (7) we obtain
From equation 8 follows that decision confidence in the sequential diffusion model does not depend on the total decision time , but on their difference
. Specifically, a longer observation time of option 1 implies a higher confidence that this option will exhibit the highest value.
These derivations are based on a specific mathematical formulation of confidence in the context of the drift diffusion model (Please, see ref. 44 for the full analysis). The reasoning hereby introduced is merely intended to allow a qualitative comparison of the parallel vs sequential operation of oculomotor dynamics and confidence, in the terms described in the experiment. In this light, we simplified the two strictly different hypotheses into the parallel (Eq. 9) and sequential (Eq. 10) expressions, while maintaining the same behaviour of equations 6 and 8. Furthermore, because it is also possible that evidence accumulation abides to a compromise between both models, we also proposed a third mixed sequential and parallel model (Eq. 11), where the accumulation of evidence is a combination of both. In this case, the decision confidence (C in formulae 9–11) will covary with both total decision time and the difference of observation times between them.
The comparison procedure is described next.
Bayesian model selection.
We used Bayesian Model Selection (BMS) to assess which formulation of evidence accumulation best explained the decision times obtained experimentally and the confidence reported by each participant after each trial, according to the hypotheses/models described in the previous section. Comparison between hypotheses was performed as a function of the individual free energy, by means of a Random Effects Analysis (RFX) fitting the [0–10] confidence index reported by the participants, normalized between 0 and 1. We performed Bayesian Model Selection (BMS) across the RFXs at the group level [36]. Briefly, models are treated as random effects, which could differ per participant, though assuming a fixed distribution across the population. To make the test possible, we first needed to use Variational Bayes (VB) [70] to obtain the posteriors and a frequency metric for each model (M1-M3, cf. eqs. 9–11) across the population [34,71]. The advantage of the VB algorithm is that it does not only invert models with a robust parameter estimation, but also estimates the model evidence, representing a trade-off between accuracy (goodness-of-fit) and complexity (degrees of freedom) [72]. The free energies estimated for each participant and model were submitted to a random effect analysis (RFX), which assumes that models could differ between participants and that they have a fixed (unknown) distribution across the population.
The BMS procedure yielded the exceedance probability [36], which measures how likely it is that any given model is more frequent than all other models in the comparison set. The BMS yields one additional metric, the model frequencies (MF) across that population, which explains the larger amount of variance for each participant’s data. The specific model comparison was based on the distribution of PFs, assumed to be Gaussian, characterized by two parameters: a mean and a standard deviation. We performed the BMS procedure on the three aforementioned models.
References
- 1. Gold JI, Shadlen MN. The neural basis of decision making. Annu Rev Neurosci. 2007;30:535–74. pmid:17600525
- 2. Cisek P, Puskas GA, El-Murr S. Decisions in changing conditions: the urgency-gating model. J Neurosci. 2009;29(37):11560–71. pmid:19759303
- 3. Krajbich I, Rangel A. Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc Natl Acad Sci U S A. 2011;108(33):13852–7. pmid:21808009
- 4. Drugowitsch J, Moreno-Bote R, Pouget A. Relation between belief and performance in perceptual decision making. PLoS ONE. 2014;9(5):e96511.
- 5. Kiani R, Shadlen MN. Representation of confidence associated with a decision by neurons in the parietal cortex. Science. 2009;324(5928):759–64. pmid:19423820
- 6. Shadlen MN, Kiani R. Decision making as a window on cognition. Neuron. 2013;80(3):791–806. pmid:24183028
- 7. Moreno-Bote R. Decision confidence and uncertainty in diffusion models with partially correlated neuronal integrators. Neural Comput. 2010;22(7):1786–811. pmid:20141474
- 8. Khalvati K, Kiani R, Rao RPN. Bayesian inference with incomplete knowledge explains perceptual confidence and its deviations from accuracy. Nat Commun. 2021;12(1):5704. pmid:34588440
- 9. Fetsch CR, Kiani R, Shadlen MN. Predicting the accuracy of a decision: a neural mechanism of confidence. Cold Spring Harb Symp Quant Biol. 2014;79:185–97. pmid:25922477
- 10. Brus J, Aebersold H, Grueschow M, Polania R. Sources of confidence in value-based choice. Nat Commun. 2021;12(1):7337. pmid:34921144
- 11. Rouault M, Lebreton M, Pessiglione M. A shared brain system forming confidence judgment across cognitive domains. Cereb Cortex. 2023;33(4):1426–39. pmid:35552662
- 12. Pleskac TJ, Busemeyer JR. Two-stage dynamic signal detection: a theory of choice, decision time, and confidence. Psychol Rev. 2010;117(3):864–901. pmid:20658856
- 13. Badami R, VaezMousavi M, Wulf G, Namazizadeh M. Feedback about more accurate versus less accurate trials: differential effects on self-confidence and activation. Res Q Exerc Sport. 2012;83(2):196–203. pmid:22808705
- 14. Desender K, Boldt A, Verguts T, Donner TH. Confidence predicts speed-accuracy tradeoff for subsequent decisions. Elife. 2019;8:e43499. pmid:31429827
- 15. Cos I, Bélanger N, Cisek P. The influence of predicted arm biomechanics on decision making. J Neurophysiol. 2011;105(6):3022–33. pmid:21451055
- 16. Cos I, Duque J, Cisek P. Rapid prediction of biomechanical costs during action decisions. J Neurophysiol. 2014;112(6):1256–66. pmid:24899673
- 17. Reynaud AJ, Saleri Lunazzi C, Thura D. Humans sacrifice decision-making for action execution when a demanding control of movement is required. J Neurophysiol. 2020;124(2):497–509. pmid:32639900
- 18. Thura D, Cabana J-F, Feghaly A, Cisek P. Integrated neural dynamics of sensorimotor decisions and actions. PLoS Biol. 2022;20(12):e3001861. pmid:36520685
- 19. Li L, Wang D. Adaptive neurons compute confidence in a decision network. Sci Rep. 2021;11(1):22190. pmid:34772969
- 20. Gold JI, Stocker AA. Visual Decision-Making in an Uncertain and Dynamic World. Annu Rev Vis Sci. 2017;3(1):227–50.
- 21. Boundy-Singer ZM, Ziemba CM, Goris RLT. Confidence reflects a noisy decision reliability estimate. Nat Hum Behav. 2023;7(1):142–54. pmid:36344656
- 22. Gluth S, Rieskamp J, Büchel C. Deciding when to decide: time-variant sequential sampling models explain the emergence of value-based decisions in the human brain. J Neurosci. 2012;32(31):10686–98. pmid:22855817
- 23. Ghaffari M, Fiedler S. The Power of Attention: Using Eye Gaze to Predict Other-Regarding and Moral Choices. Psychol Sci. 2018;29(11):1878–89. pmid:30295569
- 24. Falandays JB, Spevack S, Pärnamets P, Spivey M. Decision-Making in the Human-Machine Interface. Front Psychol. 2021;12:624111. pmid:33643152
- 25. Pärnamets P, Johansson P, Hall L, Balkenius C, Spivey MJ, Richardson DC. Biasing moral decisions by exploiting the dynamics of eye gaze. Proc Natl Acad Sci USA. 2015;112(13):4170–5.
- 26. Rich EL, Wallis JD. Decoding subjective decisions from orbitofrontal cortex. Nat Neurosci. 2016;19(7):973–80. pmid:27273768
- 27. Ferreira F, Apel J, Henderson JM. Taking a new look at looking at nothing. Trends Cogn Sci. 2008;12(11):405–10. pmid:18805041
- 28. Richardson DC, Spivey MJ. Representation, space and Hollywood Squares: looking at things that aren’t there anymore. Cognition. 2000;76(3):269–95. pmid:10913578
- 29. Altmann GTM. Language-mediated eye movements in the absence of a visual world: the “blank screen paradigm”. Cognition. 2004;93(2):B79-87. pmid:15147941
- 30. Johansson R, Johansson M. Look here, eye movements play a functional role in memory retrieval. Psychol Sci. 2014;25(1):236–42. pmid:24166856
- 31. Krajbich I, Armel C, Rangel A. Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci. 2010;13(10):1292–8. pmid:20835253
- 32. Shimojo S, Simion C, Shimojo E, Scheier C. Gaze bias both reflects and influences preference. Nat Neurosci. 2003;6(12):1317–22. pmid:14608360
- 33. Cisek P, Kalaska JF. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci. 2010;33:269–98. pmid:20345247
- 34. Daunizeau J, Adam V, Rigoux L. VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data. PLoS Comput Biol. 2014;10(1):e1003441. pmid:24465198
- 35. Friston K. The free-energy principle: a rough guide to the brain?. Trends Cogn Sci. 2009;13(7):293–301. pmid:19559644
- 36. Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ. Bayesian model selection for group studies. Neuroimage. 2009;46(4):1004–17. pmid:19306932
- 37. Cisek P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos Trans R Soc Lond B Biol Sci. 2007;362(1485):1585–99. pmid:17428779
- 38. McSorley E, Lyne C, McCloy R. Dissociation between the impact of evidence on eye movement target choice and confidence judgements. Exp Brain Res. 2014;232(6):1927–40. pmid:24682429
- 39. Lee DG, Daunizeau J, Pezzulo G. Evidence or Confidence: What Is Really Monitored during a Decision?. Psychon Bull Rev. 2023;30(4):1360–79. pmid:36917370
- 40. Ratcliff R, McKoon G. The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput. 2008;20(4):873–922. pmid:18085991
- 41. Wang X-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron. 2002;36(5):955–68. pmid:12467598
- 42. Shpiro A, Moreno-Bote R, Rubin N, Rinzel J. Balance between noise and adaptation in competition models of perceptual bistability. J Comput Neurosci. 2009;27(1):37–54. pmid:19125318
- 43. Hayden BY, Moreno-Bote R. A neuronal theory of sequential economic choice. Brain Neurosci Adv. 2018;2:2398212818766675. pmid:32166137
- 44. Roe RM, Busemeyer JR, Townsend JT. Multialternative decision field theory: a dynamic connectionist model of decision making. Psychol Rev. 2001;108(2):370–92. pmid:11381834
- 45. Hellmann S, Zehetleitner M, Rausch M. Simultaneous modeling of choice, confidence, and response time in visual perception. Psychol Rev. 2023;130(6):1521–43. pmid:36913292
- 46. Treue T, Martinez-Trujillo JC. Cognitive physiology: moving the mind’s eye before the head’s eye. Curr Biol. 2003;13(11):R442-4. pmid:12781154
- 47. Zylberberg A, Fernández Slezak D, Roelfsema PR, Dehaene S, Sigman M. The brain’s router: a cortical network model of serial processing in the primate brain. PLoS Comput Biol. 2010;6(4):e1000765. pmid:20442869
- 48. Zylberberg A, Ouellette B, Sigman M, Roelfsema PR. Decision making during the psychological refractory period. Curr Biol. 2012;22(19):1795–9. pmid:22921368
- 49. Grimaldi P, Lau H, Basso MA. There are things that we know that we know, and there are things that we do not know we do not know: Confidence in decision-making. Neuroscience & Biobehavioral Reviews. 2015;55:88–97.
- 50. Padoa-Schioppa C. Neuronal origins of choice variability in economic decisions. Neuron. 2013;80(5):1322–36. pmid:24314733
- 51. Schultz W, Apicella P, Scarnati E, Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci. 1992;12(12):4595–610. pmid:1464759
- 52. Glimcher P. Decisions, decisions, decisions: choosing a biological science of choice. Neuron. 2002;36(2):323–32. pmid:12383785
- 53. Balleine BW, Delgado MR, Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci. 2007;27(31):8161–5. pmid:17670959
- 54. FitzGerald THB, Schwartenbeck P, Dolan RJ. Reward-Related Activity in Ventral Striatum Is Action Contingent and Modulated by Behavioral Relevance. J Neurosci. 2014;34(4):1271–9.
- 55. Gluth S, Rieskamp J, Büchel C. Neural evidence for adaptive strategy selection in value-based decision-making. Cereb Cortex. 2014;24(8):2009–21. pmid:23476024
- 56. Desender K, Boldt A, Yeung N. Subjective Confidence Predicts Information Seeking in Decision Making. Psychol Sci. 2018;29(5):761–78. pmid:29608411
- 57. Noudoost B, Clark KL, Moore T. A distinct contribution of the frontal eye field to the visual representation of saccadic targets. J Neurosci. 2014;34(10):3687–98. pmid:24599467
- 58. Costello MG, Zhu D, Salinas E, Stanford TR. Perceptual modulation of motor--but not visual--responses in the frontal eye field during an urgent-decision task. J Neurosci. 2013;33(41):16394–408. pmid:24107969
- 59. Lowe KA, Zinke W, Cosman JD, Schall JD. Frontal eye fields in macaque monkeys: prefrontal and premotor contributions to visually guided saccades. Cereb Cortex. 2022;32(22):5083–107. pmid:35176752
- 60. Bang D, Fleming SM. Distinct encoding of decision confidence in human medial prefrontal cortex. Proc Natl Acad Sci U S A. 2018;115(23):6082–7. pmid:29784814
- 61. Weilbächer RA, Krajbich I, Rieskamp J, Gluth S. The influence of visual attention on memory-based preferential choice. Cognition. 2021;215:104804. pmid:34167016
- 62. Folke T, Jacobsen C, Fleming SM, De Martino B. Explicit representation of confidence informs future value-based decisions. Nat Hum Behav. 2016;1(1).
- 63. De Martino B, Fleming SM, Garrett N, Dolan RJ. Confidence in value-based choice. Nat Neurosci. 2012;16(1):105–10.
- 64. Hanks TD, Mazurek ME, Kiani R, Hopp E, Shadlen MN. Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task. J Neurosci. 2011;31(17):6339–52.
- 65.
Henderson JM, Hollingworth A. Eye movements, visual memory, and scene representation. Perception of faces, objects, and scenes: Analytic and holistic processes. New York, NY, US: Oxford University Press. 2003. p. 356–77.
- 66. Renkewitz F, Jahn G. Memory indexing: a novel method for tracing memory processes in complex cognitive tasks. J Exp Psychol Learn Mem Cogn. 2012;38(6):1622–39. pmid:22545615
- 67. Heck DH, De Zeeuw CI, Jaeger D, Khodakhah K, Person AL. The neuronal code(s) of the cerebellum. J Neurosci. 2013;33(45):17603–9. pmid:24198351
- 68. Scholz A, von Helversen B, Rieskamp J. Eye movements reveal memory processes during similarity- and rule-based decision making. Cognition. 2015;136:228–46. pmid:25498749
- 69. Moreno-Bote R. Decision confidence and uncertainty in diffusion models with partially correlated neuronal integrators. Neural Comput. 2010;22(7):1786–811. pmid:20141474
- 70. Penny WD, Stephan KE, Daunizeau J, Rosa MJ, Friston KJ, Schofield TM, et al. Comparing families of dynamic causal models. PLoS Comput Biol. 2010;6(3):e1000709. pmid:20300649
- 71. Friston K, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W. Variational free energy and the Laplace approximation. NeuroImage. 2007;34(1):220–34.
- 72.
Robert C. The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer. 2001.