Fig 1.
(a) Trial structure of the probability-estimates beads task. Participants are first shown two boxes, a ‘green box’ mostly filled with green beads and a ‘blue box’ mostly filled with blue beads. The ratio of blue to green beads (bead ratio) is shown. Participants are instructed that one of these two boxes, referred to as the “hidden box”, is selected at random, and that their task is to estimate which box is the hidden box based on beads drawn from it. Next, they are shown an obscured representation of the hidden box, but no bead is drawn. Participants then make a first probability estimate using a slider to indicate their perceived probability that the hidden box is either the blue or green box. White circles on top of the screen are used as placeholders to illustrate the remaining samples that will be drawn during the trial. After this first estimate, participants see the hidden box again but this time a bead rises out of the box. Participants are then asked to report a second probability estimate after seeing the first bead. The drawn bead replaces the leftmost available placeholder, starting a sequential visual record of beads drawn during a trial. This process of drawing and estimating repeats until participants have observed 8 samples and reported 9 estimates per trial. At the end of the trial, participants make a binary choice of the box they believe is the hidden box. After this choice, a new trial begins. (b) Task variable space showing bead-ratio conditions on the y-axis (each shown in a different shade of blue) and an evidence-order metric (evidence asymmetry) on the x-axis, with negative values indicating front-loading of majority beads (more majority beads, beads consistent with the identity of the hidden box, in the first half of the 8-bead sequence) and positive values indicating back-loading (more majority beads in the second half). The absolute value of the x axis corresponds to more extreme front- or back-loading (the most extreme being a sequence where 5 majority beads are all in the front or all in the back, respectively, and the least extreme being sequences where beads are evenly distributed around the middle). Larger circles reflect sequences with more majority beads. Sequences are organized in mirror-opposite pairs, with two example pairs shown on the right. Note that the examples illustrate majority beads as black and minority beads as white (albeit in the task majority beads were green or blue consistent with the identity of the hidden box in a given trial). Trials were selected to span the full range of the evidence asymmetry space while avoiding confounds with the bead-ratio condition (Fig 1B) and cumulative evidence (S1 Fig).
Fig 2.
Model predictions for sequential base-rate neglect under the weighted Bayesian model.
An agent with sequential base-rate neglect (ω1 < 1.00; for these simulations: ω1 = 0.88), in blue, is compared with a Bayesian ideal observer (ω1 = 1), in grey, on the 60:40 bead-ratio condition. Values of ω2(60:40) are 0.51 (or between 0.31 and 0.66 in the shaded regions in panels d-e) consistent with observed mean values (and 25th to 75th percentile range) in our prior work with a similar beads task [31]. (a) Simulated sequential probability estimates for two mirror-opposite sequences for a base-rate neglecting agent (blue/solid) and the ideal Bayesian observer (grey/dashed). Majority beads are shown as black and minority beads as white for illustrative purposes. Belief trajectories for front-loaded sequences show a gradient from dark to light and those for the back-loaded sequences transition from light to dark (b) Simulation of the recency bias, defined as the difference between the final probability estimate after 8 beads between mirror-opposite pairs, as a function of the absolute evidence asymmetry of the pairs. As in Fig 1, larger circles reflect sequences with more majority beads. The fit line shows the fixed effect of absolute evidence asymmetry on the final estimate difference. The simulated base-rate neglecter shows higher estimates for back-loaded sequences (compared to their front-loaded mirror opposites), particularly for sequence pairs with more evidence asymmetry. This effect varies with evidence strength and is strongest in the 90:10 condition (S2A Fig). See S2E Fig for a simulation of an agent that overweights the prior. The lower-case delta shows the example from (a). (c) Simulation of the magnitude of logit-belief updates as a function of the prior with respect to the color of the current evidence. For illustrative purposes, the x-axis has been discretized into bins equivalent to 0.1 increments of prior beliefs in probability space. The y-axis represents the mean magnitude of the logit belief updates (the difference in the log-odds of the prior and the posterior belief). The Bayesian ideal observer has constant logit belief-updates. In contrast, the simulated base-rate neglecter shows logit-belief updates that depend upon the prior belief, with relatively larger updates for prior-inconsistent evidence (left of the vertical dashed line) and smaller for prior-consistent evidence (right) (see S2B Fig for a condition-wise simulation). The fit line reflects the fixed effect of logit-prior on the logit-belief update, which we refer to as prior-dependent belief updating. The model predicts main effects of logit-prior and bead-ratio condition, but no interaction S2B Fig. See S2D Fig for a simulation illustrating the distinct scaling effects of ω2 and S2F Fig for a simulation of an agent that overweights the prior. (d) Simulation demonstrating the predicted relationship between ω1 and the evidence asymmetry slope (blue fit line from b). (e) Simulation of the predicted relationship between ω1 and mean final estimate difference (average of blue data points in b). (d,e) The blue and grey dots show the values for the base-rate neglecting and Bayesian ideal observers simulated in (a,b,c).
Fig 3.
Study 1 participants show behavioral signatures of sequential base-rate neglect which scale with model-derived prior underweighting.
(a) Group mean of average probability estimates over bead draws for each bead-ratio condition. Participants updated beliefs progressively toward the correct hidden box with steeper slopes for stronger evidence. The inset shows the same data limited to matched (identical) sequences for the 60:40 and 90:10 conditions. Solid lines and shaded regions reflect the mean and standard error of the mean (SEM) of the weighted Bayesian model fits across participants. (b) Group mean of final estimate difference as a function of evidence asymmetry. Each data point shows the difference in the probability estimate after 8 beads for a back-loaded and a front-loaded sequence comprising a mirror-opposite pair, with positive values indicating higher estimates for back-loaded sequences consistent with recency bias. Solid lines and shaded regions reflect the mean and SEM of the weighted Bayesian model fits. Consistent with model predictions (Fig 2B), the data shows a recency bias scaling with evidence asymmetry. (c) Group median of individual medians for the magnitude of logit-belief updates as a function of the logit prior with respect to the color of the current bead sample, divided by bead-ratio condition. The x-axis is discretized into bins equivalent to 0.1 increments of the prior belief in probability space (with a lower limit of 0.01 and an upper limit of 0.99; data only binned for visualization). The y-axis represents the magnitude of the logit-belief updates (the difference in the log-odds of the prior and posterior beliefs). Solid lines and shaded regions reflect medians and 95% bootstrapped confidence intervals of the weighted Bayesian model fits. Although not displayed for visual clarity, the confidence intervals for the raw data overlap substantially with the model fits. For visualization only, we excluded extreme outlier or noisy data points (logit belief updates > 2, individual median values based on less than 3 data points for a given bin, group median values based on less than 25% of individuals) for a total exclusion of 6.96% of the data. Consistent with model predictions (Fig 2C), the data shows prior-dependent belief-updating with less updating for prior-consistent evidence (right of the vertical dashed line; i.e. an overall negative slope). Note that at the group level this effect appears to be non-monotonic (with slightly positive slope towards the rightmost end) due to aggregation of data across individuals with different ω1 values, since individuals with ω1 > 1 are predicted to have and exhibit more updating to prior-consistent evidence (i.e., positive slopes; S2 Fig). (d) Formal model comparison for data from study 1. We compared 10 different models as in our previous work [31]. Each model is defined by its free parameters, which are reflected on the x-axis. See S28 Table for details. The winning model was defined as the model with the highest protected exceedance probability, which was the same as in our previous work [31] and in study 2 (S6 Fig). (e) The evidence asymmetry slope (equivalent to a single line fitted across all conditions in panel b) is plotted against the prior-weight ω1, showing a negative correlation. This correlation closely follows model predictions indicated by the black line (as in Fig 2D but with shaded regions including variability in likelihood-weight ω2 parameters between the 25th and 75th percentile range of observed values in our previous work [31]). Marginal violin plots show group medians and interquartile ranges. (f) The mean final estimate difference is shown against ω1, again showing a correlation that follows the model prediction (black line as in Fig 2E). Marginal violin plots show group medians and interquartile ranges. (e, f) Asterisks indicate a significant sign-rank tests of group medians against the corresponding reference values indicated by the dashed lines. Note that results in (e) and (f) were robust to the exclusion of outliers with an ω1 more than 3 scaled median absolute deviations [52] from the median (ω1<0.75; 11 outliers): after their removal, the correlation between ω1 and the evidence asymmetry slope was still significant (ρ = -0.58, p < 10−307), as was the correlation between ω1 and the mean final estimate difference (ρ = -0.53, p = 2.32 x 10−12). Posterior predictive checks further recapitulate the range of values in the data (S10–S12 Figs).
Fig 4.
Replication in study 2 of results from study 1.
(a, b) Mean final estimate difference as a function of evidence asymmetry for the low and high PDI groups independently (S18 and S19 Tables). Solid lines and shaded regions reflect the mean and SEM of the weighted Bayesian model fits. The center inset shows the exponential fit of the distribution of PDI global scores from study 1 (grey line) and study 2 (black line), indicating the cutoffs for high and low PDI by vertical dashed lines. (c, d) Logit-belief updates as a function of logit prior by bead ratio for the low (c; S20 Table) and high (d; S21 Table) PDI groups. Group medians of individual medians for logit-belief updates are shown and other conventions follow Fig 3C. Solid lines and shaded regions reflect medians and 95% bootstrapped confidence intervals of the weighted Bayesian model fits. (e) Evidence asymmetry slopes are plotted against ω1 by group. Other conventions as in Fig 3E. Marginal violin plots show the group medians and interquartile ranges. The asterisk indicates a significant rank-sum test comparing group medians of ω1. (f) Mean final estimate differences are plotted against ω1. The marginal violin plot shows the group medians and interquartile ranges. (e, f) The solid black line shows model predictions as in Fig 2E and 2F. As in Fig 3, after excluding outliers [52] (ω1<0.82; 11 outliers), the correlation between ω1 and the evidence asymmetry slope was still significant (ρ = -0.37, p = 1.08 x 10−4), as was the correlation between ω1 and the mean final estimate difference (ρ = -0.47, p = 7.32 x 10−7).
Fig 5.
Prior underweighting relates to belief oddity outside the laboratory.
(a) Summary of group-level differences (S22 Table) for model-based and model-agnostic measures between the low (n = 57) and high (n = 34) PDI groups. Bar plots are effect size (Cliff’s delta, δ) and 95% confidence intervals. (b) Scatterplot showing a negative correlation between ranked mean PDI scores and ω1 (n = 116). Values are adjusted by parameter values and the model fit (RMSE) for specificity as in the partial Spearman correlation analysis. Boxplots show medians (blue lines) and 25th and 75th percentiles (bottom and top edges, respectively). The solid black line reflects the least-squares linear fit to the data points. Mean PDI is the average of the global PDI scores across the pre-screening and the experimental sessions for each participant.
Fig 6.
Relationship between prior underweighting, prior noise, and response variance.
(a) Visual Schematic of Noisy Sampling Model. The noisy-sampling model captures an iterative sequential belief updating process where the internal representation of prior (and likelihood information) is noisy (see Methods). This is based on an agents’ uncertainty about the true values of the prior and likelihood, given recent evidence, with variances and
, and their assumed distributions of priors and likelihoods, with variances
and
. Note that variables are in logit space and noise consists of an additive zero-mean Gaussian distribution. Critical to the model are a noisy representation of the prior and likelihood where the noise is given by the functions
and
. More noise (e.g., due to higher
) leads to more random variability in responses reflecting the posterior belief (even for repetitions of identical sequence fragments, as captured by the model-agnostic measure of response variance). Optimal inference results from adjusting weighting commensurate with the degree of noise, with optimal weights given by the functions
and
. Finally, the optimal posterior is a weighted sum of the noisy prior and noisy likelihood in logit space. Model fitting used 4 free parameters, 1 shared
parameter and condition-specific
parameters (3), and a grid search with 4 fixed parameters for
(1) and
(3) (Methods). (b) Scatterplot of ranked response variance rank and ω1 showing a negative relationship indicating that individuals with more sequential base-rate neglect have more variability in their probability estimates for identical sequence fragments (Methods). (c) Scatterplot of ranked response variance and prior noise
showing a positive relationship indicating that the model-agnostic measure of response variability scales with the model-derived measure of prior noise. (b, c) Boxplots reflect median (blue) and 25th and 75th percentiles (bottom and top edges, respectively). Black lines show the least-squares linear fit of the data points. (e, f) Noise-corrupted parameter recovery analysis for the weighted Bayesian model (e) and the noisy-sampling model (f). The y-axis shows the percent deviation in the recovered versus the original parameter values. The x-axis shows the magnitude of the late Gaussian noise added at the response level in the model simulations in standard deviation. Each grey line depicts a single agent defined by a set of parameter values across a range of noise levels. The red shaded area indicates the estimated range of response variance found in the actual data as a 95% confidence interval based on the median response variance (see Methods). On average (black line), the critical parameters are adequately recovered, without systematic biases in their estimation for meaningful levels of late Gaussian noise (particularly for the weighted Bayesian model), indicating that low-level factors such as general inattention or random responding are unlikely to explain variability in ω1 values.
Fig 7.
Prior noise relates to belief oddity outside the laboratory.
(a) Summary of group-level differences for noisy-sampling model-based and model-agnostic measures between the low (n = 57) and high (n = 34) PDI groups. Bar plots are effect size (Cliff’s delta, δ) and 95% confidence intervals. (b) Scatterplot showing a positive correlation between ranked mean PDI scores and (n = 116). Values are adjusted by
parameter values and the model fit (RMSE) for specificity as in the partial Spearman correlation analysis. Boxplots show medians (blue lines) and 25th and 75th percentiles (bottom and top edges, respectively). The solid black line reflects the least-squares linear fit to the data points. Mean PDI is the average of the global PDI scores across the pre-screening and the experimental sessions for each participant.