Faithful tracking of trial-by-trial uncertainty

First, we tested whether participants demonstrate the ability to faithfully estimate the dispersion of the centered normal distribution assumed to be responsible for the observations. The MLE of the Gaussian, σ_ML (Fig 3A, red), is the sufficient statistic to inform the optimal response (green).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Human behavior closely tracks trial-by-trial uncertainty of future events.

(A) Mean responses across participants plotted as a function of the MLE of the sample, σ_ML(d), in ten equally spaced bins (black; error bars, 95% CI). Basing behavior on a Gaussian estimated by ML (red, N(x|0,σ_ML(d))) results in responses proportional to the estimate. The prior distribution that is assumed by the devised Bayesian benchmark model (green) biases responses towards intermediate values (see Methods). (B) Individual response curves of all 23 participants tested (gray lines). Three participants displaying poor compliance with the instructed task (dotted) were excluded from further analysis. The average across the remaining participants is superimposed (black).

https://doi.org/10.1371/journal.pcbi.1006205.g003

The averaged mean response across participants (black) is related to the ML approach in a approximately linear relationship (Methods). Assuming that participants use the Gaussian distribution for inference (Methods, normal model) yields good predictive performance and accounts for a substantial amount of the variance (regression, cross-validated median R² = 0.80, 95%-CI (0.73,0.82), across participants). Hence their judgments correlated tightly with the uncertainty about the abilities of the supposed dart players. Such uncertainty tracking is also apparent on an individual participant level (Fig 3B) (cross-validated median R² ranging from 0.47 to 0.93). On average, the responses appear to be systematically biased toward intermediate values with respect to the ML approach (Fig 3A, red) resembling the effect of a prior distribution (green) incorporating knowledge about the range of dispersions across trials. To quantify this effect, we used two variants of a model (Methods, Weighting model) that effectively determines whether judgments may still be predicted well if they are assumed to be proportional to the estimated dispersion (Eq 7). However, this was found to be strongly inferior to a linear relationship (Methods, Eq 5), even on an individual level (cross validation log likelihood (CVLL) difference Δ ≥ 20 for 12 participants, Δ ≥ 10 for 17 participants).

Evidence for an internal trial-by-trial objective

Next, behavior is examined with respect to the objective participants were instructed to obey. Namely, if their estimates are quantitatively accurate and correspond to the 65% target percentage. For independent trials, participants must infer the dispersion anew on each trial. Inferring a probability distribution over future events allows behavior to be derived from a principled trial-by-trial objective regarding the target percentage (see Fig 1B and Methods, Eqs 3 and 4). By construction, our task objective demands a quantification of the relative frequency of all future events and was intended to require participants to approximate distributional estimates.

To examine how well participants performed with respect to the devised optimal inference strategy, we calculated the capture percentage by evaluating (Eq 3) with respect to the optimally inferred probability distribution (Eqs 1 and 2). The distribution of the per participant median capture percentage across all trials is clustered close to the target of 65% (Fig 4A). In this measure opposing deviations cancel, so that it evidences an overall compliance to the target percentage across all trials. The median across participants is close to the target percentage, which indicates that participants quantify uncertainty in a quantitatively similar manner as the probabilistic benchmark model. The median of the absolute deviation per response is 6.54% (95% CI, (5.83,7.28) %) with respect to the external objective of the task. However, it is possible that behavior has been produced from an internal objective (see Eq 4) in which the percentage is matched much more closely to 65%. There are at least two contributions that inflate the deviation from the external measure (Fig 4A). First, there is intrinsic response noise which would even occur for fixed stimuli on the screen, e.g. through motor-related variability. Second, there are deviations due to mismatched inference with respect to our benchmark model [26]. The latter are deterministic and the result of e.g. different prior knowledge from the one assumed by our benchmark model. Altogether, the median absolute deviation (Fig 4A) is a conservative upper bound estimate for an internal trial-by-trial objective of the capture percentage such that the quantitative match with the target percentage can be considered high.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Behavior is consistent with participants possessing a subjective but well calibrated trial-by-trial internal objective that remains stable over the experiment.

(A) Across trials participants tend to comply well to the objective despite per trial deviations due to systematic biases and response noise, as the capture percentage c is typically around the target value 65% (vertical axis) and the median deviance is relatively small (horizontal axis). Histograms correspond to marginal distributions. (B) Participants display stable behavior throughout the experiment, as they do not appear to adjust their responses closer to the task objective over time. Median capture percentages c are calculated separately for the first and second halves of the experimental session.

https://doi.org/10.1371/journal.pcbi.1006205.g004

If participants did not possess an internal trial-by-trial objective, they could instead associate stimuli with suitable responses by a learning a behavioral function. Next, we tested whether behavior is consistent with this alternative approach, which should result in across-trial and feedback dependencies. Even though barely informative, the feedback may have been used to adjust behavior. Remarkably, however, the median capture percentage appears not to adjust closer to the target percentage as indicated by similar values calculated separately for the first and second half of the experimental session for each participant (Fig 4B). The absolute difference of the median capture deviation is small and not significantly different from zero (right-tailed Wilcoxon signed rank test, p = 0.48) despite the fact that the trial-averaged feedback about the capture percentage in the experimental session may have allowed to derive some global adjustments. Accordingly, too high a capture percentage on average should subsequently lead to the choice of smaller response frames. Hence, a decrease of the feedback error would be expected over time. The results, on the other hand, suggest that participants did not even use feedback to calibrate their probability estimates. We also confirmed that the previously presented feedback about the capture percentage did not influence behavior (regression, exceedance probability 2.04 ⋅ 10⁻⁴ compared to baseline model, see Methods). Similarly, no considerable dependencies across trials were found (Methods). Consequently, it appears unlikely that the feedback scheme had an important influence on behavior.

Overall, participants typically predict the dispersion of future darts in a quantitatively accurate manner. They appear to have relied on an internal trial-by-trial objective regarding the target percentage as they largely conform to trial independence, feature stable processing across time and virtually ignore feedback. This is consistent with internal probabilistic processing.

Systematic deviations from inference of a Gaussian

Thus far, behavior appears to be close to the optimal inference strategy defined by the benchmark model, but we have also observed deviations (Figs 3 and 4A). If behavior follows from inference of a normal distribution, it can only depend on the sample via the sufficient statistic, . This means that the squared position of each point should contribute equally to the final estimate. We tested this with a weighting model that generalizes σ_ML by assigning a tunable weight ω_n to each input depending on its excentricity, . Excentricity refers to the distance from the center irrespective of the side where the sample occurs.

Experimentally, the weights of the individual points tend to take unequal values when we index them with respect to their distance from the center (Fig 5A). Participants put more emphasis on the third most excentric point and down-weigh the first and the fourth point. We also tested whether other models of behavior, such as the KDE model, are able to reproduce this pattern (Fig 5B). For that purpose, we used those fitted models to generate surrogate responses for every actual experimental response. Subsequently, for the comparison, the weighting model was fitted to the surrogate responses. In the following, models will be compared by both the (i) weighting pattern (Fig 5B) as well as their (ii) overall ability to predict behavior (Fig 6). Consistent with the weighting pattern observed in our data, the normal model (nm) is far from providing the best predictions of behavior. This can be seen from the pairwise model comparison matrix (Fig 6). There the binomial probability that the model indexing the row (vs. the model indexing the column) is more likely to account for the data of a randomly chosen participant is depicted as color code. Additionally, entries with high exceedance probabilities are considered significant (Methods) and marked with asterisks. For instance, the comparison between the weighting model in row (wgt) to the normal model in column (nm) shows that the latter is clearly rejected (p_exc > 0.999). Beyond the group level, the normal model can be decisively ruled out individually for many participants despite the fact that generally different participants are best described by different models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The weighting pattern of the observed samples deviates from inference of a close-to-normal distribution and matches kernel density estimation (KDE).

Evaluation of the normalized weights ω_n of the weighting-model as a generalization of the MLE of a zero centered Gaussian. The points are indexed according to their distance from the center. (A) Input weight that each participant (gray lines) assigns as a function of the weight index. If participants followed optimal MLE based on a Gaussian centered at zero, all input weights should be equal (black line). Fitting of the weighting model (see Methods) shows a systematic deviation of the across participant median (red, error bars, 95% CI). Participants tend to overweigh the third most extreme value compared to the others. (B) Among all models tested, only KDE (blue) qualitatively matches the characteristics of the experimental weighting pattern (red, same as panel A). The other models fail to capture the behavioral weighting pattern (fits of the weighting model to the other indicated models’ output). Model abbreviations: kde—kernel density estimation, tlg—tiling, gnm—generalized normal, max–maximum.

https://doi.org/10.1371/journal.pcbi.1006205.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Pairwise model comparison evidences an inclination to resort to instance-based generalization, indicating that fluctuations have a profound effect on the inferred representations.

Summarized results of a hierarchical Bayesian model comparison procedure that estimates probability distributions over models. Pairwise comparisons (each square) are performed to evidence relative differences in prediction for models with different features. The color code over each square shows estimates of the parameter of the binomial distribution governing the probability by which the model indexed by the row is more likely than the one indexed by the column. This corresponds to the expectation value that a given model is considered responsible for generating the data of a randomly chosen participant. Superimposed are large differences of the exceedance probability (; ) which quantifies the belief that the row model is more likely to have generated the data of a randomly chosen participant compared to the column model. Model abbreviations: gpr—Gaussian process regression, wgt—weighting, kde—kernel density estimation, gnm—generalized normal, tlg—tiling, nm—normal, max—maximum, rng–range.

https://doi.org/10.1371/journal.pcbi.1006205.g006

We tested whether generalizations of the Gaussian can account for the systematic deviations that were observed before. The generalized normal model (gnm) allows for more freedom in the representation of the inferred density through a shape parameter governing its kurtosis (see Fig 1B) by generalizing the square in the exponential function to other powers than two leading to an unequal weighting pattern of the samples (Fig 5B). This model predicts significantly better than the Normal-model (Fig 6, p_exc > 0.999) by making use of the additional shape parameter to represent heavier tailed distributions (quartiles across participants Q = (0.79,1.24,1.68)). Heavier tailed distributions discount outlying and enhance the influence of inlying points on judgments (Fig 5B, black line). The experimental pattern (red) is not matched well suggesting that it does not reflect how participants behave. In addition, the weighting model still outperforms the generalized normal model (Fig 6).

Simple heuristics are poor predictors

We determined above that responses are on average relatively close to optimal but that the finer-grained behavioral patterns are inconsistent with inference of a Gaussian. That raises the question whether simpler, heuristic strategies that unequally weigh sample information might offer a better account of behavior.

We first tested the established heuristic models that use perceptually simple statistics and only a subset of the available information. The maximum model (max) only depends on the most excentric point which leads to a weighting pattern (Fig 5B, yellow) which is highly inconsistent with the experimental one (red). The participants’ weighting is more balanced and typically features weights smaller than four (normalization to number of sample points). The range model (rng) is based on the sample's range and predicts worse than the maximum model (Fig 6). On the group level, both are clearly refuted by all other models.

Another heuristic strategy is attending to just one point when sorting them according to their excentricity. In particular, the third most excentric point is important as it closely corresponds to the target percentage of 65% on the sample and is the response in the limiting case of pure instance-based generalization (see δ-KDE model, Methods). Participants typically take all point positions into account. The four unnormalized weights (w₁,…,w₄) are significantly different from zero for the respective number of (14, 20, 20, 19) of all 20 participants (Weighting model, 10000-fold permutation test, p-value threshold 0.05). Furthermore, for each individual at most one weight is non-significant showing that it is not an effect of grouping. Consistent with integration of the whole sample, the maximum of the normalized weights is considerably lower than four (Fig 5A).

Altogether, this is evidence that among all participants few exploit heuristics. The clear majority however resorted to some more sophisticated weighting inconsistent with the simple heuristics tested.

Behavior relies on instance-based generalization

So far, participants appear to violate the assumptions of a close to Gaussian distribution centered at zero that was suggested by the task instructions and the dart metaphor. Alternatively, the probability distribution to be inferred may be directly constructed from the observed instances by imposing only minimal structural constraints on the data. That corresponds to the assumption that the sample is representative of the unknown population to be estimated.

Our tiling model (tlg) implements such an approach with spatially confined basis distributions. It places a uniform distribution in between observations and hence the resulting density is increased around clusters and reduced elsewhere (Methods). It adapts to the fluctuations which are present in the sample. Consequently, the target capture percentage of 65% is by construction very close to the third most excentric point. As a result, this model emphasizes the third most excentric point (Fig 5B, green) and thus captures an important characteristic of behavior (red).

The kernel density estimation (KDE) model uses Gaussian basis functions to implement instance-based generalization. It centers a Gaussian distribution on each data point and thus assigns density to its vicinity depending on the standard deviation parameter. The experimental weighting pattern (black) is closely captured by KDE (Fig 5B, blue). It is very successful at predicting behavior and superior to both the normal and the generalized normal model considered before (Fig 6). The small and insignificant difference of the model probability (Fig 6, wgt vs. kde) indicates that KDE predicts on a similar level as the weighting model even though the latter has more adaptable parameters and thus may be considered more flexible. The weighting model does not explicitly construct a probability density but can be viewed as a functional approximation that can capture similar dependencies of behavior on the sample.

In summary, participants do not sufficiently exploit the structural constraints suggested by the task but instead give more freedom to the specific instances of the observations to determine their responses. The tendency to assume that even small samples are representative of the population could be well captured by nonparametric kernel density estimation.

Inferred representations feature overlapping and redundant kernels

Probability distributions over perceptual variables should be embedded in the context of more general knowledge of the task’s context. From a causal inference perspective, they should be attributed to the causal variables already known to exist. Treating all observations as if they originate from their own cause, i.e. as new causal variables, makes purely nonparametric methods seem of limited applicability in wider contexts. In this sense, KDE itself may be considered a heuristic approach as it largely ignores prior (structural) knowledge. Examining the inferred representations, we argue here that there is reason to believe that behavior is not purely nonparametric but can rather be conceived of as an instance-based modulation, or bias, to causal inference.

If we infer very narrow kernel functions for our participants that indicates that there is very little generalization from the sample. For close to orthogonal kernel functions with virtually no overlap (e.g. delta-distributions) the output reduces to a mere counting of observations. First, we tested how strong this instance-based bias is on the level of raw responses by comparing them to the predictions of δ-KDE (Fig 7A). Both axes are normalized to the MLE, σ_ML, of the sample (i.e. the draws from the standard normal distribution, see Methods). All responses are plotted as a function of the δ-KDE output. Thus, by construction, predictions of δ-KDE (green) itself follow the unity line while predictions of inference using a Gaussian likelihood function follow a constant line of slope zero (red). Values of the optimal benchmark model would fluctuate because of varying prior beliefs that average to a constant independent of the sample given the MLE. The slope of a linear function fitted to the experimental responses is far from one as expected from δ-KDE (Fig 7A, regression, median slope across participants 0.27, 95%-CI (0.24,0.38)). As opposed to the δ-KDE model, the KDE model (cyan) can predict the behavioral pattern (black) well because its kernel width parameter takes large values (Fig 7B, red) (median across participants 0.40, 95%-CI (0.35,0.59), in units of σ_ML). Participants capture a varying number of points with the response frame (Fig 7A, inset) which is only possible if the constructed density is a non-local function of the specific sample configuration on the screen. This slope pattern is not entirely inconsistent with inference of a Gaussian likelihood function as responses actually vary around its value as a function of the sample configuration. On the contrary, the normal model reaches high predictive performance in absolute values as shown before. However, additional to the responses derived from Gaussian inference, there are subtle instance-based variations which can be captured by the KDE model. At the level of the responses, behavior may be understood as inference of a normal distribution that is modulated by KDE.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Strong generalization is consistent with the possibility of integrating prior knowledge about the task structure.

(A) Responses (black) show higher consistency with inference of a single Gaussian than with approaches generalizing only weakly beyond the sample such as δ-KDE (limit of vanishing kernel widths; third most excentric sample point). The plot shows aggregated (median across participants, 95% CI) bin medians of the responses (normalized by σ_ML) and the fitted KDE model (cyan) as a function of the δ-KDE output (approximately equally filled bins). By construction, inference of a Gaussian results in a horizontal line (red) while δ-KDE (green) yields a linear function of slope one. The experimental curves are less steep indicating a rather moderate instance-based modulation compared to a Gaussian model. The inset is a zoomed-out version additionally showing the relationship of the responses to the distribution of sample points (median of absolute value within each bin). (B) The KDE model infers internal distributions that are smoothed and spatially extended around the sample points. The mean probability density function across participants (black, 95% CI) is shown for four different samples (blue circles). The inferred density is smooth featuring fewer modes than the number of basis distributions (red curves). This is a consequence of the large fitted Gaussian kernel widths which lead to substantial overlap of the basis distributions.

https://doi.org/10.1371/journal.pcbi.1006205.g007

Interestingly, KDE predicts behavior significantly better than the tiling model (Fig 6). The main difference is that the tiling model relies on spatially confined basis functions while Gaussian kernels are spatially extended. The weighting pattern shows that the tiling model (Fig 5B, green) overweighs the third most excentric point even more than behavior (red). The tiling model too closely resembles the purely instance-based approach of δ-KDE while behavior is not so strongly influenced by the third most excentric point. Of all models tested KDE (blue) best captures the weighting pattern (Fig 5B) because the large kernel width exhibits a non-local effect so that the positions of all points influence judgments leading to a more balanced pattern.

A large kernel width makes spatially extended basis distributions overlap (Fig 7B, red). Accordingly, we typically find fewer than four modes in the inferred densities of the participants (median of the per participant mean across trials 2.0375,95% CI = (1.50,2.27)). Thus, increasing the kernel width may be understood as a reduction of the effective number of components in the mixture distribution as measured by the number of modes (Pearson correlation coefficient, ρ = −0.95, p = 6.01 ⋅ 10⁻¹¹). Our data requires the KDE model to perform close to a regime where it must approximate inference of some smooth distribution which is closer to unimodal. Despite being the best approximation explored, it is nevertheless possible that the inference method used by our participants is structurally more constrained than KDE and uses some prior knowledge of the task structure.

From a representational point of view, the large overlap of the basis distributions (Fig 7B, red) is a rather redundant and thus inefficient way of representing the whole distribution. For a large degree of overlap, several kernel functions could be well represented by a single kernel function whose free parameters are tuned to accommodate all their contributions. Bayesian nonparametric mixture models [27] can effectively reduce the number of redundant mixture components and minimize shared responsibility to account for the data points. The number of components can adapt to the position and number of data points in the sample. It gives less freedom to the data than KDE but implements soft and gradual constraints towards sparsity. A preference for sparser or denser representations can be specified by a prior. Likewise, prior knowledge such as a zero-centered population may be included in this way. We suggest this as a connection to theoretical principles.

We found that participants show different preferences for instance-based generalization. The average number of modes of the inferred densities according to the KDE-model almost covers the full range of possible values (minimum 1.01, median 2.04, maximum 3.63, across participants). Even with wide kernels, KDE is limited in its ability to represent unimodal near-Gaussian distributions. Correspondingly, the difference in predictive performance (CVLL) between the KDE and the normal model is larger for smaller kernel widths (linear correlation coefficient, ρ = −0.54,p = 0.0075). Consistent with previous results, the slope in Fig 7A decreases with the kernel width (Pearson correlation coefficient, ρ = −0.66, p = 7.30 ⋅ 10⁻⁴). The determinants of the participants' preferences are unclear from this experiment. We remark however, that participants who infer more redundant densities tend to respond faster (Spearman correlation coefficient, ρ = 0.35,p = 0.064) although the result does not reach significance.

In summary, using KDE we found very wide overlapping kernels leading to densities which could be more sparsely represented. This hints at a more sophisticated inference approach than pure instance-based generalization. It may be considered a modulation of causal inference by a kernel-based approach. We suggest a connection to Bayesian nonparametric methods in statistics that allow to incorporate prior knowledge and sparsity constraints.

Explanation close to ceiling level

There are many possible ways in which this task might be approached by our participants. Thus, we attempt to estimate an upper bound of the predictable structure in the data regardless of how the task was solved by the participant. Gaussian process regression (GPR) is used to find a low-bias functional approximation between input d and behavior y. Hence, if a model reaches similar predictive levels, this is indication that it captures the most relevant computational operations. GPR is indeed found to be the best model (Fig 6) on the group level. However, the differences to the KDE-model are not disconcertingly large (median CVLL difference across participants, 13.3 dHart, 95%-CI, (−3.1, 23.5) dHart). Overall, KDE can predict on a comparable level as GPR. This is remarkable as for interpretable models, all factors need to be specified explicitly. For instance, even motor related variations with d would have to be incorporated. Moreover, as probability densities are high dimensional and subjective, the achieved match is not trivial.

Ethics statement

Comité Ético de Investigación Clínica, Parc de Salut MAR, Barcelona Spain, 2013/5464/I, titulado “Del laboratorio a la calle: El impacto de la integración multisensorial en la vida cotidiana”. Written informed consent was obtained from all participants.

Sampling scheme to generate observations

On each of the 320 trials, the horizontal positions of the points with respect to the center were generated as follows (Fig 1C). First, always N = 4 sample values r = (r₁,…,r₄) are independently drawn from a standard normal distribution r_n ~ N(0,1). Second, the samples were scaled by the factor ν/σ_ML(r), where is the maximum likelihood estimator (MLE) for a normal distribution centered at zero of the samples r and ν is drawn from a uniform probability distribution over the range of [10,140] pixels. The scaled sample d = ν/σ_ML(r) ⋅ r always has a MLE given by . This method allows choosing any desired value of σ_ML(d) by setting ν correspondingly. Setting σ_ML(d) directly, which is the main determinant for inference, has the advantage that observations d and the MLE σ_ML(d) take less extreme values which translates into increased numerical stability for model comparison. Defining an explicit latent σ-variable over a finite range instead would have led to a long-tailed σ_ML(d) distribution with undesirable properties (s. Fig 1D). The ability to tell apart models with similar predictions is enhanced if response noise and outlying conditions are kept at a minimum.

However, because of this way of generating the dots, the optimal inference model with respect to the actual generative model in the environment is not readily defined. Nevertheless, participants do not know these alterations to how the dots were generated. The best they can do is to follow the instructions and their prior knowledge suggested by the dart metaphor to explain the data. We do not define the optimal model with respect to the generative model in the environment. Instead, we define it as an optimal inference strategy based on a normal distribution whose width varies parametrically across trials. It follows the inference strategy of Eqs (1 and 2) and assumes a uniform prior over the range of [0,140] pixels. As this prior arguably matches the task instructions it was chosen as a basis for our Bayesian benchmark model and the feedback in the experiment.

Participants and experimental procedure

In total 23 participants (15 female, 8 male) were recruited mainly among students from the Pompeu Fabra University in Barcelona. We accepted all healthy adults with normal or corrected to normal vision. We obtained written confirmation of informed consent to the conditions and the payment modalities of the task. The training and the experimental session were carried out on a single appointment that nominally lasted 75 min. First, participants read detailed written instructions of the task. In a brief training session, they were given 40 trials to familiarize with the handling of the task through a short interactive session with feedback after every trial. The feedback consisted of the actual percentage c_t (using Eqs 1–3) they would have captured in trial t according their response y_t and our benchmark model. In addition, they were given a deviation score (mean squared error (MSE)) from the target percentage δ_t = (c_t − 0.65)² ⋅ 1000.

In principle, a subject could learn how a pair consisting of observations d together with his response y, (d,y), relates to the capture probability p from experience in the 40 training trials. For a given learned mapping (d,y) → p he would have to adjust y such that p = 0.65. We regard this as unlikely for the following reasons. First, 40 trials do not provide a lot of data to learn from. Second, the mapping is high-dimensional and nonlinear which makes it hard to learn and susceptible to the specific instantiations of d across trials–as well as the choice of y. (d,y) and p are never simultaneously visible on the screen. And finally, batch learning requires memorizing all presented pairs which seems infeasible for participants. While on-line learning is possible, it typically suffers from slower convergence rates.

Participants could ask any questions to the experimenter prior to the experiment. The subsequent experimental session consisted of 320 trials with pauses together with feedback after every 5 trials. In the experiment, the feedback consisted of 5-trial averages of the quantities c_t and δ_t above that were computed since the last pause. Participants were supposed to minimize the deviation score and were payed more compensation when having a smaller deviation score to incentivize optimization. This supposedly promoted high motivation to prevent participants from resorting to computationally cheaper heuristic shortcuts. The task circumvents risk aversion since there is practically nothing that the participant can do to prevent losses other than stating the response as accurately as possible.

The bonus payment was determined by the mean of their final deviation score after removing the eight worst trials. The payment was determined by comparison to an array of five thresholds that were set according to the {0.1,0.2,0.3,0.4,0.5} cumulative quantiles of the empirical deviation score distribution across prior participants. A lower score corresponds to a better performance so that participants were payed an additional bonus of {5,4,3,2,1} € if their final deviation score was less or equal to the quantile thresholds. This is a relative way of rewarding their efforts to optimize their responses. Irrespective of their performance they were paid 10 € and hence on average received 11,50 € per session. The experiment was carried out with 23 participants. Later we excluded three of them because their behavior had little dependence on the stimulus.

The task was presented with Matlab Psychtoolbox 3.0.12. Participants made input with an USB-mouse that allowed them to precisely adjust the width of the response frame and confirm it with a click. Immediately after trial onset, they were presented with the dots and could start to expand/shrink the frame from a random initial width by moving the mouse up/down-wards. The points were visible throughout the entire time until the participant confirmed his response with a click. The program then either proceeded to the next trial or to the feedback/pause screen that indicates the averages over the five last trials of the percentage the participant would have captured as well as the numerical deviation score. In addition, information about the how many of all trials have already been completed was presented. The participant could proceed at his own pace.

Computational models

We attempt to examine whether the behavior of the participants can be described by inference of probability distributions. More specifically, we attempt to infer whether their internal structural assumptions correspond to unimodal near-Gaussian distributions (Fig 2A) or might be better described by instance-based, nonparametric approaches (Fig 2B–2D) such as kernel density estimation. In addition, we checked whether selected heuristics can also account for the behavioral data.

Response mapping accounts for nuisance factors.

Behavior is influenced by various factors and subjective assumptions of the participant which are difficult to model explicitly. Among these are subjective prior knowledge and probability distortion. Even for a probabilistic agent there exists some mathematical freedom as to what prior distribution over the latent variables to use. We did not explicitly include prior knowledge into our models but instead endowed the model with flexibility to approximately account for such effects.

We make use of the fact that ultimately, behavior such as the one derived from a probabilistic inference model just amounts to a specific mapping from inputs onto the response . Generally, for probabilistic models the mapping can be written in two steps. (i) Computing the sufficient statistic which is then (ii) mapped onto the response, , such as for the Gaussian. We use to refer to any dispersion estimate and call the response mapping. For non-probabilistic estimators, it just allows for additional tuning of the dispersion estimate. The introduction of the response mapping permits the construction of computationally simple models that may accommodate subjective knowledge of latent variables like σ in the second step.

This is illustrated in Fig 3A for the theoretical response curves (red, green). For maximum likelihood estimation (MLE) the response (red) is nothing but a linear mapping of the sufficient statistic σ_ML(d) onto its output . The Bayesian benchmark model (green) also takes the sample size N = 4 and a uniform prior distribution over σ into account. Compared to MLE, its main effect is a bias of the responses towards intermediate values. The effect of a different prior on σ would merely manifest as a somewhat different mapping onto the response because σ_ML(d) and N are sufficient statistics for σ. In other words, the model will produce the same results even when input d changes as long as the sufficient statistics remain the same. They compactly sum up all the information that is to be known about the hidden variables of a probabilistic model from the sample d. Hence, distributions such as the posterior p(σ|d) or the prior p(σ) do not have to be explicitly represented in our model. Instead they are implicitly considered through the effects they exert on the response by allowing for additional freedom through a mapping. Apart from that, the mapping also depends on the target percentage that the model is required to capture. A larger target percentage leads to a larger dependence on σ_ML(d) and would e.g. manifest as a larger slope of the ML response (Fig 3A, red). The model may however account for the fact that participants suffer from probability distortion such that their internal target probability does not exactly match the one of a probabilistic agent (Eq 4).

The response mapping from the dispersion estimate to the response, , is chosen to be the same for all models and is intended to be flexible enough to account for these implicit effects. Empirically we found that a quadratic polynomial is only minimally better than a linear mapping (using the weighting-model, below). The improvements on the group level are significant (increased median cross-validation log likelihood (CVLL) across participants, Wilcoxon sign rank test, p = 0.0027) but small in absolute terms (median CVLL difference 3.66 dHart, 95% CI (0.34, 7.15) dHart, below). For this weak nonlinearity and to obtain a sparse model formulation, we consider a polynomial of first order to be a sufficiently good approximation to represent the response mapping.

(5)

The models that we consider differ only in how they compute the dispersion measure . They may introduce additional parameters which are detailed below. We start by describing approximative models that do not make use of distributions first. We will explicitly consider heuristic models. In general, heuristics are not linked to optimal responses in a principled way but nevertheless might yield satisfactory results. Every estimator that correlates with σ_ML contains some useful information about the dispersion and may thus be used. As heuristics are frequently associated with less effortful processing, we consider simple and visually salient quantities that may be readily assessed by the participants. As another approximate model, we test a weighting model that emphasizes certain stimulus features. We will then describe probabilistic models that derive responses from different distributional estimates and conclude with a predictive model intended to serve as an estimator for the upper bound on predictability given our data.

Gaussian kernel density estimation model.

If one imposes only minimal structural constraints, more freedom is given to the data to determine the inferred density. One may assume that even small samples represent the population well and that future observations will cluster around the already observed instances. One way to do so is to estimate p(x|d) over future events x based on a kernel method. It generalizes observed data points d_n by assigning probability density proportional to a kernel function k(x,d_n) to their vicinity and thus constitutes a data smoothing problem (Fig 2D). For the whole training set d, kernel density estimation centers a kernel on each observation and sums up their contributions to determine p(x|d) as: (9) It is a nonparametric method because it does not assume a certain parameterized family of probability distributions for p(x) apart from the kernel. The kernel function k typically decays with the distance between x and d_n. Here we assume that it has the shape of a normal distribution k(x|d_n,η) = N(x|d_n,η). The kernel width η = η(d) is in principle a free parameter but needs to be sensibly chosen with respect to the dispersion of the data. Manual testing revealed that η = a ⋅ (d₃ + d₄)/2 is a reasonably good approximation to the unknown η(d) function. Thus, potentially even better performance might be achievable than the one reported here. The model’s dispersion estimate, , regarding the 65% capture probability is determined by inserting the inferred distribution Eq 9 into Eq 3 and then solving Eq 4.

In the limit of vanishing kernel widths η → 0 (δ-distributions) the response for the target percentage of p_t = 0.65% converges to the third most excentric point. We refer to this approach as δ-KDE (Fig 2C). In this limiting case, one would merely capture the target fraction p_t of observed points on the screen, thus replacing an estimation of the target fraction p_t of the population with a corresponding estimation of p_t on the sample.

Tiling model. To capture a certain percentage of points of the sample, one must have some sort of quantile function that outputs the region containing the desired percentage. Explicit density models such as KDE entail a quantile function. A simple alternative is to construct some normalized histogram. We attempt to do so with the constraint that an observation point only exhibits a local effect on the constructed density (Fig 2B). Specifically, the contribution to the overall density of one data point only depends on its own position and on the position of its adjacent points.

More formally, this can be achieved by tiling the space between observations into rectangular, adjacent but non-overlapping basis distributions. We adhere to the additional constraint that the N ordered points correspond to the (0.5/N,1.5/N,…,(N − 0.5)/N) cumulative quantiles. Hence each basis distribution spanned between points has to be normalized by N. To assign the remaining probability 0.5/N below the lowest point d₁ we use a uniform distribution U(d₁ − d₂,d₁) whose support equals the distance to its only adjacent point d₂ (and likewise for the largest point). Representations of probability densities based on orthogonal basis functions are suggested as a solution to tractably represent complex densities [6].

Gaussian process regression. Gaussian Process Regression (GPR) [49] is used to estimate the upper bound on predictability of the participants’ behavior. It does not lend itself readily to an interpretation of how participants solve the problem on a given trial. It is however very flexible and successful in prediction by exploiting consistency between input d and output y across pairs of trials (i,j). We used GPR since it is a bias free estimator of the distribution p(y|d) which is assumed to be normally distributed with a constant intrinsic noise parameter σ_I. We chose a Gaussian kernel function (10) that defines a scalar measure of similarity and the entries of the covariance matrix of the GP as . Input pairs (d_i,d_j) that are considered similar in this sense should result in comparable responses (y_i,y_j) if the process p(y|d) is consistent. Prediction is more strongly influenced by those trials’ responses y for which (d_i,d_j) are similar. To make predictions for a new input d_ν, we evaluate the mean of the predictive distribution . Here k has the entries k(d_i,d_ν) with i indexing all trials in the training data. Likewise, C and y are constructed from all the training data used to derive predictions. For each trial d_t = (d_t1,…,d_tN), symmetry is exploited by sorting the points in ascending order of excentricity. To set the hyperparameters of the GP, (θ,σ₁,…,σ_N,σ_I), its generalization error is minimized. To do so, the mean of the test sets of Eq 13 of a 5-fold cross validation (CV) procedure is calculated. This procedure is part of training the GPR. We also attempted to predict behavior using a simple 1-hidden-layer feedforward neural network. Despite being a successful predictor, its performance was inferior to the GPR which is why we chose to only report the latter.

Baseline model. The baseline model is chosen to provide a simple lower bound estimate for predictability that is independent of the trial-by-trial variations of the stimulus. This model calculates the mean of the responses of all its input y_in (training set). It thus makes the same prediction on every trial t.

(11)

Inter-trial and feedback dependence. We investigated the influence of other quantities on behavior that participants might have (erroneously) utilized to guide their responses. To test for a dependence on the preceding trial, the estimator is chosen to be the previously stated response. (12) There is a significant effect with respect to baseline (exceedance probability, p_exc > 0.99), yet the effect on behavior is virtually negligible as the overall predictive performance is very low (median cross-validation log likelihood across participants −318 dHart, 95%-CI (−356,−300) dHart, with respect to the best model for each participant). The influence of the previously presented feedback about the capture percentage is similarly tested but its effect is found to be even weaker (−327 dHart, 95%-CI (−368,−312) dHart). Together with the evidence that participants did not adjust closer to the target capture percentage of the task (Fig 4B), we consider it unlikely that feedback affected behavior to a considerable extent.

Overview of model parameters. The models used have a different number of parameters depending on the dispersion estimate . The ones reported in the main text are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Overview of model parameters.

https://doi.org/10.1371/journal.pcbi.1006205.t001

The response distribution. The probability of obtaining the response y_t on trial t conditional on the data d_t and the model parameters is assumed to be a mixture distribution of two contributions. The first and dominant term is a normal distribution centered on the model prediction , modeling task-intrinsic noise around the estimates. Upon preliminary inspection of the data we found considerable heteroscedasticity with higher response variability for larger sample dispersions.

To take this feature of the response data into account, we assume that the standard deviation (SD), θ, of the distribution over response y_t, , is a function of the model output . The model output is denoted by to distinguish it from the response y of the participant which is formally represented by a draw from the response distribution to account for behavioral variability. Instead of assuming a parametric relationship and the need of further parameters to be fitted in the model, we make a parameter free estimate by assuming a discretized function, as follows. We divide the whole model output into Q equally filled quantiles q ∈ {1,…,Q} by assigning trial t to quantile q_t. For every quantile q, the SD is estimated separately by calculating (j = 1,…,J indexes trials belonging to quantile q). Hence, whenever there is heteroscedasticity, the true function is approximated by the estimated bin values. For homoscedasticity all θ_q are the same and collapsing bins would make no difference. The resolution of the function is higher when many quantile divisions are used provided the θ_q can still be estimated faithfully. We consider Q = 5 a suitable choice for our problem.

As our data might be contaminated by processes other than dispersion estimation, such as lapses, we take precaution against far outlying responses. We calculate a trimmed standard deviation, i.e. before calculating θ_q we remove values below or above two interquartile ranges from the lower or upper quartile respectively. However, this applies to θ_q estimation only. No points are removed from calculating the response likelihood (13) Additionally, to prevent isolated points from being assigned virtually zero probability, we generally add a small probability of ϵ = 1.34 × 10⁻⁴ to all. This corresponds to the probability of a point at four standard deviations from the standard normal distribution. For non-outlying points this alteration is considered negligible.

Estimating model evidence. The evidence that each participant’s data lends to each model is derived from predictive performance in terms of the cross-validation log likelihood (CVLL). For training, we maximized the logarithm of the response likelihood (Eq 13). To maximize the chances of finding the global maximum even for non-convex problems or shallow gradients, every training run first uses a genetic algorithm and then refines its estimate with gradient based search (MATLAB ga, fmincon). The CVLL for each participant and model is summarized by the mean of the logarithm of the response likelihood (Eq 13) on the test set across all cross validation (CV) folds.

As cross validation is a computationally expensive method, we use a random 5-fold split of data into training and test sets such that each training point is used four times for training and once for testing. However, to make splits more representative of the sample we use a stratified version of CV by ensuring that the mean target variable is approximately equal in all folds. This is done by assigning data points to one of the 8-quantiles of the distribution of the target variable. We constructed slices that contain one value from each quantile. Subsequently, we sampled strata to create the 5-fold CV splits. To improve the reliability of per participant estimates of the model evidence (CVLL) we repeated this procedure with different random splits and aggregated the output so that in total 10 CV splits are performed for each participant and model.

Differences in model evidence, Δ, are reported on a log-scale in decibans (also decihartleys, abbreviated dHart) that may be used to interpret the significance of the results of individual participants. According to standard conventions, we consider a value of 5 > Δ barely worth mentioning, 10 > Δ ≥ 5 substantial, 15 > Δ ≥ 10 strong, 20 > Δ ≥ 15 very strong and Δ ≥ 20 decisive.

Group level comparison. Instead of making the assumption that all participants can be described by the same model, we use a hierarchical Bayesian model selection method (BMS) [50] that assigns probabilities to the models themselves. This way, we assume that participants may be described by different models. That is a more suitable approach for group heterogeneity and outliers which are certainly present in the data. The algorithm operates on the CVLL for each participant (p = {1,…,P}) and each model (m = {1,…,M}) under consideration and estimates a Dirichlet distribution Dir(r|α₁,…,α_M) that acts as a prior for the multinomial model switches u_pm. The latter are represented individually for each subject by a draw from a multinomial distribution u_pm ~ Mult(1,r) whose parameters are r_m = α_m/(α₁+…+α_M). We use the CVLL and assume an uninformative Dirichlet prior α₀ = 1 on the model probabilities. Later, for model comparison, exceedance probabilities, , are calculated corresponding to the belief that a given model is more likely to have generated the data than any other model under consideration. High exceedance probabilities indicate large differences on the group level. We consider values of p_exc ≥ 0.95 significant (marked with *) and values of p_exc ≥ 0.99 very significant (marked with **).