Fig 1.
Type 1 and Type 2 performance.
(A) Type 1 performance. The proportion of responding ‘Right’ is plotted for each of the two stimuli , thus providing two points on the psychometric function. In this figure, sensory noise is
. (B) Type 2 performance. For each stimulus (left and right panels), confidence probability is plotted for each confidence level (‘low’ and ‘high’). Blue bars represent ‘Right’ perceptual responses (i.e., correct when
), and orange bars, running downwards, ‘Left’ responses (i.e., incorrect when
). In this figure, confidence noise is
and confidence boost is
. (C) Left panel: joint distribution of sensory and confidence evidence for one of the two stimuli,
. When the sensory criterion is
, perceptual responses are correct when sensory evidence is positive (blue), and incorrect when it is negative (orange). The confidence boundary separates high from low confidence judgments. It is shown as a dashed blue line for correct responses and negative dashed orange line for incorrect responses. To obtain the same fraction of high and low confidence judgments (see panels B), the confidence boundary was set to
. Right panel: marginal confidence probability distributions for correct responses (in blue) and incorrect responses (in orange). Higher levels of confidence are shown in more saturated colours. (D) Type 2 ROC. The Type 2 ROC curve plots the Type 2 hit rate against the Type 2 false alarm rate. The area under the Type 2 ROC (shown in shaded grey) is a measure of confidence sensitivity.
Fig 2.
Trade-off between confidence noise and confidence boost.
(A) Type 2 ROC. The blue dot shows the Type 2 hit and false alarm rates for the simulated human observer of Fig 1 (,
, and
). The smooth blue curve corresponds to the theoretical ROC curve obtained with the same values of confidence noise and confidence boost, but by varying the value of the confidence boundary. The dashed green curve corresponds to another observer with different confidence noise and confidence boost (
and
), chosen such that Type 2 hit and false alarm rates are identical to the simulated human observer. Note that these two curves intersect in a single point (blue dot), and slightly differ outside this point. In fact, there is an infinite number of pairs of confidence noise and confidence boost that will give the same set of Type 2 hit and false alarm rates, as illustrated in panel B. (B) Equivalent pairs of confidence noise and confidence boost. Each coloured line represents an observer with a particular set of Type 2 hit and false alarm rates. The blue curve shows the pairs of confidence noise and confidence boost that are equivalent to the simulated human observer shown by the blue dot that was also the example shown in panel A. The black curve shows the equivalent pairs of confidence noise and confidence boost for the ideal confidence observer (i.e., for whom
and
). The equivalent confidence noise is obtained when the confidence boost
is set to 1, and is shown as a green dot for the human observer (
), and as a black dot for the ideal observer (
). The red and yellow curves are equivalent pairs of confidence noise and confidence boost for other simulated observers that are presented later in this paper, namely
in red and
in yellow.
Fig 3.
Effect of multiple confidence levels on Type 2 ROC.
(A) Psychometric function from two stimulus strengths , with sensory noise
and no sensory bias (
). (B) For each stimulus strength (left and right panels), confidence probability is plotted for each confidence level (1 to 4). Blue bars represent ‘Right’ perceptual responses, and orange bars ‘Left’ responses (running downwards). In this figure, confidence noise is
and confidence boost is
. Confidence boundaries were chosen to produce the same fraction of confidence responses in each of the four confidence levels (
). (C) Left panel: joint distribution of sensory and confidence evidence for one of the stimulus strengths (
). Perceptual responses are correct to the right of the criterion (in blue), and incorrect to its left (in orange). When confidence is judged on a 4-point scale, there are three confidence boundaries as shown by the dashed lines. Right panel: confidence probabilities for correct and incorrect responses in blue and orange, respectively. Higher levels of confidence are shown in more saturated colours. (D) Type 2 ROC. When confidence is rated on 4 levels, the Type 2 ROC is constrained by 3 points.
Fig 4.
Confidence noise and confidence boost from multiple confidence levels.
(A) Lower panel: CNCB efficiency estimates as a function of number of confidence levels in the simulation. The colours represent three conditions for the confidence noise and confidence boost parameters , namely
in blue,
in red, and
in yellow. Upper panel: Discriminability of the blue and red simulations. In all lower panels, solid curves show median values over 100 repeated simulations, and shaded regions represent the interquartile range. The dashed lines correspond to the median CNCB efficiency obtained for the largest number of confidence levels (presumably the best estimates). Other parameters are listed in the Methods section. (B) Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (C) Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations.
Fig 5.
Effect of multiple difficulty levels on Type 2 ROC.
(A) Psychometric function from six stimulus strengths chosen to be equally spaced between and
. In this figure, sensory noise is
and sensory criterion is
. (B) For each stimulus strength, confidence probability is plotted for each confidence level (‘high’ and ‘low’). Blue bars represent ‘Right’ perceptual responses, and orange bars ‘Left’ responses (running downwards). In this figure, confidence noise is
and confidence boost is
. (C) The sensory evidence for each stimulus strength has an associated confidence evidence. The full bivariate distribution of sensory and confidence evidence is shown for one of the six stimulus strengths. Positive stimulus strengths are shown in solid lines, negative ones in dotted lines. (D) When the observer is unbiased (
), there are now three Type II ROC curves. The three dots correspond to the Type 2 hit and false alarms rates obtained when the confidence boundary was set to equate the overall fraction of high and low confidence ratings (
) over all six stimulus strengths.
Fig 6.
Confidence noise and confidence boost from multiple stimulus strengths.
(A) Lower panel: CNCB efficiency estimates as a function of number of stimulus strengths in the simulation. The colours represent three conditions for the confidence noise and confidence boost parameters , namely
in blue,
in red, and
in yellow. Upper panel: Discriminability of the blue and red simulations. In all lower panels, solid curves show median values over 100 repeated simulations, and shaded regions represent the interquartile range. The dashed lines correspond to the median CNCB efficiency obtained for the largest number of stimulus strengths (presumably the best estimates). Other parameters are listed in the Methods section. (B) Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (C) Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations.
Fig 7.
(A) Fraction of estimated models for each simulated model. Model 1 does not have a confidence boost parameter and Model 2 does. Blue bars correspond to fractions of estimated Model 1, and red bars Model 2. (B) Simulations of Model 1. Heatmaps are parameters’ estimates from Model 2, with saturation showing the frequency of estimated parameter over 2,500 simulations. Left panel, left axis: estimated confidence noise as a function of simulated confidence noise. The diagonal dashed line shows perfect performance. The axis on the right and the red curve show the fraction of times Model 1 was favoured over Model 2 as a function of confidence noise. Right panel: same analysis as that in the left panel, but for confidence boost instead of confidence noise. (C) Simulations of Model 2. Left and right panels show the effects of simulated confidence noise and confidence boost, respectively, on the estimated confidence noise and confidence boost, and on the fraction of times Model 2 was favoured over Model 1.
Fig 8.
Parameters recovery for varying confidence noise, confidence boost, and number of confidence judgments.
(A-C) Effect of different values of simulated confidence noise, for two different values of confidence boost ( in blue, and
in red). (D-F) Effect of different values of simulated confidence boost, for two different values of confidence noise (
in blue, and
in yellow). (G-I) Effect of the number of confidence judgments in the simulation, for three pairs of confidence noise and confidence boost parameters
, namely
in blue,
in red, and
in yellow. Upper panels show discriminability of the blue and red simulations (G), blue and yellow (H), and blue and red (I). Panels show estimated confidence efficiency (A, D, G), estimated confidence noise (B, E, H), and estimated confidence boost (C, F, I). In lower panel (G), the dashed lines correspond to the median CNCB efficiency obtained for the largest number of confidence judgments (presumably the best estimates). In all panels, solid curves show median values over 100 repeated simulations, and shaded regions represent the interquartile range. Unless specified otherwise, simulations consisted in 10,000 confidence judgments. Other parameters are listed in the Methods section.
Fig 9.
Effect of sensory sensitivity.
(A) M-ratio as a function of different values of sensory noise. The colours represent three conditions for the confidence noise and confidence boost parameters , namely
in blue,
in red, and
in yellow. Continuous lines and shaded areas show the medians and interquartile ranges, respectively, over 100 repeated simulations. The dashed lines correspond to the median M-ratio obtained for the sensory noise that has been used in the other simulations (
). (B) CNCB efficiency estimates for the same conditions as in (A). (C) Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (D) Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations. Other parameters are listed in the Methods section.
Fig 10.
(A) Probability densities for sensory strengths . Perceptual decisions are made relative to a biased sensory criterion (here,
). (B) M-ratio as a function of different values of sensory criteria. The colours represent three conditions for the confidence noise and confidence boost parameters
, namely
in blue,
in red, and
in yellow. Continuous lines and shaded areas show the medians and interquartile ranges, respectively, over 100 repeated simulations. The dashed lines correspond to the median M-ratio obtained for the optimal sensory criterion (
), presumably providing the best estimates. The M-ratio is strongly biased by non-optimal sensory criteria. (C) CNCB efficiency estimates for the same conditions as in (B). The CNCB efficiency seems stable across sensory criteria. (D) Lower panel: Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (E) Lower panel: Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations. Other parameters are listed in the Methods section.
Fig 11.
(A) M-ratio as a function of different values of confidence bias. The colours represent three conditions for the confidence noise and confidence boost parameters , namely
in blue,
in red, and
in yellow. Continuous lines and shaded areas show the medians and interquartile ranges, respectively, over 100 repeated simulations. The dashed lines correspond to the median M-ratio obtained for a uniform distribution of confidence judgments across all levels, presumably leading to the least bias in M-ratio estimate. (B) CNCB efficiency estimates for the same conditions as in (A). (C) Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (D) Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations. Other parameters are listed in the Methods section.
Fig 12.
Relationship between confidence efficiency and M-ratio.
(A) The heatmap shows CNCB efficiencies and M-ratios from 2,500 simulated experiments where there were only two stimulus strengths and four confidence levels. Other parameters were sampled from the ranges described in the Materials and Methods section. The diagonal shows a squared relationship between M-ratio and confidence efficiency in log-log coordinates (see Eq. 7).
Fig 13.
Continuous confidence ratings.
(A) Non-biased relationship between objective and subjective confidence probabilities. (B) Confidence ratings for the two categories of stimuli (). The blue distribution corresponds to perceptual responses ‘Right’ (i.e., correct when
, and incorrect when
), and the orange distributions to responses ‘Left’ (running downwards). With the non-biased relationship in (A), confidence ratings are strongly skewed towards high confidence levels. For these simulations,
and
. (C) Log-odds transformation between objective and subjective confidence probabilities with
and
. (D) With the biased relationship in (C), and the same confidence noise and confidence boost parameters as in (B), confidence ratings are more concentrated in the middle of the range of confidence levels. Other parameters are listed in the Methods section.
Fig 14.
Recovery and influence of probability transformation.
(A) Lower panel: Confidence efficiency for different values of the parameter. Upper panel: Discriminability of the blue and red simulations. (B) Estimated confidence noise. Upper panel: Discriminability of the blue and yellow simulations. (C) Estimated confidence boost. Upper panel: Discriminability of the blue and red simulations. (D) Recovery of the
parameter of the log-odds transformation. (E) Recovery of the
parameter of the log-odds transformation for different values of the
parameter. When
,the parameter
is undetermined. The colours represent three conditions for the confidence noise and confidence boost parameters
, namely
in blue,
in red, and
in yellow. In all panels, solid curves show median values over 100 repeated simulations, and shaded regions represent the interquartile range. Other parameters are listed in the Methods section.