Pharmacological Fingerprints of Contextual Uncertainty

Louise Marshall; Christoph Mathys; Diane Ruge; Archy O. de Berker; Peter Dayan; Klaas E. Stephan; Sven Bestmann

doi:10.1371/journal.pbio.1002575

Abstract

Successful interaction with the environment requires flexible updating of our beliefs about the world. By estimating the likelihood of future events, it is possible to prepare appropriate actions in advance and execute fast, accurate motor responses. According to theoretical proposals, agents track the variability arising from changing environments by computing various forms of uncertainty. Several neuromodulators have been linked to uncertainty signalling, but comprehensive empirical characterisation of their relative contributions to perceptual belief updating, and to the selection of motor responses, is lacking. Here we assess the roles of noradrenaline, acetylcholine, and dopamine within a single, unified computational framework of uncertainty. Using pharmacological interventions in a sample of 128 healthy human volunteers and a hierarchical Bayesian learning model, we characterise the influences of noradrenergic, cholinergic, and dopaminergic receptor antagonism on individual computations of uncertainty during a probabilistic serial reaction time task. We propose that noradrenaline influences learning of uncertain events arising from unexpected changes in the environment. In contrast, acetylcholine balances attribution of uncertainty to chance fluctuations within an environmental context, defined by a stable set of probabilistic associations, or to gross environmental violations following a contextual switch. Dopamine supports the use of uncertainty representations to engender fast, adaptive responses.

Author Summary

Interacting with dynamic and ever-changing environments requires frequent updating of our beliefs about the world. By learning the relationships that link events in the current environmental context, it is possible to prepare and execute fast, accurate responses to those events that are predictable. However, the world’s complex dynamics give rise to uncertainty about the relationships that exist between events and uncertainty about how these relationships might change over time. Several neuromodulators have been proposed to signal these different forms of uncertainty, but their relative contributions to updating beliefs and modulating responses have remained elusive. Here we combine a probabilistic reaction time task, pharmacological interventions, and a hierarchical Bayesian learning model to identify the roles of noradrenaline, acetylcholine, and dopamine in individual computations of uncertainty. We propose that noradrenaline modulates learning about the instability of the relationships that link environmental events. Acetylcholine balances the attribution of uncertainty to unexpected events occurring within an environmental context or to gross violations of our expectations following a context change. In contrast, dopamine sensitises our actions to our beliefs about uncertainty.

Citation: Marshall L, Mathys C, Ruge D, de Berker AO, Dayan P, Stephan KE, et al. (2016) Pharmacological Fingerprints of Contextual Uncertainty. PLoS Biol 14(11): e1002575. https://doi.org/10.1371/journal.pbio.1002575

Academic Editor: Matthew F. S. Rushworth, Oxford University, UNITED KINGDOM

Received: April 11, 2016; Accepted: October 5, 2016; Published: November 15, 2016

Copyright: © 2016 Marshall et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work was supported by the Medical Research Council (LM and AOdB), the European Research Council, grant number ActSelectContext 260424 (SB), the Gatsby Charitable Foundation (PD), the Clinical Research Priority Program “Molecular Imaging” (KES), the René and Susanne Braginsky Foundation (KES), the Tourette Syndrome Association (DR), a Dorothy Feiss Scientific Research Grant (DR), and core funding from the Wellcome Trust Centre for Neuroimaging (CM) and the University of Zurich (KES). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: ACh, acetylcholine; BIS-11, Barratt Impulsiveness Scale; BOLD, blood-oxygen-level-dependent; BP, blood pressure; BPA, Bayesian parameter average; CFQ, Cognitive Failures Questionnaire; DA, dopamine; DOSPERT, Doman-Specific Risk Taking; HGF, Hierarchical Gaussian Filter; HR, heart rate; ITI, intertrial interval; LC, locus coeruleus; NA, noradrenaline; ns, nonsignificant; PE, prediction error; RM-ANOVA, repeated-measures analysis of variance; RT, reaction time; SD, standard deviation; SED, standard error of the difference; SEM, standard error of the mean; TM, transition matrix; VAS, visual analogue scale; YD, Yu and Dayan

Introduction

Adaptive performance in dynamic environments depends on our ability to represent and manipulate internal estimates of the world’s statistical structure [1–4]. By tracking the environment’s underlying regularities, an agent can learn the causes of its sensory input and thus the likelihood that a particular event will occur. In turn, this permits anticipatory action preparation and the rapid execution of responses [5].

However, the environment’s richly complicated sources of noise and latent structure present us with various forms of uncertainty. For instance, a London commuter predicting her journey time to work faces three distinct forms. First, there is irreducible uncertainty, which captures the randomness inherent in any complex environment and is undiminished by learning; an unplanned station closure could cause journey delays and thus influence the accuracy of the commuter’s estimated arrival time on any given day. Second, after moving to a new part of town, the duration of the commuter’s chosen route to work may be unclear, producing uncertainty about how likely she is to arrive at work on time. Over repeated journeys, this estimation uncertainty falls as the commuter learns about the probabilistic relationships within the current environmental context. For example, she learns to predict the frequent delays on this new route due to congestion during the morning rush hour, although these delays may vary with local dips and surges in the number of passengers using the service. Volatility uncertainty arises from our beliefs about the stability of the environment and thus how quickly probabilistic relationships are changing between contexts. A major sporting event, such as the London Olympics, may bring a large influx of additional passengers for an unknown period of time and with unexpected effects on transport performance, making it harder to predict future journey times until these changes have been taken into account.

The Brain Computes Different Forms of Uncertainty

An assortment of theoretical, behavioural, and neurobiological evidence suggests that the brain computes uncertainty estimates relating to the environment’s sensory events, contextual associations, and their changes over time [6–17]. Uncertainty about the validity of one’s own perceptual beliefs about the world should have the effect of suppressing top-down prior expectations relative to new bottom-up sensory evidence, promoting learning about the current environmental context [18]. With their broad distribution and extensive connectivity, neuromodulatory networks are well placed to facilitate the widespread changes in gain required to support such a function [19,20]. In particular, acetylcholine (ACh) and noradrenaline (NA) are known to enhance bottom-up, feedforward thalamocortical transmission of sensory information relative to top-down, intracortical, and feedback processing [4,21–30].

Proposed roles for NA and ACh in computations of uncertainty.

Within a stable environmental context, uncertainty arises from ignorance about, and the unreliability of, probabilistic cues predicting upcoming sensory events. Learning the environment’s underlying statistical regularities means that responses to explicitly and predictably cued events are typically faster than to those believed improbable [31–34]. This effect is modulated by pharmacological [35,36], surgical [37,38], and neurodegenerative [39] manipulations of ACh. For example, human cholinergic basal forebrain activity has been shown to reflect an individual’s uncertainty about probabilistic cue-outcome relationships [15]. More specifically, ACh appears to increase the rate at which humans learn probabilistic relationships under estimation uncertainty, supporting the idea that ACh enhances learning accorded to stimuli with uncertain predictive consequences [40] by suppressing the use of outdated top-down cues and boosting bottom-up sensory processing [4].

While NA plays no consistent role in probabilistic cueing [41,42], it is thought to offer an interrupt signal when volatility uncertainty arises between contexts [41–47]. Learning to make accurate predictions from the strongly unexpected observations that follow a contextual switch necessitates heightened sensory vigilance and a disregard for outdated top-down expectations. NA, with its broad neural network capable of triggering multiple, simultaneous changes across the brain [48], is well placed to rapidly coordinate this process. Indeed, neurons in the locus coeruleus (LC), the primary source of cortical NA, show strong responses to unexpected environmental changes [49,50]. Pharmacologically up-regulating NA accelerates the detection of unexpected switches in the predictive properties of sensory stimuli [51], while noradrenergic deafferentation of rat medial frontal cortex impairs behavioural adaptation to contextual switches [52]. Moreover, blood-oxygen-level-dependent (BOLD) activity in human LC has been shown to dynamically track volatility uncertainty [16], and pupil dilation—which is influenced by (nor)adrenergic afferents [53]—correlates with unexpected changes in probabilistic context [54,55].

Motor Responses Are Sensitive to Uncertainty

Thus, uncertainty representations existing within and between environmental contexts are crucial for optimal predictions about the probability of future events. While good predictions facilitate anticipatory preparation of appropriate behavioural responses [5], they are not sufficient for adaptive performance in dynamic environments. An additional mechanism is required to modify action selection based on one’s own beliefs about the latent changes in the environment and/or the occurrence of unexpected events. Indeed, when an unexpected event occurs, humans are capable of engaging resources to inhibit a prepared response and replace it with an alternative [56,57], albeit at the expense of a prolonged reaction time (RT) [58,59].

Proposed role for dopamine in response modulation.

There is considerable evidence linking the neuromodulator dopamine (DA) to flexible behaviour [60–64]. Dopaminergic deficits due to Parkinson’s disease are associated with specific flexibility impairments in both motor [59,65] and cognitive domains [60,66], with performance restored by dopaminergic medication [59,67]. In healthy individuals, pharmacological DA depletions impair adaptive reactions to unexpected events occurring within a broadly predictable context [58]. However, it remains unclear whether DA supports accurate response selection by facilitating perceptual belief updating [15] or by modulating the sensitivity of response selection to perceptual beliefs.

A Unified Framework of Uncertainty

In sum, while physiological, pharmacological, behavioural, and theoretical work has suggested separable neuromodulatory involvement in different uncertainty computations, attempts to characterise the relative contributions of NA, ACh, and DA within a single computational scheme are lacking (but see noticeable exceptions from Varazzani et al. [68] contrasting the roles of NA and DA in motivation and Brown et al. [69] assessing NA and ACh involvement in orienting responses to environmental novelty). Here we employ a Hierarchical Gaussian Ffilter (HGF) model [9,10] to track human learning and response modulation in dynamic, probabilistic environments under pharmacological NA, ACh, and DA interventions.

The HGF has been successfully applied in several recent studies of learning under volatility [15,17,70–74]. We developed a novel instantiation of the HGF with two components: a three-level perceptual model of an agent’s mapping from environmental causes to sensory inputs and a response model that maps those inferred environmental causes to observed RT responses [75]. Thus, we sought to disentangle the effects of pharmacological interventions of the NA, ACh, and DA neuromodulatory systems on participant-specific perceptual belief updating from those on the sensitivity of motor responses to perceptual estimates.

Results

According to a double-blind, between-subject design, 128 healthy participants undertook a serial probabilistic RT task (Fig 1) after having received a noradrenergic α1-receptor antagonist (prazosin; NA- group), a cholinergic M1-receptor antagonist (biperiden; ACh- group) or a dopaminergic D1/D2 receptor antagonist (haloperidol; DA- group), or a placebo. Data from 124 participants are reported. Four participants were excluded from analyses, three due to high missed response rates (≥11%) and one because the behavioural model parameter estimation (using the Broyden-Fletcher-Goldfarb-Shanno algorithm) did not converge. The four drug groups were matched for gender (Kruskal-Wallis test: H₃ = 0.53, p = 0.912), age (one-way ANOVA: F_3,120 = 0.46, p = 0.714), body weight (F_3,120 = 2.24, p = 0.087), education level (H₃ = 1.31, p = 0.727), and all baseline psychometric measures taken (Table 1).

Download:

Fig 1. Task design.

(A) Trial sequence. A trial began with the presentation of a central white fixation cross against a black background. After an initial delay of 1,500 ms at the start of each block, one of four visual stimuli was presented for 200 ms. Participants were required to make a speeded button-press response before the end of a 1,200 ms intertrial interval (ITI). (B) Stimulus transitions were generated by one of eight different transition matrices (TMs), which changed every 50 trials without explicit indication to the participant. These TMs comprised two different first-order sequences, two alternating sequences, and four zeroth-order sequences, each of which occurred three times in a pseudorandom order across 1,200 trials. The overall probability of each stimulus was equal across the 1,200 trials. For full details, see S1 Fig. (C) Example trial sequences generated from the three example TMs in 1B. (D) By tracking the transition probabilities, participants could learn to predict high probability events and prepare to make the correct button press accordingly. Faster responses were observed for predictable stimuli compared to unexpected stimuli. Here we depict Placebo group log(RTs) (mean ± standard error of the mean [SEM]) for each of the 16 possible combinations between consecutive stimuli for the first-order sequence shown in B. Grey boxes indicate stimulus combinations with a high transition probability. (E) Indeed, across all types of TM, responses were faster for stimuli with higher transition probabilities (mean ± SEM). http://dx.doi.org/10.6084/m9.figshare.3796314.v1.

https://doi.org/10.1371/journal.pbio.1002575.g001

Download:

Table 1. Participant details for each experimental group.

https://doi.org/10.1371/journal.pbio.1002575.t001

On each trial, participants were required to respond to the presentation of one of four visual stimuli by making a speeded button press before the end of a 1,200 ms intertrial interval (ITI) (Fig 1). Participants were trained on the stimulus-response mappings, which remained consistent within an experimental session but were counterbalanced across participants. The experimental sequence of 1,200 trials was generated by a hidden probabilistic rule that switched, without explicit indication to the participant, every 50 trials. At any given time, stimulus transitions were generated by one of eight different transition matrices (TMs). Trials were drawn from each TM three times. The order of TMs was pseudorandom, with no consecutive repeats. The overall probability of each stimulus was equal across the 1,200 trials.

This created transient contexts that participants could infer from stimulus observations, allowing them to reduce their uncertainty about events before they occurred [76]. Nonetheless, the probabilistic nature of these contexts also produced unexpected stimulus outcomes, i.e., a sensory prediction error (PE). For fast and accurate responses, participants had to track three forms of uncertainty: irreducible uncertainty arising from the inherent randomness of the probabilistic transitions between consecutive stimuli, estimation uncertainty arising from their imperfect knowledge of the probabilistic relationships governing stimulus transition contingencies within contexts, and volatility uncertainty maintained by the unsignalled contextual instability.

In total, participants responded correctly on 90.3 ± 0.79% (mean ± standard error of the mean [SEM]), 88.4 ± 1.23%, 87.7 ± 1.27%, and 89.2 ± 0.91% of the trials in the Placebo, NA-, ACh-, and DA- groups, respectively. The percentages of correct responses did not differ between groups (F_3,123 = 1.12, p = 0.345). Since a significant time x drug interaction on self-reported alertness was identified (see the Physiological and Subjective Control Measures results in S1 Text), the participant-specific difference in alertness between baseline and the time corresponding to peak drug concentration, Δalertness, was used as a covariate in our analyses to control for any interparticipant variability in subjective drug effect. Since psychopharmacological drugs can have dose-dependent effects [77–79], we also included body weight as an additional covariate in our analyses.

Model-Agnostic Analyses

We first conducted a series of conventional, model-agnostic analyses of behaviour, to assess whether participants learned about the underlying stimulus transition contingencies and whether learning was influenced by our pharmacological interventions (Fig 2). First, we conducted a repeated-measures analysis of variance (RM-ANOVA) of the log(RTs) for correct responses on trials binned according to the five true conditional probabilities that existed in each of the TMs, grouped into High (0.85 and 0.70), Mid (0.25 and 0.20), and Low (0.05) transition probabilities, with drug as a between-subject factor. This revealed a significant decrease in log(RTs) with increasing transition probability (main effect of probability: F_1.27,150.10 = 28.32, p < 0.001, effect size η_p² = 0.19), which was modulated by drug type (probability x drug interaction: F_3.82,150.10 = 12.33, p < 0.001, η_p² = 0.24).

Download:

Fig 2. Model-agnostic changes in log(RT) indicate that participants learned to predict the stimulus transitions.

(A) In all four groups, log(RT) increased as a stimulus’ true transition probability decreased. (B) A median split on each 50-trial contextual block was used to compare mean log(RTs) on Early (1–25) and Late (25–50) trials at each probability level. Over the course of a context, participants became faster at responding to High and Mid probability stimuli and slower at responding to Low probability stimuli. (C) Across drug groups, participants showed evidence of post-error slowing on correct trials that followed an erroneous response compared to those following correct responses. (D) They also showed evidence of slowing on correct trials that followed an infrequent stimulus transition. Results are mean ± SEM, corrected for the covariates Δalertness and body weight. Results shown in A, B, and D were modulated by drug group. http://dx.doi.org/10.6084/m9.figshare.3796353.v1.

https://doi.org/10.1371/journal.pbio.1002575.g002

Moreover, across the course of a contextual block (Fig 2B), participants became faster at responding to High and Mid probability stimuli and slower at responding to the Low probability stimuli (significant main effects of probability: F_1.28,151.41 = 27.80, p < 0.001, η_p² = 0.19, and time: F_1,118 = 12.01, p = 0.001, η_p² = 0.09; probability x time interaction: F_2,236 = 6.55, p = 0.002, η_p² = 0.05). The effect was modulated by drug type (probability x time x drug interaction: F_6,236 = 3.16, p = 0.005, η_p² = 0.07) but again not systematically related to differences in Δalertness or body weight between drug groups (all p > 0.05). Post hoc (Benjamini-Hochberg-corrected) pairwise comparisons indicated that the impact of drug was driven by the ACh- group, which showed significant log(RT) slowing compared to Placebo (t₅₇ = 2.98, p = 0.003, effect size Cohen’s d = 0.79). Together, these results indicate that participants learned about the true stimulus transition contingencies and that this learning was modulated by our pharmacological manipulations.

As is common in such behavioural tasks [80–83], participants showed evidence of post-error slowing on correct trials following those on which they made an error (F_1,118 = 10.92, p = 0.001, η_p² = 0.09; Fig 2C). This effect was not modulated by drug group (trial-type x drug interaction: p = 0.933) or by Δalertness or body weight (both p > 0.29). Participants also demonstrated significant log(RT) slowing on correct post-infrequent trials (true transition probability = 0.05) compared to all other correct trials (F_1,118 = 36.00, p < 0.001, η_p² = 0.23; Fig 2D), which was modulated by drug group (F_3,118 = 4.58, p = 0.005, η_p² = 0.10), but not by Δalertness or body weight (both p > 0.15). This effect was driven by the ACh- group, with pairwise comparisons revealing significant slowing compared to Placebo (t₅₇ = 3.38, p = 0.001, d = 0.90). Error rates significantly decreased with increasing transition probability (main effect of probability: F_{1.57, 182.11} = 5.04, p = 0.013, η_p² = 0.04). The effect was again modulated by drug type (probability x drug interaction: F_{4.71, 182.11} = 4.72, p = 0.001, η_p² = 0.11), but not by Δalertness or body weight (both p > 0.78). There was no between-subject effect of drug group (p = 0.768). For additional model-agnostic analyses, please refer to S1 Text. Raw RT data can be found at https://dx.doi.org/10.6084/m9.figshare.3793410.v2.

Model-Based Analyses

The HGF model (Fig 3) allows us to map an individual’s beliefs about stimulus transitions, transition contingencies, and volatility—and the respective irreducible, estimation, and volatility uncertainty about these beliefs—onto his/her observed RT responses. The HGF is hierarchical not only in that learning occurs simultaneously at multiple levels, but also in that belief updating at one level is constrained by beliefs at the level above. This provides a generic framework for implementing dynamic learning rates, which underlie learning in volatile environments [1,84]. We predicted that antagonising NA and ACh would impact on participants’ computation of volatility uncertainty and estimation uncertainty respectively, while DA antagonism would impede motor adaptation to uncertain outcomes without perturbing the course of learning.

Download:

Fig 3. Hierarchical Gaussian Filter (HGF).

(A) The perceptual model tracks an individual’s learning of the task’s structure across three levels. State x₁ represents trialwise stimulus transitions from one stimulus to the next, x₂ the probability of the transitions (i.e., transition contingencies), and x₃ the phasic volatility, where t is the current trial number. Participants hold and update beliefs about the true quantities at each level, with a mean μ and a variance σ. ϑ and ω are participant-specific parameters that couple the levels and determine the respective speed of belief updating about phasic volatility and transition contingencies. The response model describes the mapping from a participant’s trialwise beliefs onto their observed log(RT) responses. (B) Example of the trialwise dynamics at level 3 from Placebo Participant 2. μ₃ reflects the participant’s belief about the true phasic volatility (x₃). Vertical dashed lines indicate true context switches. μ₃ tends to increase following a context change and then decreases over the course of a context as the participant learns the new contextual rule and thus perceives the environment to be increasingly stable. √ϑ is a variance determining the step-size of μ₃ and therefore how quickly the participant updates their phasic volatility estimates. (C) As in B, but for precision-weighted contingency prediction error [PE] (ε₃) at level 2. This estimate results from weighting the contingency PE (δ₂) by a precision ratio that captures uncertainty about input from the level below relative to the level above. The higher the precision at level 2, the more meaningful a deviation from the predicted stimulus transition contingency. This in turn increases the impact on phasic volatility belief updating at level 3. For simplicity, we only depict the ε₃ trajectory for true transition changes. (D) As in B and C, but for sensory PE (δ₁) at level 1. This estimate arises from irreducible uncertainty about stimulus transitions. Trialwise values are equivalent to , where is equal to the participant’s trialwise prediction of the occurring transition. Again, for simplicity we only show δ₁ values for true transitions here. (E) Mean β values for the Placebo group indicate that increases in sensory PE (β₁), precision-weighted contingency PE (β₂), and phasic volatility estimates (β₃) slowed participants’ trialwise log(RTs). There was also evidence of post-error slowing (β₄). Results are mean ± SEM. * p < 0.05, ** p < 0.01, *** p < 0.001. http://dx.doi.org/10.6084/m9.figshare.3796356.v1.

https://doi.org/10.1371/journal.pbio.1002575.g003

Perceptual model.

The HGF’s perceptual model tracks a participant’s learning of the task’s structure: the trialwise stimulus transitions at level 1, the probability of the transitions (i.e., transition contingencies) at level 2, and the volatility of transition contingencies at level 3 (Fig 3A). Trialwise trajectories of each participant’s perceptual estimates at each level evolve according to the predictions made and outcomes experienced by that individual (Fig 3B–3D). At levels 2 and 3, these estimates are modelled by Gaussian distributions with a mean (μ) and variance (σ), the latter reflecting the uncertainty of the estimate. Precision (π) of the estimate is equal to inverse variance (1/σ). Irreducible uncertainty at level 1 gives rise to sensory PE, δ₁. Estimation uncertainty at level 2 gives rise to contingency PE, δ₂. We can weight PEs according to their precision (inverse uncertainty). At level 1, this gives us precision-weighted sensory PE, ε₂, and at level 2 precision-weighted contingency PE, ε₃. Volatility uncertainty arises from phasic volatility beliefs, μ₃, at level 3. See Materials and Methods and S1 Text for full details.

Importantly, the HGF does not assume fixed learning across the population but rather contains participant-specific parameters that couple the hierarchical levels and allow for individual expression of approximate Bayes-optimal learning. ϑ determines the speed of learning about volatility, i.e., the rate at which estimates of phasic volatility (μ₃) are updated. As such, ϑ encapsulates metavolatility, i.e., the rate at which volatility changes, with higher values implying a belief in a more unstable world and leading to a more variable learning rate (as expressed in phasic volatility belief updating). By contrast, ω is a constant component of the volatility and captures how rapidly individuals generally update their beliefs about transition contingencies at level 2. Changes in ω therefore lead to a tonic alteration of the learning rate. Comparing ϑ and ω estimates for each of the drug groups to the Placebo group allowed us to assess the effects of NA, ACh, and DA antagonism on perceptual belief updating.

We note that while model-agnostic analyses can provide an indication of learning and possible drug effects, the model we endorse here permits us to probe (our best guess at) subjective expectations about the transitions that are driven by data-limited observations. Moreover, it separates the set of relatively complex and interacting factors that influence RTs in a computationally limpid way and provides us with insight into the individual effects of our pharmacological manipulations.

Overall, the model tracked the true stimulus transitions well (Fig 4). We note that the model is uninformed about the true stimulus transition probabilities but rather bases its estimates on the observed stimulus transitions only. The punctate change points contained in the true generative process are detected implicitly by the HGF as an increase in learning rate (α₁), which reflects the influence of increased uncertainty and formally corresponds to a reduced contribution of belief precision (denominator in Eq 3) to the weighting of PE.

Download:

Fig 4. True and estimated stimulus transition contingencies for two example participants.

(A) Transitions between pairs of stimuli, from trial t-1 to trial t, were defined by TMs. Every 50 trials, the TM switched to a different matrix. (B) Each panel corresponds to 1 of 16 possible transitions between stimuli across 1,200 trials. The black lines indicate the true transition contingencies. The blue lines reflect the participant’s inferred estimates (i.e., the posterior expectation of these contingencies, ), before seeing the stimulus outcome on each trial. The model tracked the true underlying contingencies and detected change points. In a representative participant from the Placebo group, the model tracked the true transition contingencies closely. An example participant from the ACh- group showed a greater discrepancy in the tracking of the true transition contingencies. This is reflected in the two participants’ ω estimates: Placebo Participant 2 showed a higher transition contingency learning rate (ω = −3.27) than ACh- Participant 16 (ω = −5.84). http://dx.doi.org/10.6084/m9.figshare.3796362.v2.

https://doi.org/10.1371/journal.pbio.1002575.g004

Moreover, when we categorise trials according to participants’ trialwise estimates of transition contingencies, as provided by model parameter (five bins: 0.8–1, 0.6–0.8, 0.4–0.6, 0.2–0.4, and 0–0.2), we observe the same decrease in log(RT) with increasing transition probability established in our model-agnostic results (cf. Fig 5 with Fig 2A; significant effect of : F_1.67,185.64 = 17.56, p < 0.001, η_p² = 0.14). As in the model-agnostic results, this was modulated by drug group (significant x drug interaction: F_5.02,185.64 = 9.51, p < 0.001, η_p² = 0.20), but not by Δalertness or body weight (both p > 0.11).

Download:

Fig 5. Model-based changes in log(RT) mirror our model-agnostic finding that participants learned to predict the stimulus transitions.

In all four groups, faster responses were observed as participants’ estimates of the true transition contingencies increased, mirroring our model-agnostic result (see Fig 2A). Results are mean ± SEM, corrected for Δalertness and body weight. http://dx.doi.org/10.6084/m9.figshare.3796401.v2.

https://doi.org/10.1371/journal.pbio.1002575.g005

Response model.

The response model describes the mapping from a participant’s perceptual trialwise beliefs, as provided by the perceptual model, onto their observed log(RT) responses (Fig 3A). The response model we report in the following (Eq 1) describes log(RT) as a linear function of a constant component of log(RT), sensory PE (δ₁), precision-weighted contingency PE (ε₃), phasic volatility (μ₃), post-error slowing, and Gaussian noise (ζ). (See Materials and Methods and S1 Text for random effects Bayesian model comparison of alternative response models.) (1)

All regression coefficients for the Placebo group were significantly greater than 0 (Fig 3E), meaning that sensory PE (β₁(δ₁): t₂₉ = 2.63, p = 0.010, d = 1.38), precision-weighted contingency PE (β₂(ε₃): t₂₉ = 3.74, p < 0.001, d = 1.97), and phasic volatility estimates (β₃(μ₃): t₂₉ = 2.05, p = 0.043, d = 1.08) all had slowing influences on log(RT) and that there was evidence of post-error slowing (β₄(PostError): t₂₉ = 3.09, p = 0.003, d = 1.62). Each of the drug groups showed equivalent post-error slowing to the Placebo group (all p > 0.48; Fig 6H), mirroring our model-agnostic result. The lack of a difference in the noise parameter ζ between the Placebo group and any of the drug groups (all p > 0.34; Fig 6C) indicates that the model’s ability to predict log(RT) was unaltered under our drug manipulations. Comparing the regression coefficients for each drug group to the Placebo group allowed us to assess whether NA, ACh, and DA antagonism altered the mapping from perceptual beliefs onto motor responses.

Download:

Fig 6. Model parameter results.

(A–H) Compared to the Placebo group, NA and ACh antagonism modulated participants’ perceptual belief updating. NA- increased the rate at which participants updated their volatility estimates (increased ϑ). ACh- decreased the rate at which participants learned about stimulus contingencies (decreased ω) and increased participants’ ϑ estimates. DA- antagonism decreased the sensitivity of participants’ trialwise responses to their phasic volatility estimates (β₃). DA- and ACh- antagonism also caused some general response slowing (β₀). The three drug groups and the Placebo group showed equivalent post-error slowing (β₄) and Gaussian noise (ζ). Results are (mean Drug) − (mean Placebo), ± the standard error of the difference (SED) between the means of the two samples and corrected for the covariates Δalertness and body weight. For uncorrected comparisons: * p < 0.05, ** p < 0.01, *** p < 0.001. For corrections for three comparisons, ⁺’s are used for Benjamini-Hochberg-corrected significance thresholds at ⁺ p < 0.05, ⁺⁺ p < 0.01, ⁺⁺⁺ p < 0.001. See S2 Table for Placebo group means. (I) First-level learning rate (α₁) trajectories for truly occurring trials from the Placebo group. Increases in α₁ are observed following a true change in context. This α₁ increase is amplified for a more obvious switch from one easy-to-detect zeroth-order context to a different zeroth-order context. In contrast, a switch to an alternating context, which is trickier to detect, is accompanied by a modest, more gradual increase in α₁. Results are mean ± SEM. http://dx.doi.org/10.6084/m9.figshare.3796407.v1.

https://doi.org/10.1371/journal.pbio.1002575.g006

The Influence of NA and ACh in Perceptual Uncertainty Computations

NA antagonism increased phasic volatility learning rate.

The noradrenergic (α1-receptor) antagonist prazosin increased the rate at which individuals updated their volatility estimates, as reflected by an increase in ϑ (linear model: t₅₉ = 2.11, p = 0.037, effect size Cohen’s d = 0.56; Fig 6A). A higher ϑ leads to greater fluctuations in participants’ phasic volatility estimates, μ₃, resulting in a more variable phasic learning rate. By contrast, there was no effect on ω (p = 0.574; Fig 6B), indicating that the tonic learning rate about the probabilistic contexts remained unchanged. We note that when we apply a Benjamini-Hochberg correction for three comparisons, the reported effect of NA- on ϑ does not survive. However, we do observe a significant increase in ϑ for the NA- group when we run permutation tests randomising over drug assignment (see S1 Text and S5 Table for details). Further, when we rerun the linear model analyses excluding body weight, which was equivalent across drug groups (F_3,120 = 2.24, p = 0.087), as a covariate, the significant increase in ϑ for the NA- group survives the more stringent correction for three comparisons (t₆₀ = 2.32, p = 0.022, d = 0.60), and all other reported significant effects are unchanged.

ACh antagonism slowed learning about stimulus transition contingencies.

Muscarinic ACh receptor antagonism under biperiden had more widespread perceptual effects. While ϑ was again significantly increased compared to Placebo (t₅₇ = 2.86, p = 0.005, d = 0.79), ω estimates in the ACh- group were significantly reduced (t₅₇ = −2.59, p = 0.011, d = −0.71). The lower estimate of ω indicates that participants were slower to update their transition contingency estimates under biperiden and thus slower to adapt to the probabilistic contexts.

DA antagonism had no effect on learning about task structure.

The D1/D2 DA receptor antagonist haloperidol did not influence the rate at which participants learned about the task’s volatility or contextual transition contingencies compared to Placebo (ϑ and ω: both p > 0.23).

To summarise, both NA and ACh antagonism altered learning of uncertain events arising from unexpected contextual changes in the environment. Only ACh antagonism disrupted learning of transition contingencies within probabilistic contexts.

Neuromodulatory Effects on Response Modulation

NA antagonism had no influence on responses.

The response model output revealed no significant effects of NA antagonism on participants’ capacity to modulate their motor responses according to their perceptual estimates of uncertainty (Fig 6E–6G).

ACh antagonism reduced response sensitivity to perceptual beliefs.

Compared to Placebo, ACh antagonism reduced the sensitivity of participants’ motor responses to sensory PE (β₁: t₅₇ = −3.25, p = 0.002, d = −0.90), precision-weighted contingency PE (β₂: t₅₇ = −2.56, p = 0.012, d = −0.71), and phasic volatility estimates (β₃: t₅₇ = −3.91, p < 0.001, d = −1.08) (Fig 6E–6G).

DA antagonism reduced response sensitivity to phasic volatility.

Compared to Placebo, DA antagonism led to a decrease in the influence of phasic volatility estimates on log(RT) (β₃: t₆₀ = −2.65, p = 0.009, d = −0.67; Fig 6G). This indicates that DA antagonism suppressed the sensitivity of motor responses to higher-level inference.

In addition to the effects reported above, the log(RT) constant output indicated that suppressing DA and ACh also led to some general log(RT) slowing (β₀: t₆₀ = 2.48, p = 0.015, d = 0.62; t₅₇ = 4.78, p < 0.001, d = 1.32, respectively; Fig 6D). Subjective Δalertness systematically modulated the effects observed on ϑ (t₁₁₈ = 2.62, p = 0.010, d = 0.02), sensory PE (β₁: t₁₁₈ = 2.52, p = 0.013, d = 0.02) and precision-weighted contingency PE (β₂: t₁₁₈ = −3.21, p = 0.002, d = −0.02). None of the effects were systematically modulated by any between-group differences in body weight (all p > 0.10).

Control Analyses

In S1 Text, we provide details of the physiological and subjective control measures. We also provide additional assessments of the HGF model and its ability to provide a good fit to the behavioural data by examining model parameter correlations, residuals and simulations (S2–S6 Figs). Finally, we report the results of additional permutation tests (S5 Table), which allow us to make distribution-free comparisons of the effects of our drug manipulations on the model parameters.

Discussion

By implementing a novel probabilistic serial RT task in conjunction with three pharmacological manipulations and placebo, we characterised the roles of three neuromodulatory systems during perceptual belief updating and response selection. Leveraging a hierarchical Bayesian learning model to decompose hierarchically related forms of uncertainty enabled us to pinpoint processes linked to NA, ACh, and DA. While manipulating NA and ACh modulated perceptual uncertainty computations, DA receptor antagonism reduced the sensitivity of the motor system to uncertainty estimates.

A key benefit of our pharmacological approach is that we were able to directly manipulate the function of three different neuromodulatory systems and compare the resulting psychopharmacological effects to a placebo condition. This is relevant given likely functional overlap between the different neuromodulatory systems, as observed here. Indeed, manipulation of a single neuromodulatory system, or use of a single drug, will be agnostic to such an overlap and may make any one effect appear more relevant and specific than it is. We are also able to extend interpretations of earlier neuroimaging studies [15,16], from which it is not possible to infer with certainty that activations in particular brain regions, with inhomogeneous cellular compositions, reflect the activity of specific neuromodulatory neurons.

Overlapping, but Dissociable, Noradrenergic and Cholinergic Influences on Perceptual Belief Updating

We found considerable overlap in the influence of NA and ACh antagonism on perceptual belief updating but also quantitative differences between drug conditions. While part-synergistic, part-antagonistic interactions between the two neuromodulators during uncertainty processing have been theorised previously [4], to our knowledge this is the first study to directly assess these putative computational roles, and to distinguish them from dopaminergic effects, under three pharmacological manipulations and within the same computational framework. We propose that ACh guides probabilistic learning within environmental contexts, while NA has a more circumscribed role in modulating the rate at which an agent learns about the volatility latent in the environment.

NA influences beliefs about unexpected environmental changes.

Our results suggest that NA antagonism under prazosin altered the rate at which individuals updated their volatility beliefs, as indicated by an increase in the model parameter ϑ. We note, however, that this result should be interpreted conservatively since it did not survive correction for three comparisons. Nonetheless, a permutation test, without any distributional assumptions, mirrored the result of the uncorrected comparisons (see S5 Table). Moreover, the result did survive correction when body weight (which was equivalent across drug groups) was excluded as a covariate from the linear model analyses. An influence of NA- on ϑ fits with the theorised role for NA in computing uncertainty arising from changes in environmental context [4]. Numerous studies have offered evidence that the NA system is sensitive to highly unexpected events that arise from a hidden contextual change. Noradrenergic neurons in the rat and nonhuman primate LC are responsive to environmental novelty and unexpected changes in reward contingencies [48–50,85]. Additionally, changes in pupil diameter, attributed at least in part to noradrenergic LC activity [53,68,86], have been shown to correlate with unexpected outcomes [54,55].

More specifically, in the present study we observed faster volatility belief updating following NA antagonism. In our model, ϑ represents the volatility of the volatility, and thus, our results suggest that NA stabilises an agent’s estimate of environmental volatility. This is compatible with the notion that the volatility estimate has a relatively low baseline level, to which it returns after being pushed away. In a volatile environment, this is not an adaptive feature. Rather, the volatility estimate should remain high to enable revision of one’s beliefs. It is possible that NA prevents the volatility estimate from falling by reducing an agent’s ϑ estimate.

The neurophysiological literature has distinguished two functional modes of LC noradrenergic release [44,87]. A phasic mode, characterised by a relatively low baseline firing rate and high phasic responsiveness to task relevant stimuli, has been linked to enhanced task engagement, and a tonic mode to increased distractibility, attention shifting, and exploratory behaviour [87–89] (but see Jepma et al. [90]). More recently, human LC BOLD activity was demonstrated to correlate with “unexpected uncertainty” induced by a switch in reward probabilities associated with familiar stimuli [16], although the negative sign of this correlation still seems to lack explanation. In both our task and that used by Payzan-LeNestour et al., contextual switches required participants to identify discrete changes in underlying transitions between familiar stimuli. To continue making accurate predictions in light of new transition probabilities, participants had to increase their attentional engagement to facilitate an augmented learning rate. It is likely that in both cases a phasic LC activity mode was recruited and that this would be recognised as a decrease in BOLD activity at a neural population level. Speculatively, it also suggests that our pharmacological NA manipulation may have enabled more phasic NA responsiveness to emerge under suppression of tonic NA firing. Future investigations of the impact of noradrenergic drugs on LC activity profiles are needed to validate this theory.

ACh balances the attribution of uncertainty within and between environmental contexts.

Muscarinic ACh receptor antagonism by biperiden led to slower updating of beliefs about stimulus transition contingencies, and therefore slower adaptation to the probabilistic contexts, as reflected by a decrease in the model parameter ω. We argue that this slowed adaptation also had knock-on effects higher up in the inferential hierarchy. Namely, we propose participants attributed perceived violations of their expectations to gross contextual switches as opposed to chance fluctuations in stimulus outcomes, which would be expressed as an increase in ϑ. In light of previous work, which we discuss next, it seems reasonable to suggest that by setting the rate at which an agent learns probabilistic associations, ACh facilitates the appropriate attribution of violated expectations to chance fluctuations in an environment’s statistical regularities or to gross switches in environmental context.

According to the structure of our Bayesian model, a reduction in ω maps onto a reduced precision weighting of perceptual belief updates at level 2; compare Eq 2 and Equation B in S1 Text. Our findings indicate that under biperiden less weight was given to sensory evidence, and updates of probability estimates became more reliant on current beliefs. This supports proposed roles for ACh in regulating the relative influences of stimulus-driven versus expectation-guided processing [52,91] and attentional deployment [37,40]. For instance, it has been shown that pharmacologically stimulating ACh augments bottom-up sensory signalling in human primary auditory cortex in response to auditory stimuli, possibly by enhancing the gain of superficial pyramidal cells, to bias inference towards sensory data [30].

In a recent study, Vossel et al. examined perceptual belief updating during a probabilistic attentional cueing paradigm. By applying a similar instantiation of the HGF to saccadic RTs, the authors demonstrated faster learning about contextual probabilities following administration of galantamine, an acetylcholinesterase inhibitor which increases the synaptic availability of ACh, as indicated by an increase in model parameter ω [17]. In the present study, we observe the opposite behavioural effect with the opposite pharmacological manipulation (ACh receptor antagonism), offering independent evidence that ACh signalling guides belief updating about probabilistic associations within environmental contexts.

Our results also indicate that ACh antagonism led individuals to update their volatility estimates more rapidly, reflected by an increase in the model parameter ϑ. This is consistent with the notion that ACh- participants’ impaired ability to learn transition contingencies led them to infer that contexts changed at a faster rate. Notably, in their theoretical framework, Yu and Dayan predicted that ACh depletions should cause an agent to underestimate the amount of randomness in a given context. In turn, this causes chance events occurring within a context to seem more significant than they are, and thus, they are more likely to be incorrectly taken as indicative of a context change (see Fig 6D in Yu and Dayan 2005 [4]). Our experimental observations support this hypothesis and are compatible with data indicating that cholinergic antagonists increase distractibility [92], while agonists suppress it [93–95].

We note that, although the quantities used in this current work are not identical to those previously introduced by Yu and Dayan 2005 [4], the HGF does embody versions of the same forms of uncertainty. The highest level of uncertainty in Yu and Dayan’s (YD’s) framework was induced by abrupt, discrete changes in contingencies, which induced what YD call “unexpected uncertainty” (and ascribed to NA). By contrast, the highest level of uncertainty in the HGF is the overall instability of the world, i.e., the rate at which volatility changes. It is this that we found to be modulated by our NA antagonist. Conversely, YD’s notion of “expected uncertainty” (ascribed to ACh) suggests that it arises from the known unreliability of predictive relationships within a familiar environmental context. Amongst other effects, the lower the expected uncertainty, the slower the learning—consistent with the effect of parameter ω in the HGF, which was found to decrease under cholinergic antagonism. Along with YD, we also argue that this change in learning has further knock-on effects for what participants perceive to be a chance random event or a change of context (and hence unexpected uncertainty).

In sum, our findings offer empirical support for the theoretical proposal that ACh and NA interact to construct appropriate cortical representations of volatile contexts, which facilitates optimal inferences about the current environment [4]. By regulating high-level uncertainty representations, the two neuromodulators contribute to the updating of one’s perceptual beliefs, both within and between environmental contexts, an idea that is broadly supported by recent neuroimaging [15,16] and pharmacological [17] evidence.

DA Sensitises Motor Responses to Environmental Volatility

As per its construction, our computational model allowed us not only to characterise perceptual belief updating under our three pharmacological manipulations but also to assess how each intervention influenced the deployment of motor responses in response to individual estimates of uncertainty. Pharmacologically manipulating DA and ACh altered the degree to which participants’ perceptual beliefs modulated the preparation of their speeded responses to uncertain stimuli. In contrast, NA antagonism had no significant impact on the sensitivity of participants’ motor responses to their current perceptual beliefs relative to placebo.

We had originally predicted that an individual’s capacity to modulate response selection following a sensory PE would be dependent on DA. Indeed, it has previously been shown that pharmacological DA depletion impedes adaptive reactions to unexpected events occurring within predictable contexts [58]. However, in the present study, we found no evidence that DA receptor antagonism influenced participants’ reactions to low-level sensory PE (δ₁). Rather, we observed that suppressing DA significantly reduced β₃, which we interpret as a reduction in the sensitivity of participants’ motor responses to their higher-level volatility estimates (μ₃).

It is important to note that some key differences distinguish our present experimental design from previous paradigms. In earlier work, participants were pre-trained to respond to stimuli presented within one predictable context, defined by one TM. Furthermore, switches from predictable to unpredictable contexts, consisting of random presentations of stimuli, were explicitly signalled [58]. Therefore, any probabilistic learning and higher-level perceptual uncertainty were removed. In this earlier setting, dopaminergic antagonism under haloperidol selectively impaired participants’ reactions to unexpected events that elicited large sensory PEs.

In contrast, our present task created a more complex, and arguably more ecologically valid, scenario in which individuals had to infer the current context for themselves and adapt to any contextual changes. Here, uncertainty representations had to be acquired through direct sampling from a distribution of observations. To our knowledge, our study is the first attempt to interrogate the impact of DA, ACh, and NA on nonrewarded probabilistic learning within a single behavioural paradigm and a unified Bayesian framework. By estimating beliefs about various forms of uncertainty, we sought to identify neuromodulatory contributions specifically related to particular forms of uncertainty, as opposed to any confounding variables.

Our finding that haloperidol reduced the sensitivity of participants’ responses to their phasic volatility estimates does sit well with an alternative line of work highlighting the importance of DA in cognitive switching [61,62]. For instance, Parkinson’s disease patients with DA dysfunction have an impaired capacity to switch from naming digits to letters when both types of stimuli are presented simultaneously, even when the task shift is explicitly cued [60]. In summary, we propose that our DA manipulation suppressed response modulation by impeding cognitive switching following complex contextual rule changes.

Muscarinic ACh receptor antagonism under biperiden also led to decreased response modulation by parameters at all three hierarchical levels: sensory PE (δ₁), precision-weighted contingency PE (ε₃), and phasic volatility estimates (μ₃) compared to Placebo. We propose that ACh receptor antagonism impeded participants’ abilities to learn the statistical structure of the behavioural task, which in turn impaired their capacities to respond accordingly. Although both ACh and DA had effects on response modulation, in light of previous work, we suggest that DA’s role is to modulate motor responses according to the widespread perceptual effects of ACh.

Limitations and Future Work

One of the main constraints of our study is that although prazosin, biperiden, and haloperidol are rather selective for NA, ACh, and DA receptors, respectively, there are complex interactions and dependencies between noradrenergic, cholinergic, and dopaminergic systems. Such interactions are a main reason why direct quantitative comparison between drug groups would not have provided direct comparisons between the action of different neuromodulators and therefore why our study was designed to detect changes relative to placebo instead. While our results highlight qualitative differences in how NA, ACh, and DA influence perceptual belief updating, future work will have to conduct direct quantitative comparisons of their roles.

Further, it is the receptors rather than the neuromodulators themselves that bring about psychophysiological effects, and there are dissociable roles of different receptor subtypes. For instance, the functions of nicotinic versus muscarinic cholinergic receptors in uncertainty signalling have yet to be directly compared. Distinctions have also been made between D1 and D2 dopaminergic receptor subtypes in regulating adaptive responses to unexpected stimuli [58]. Thus, future work could usefully be extended with genetic profiling and a range of selective agonists and antagonists for different receptor subtypes.

Finally, it is likely that all the neuromodulators operate over multiple timescales. For instance, separate, even competing, tonic and phasic effects have been a special target of investigation for NA. Teasing these timescales apart more fully is an ambition for the future, requiring a temporally richer design.

Nevertheless, our findings emphasise the necessity of studying the systems conjointly, as tasks associated with uncertainty will tend to involve them all.

Conclusion

In summary, our results offer novel and direct insight into the complex and intricate effects of NA, ACh, and DA during a probabilistic serial RT task. Employing a hierarchical Bayesian learning model that allowed us to assess various forms of uncertainty and PEs, we provide interventional evidence linking ACh and NA to uncertainty computations within and between behavioural contexts. In contrast, DA appears to be involved in sensitising motor responses to perceptual volatility estimates. While pharmacological manipulations do not selectively target particular neuromodulatory systems, our results offer a fresh perspective on the effects of noradrenergic, cholinergic, and dopaminergic neurotransmission on the computational mechanics of perceptual belief updating according to Bayesian principles. Future studies will verify the generality of our observed effects to different behavioural paradigms with and without learning, reward, prediction, and action. By characterising uncertainty computations and response modulation, our methodology can also be used to offer fresh insight into the numerous neurological and psychiatric disorders in which there is dysregulation of processes dependent on NA, ACh, and DA.

Materials and Methods

Participants

A total of 128 healthy participants (56 male, aged 18–38 y, 119 right-handed) with normal or corrected-to-normal vision undertook this study. Participants gave written informed consent in accordance with the Declaration of Helsinki. The experimental protocol was approved by the UCL Research Ethics Committee (Project ID: 3491/001). The following exclusion criteria applied: history of neurological or psychiatric disease, intake of medication (other than contraceptives), smoking, recreational drug use, and current participation in other pharmacological studies. Following a screening interview to rule out intolerances or contraindications, the study clinician (DR) assigned participants pseudorandomly (i.e., ensuring a balanced distribution of gender, age, and body weight) to receive a NA, ACh, or DA antagonist or a placebo. All other authors, including the experimenter (LM), were blind to the drug conditions.

General Procedure

We employed a double-blind, between-subject design. Each participant attended one experimental session during which they received a single oral dose of one of the following: 1 mg prazosin (α1-receptor antagonist; NA- group), 6 mg biperiden (M1-receptor antagonist; ACh- group), 2.5 mg haloperidol (D1/D2-receptor antagonist; DA- group), or a placebo. We selected doses that were in line with previous studies showing clear behavioural and neurophysiological effects [58,96–98]. On arrival, participants completed computerised versions of the Digit Span test, Barratt Impulsiveness Scale (BIS-11) [99], Doman-Specific Risk-Taking (DOSPERT) Scale [100], and Cognitive Failures Questionnaire (CFQ) [101]. Participants also self-reported their baseline mood (alertness, calmness, and contentedness) with visual analogue scales (VASs) [102], and we measured their baseline heart rate (HR) and blood pressure (BP). To assess any subjective and/or physiological drug effects, the VAS, HR, and BP measurements were repeated before participants started the RT task and again once they completed it.

Two different drug administration times were used to match peak plasma concentration across drugs, based on previous pharmacokinetic data. To ensure that participants undertook the RT task when the drug was at its most active, haloperidol was administered 2 h in advance, while prazosin and biperiden were administered 1.5 h before the main experimental session [96–98]. A random 50% of participants from the Placebo group were administered a placebo tablet at the first time point, and the other 50% at the second time point. The study clinician administered the drug or placebo while the experimenter was away from the testing room. Participants were asked not to eat for at least 1 h before the first drug administration time.

Probabilistic Serial RT Task

Participants sat facing a computer screen positioned approximately 60 cm away. They were instructed to rest their left and right index and middle fingers on the four buttons of a custom-made button box placed in front of them and to maintain this position throughout the task. On each trial, participants were required to respond to the presentation of one of four visual stimuli by pressing an appropriate button as quickly as possible. Each stimulus was associated with one particular button. The stimulus-response mappings remained consistent within an experimental session but were counterbalanced across participants. Each participant acquired the stimulus-response mappings for their session during a training block in which they received visual error feedback after each trial. The training session comprised at least 100 trials and did not finish until the participant had reached a minimum performance criterion of 85% accuracy on the last 20 trials. Participants were then given 40 practice trials, in which the stimuli were presented in a random order and without error feedback, to familiarise them with the timings of the main experiment. An additional refresher block, consisting of at least 26 trials with error feedback, was completed immediately before the main experiment. Again, participants had to achieve 85% accuracy in the last 20 trials to proceed. On average, participants reached this criterion in 28.1 ± 1.1 trials, indicating adequate learning and retention of the mappings. There was no difference in the number of refresher trials required between groups (F_3,120 = 1.17, p = 0.324).

Each participant performed 1,200 trials of the probabilistic RT task. Fig 1A shows an example trial sequence. Anticipatory responses (<80 ms RT) were recorded as incorrect. At any given time, the trial sequence was generated by one of eight TMs, which changed every 50 trials. In each case, there were 16 combinations that determined the probabilistic relationship between the stimuli presented on the current trial, t, and the previous trial, t-1. Three types of TM were utilised: first-order sequences, alternating sequences, and zeroth-order sequences (see Fig 1B and S1 Fig for further details). The order of TMs was pseudorandom, with no consecutive repeats. Importantly, the overall probability of each stimulus was equal across the 1,200 trials.

The pseudorandom order of TMs was used to generate one stimulus sequence that was used for all participants to ensure comparable learning processes and model parameter estimates. Rest periods occurred every 185 trials, orthogonal to TM switches. The importance of fast responses was stressed. Participants were told that by paying attention to any patterns in the order in which stimuli were presented and to any switches in these patterns, it might be possible to respond faster. No further information about the nature of the experiment was provided.

Combining our behavioural paradigm with three pharmacological manipulations allowed us to assess directly any separable roles for NA, ACh, and DA in belief updating under irreducible uncertainty, estimation uncertainty and volatility uncertainty and in sensitising the motor system to participants’ individual perceptual beliefs. At the end of the experimental session, participants were debriefed, indicated whether they thought they had taken an active drug or placebo, and reported the quality and quantity of their sleep on the previous night [103].