Participants and procedure

Twenty-nine participants, screened for any counter indications to TUS and magnetic resonance imaging (MRI) (S2 Appendix) completed a four-session study protocol. The first session involved collecting MR images for neuronavigation and generating a pseudo-computed tomography (pseudo-CT) image for personalized acoustic planning [29] as well as a task practice (Fig 1A). Participants were then assigned to one of three TUS conditions: Sham, dACC, or aIns (Fig 1B), using a counterbalanced design. Each of these subsequent sessions included an 80-s-long 5 Hz-patterned TUS intervention, with a spatial peak pulse average intensity (ISPPA) of approximately 50 W/cm² in water, immediately followed (maximal latency: 10 min) by a motivational Go/NoGo learning task [8–10]. While the task structure remained consistent across sessions, the cue sets were different and individually randomized to prevent learning effects. Each session lasted approximately 1.5 hours.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Study design, TUS targeting, task structure, and acoustic simulations.

A. Study Design: In the first session, a T1-weighted image was acquired for neuronavigation and acoustic planning. Participants also practised a short version of the task. In sessions two to four, participants were randomly allocated to one of three conditions: Sham, dACC, or bilateral aIns. Post-stimulation safety questionnaires were completed on the same day and the following day. B. TUS positioning: Visualization of the positioning of the transducer on the head for the active TUS sessions. C. Task design: On each trial, participants saw an abstract cue and needed to learn from trial-and-error to respond with either a Go or NoGo response to that cue. The cue disappeared at the response onset, followed by a jittered interval and finally response-dependent feedback. Each cue had one correct action (Go or NoGo) and was either a Win cue (leading to rewards or neutral outcomes) or an Avoid cue (leading to neutral outcomes or punishments). Rewards and punishments were represented as money falling into or out of a bucket. D. Feedback validity and cue types: Each block of session included 4 novel cues, varying in valence (Win/Avoid) and required action (Go/NoGo). Incongruent cues, where the required action was in opposition to the Pavlovian bias (NoGo to Win, Go to Avoid punishment), are highlighted in gray. Feedback was probabilistic. Correct responses to Win cues resulted in rewards 80% of the time and neutral outcomes 20% of the time. Correct actions to Avoid cues led to neutral outcomes in 80% of cases and punishments in 20%. Incorrect responses led to feedback with the reverse probabilities. E. Regions: Targeted brain regions for TUS (dACC and bilateral aIns) are shown (top), with post-stimulation pressure simulated through k-Plan software (BrainBox) expressed in megapascals on a representative participant (bottom). E. Regions: Targeted brain regions for TUS (dACC and bilateral aIns) are shown (top), with post-stimulation pressure simulated through k-Plan software (BrainBox) expressed in megapascals on a representative participant (bottom). F. TUS simulations: Maximum pressure in the brain target volume (top panel) and maximum temperature rise in the brain target volume (bottom panel), (n = 29). Box plots show the mean, and the standard error (bounds of the box). Data from each individual participant are presented as small black circles. The data underlying this figure can be found in S1 Data.

https://doi.org/10.1371/journal.pbio.3003767.g001

During the motivational Go/NoGo learning task, on each trial, participants were presented with one of several cues. They had to learn from probabilistic feedback via trial-and-error to either perform a Go (button press) or a NoGo action (no button press) to four cues per block. Half of these cues were Win cues for which correct responses (mostly) returned a reward (increase in point score), while incorrect responses (mostly) led to neutral feedback (no change in point score). For the other half of cues, called Avoid cues, correct responses (mostly) led to neutral feedback, while incorrect responses (mostly) led to punishments (decrease in point score). The task timeline is presented in Fig 1D. Feedback was probabilistic such that correct responses resulted in the desired outcome (reward or omitted punishment) 80% of the time, whereas incorrect actions led to the desired outcome in only 20% of trials. On each session, participants completed 320 trials, split into four blocks of 80 trials. In each block, a distinct set of four cues was presented, each repeated 20 times. The order of cue presentations was randomized for each participant within each block (Fig 1D).

Targeting of the bilateral aIns and dACC was individually adjusted using T1-weighted MRI (structural scans) during planning for target nd transducer placement, as well as acoustic and thermal simulation. The dACC was defined anatomically as the region anterior to the anterior commissure, dorsal to the genu of the corpus callosum, and bounded dorsally by the cingulate sulcus. The aIns was identified as the anterior portion of the insular cortex, located deep within the Sylvian fissure, anterior to the central insular sulcus, and bordering the frontal operculum. S1 Fig shows a representative participant for the region targeted. Repetitive TUS followed a 5 Hz-patterned protocol with a 10% duty cycle (Fundamental Frequency = 500 kHz, Pulse Duration = 20 ms, pulsed every 200 ms), applied for 80 s (400 pulses in total). The ISPPA in water was maintained at 50 W/cm² across participants (validated in a hydrophone tank [22]). We took care to remain within guidelines for human ultrasound exposure as defined the International Thermal and Radiological Ultrasound Safety Standards and Thresholds (ITRUSST; [30]). Importantly, the transcranial Mechanical Index (MItc) was way below 1.9 and the results of our acoustic simulations predominantly indicated a maximum soft tissue temperature rise below 2 °C (S1 Table). We also calculated the Cumulative Equivalent Minutes at 43 °C (CEM43), a metric reflecting both duration and intensity of heating relative to 43 °C, the critical threshold for thermal cell damage. The CEM43 values were always below 0.1, i.e., well below the safety threshold of 0.25 [30].

Sham was administered in the same manner as active aIns-TUS, placing the transducer over the site but without actual stimulation. Instead, participants listened to a sound that mimicked the TUS protocol’s acoustic output (validated across 6 participants independent from this study), played through bone conduction headphones. Acoustic simulations for each participant were used to plan the stimulation to ensure accurate pressure delivery at the target area and adherence to safety protocols (an example participant’s targeted areas are shown in Fig 1E). Detailed acoustic simulation parameters and outputs for all study participants can be found in Fig 1F, S1 and S2 Tables, and are summarized in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Ultrasound parameters across different target tissues for dACC and aIns stimulation. A sphere (Target ROI) with a 10mm radius centered on the target coordinates is used to reliably extract peak values from the brain region where the focus is located.

https://doi.org/10.1371/journal.pbio.3003767.t001

Side effects

Participants completed a post-TUS questionnaire for side effects twice: once at the end of each session, and a second time 12–48 hours after the session. They were presented with a list of symptoms for which they had to rate the intensity of the symptom and the degree to which they believe that symptom was related to the stimulation (refer to the Methods section under TUS protocol and procedure for further safety details; see also S1 Appendix for the full safety questionnaire). Immediately after stimulation of the insula, participants reported the following side effects: anxiety (N = 1), difficulty paying attention (N = 2), forgetfulness (N = 1), headache (N = 1), and sleepiness (N = 3). All side effects were rated as mild and unrelated to the stimulation. Following stimulation of the dACC, moderate anxiety (N = 1), mild forgetfulness (N = 1), moderate neck pain (N = 1), mild to moderate sleepiness (N = 2), and mild vision problem (N = 1) were reported. Only the participant reporting vision problems considered those as possibly related to the stimulation. Following sham stimulation, one participant reported mild itchiness (N = 1) and another moderate sleepiness (N = 1), again both problems were rated as unrelated to the stimulation. Overall, no qualitative difference in the number and quality of side effects was observed between sham and active stimulation. In most cases of side effects, participants did not attribute them to the stimulation.

Pavlovian biases in responding and learning in the sham session

We first focus on the data from the sham session to replicate Pavlovian biases in responding and learning as typically observed in this task [8–10].

First, in line with previous work [9,10,31,32], we used mixed-effects logistic regression to fit Go/NoGo responses as a function of the required action of a given cue (Go versus NoGo), the cue valence (Win versus Avoid cues), and their interaction. Participants successfully learned the task, as they made more Go responses to Go than NoGo cues (main effect of required action: b = 2.132, 95%-CI [1.797, 2.467], χ²(1) = 53.812, p < .001; Fig 2D). Furthermore, there was a Pavlovian bias in behavior, indexed by significantly more Go responses to Win than Avoid cues (main effect of cue valence: b = 0.689, 95%-CI [0.421, 0.966], χ²(1) = 17.173, p < .001). Lastly, the interaction between both terms was significant (b = 0.269, 95%-CI [0.104, 0.434], χ²(1) = 9.646, p = .002), reflecting a stronger Pavlovian bias for Go than for NoGo cues. Similar analyses of the reaction times (RTs) also revealed a bias (S2 Fig). These analyses confirm that participants’ behavior was influenced by Pavlovian biases in the sham session, which constitutes a prerequisite for modulating them in the other two active sonication sessions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Behavioral performance in the sham dataset.

A. Response bias: Humans exhibit a tendency to show actions (i.e., button presses) for Win cues (blue), aligning reward seeking with action invigoration, and to withhold actions for Avoid cues (red), aligning punishment avoidance with action suppression/inaction. B. Learning bias: Learning is enhanced for rewarded Go actions compared to rewarded NoGo actions, promoting faster acquisition of Go actions from rewards. Conversely, learning is reduced for punished NoGo actions compared to punished Go actions, slowing the extinction of NoGo responses via punishments. C. Persistence bias: Seeing a Win cue partly acts as a reward in itself, leading to a higher tendency to repeat responses to such cues, even when a neutral outcome (negative feedback) suggests that one should switch the response. In contrast, seeing an Avoid cue partly acts as a punishment in itself, leading to a higher tendency to switch responses, even if a neutral outcome (positive feedback) suggests repeating the response. D. Trial-by-trial proportion of Go responses (error bands = ± SEM, n = 29) for Go cues (solid lines) and NoGo cues (dashed lines). A clear response bias is evident from the start, with participants making more Go responses to Win (green lines) than Avoid (red lines) cues. Participants show a general bias towards Go responses, with initial response rates around 80%. Additionally, instrumental learning is apparent in Go responses increasing for Go cues and decreasing for NoGo cues over time. E. Probability to repeat a response on the next encounter of the same cue as a function of the response and outcome for valenced outcomes (rewards, punishments) only (error bars are ± SEM across participants, n = 29, dots indicate individual participants). Learning was reflected in the higher probability of staying after rewarded responses than after being punished (main effect of outcome valence). Biased learning was evident in the stronger outcome effect after Go responses than after NoGo responses, indicating the presence of a learning bias. F. Probability to repeat a response on the next encounter of the same cue as a function of cue valence (error bars are ± SEM across participants, n = 29, dots indicate individual participants). Participants showed a higher tendency to repeat responses for Win than Avoid cues, indicative of a persistence bias. G. Laplace approximation to the log-model evidence favors the model incorporating all three Pavlovian biases—response bias, learning bias, and persistence bias (M7)—over simpler models (M1–M6); error bars are ± SEM across participants, n = 29. H–J. Posterior predictive checks: one-step-ahead predictions based on the best fitting parameters (estimated with hierarchical Bayesian inference) for M7. K. Bayesian model selection. The winning model is M7 as determined by model frequency and protected exceedance probability, in line with log model evidence. The data underlying this figure can be found in S2 Data.

https://doi.org/10.1371/journal.pbio.3003767.g002

Second, we used mixed-effects logistic regression to analyze participants’ propensity to repeat versus switch a response on the next trial as a function of the obtained outcome and performed response on the current trial. We have previously found that Pavlovian biases are present not only in action selection, but also in learning, with a larger propensity to take valenced feedback into account (i.e., repeat actions after rewards, switch actions after punishments) for Go actions compared to NoGo actions. Specifically, learning is increased for rewarded Go (relative to rewarded NoGo) actions and decreased for punished NoGo (relative to punished Go) actions (see detailed explanation in the computational modeling) [9,10]. Because this bias should only occur for valenced outcomes (rewards, punishments), but not for neutral outcomes, we only included valenced outcomes in this analysis. We observed a significant main effect of the obtained outcome (reward versus punishment), b = 1.436, 95%-CI [1.190, 1.683], χ²(1) = 50.170, p < .001, with more response repetitions after rewards than after punishments, and a main effect of performed response, b = 0.211, 95%-CI [0.081, 0.341], χ²(1) = 6.422, p = .001, with more response repetitions for Go than NoGo responses (likely due to the fact that participants showed overall more Go than NoGo responses in this task). Crucially, the interaction between the obtained outcome and performed response was significant, b = 0.423, 95%-CI [0.195, 0.650], χ²(1) = 10.051, p < .001, with a stronger outcome effect after Go than after NoGo responses (Fig 2E). This result confirmed the presence of Pavlovian biases also in learning from feedback.

Thirdly, we used mixed-effects logistic regression to analyze participants’ propensity to repeat versus switch a response on the next trial as a function of the cue valence. We have previously found that participants show a propensity to repeat responses for Win cues than for Avoid cues, suggesting that they also partly interpret the cue valence as a feedback signal [10]. This leads to response repetition (persistence) tendencies that depend on the cue valence, a tendency that was not well captured by previous computational models (S4 Fig), but can only be accounted for by a third, persistence bias. Indeed, we found a strong main effect of cue valence on response repetitions, b = 0.351, 95%-CI [0.262, 0.440], χ²(1) = 60.117, p < .001, with more repetitions for Win than Avoid cues (Fig 2F). This result indicated the presence of a third Pavlovian bias in which the cue valence affected persistence tendencies. Taken together, these results motivated us to further interrogate these biases using computational reinforcement learning models.

As control analyses, we tested for age or gender effects on any of the reported effects. For Go/NoGo responses, there were significant interaction between age and required action (b = −0.415, 95%-CI [−0.685, −0.145], χ²(1) = 9.061, p = .003) and gender and required action (b = −0.293, 95%-CI [−0.573, −0.013], χ²(1) = 4.206, p = .040), reflecting effects on task accuracy: Male participants achieved overall higher accuracy than female participants, and younger participants higher accuracy than older participants. For repeat/switch responses, there was a main effect of gender (b = −0.276, 95%-CI [−0.515, −0.038], χ²(1) = 5.157, p = .023) as well as significant interaction between gender and previous outcome (b = −0.310, 95%-CI [−0.528, −0.092], χ²(1) = 7.743, p = .005): Male participants showed overall more repetitions than female participants as well as a stronger effect of past feedback. Specifically, male participants more often repeated responses after positive feedback, which might have contributed to higher performance. Furthermore, there was a significant main effect of age (b = −0.313, 95%-CI [−0.540, −0.085], χ²(1) = 7.250, p = .007), with older participants repeating responses less often than younger participants. This propensity to more often switch responses might have contributed to lower accuracy in older participants.

As a further control analyses, we tested for effects of session identity (1st, 2nd, 3rd session, using data from all sonication conditions to increase power) on the reported effects. For Go responses, there was an interaction between session ID and required action (χ²(1) = 18.189, p < .001), reflecting that accuracy increased over sessions. For repeat/switch responses, there was a significant main effect of session (χ²(1) = 18.916, p < .001) and a significant interaction between session and past outcome (χ²(1) = 7.898, p = .019): participants overall repeated their responses more often in later sessions and showed a stronger effect of past outcomes on choice repetitions, mostly driven by more repetitions after positive feedback. This pattern might have contributed to higher accuracy in later sessions. We furthermore tested for effects of session order (i.e., order in which all three stimulation conditions were performed) and effects of cue set (i.e., the set of geometric cues which participants saw in a given block), but did not find any effects on Go/NoGo or repeat/switch responses.

Computational modeling of the sham data

We next fit a series of increasingly complex reinforcement learning models to participants’ responses in the sham sessions. These models follow previous publications on this task [8–10]. These models incorporate three previously described forms of Pavlovian biases: (a) a response bias, (b) a learning bias, and (c) a persistence bias.

The first bias, called response bias, is a bonus/malus added to the learned action values (Fig 2A): For Win cues, a bonus is added to the value of Go actions, and for Avoid cues, a malus is subtracted from the value of Go actions [8]. This bias is present at the onset of the task and constant throughout learning. Since action values and the bias jointly determine which action is selected, and since action values will increasingly separate with learning over the course of the task, the role of this response bias will, in net, decrease over the time [9].

The second bias, called learning bias, captures two tendencies well described in the literature: people ascribe positive outcomes to their own actions (which in other contexts can lead to “illusions of control” [33,34], but tend to ignore negative outcomes after having remained passive (a form of “omission bias” [35,36]; Fig 2B). These tendencies are captured by a boost in the learning rate for rewarded Go actions and a decrement in the learning rate for punished NoGo actions [9]. The impact of this bias on behavior only develops over time, where the action values become increasingly distorted relative to unbiased learning.

The third bias, called persistence bias, captures the fact that the cue valence (Win or Avoid cue) can act as a reinforcer (Fig 2C) [10]: For Win cues, rewards signal positive feedback, and neutral outcomes signal negative feedback. However, the Win cue itself can erroneously be interpreted as a feedback signal and reinforce behavior irrespective of the eventual outcome. Thus, while a neutral outcome for a Win cue signals negative feedback, which suggests switching behavior, the cue valence itself might be taken for positive feedback, which usually translates into a response repetition. Vice versa, for Avoid cues, neutral outcomes signal positive feedback and punishments signal negative feedback. The Avoid cue itself can erroneously be interpreted as negative feedback. Thus, while a neutral outcome for an Avoid cue signals positive feedback, which suggests repeating behavior, the Avoid cue itself can be taken for negative feedback, which usually translates into a response switch. We have previously observed that this bias can dominate repeat/switch behavior after neutral outcomes, resulting in the paradoxical observation of more response repetition after negative (i.e., no reward) feedback for Win cues than positive (i.e., no punishment) feedback to Avoid cues [10]. Such a bias should be invisible at the beginning of the task and only arise with learning, leading to increasingly distorted learning curves over time.

We tested the presence of these three biases in behavior using a series of seven reinforcement learning models. The base model (M1) was a simple Q-learning model that learned the values of Go and NoGo actions for each cue. To generate choices, action values were turned into action probabilities via a softmax transform. Action values were subsequently updated based on the obtained outcomes via reward prediction errors, which were scaled by a free learning rate parameter ε and then added to the old action values. In line with previous implementations [8–10], objective outcomes were scaled by a free feedback sensitivity parameter ρ. High values on this parameter lead to learned action values developing further apart and thus more deterministic choices, similar to a high inverse temperature parameter used in alternative models. We extended this model with a constant Go bias parameter b to capture overall tendencies towards Go or NoGo responses (M2), a response bias parameter π that added a bonus/malus to the action value of Go depending on the valence of a cue (M3), and finally, a learning bias parameter κ capturing increased learning rates for rewarded Go responses and decreased learning rates for punished NoGo responses (M4). M5 combined all three parameters and fitted the data better than any of the simpler models, suggesting that both a response bias and learning bias were present in the data. Finally, we extended M5 by adding a persistence parameter φ_INT that captured overall tendencies to repeat versus switch actions (M6), and a persistence bias φ_DIFF that captured the higher tendency to repeat actions for Win cues than for Avoid cues (M7).

We compared these seven models using Bayesian model selection [37]. In line with our previous work [10], M7, implementing all three Pavlovian biases, was the best-fitting model according to the log-model evidence (Fig 2G), the model frequency (M7: 61%; M6: 14%; M5: 25%), and crucially, according to the protected exceedance probability (M7: 98%; M5: 2%; Fig 2K). We performed posterior-predictive checks by simulating the action probabilities based on the fitted parameters combined with participants’ actions and outcomes (one-step ahead predictions), which captured the empirical data patterns very well (Fig 2H–2J). Simpler models failed to capture core features of the empirical data patterns. Model recovery showed that all models were distinguishable from each other, and parameter recovery on M7 showed that all parameters were well identifiable (S6 and S7 Figs). We also explored alternative models in which cue valence did not affect persistence directly, but biased the outcomes in the prediction error updating equation. Such models predicted, qualitatively, a very similar pattern as M7, but showed slightly inferior fit (S5 Fig). In sum, these results confirmed the presence of all three Pavlovian biases in participants’ behavior in the sham sessions.

We again tested for effects of age, gender, session ID, and session order on parameter values (using estimates from all three sonication conditions). Male participants showed overall significantly higher values for the reward sensitivity ρ (χ²(1) = 6.037, p = .014), in line with more deterministic behavior and higher accuracy. There were no significant effects of age on parameter values. Across session IDs, values of both ρ (χ²(1) = 10.336, p = .006) and ε (χ²(1) = 6.267, p = .044) significantly increased, in line with increasing accuracy across sessions. In consequence, values of π significantly decreased across sessions (χ²(1) = 8.423, p = .015), which is expected since a lower number of errors will lead to a smaller Pavlovian response bias. Finally, there was no effects of session order on any parameter value.

Sonication effects on Go/NoGo choices

We next analyzed the effect of TUS on Pavlovian biases in responding and learning. We again first used regression models based on participants’ Go/NoGo responses and their propensity to repeat/switch responses, and then subsequently contrasted parameters obtained from the best-fitting reinforcement learning model across all sonication conditions.

We used a mixed-effects logistic regression model to test for effects of TUS on (biased) responding, i.e., their Go/NoGo responses. This model included the required action, cue valence, the TUS sonication condition, as well as the block half (first versus second half of a block) to account for the possibility that TUS might affect learning processes, which might only become visible in the second half of each block. Beyond the main effects of required action and cue valence and their interaction, this model yielded a significant 3-way interaction between required action, cue valence, and sonication condition, χ²(2) = 14.277, p < .001, a significant 3-way interaction between cue valence, sonication condition, and block half, χ²(2) = 9.191, p = .010, and a significant 4-way interaction between required action, cue valence, sonication condition, and block half, χ²(2) = 21.548, p < .001 (Fig 3A–3C). See S3 Table for results when analyzing each cue condition (Go-to-Win, Go-to-Avoid, NoGo-to-Win, NoGo-to-Avoid) separately. To better understand these effects, we followed them up with the regression analyses of response repetitions versus switches.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Empirical and model-based effects of TUS on Go responding and response repetition biases.

A, B Empirical Probability of Go. Trial-by-trial proportion of Go responses across sonication conditions (TUS-dACC, and TUS-aIns, gray lines represent the sham condition). TUS-dACC decreased Go responses in the Go-to-Win condition but increased them in the NoGo-to-Win condition. TUS-aIns decreased Go responses in the Go-to-Win condition but increased them in the Go-to-Avoid and NoGo-to-Win conditions. Coloured lines display the active sonication condition. C, D Probability of Stay. TUS effects on response repetitions based on outcomes (rewarded Go vs. punished NoGo) and cue valence (Win vs. Avoid). TUS-aIns (but not TUS-dACC) attenuated the tendency to repeat rewarded Go responses, but increased the tendency to repeat punished NoGo responses compared to sham. TUS-dACC (but not TUS-aIns) increased the cue valence effect on response repetitions, with stronger persistence for Win cues relative to Avoid cues. E–H Posterior predictive checks reproduced the empirical data patterns for the probability of Going and staying. The data underlying this figure can be found in S3 Data.

https://doi.org/10.1371/journal.pbio.3003767.g003

Sonication effects on response repetitions/switches

Analyses of Go/NoGo choices suggested effects of TUS on learning (i.e., changes in Go/NoGo responses over time). To more directly investigate learning, i.e., the impact of outcomes on subsequent choices, we analyzed participants’ repeat/switch choices as a function of outcomes and responses on the current trial. We focused on the signatures of biased learning.

First, we contrasted sonication conditions selectively on those trials on which biased learning should occur, namely trials with rewarded Go responses or with punished NoGo responses. If the learning bias became stronger, participants would repeat rewarded Go responses more and punished NoGo responses less, yielding a stronger difference between both conditions (with no change in any other condition). Vice versa, if the bias became weaker, the difference between both conditions would decrease. In a mixed-effects logistic regression model fit to the response repetitions versus switches, we observed a significant main effect of outcome (reward versus punishment) qualified by a significant outcome × sonication interaction, χ²(2) = 6.162, p = .046: participants more often repeated rewarded Go responses than punished NoGo responses and this tendency was significantly attenuated after TUS-aIns relative to sham, b = −0.150, 95%-CI [−0.293, −0.006], χ²(1) = 4.197, p = .040, and not after TUS-dACC compared to sham, b = 0.001, 95%-CI [−0.058, 0.249], χ²(1) = 0.001, p = .984 (Fig 3C). These results suggest that TUS-aIns, but not TUS-dACC, reduced the learning bias.

Second, we tested whether sonications changed the effect of cue valence on response repetitions versus switches, reflecting a bias in persistence. Participants were more likely to repeat responses for Win than Avoid cues (main effect of valence: b = 0.428, 95%-CI [0.342, 0.514], χ²(1) = 94.620, p < .001], which was qualified by a cue valence × sonication interaction, χ²(2) = 7.617, p = .022: the cue valence effect was significantly stronger after TUS-dACC compared to sham, b = 0.113, 95%-CI [0.031, 0.194], χ²(1) = 7.368, p = .007, with no difference between TUS-aIns and sham, b = 0.006, 95%-CI [−0.063, 0.074], χ²(1) = 0.025, p = .873 (Fig 3D). These results suggest that TUS-dACC, but not TUS-aIns, affects the persistence bias. Next, we directly confirmed these observations by fitting these parameters using computational reinforcement learning models and comparing the fitted parameters across TUS sonication conditions.

Effects of TUS on computational modeling parameters

We fit the best-fitting model from the sham session, M7, comprising all three Pavlovian biases, namely a response bias π, a learning bias κ, and a persistence bias φ_DIFF, to the other two sonication conditions. Model comparisons suggested that, in the TUS-dACC sessions, M7 was again the best-fitting model according to both the model frequency (M7: 89%; M6: 11%) and the protected exceedance probability (M7: 100%). In the TUS-aIns sessions, a slightly different outcome occurred, with M6 being the best-fitting model according to both the model frequency (M6: 68%; M7: 32%) and protected exceedance probability (M6: 97%; M7: 3%). Given that M6 is a nested version of M7 (i.e., M7 with the φ_DIFF parameter fixed to zero is equivalent to M6), we proceeded with comparing the parameter values in M7 across all three sonication conditions.

We focused on the three parameters reflecting the three different bias parameters: the response bias π, the learning bias κ (Fig 4A), and the persistence bias φ_DIFF (Fig 4D). We compared the parameter values between all three conditions with a one-way repeated-measures ANOVA, following up any significant effects with paired t-tests. To correct for multiple comparisons (seven tests for seven parameters), we applied an adjusted alpha-level of.05/ 7 = .007. There was no significant difference in the response bias π, F(2, 56) = 1.719, p = .189, generalized η² = 0.018. However, there was a significant difference in the learning bias κ, F(2, 56) = 7.680, p = .001, η² = 0.134: this bias was significantly attenuated after TUS-aIns compared to sham, t(28) = 4.381, p < .001, Hedges’ g = −0.791, bootstrapped 95%-CI [−1.248, −0.460], but not after TUS-dACC compared to sham, t(28) = 0.489, p = .629, g = −0.088, 95%-CI [−0.494, 0.264] (Fig 4B). This observation is in line with the regression results showing an attenuated difference in repetitions of rewarded Go versus punished NoGo responses after TUS-aIns.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Effects of TUS on instrumental learning biases, as reflected by computational model parameters.

A. Learning bias illustration: Learning is stronger for rewarded Go actions, and weaker for punished NoGo actions. B. Learning bias (κ): TUS-aIns (but not TUS-dACC) significantly diminished the learning bias, i.e., the tendency to attribute rewards to actions and the reluctance to attribute punishments to inactions. C. Go bias (b): both sonication conditions significantly increased participants’ overall tendency for Go responses regardless of the cue valence. D. Persistence bias illustration: participants show an overall higher tendency to repeat actions for Win cues than for Avoid cues. For Win cues, this implies that they sometimes ignore neutral outcomes (negative feedback, which suggests to switch the response) and instead take the Win cue itself as positive feedback (which suggests to repeat the response). E. Persistence parameter (φ_INT): both sonication conditions increased participants’ overall tendency to repeat previous responses. F. Persistence bias (φ_DIFF): TUS-dACC (but not TUS-aIns) significantly increased the persistence bias, leading to a stronger tendency to repeat responses relatively more for Win cues than for Avoid cues. Asterisks indicate significance levels: ***p < 0.001; **p < 0.01; *p < 0.05; † p < 0.10. The data underlying this figure can be found in S4 Data.

https://doi.org/10.1371/journal.pbio.3003767.g004

Furthermore, there was a significant difference in the persistence bias φ_DIFF, F(2, 56) = 9.836, p < .001, η² = 0.157: this bias was significantly stronger after TUS-dACC compared to sham, t(28) = 4.028, p < .001, g = 0.728, 95%-CI [0.419, 1.146], but not after TUS-aIns compared to sham, t(28) = 1.058, p = .299, g = 0.191, 95%-CI [−0.170, 0.547] (Fig 4F). This effect was in line with the increased cue valence effect on response repetitions versus switches after TUS-dACC. In sum, TUS-aIns attenuated the learning bias, while TUS-dACC increased the persistence bias.

Beyond the bias parameters of interest, both TUS conditions also unspecifically increased the Go bias parameter, F(2, 56) = 4.958, p = .010, η² = 0.071 (Fig 4C), and the persistence parameter φ_INT, F(2, 56) = 3.517, p = .036, η² = 0.055 (Fig 4E), though neither of these changes was significant when corrected for multiple comparisons. We report follow-up t-tests for completeness. Relative to sham, the overall Go bias b was significantly increased after both TUS-dACC, t(28) = 3.148, p = .004, g = 0.569, 95%-CI [0.228, 0.979], and TUS-aIns, t(28) = 2.130, p = .042, g = 0.385, 95%-CI [0.039, 0.774]. Similarly, the overall persistence parameter φ_INT was significantly increased after TUS-dACC, t(28) = 3.070, p = .005, g = 0.555, 95%-CI [0.211, 1.000], and marginally significantly increased after TUS-aIns, t(28) = 1.972, p = .059, g = 0.356, 95%-CI [0.026, 0.683] (not significant when correcting for multiple comparisons). Notably, stimulation effects on κ and φ_DIFF were still significant when controlling for b and φ_INT using mixed-effects linear regression with a random intercept per participant overall effect of sonication on κ: χ²(2) = 18.591, p < .001; difference TUS-aIns–sham in κ: β = 0.546, 95%-CI [0.313, 0.780], χ²(1) = 21.001, p < .001; overall effect of sonication on φ_DIFF: χ²(2) = 13.378, p < .001; difference TUS-dACC–sham in φ_DIFF: β = 0.374, 95%-CI [0.126, 0.622], χ²(1) = 8.758, p = .003), suggesting that region-specific effects occurred independently of these unspecific effects. Furthermore, stimulation effects remained significant when controlling for age, gender, session ID, or session order, and did not interact with any of these variables. There was no significant difference in the feedback sensitivity parameter ρ, F(2, 56) = 0.928, p = .401, η² = 0.015, or the learning rate ε, F(2, 56) = 1.047, p = .358, η² = 0.018.

We tested whether the effect of TUS on behavior could be predicted by the dose of the ultrasound stimulation. For this purpose, we correlated the changes in parameter values (active session – sham) with the estimated dose (pressure in kPa) based on acoustic simulations of the acoustic beam trajectory. These simulations were based on the achieved target and transducer coordinates recorded during the stimulation sessions. After controlling for multiple comparisons (Bonferroni correction for eight tests, adjusted alpha-level of  .05/8 = .006), we did not observe any significant relationship between pressures and TUS effects on parameters (S11 Fig).

Taken together, the effects of dACC and aIns sonication were best captured by a decrease in the learning bias after TUS-aIns, an increase in the persistence bias after TUS-dACC, and increases in overall Go biases and persistence after both types of sonications. These effects were well captured by posterior predictive checks: Simulated learning curves closely followed the empirical learning curves (Fig 3E and 3F). These observations also match the regression results of response repetitions/switches, with an attenuated difference between rewarded Go and punished NoGo responses after TUS-aIns (Fig 3G) and a stronger cue valence effect after TUS-dACC (Fig 3H). In sum, in this study, we found a functional differentiation between the effects of TUS-aIns and TUS-dACC, with both altering different Pavlovian biases.

Participants

Thirty-four healthy participants (18 females, 16 males) aged 19–59 years (mean age = 29.73, SD = 11.16) took part in this study. One participant was excluded because they did not understand the task and three because of missing sessions. Twenty-nine participants (16 females, 13 males), aged 19–59 years (mean age = 30.24, SD = 11.63) were included in the analysis. Participants had normal or corrected-to-normal vision. All participants provided written informed consent following a full explanation of the study procedures. Ethical approval was granted by the University of Plymouth Faculty of Health Staff Research Ethics and Integrity Committee (reference ID: 3394; date: 28/06/22). The study was conducted in accordance with the principles expressed in the Declaration of Helsinki.

Prior to participation, all participants completed a structured procedure to assess eligibility for MRI and TUS. Exclusion criteria covered standard MRI contraindications and safety restrictions, pregnancy, age below 18 years, a history of major neurological conditions linked to increased seizure risk, relevant implanted or metallic devices, and current medication or substance deemed incompatible with the study safety criteria. Participants were additionally instructed to abstain from alcohol for at least 24 hours before stimulation. Adverse events were assessed after each TUS session and again 12–48 hours later using a structured symptom questionnaire in which participants rated symptom severity and their perceived relationship to stimulation. The full versions of the screening and post-TUS questionnaires are provided in the Supporting information. No participant reported a current neurological or psychiatric diagnosis or any use of psychoactive medication.

Participants were compensated an average of £85, which included a performance bonus of up to £10 based on points collected over three sessions in the Motivational Go/NoGo task. Travel expenses were reimbursed up to £5 per session. All study sessions took place at the Brain Research & Imaging Centre (BRIC) in Plymouth, UK.

Study design

The study design is summarized in Fig 1A. Participants began with an MRI-only session, which included a series of scans, including a T1-weighted MRI scan. This imaging data set was used to create a personalized head model for neuronavigation and acoustic and thermal simulations, enabling precise planning of the TUS target and transducer placement for the upcoming TUS sessions. In the first session, participants completed a practice phase to become familiar with the Motivational Go/NoGo learning task.

For the TUS sessions, participants received a randomized sequence of TUS conditions. Each session involved an 80-s TUS application with three different conditions: active stimulation of the bilateral anterior insula (TUS-aIns), active stimulation targeting the dorsal anterior cingulate cortex (TUS-dACC), and a sham condition. In the sham condition, the transducer was positioned over the temporal region bilaterally, similar to the TUS-aIns setup, but no ultrasound was delivered. To maintain a consistent procedure, both sonication and sham conditions were always applied first to the left hemisphere, followed by the right. Directly after the end of the stimulation procedure (maximal latency of 10 min), participants started the Motivational Go/NoGo Task.

MRI data acquisition

For TUS planning, including neuronavigation and acoustic simulations, MRI scans were acquired using a Siemens MAGNETOM Prisma 3T scanner (syngo MR V11E, Siemens Healthineers, Erlangen, Germany) equipped with a 64-channel head coil. The scanning protocol for this study included a T1-weighted MPRAGE sequence in the sagittal plane (TR = 2100 ms, TE = 2.26 ms, inversion time = 900 ms, flip angle = 8°, GRAPPA acceleration factor = 2, matrix size = 256 × 256, 176 slices, voxel size = 1 × 1 × 1 mm³).

TUS protocol and procedure

Targeting of the bilateral aIns and dACC was guided by personalized anatomical landmarks using each participant’s T1-weighted MRI scans during the planning of target and transducer placement (see S1 Table). Acoustic field simulations, generated from pseudo-CT skull models derived from participants’ T1-weighted images using a convolutional neural network (CNN) [29], ensured precise targeting and safety by keeping both on-target and off-target pressure within acceptable ranges.

A bespoke CTX-500 NeuroFUS TPO system (Brainbox, Cardiff, UK) with a four-element annular transducer (diameter = 64 mm, central frequency = 500 kHz, and steering range between 27.3 and 82.6 mm) (S1 Fig) was used to deliver the TUS protocol. The TUS protocol consisted of repetitive TUS applied using a 5 Hz-patterned protocol with a 10% duty cycle, lasting 80 seconds and delivering a total of 400 pulses (pulse duration of 20 ms and pulse interval of 200 ms). The initial spatial-peak pulse-average intensity (ISPPA) in free-field conditions was set at 50 W/cm² for all participants. Free-field pressures were calibrated in a water tank using an ONDA capsule hydrophone (HGL-series Golden Lipstick) mounted in a low-cost hydrophone tank adapted from [80] converting voltage from the stable portion of the waveform at focus to pressure using the hydrophone’s calibrated sensitivity to compute peak rarefactional pressure and ISPPA. No attenuation correction was applied for the free-field water calibration beyond the intrinsic frequency-dependent sensitivity correction of the hydrophone (water assumed lossless at 500 kHz).

We took care to remain within the guidelines for human ultrasound exposure as defined by ITRUSST [30]. The results of our acoustic simulations predominantly indicated a maximum skull temperature rise below 2 °C (S1 Table). In cases where this threshold was exceeded, we calculated the CEM43, a metric reflecting both duration and intensity of heating relative to 43 °C, the critical threshold for thermal cell damage. We ensured that CEM43 values remained well below 0 [30]. In our study, CEM43 was always below 0.1. The transcranial Mechanical Index (MItc) was computed as MItc = P_neg,tc/√f, where P_neg,tc is the simulated peak rarefactional (negative) pressure after skull transmission (MPa) and f is the ultrasound fundamental frequency in MHz (0.5 MHz). P_neg,tc values were extracted as voxelwise maxima within a 3 mm-radius spherical ROI centered on the stimulation target, likely to include local maxima.

To improve ultrasound transmission, a layer of ultrasound gel (Aquasonic 100, Parker Laboratories) was applied at the transducer placement site, with a 2 cm gel pad (Aquaflex, Parker Laboratories) positioned between the transducer and the participant’s head. Participants’ heads remained unshaved, and any air bubbles were carefully eliminated by smoothing and combing the hair. Neuronavigation was performed using Brainsight v2.5 (Rogue Research, Montréal, Québec, Canada) with T1-weighted anatomical MR images. During each session, focal depth measurements obtained from Brainsight were entered into the NeuroFUS TPO system (Brainbox, Cardiff, UK) before stimulation, and the trajectory was sampled to support confirmatory acoustic simulations following each session.

The dACC, was defined anatomically as the region anterior to the anterior commissure, posterior to the genu of the corpus callosum, bounded ventrally by the callosal sulcus. We aimed to stimulate just ventrally of the cingulate sulcus such as to engage the area labeled aMCC by Palomero-Gallagher and colleagues [81], with a maximum intensity on region 24c’v. The aIns target was localized in the anterior short gyrus located deep within the Sylvian fissure and bordering the frontal operculum.

Blinding procedure

Sham TUS was administered identically to active TUS, except that no actual stimulation was applied. Instead, participants listened to a sound mimicking the transducer’s pulse repetition frequency through bone conduction headphones, positioned approximately 2 cm posterior and superior to the temples. This sound was only played during the Sham condition to ensure blinding, while headphones were worn in both active and Sham sessions. To create this signal, we generated a waveform matched to the ultrasound protocol in terms of pulse repetition frequency and pulse duration, and we adjusted the sampling rate to approximate the perceptual pitch of the TUS-related sound. Six lab members who had previously experienced the full TUS protocol evaluated several candidate waveforms and selected the one that most closely resembled the real stimulation. This calibration step had to be conducted in a separate group to avoid contaminating expectations in the actual study participants. During the experiment, the audio was played only during the sham condition.

Task

The task consisted of four blocks, presented approximately 15, 25, 35, and 45 min after TUS (Fig 1B and 1C). Each block featured four cues (gems), with each trial requiring participants to determine, through trial and error, the correct action (Go or NoGo) and the potential outcome (reward or punishment) associated with each stimulus. The task design involved four different cues that varied by cue valence (Win versus Avoid) and required action (Go versus NoGo), counterbalanced across participants in three sessions of 320 trials each. Motivationally incongruent cues were included, where participants’ natural action tendencies (e.g., to act for a reward) conflicted with the task requirements (e.g., to refrain from action when a reward was expected). For each cue, correct responses resulted in positive outcomes (80% chance of reward or avoiding punishment), whereas incorrect actions had only a 20% chance of yielding positive outcomes. The valence (Win or Avoid) of each cue was not explicitly signaled and had to be learned over time.

This probabilistic task was designed to capture learning and decision-making under uncertainty, similar to real-world scenarios where outcomes are not guaranteed. The structure of probabilistic feedback allowed us to observe participants’ flexibility in adjusting strategies, sensitivity to feedback, and the cognitive and neural processes underlying decisions when faced with motivational conflicts, such as balancing exploration versus exploitation in uncertain environments.

Stimuli display

The probabilistic motivational learning task was displayed on an LCD screen (1,920 × 1,080 resolution, 60 Hz refresh rate, 8-bit color depth, RGB, standard dynamic range (SDR)), positioned 0.5 m in front of participants. The experiment was programmed using the Presentation software (Neurobehavioral Systems, Berkeley, CA, USA) and run on a Windows-based computer. Participants responded using their preferred hand by pressing the SPACE bar on a keyboard.

Post-TUS questionnaire

After each TUS session, as well as on the following day, participants were asked to report any adverse effects. They were presented with a list of symptoms and asked to rate the intensity to which they experienced each of them using a 4-point scale (absent, mild, moderate, severe) and whether they thought their experience was related to the stimulation on a 5-point scale (unrelated, unlikely, possible, probable, definite). Symptoms included those reported in [82], to which we added speech problems, vision problems, and muscle tightness of the face or arm. Respondents had the possibility of listing and rating up to two additional symptoms and to provide details for their experience.

For further insights, S1 Table present detailed information regarding the acoustic simulation parameters and the outcomes observed across participants. This includes data on mechanical indices, derated ISPPA values at the focal point, and the incidence of side effects associated with both aIns and dACC stimulation.

Regression analyses

We fit mixed-effects logistic regression models using the lme4 package in R. We always included a random intercept per subject and all possible random slopes and random correlations, achieving a maximal random effects structure [83]. We used sum-to-zero coding for all categorical predictors. We obtained p-values from Wald chi-squared tests using Type-3 sums-of-squares via the Anova() function from the car package.

Computational modeling and model comparison

We fit a series of increasingly complex computational reinforcement learning models to participants’ choices, following previous studies [8–10]. All models are Q-learners that learn the expected value of performing an action for a given cue using reward prediction errors and select actions with higher expected value. On a given trial t, the base model M1 computes choice probabilities for Go and NoGo actions (a) given a cue (s) using action weights (w; modified Q-values) turned into probabilities via a softmax transform:

(1)

For each action, the model obtains an outcome r and computes a reward prediction error as the difference between the outcome r and the expected value Q(a, s). It then uses prediction errors to update Q-values via the delta learning rule:

(2)

Outcomes can be +1 (reward) or 0 (no reward) for Win cues, or 0 (no punishment) or −1 (punishment) for Avoid cues. In this learning rule, prediction errors are scaled by the learning rate ε, with large learning rates leading to faster updating (and a stronger recency bias in value estimates) and small learning rates leading to slower updating (and a more long-lasting impact of outcomes obtained far in the past). Furthermore, outcomes themselves are scaled with a participant-specific feedback sensitivity parameter ρ, which plays a similar role as an inverse temperature parameter: high values of ρ lead to Q-values for Go and NoGo actions developing further apart with learning and leading to more deterministic choices, while small values of ρ lead to Q-values closer together and thus more stochastic choices. We initialized Q-values to the midpoint between the two possible outcomes of each cue (+0.5 for Win cues, −0.5 for Avoid cues) multiplied with participants’ respective feedback sensitivity parameter such that starting Q-values are the midpoint of each participant’s subjective value space. Given that the valence of each cue (Win/Avoid) was not instructed but had to be inferred from the attainment of valenced outcomes (rewards/punishments), participants could not interpret neutral outcomes as positive/negative feedback before having observed such a valence outcome. During this phase, Q-values were updated based on feedback, but “muted” in the action selection process (multiplied with zero when computing choice probabilities), assuming that participants could retrospectively infer the meaning of neutral outcomes once they determined the valence of a given cue. Upon the first encounter of a valenced outcome (reward, punishment), Q-values for this cue were “unmuted” and subsequently used for action selection.

We extended this base model using several bias terms. In M2, we added a Go bias term to the Q-values of making a Go response for each cue, capturing participants’ overall propensity of making a Go response:

(3)

In M3, in addition to the Go bias, we added the response bias π multiplied with the cue valence V (+0.5 for Win; −0.5 for Avoid; value of 0.5 arbitrarily chosen for scaling) to the Q-values of making a Go response:

(4)

Note that the cue valence was fixed and participants instructed that each cue was either a Win or an Avoid cue. Hence, no incremental learning of the cue valence was required, but the valence could be inferred as soon as the first valenced outcome was encountered. Similar to early learning from neutral outcomes, the response bias was muted (multiplied with zero) up until the first encounter of a valenced outcome and only subsequently unmuted.

In M4, instead of a response bias, we added a learning bias κ, capturing participants’ tendency to use an increased learning rate for rewarded Go actions (i.e., attribute rewards to their own actions) and a decreased learning rate for punished NoGo actions (i.e., an unwillingness to attribute punishments to own inactions):

(5)

Since learning rates were sampled in a continuous space and later inverse-logit transformed to constrain them to the range of [0, 1], the impact of κ might be asymmetric after the transformation. To achieve a symmetric impact of κ, we first determined whether the base learning rate ε₀ was smaller or bigger than 0.5, then computed one half of the bias (i.e., the side closer to the range limits), took the difference between the base and the biased learning rate, and used this difference to compute the other half of the bias:

(6)

(7)

In M5, we added both the response bias π and the learning bias κ, which provided a better fit than each bias in isolation.

In M6, to capture participants’ overall tendency to repeat responses irrespective of the received outcome, we extended M5 by adding an intercept persistence parameter φ_INT to the weight of the action performed on the last encounter of the same cue:

(8)

Finally, in M7, to capture participants’ higher tendency to repeat responses for Win than for Avoid cues, we added the persistence bias φ_DIFF, which was added to the weight of the last performed action in case of a Win cue and subtracted from this weight in case of an Avoid cue:

(9)

We fitted models using weekly informative hyperpriors to , , in line with previous studies [9,10]. We constrained feedback sensitivities ρ to be positive using the exponential transform and learning rates ε to the range [0, 1] using an inverse the inverse-logit transform. Furthermore, we constrained κ and φ_DIFF to be positive in line with the hypothesized direction of the learning and persistence biases (Fig 2B and 2C) using the transform, which is y = 0 for negative numbers, smoothly asymptotes 0 for small positive numbers, and is roughly y = x for large positive numbers. Leaving these parameters unconstrained led to the same qualitative conclusions, but a slightly inferior fit. In line with recent recommendations [84], we fitted the data of each sonication condition separately.

We fitted all models using the CBM toolbox in MATLAB [85] using three steps: In the first step, we fitted the respective model to each participants’ data individually using an iterative expectation-maximization algorithm that uses the Laplacian approximation for estimating the parameter posteriors. Chances of getting stuck in a local minimum were mitigated by using multiple parameter initialisations. In a second step, we used the individual fits of all participants as the starting point in a hierarchical Bayesian inference procedure. Group-level parameters were iteratively fitted using mean-field variational Bayes and served as hyperpriors for the individual participants’ parameters. Thus, the parameter values of each participant were constrained/informed by the parameter values of all other participants, leading to more robust estimates and a lower chance of overfitting. We used these parameter values for all our posterior predictive checks. Finally, in a third step, we fitted all candidate models for all participants in one single step by iteratively (a) determining the probability that a given participant X was best fitted by a given model Y and (b) fitting the group-level parameters based on (only) those participants whose behavior was in fact best characterized by model Y (i.e., parameters weighted proportionally to participants’ latent “model responsibility” weights). This approach allows for simultaneous hierarchical Bayesian inference (with participants constraining each other’s parameter values) and random-effects model selection (allowing for the possibility that different participants are best characterized by different models). Based on this last step, we computed the model frequency and protected exceedance probability of each model and performed Bayesian model selection [37], selecting the model with the highest exceedance probability as the winning model.

We compared parameters across conditions using one-way repeated-measures ANOVAs as implemented in the ezAnova() function of the ez package in R. We computed paired t-tests using the t.test() function from the stats package in R and computed Hedges’ g as a metric of effect size by first computing Cohen’s d (mean of the condition difference divided by its standard deviation) and then multiplying it with the adjustment factor ((with N being the number of participants). We computed robust bootstrapped confidence intervals by, for 10,000 iterations, sampling N times from the condition difference vector (with replacement), computing Cohen’s d per iteration, and then extracting the 2.5% and 97.5% percentiles across iterations (and multiplying them by J).

To validate that our best fitting model M7 could reproduce key patterns observed in the empirical data, we performed posterior predictive checks [86,87] using one-step-ahead predictions [88]. We used each participant’s best fitting parameters from the hierarchical Bayesian fit and their actual choices and outcomes from the empirical data to simulate the action probabilities of each participant on each trial, which we used as synthetic data to then plot the proportions of Go response/response repetitions the same way as the empirical data.

To confirm that we could reliably measure individual differences in model parameters, we performed parameter recovery [89,90]. We first fitted a multivariate normal distribution to the best fitting parameter values of M7, sampled 1,000 new combinations of parameter values from this distribution, simulated new data based on this parameter combinations, and fitted M7 to each simulated data set. We then correlated the sampled “ground-truth” parameters to the fitted “recovered” parameters. To test whether these correlations were significantly higher than expectable by chance, we compared them against a permutation null distribution obtained by randomly permuting the assignment of ground-truth to recovered parameter values 10,000 times and computing the 95th percentile of this distribution as the upper bound of a one-sided confidence interval.

To confirm that we could reliably detect the best fitting model for a given data set, we performed model recovery [89,90]. For each of the seven candidate models, we fitted a multivariate normal distribution to the best fitting parameter values and then sampled 1,000 new parameter value combinations for each model (in total 7,000 parameter sets). We applied the following parameter constraints: ρ < 400 (otherwise possibility of infinitely large numbers), ε > 0.05 (otherwise too little learning), and ρ, π, κ, φ_INT, and φ_DIFF being far enough away from zero (discarding the lowest 10% of absolute sampled parameter values). These constraints were important to ensure that each data set expressed characteristics of the respective generating model; otherwise, if one of the bias parameters was too close to zero, a more complex model would effectively reduce to a simpler model. We then simulated a new data set for each parameter combination (in total 7,000 simulated data sets) and fitted each of the seven models to each simulated data set (in total 49,000 fits). For each data set, we used the log model evidence to determine which model fitted the data best. We then computed the forward confusion matrix with the conditional probabilities (relative frequencies) of model Y emerging as a best fitting model for a data set generated by model X. We also computed the inverse confusion matrix with the conditional probabilities of model X being the generative model for a data set best fitted by model Y. To test whether the on-diagonal probabilities of these matrices were significantly higher than expectable by chance, we created a permutation null distribution by randomly permuting the log model evidences for the different fits to a given data set, computing the respective confusion matrices, and saving the on-diagonal probabilities. We again computed the 95th percentile of these distributions as upper bounds of one-sided confidence intervals.