Introduction

Freely available supplements that promise enhancing effects on cognition are on the rise. Yet, if and how dietary supplements may affect cognitive function and performance is still poorly understood. Tyrosine (TYR) is the precursor of catecholamine neurotransmitters and is thought to facilitate the synthesis of dopamine, norepinephrine, and epinephrine [6]. Cognitively demanding activities and stress increase catecholamine demand, which in turn leads to a decrease in TYR plasma availability [41,47,48]. Thus, additional intake of TYR is assumed to provide the necessary resources to reach and maintain optimal performance under demanding conditions [41,101].

Such claims seem to be supported by findings on enhancing and protective effects of TYR on working memory (WM) and executive control (especially under conditions of heightened demand; [17,33,41]). Nonetheless, despite the central role of the catecholamine neurotransmitter dopamine (DA) in reinforcement learning (RL) and decision-making [60,64,83, Westbrook et al., 2025,90], comparatively little is known about potential TYR-related changes in these functions. An earlier report suggested that TYR may improve some aspects of decision-making [51], but this study was restricted to male participants, thus limiting generalizability. TYR effects on RL and decision-making are of considerable relevance, however, as both aberrant DA-functioning and RL deficits have been linked to several psychiatric disorders and maladaptive behaviours on the subclinical spectrum [2,12,16,32,55,56,74,95]. This resonates with the approach taken in the field of computational psychiatry, which investigates neurocomputational components underlying mental disorders [86,39,55,99,102]. Especially the balance between more elaborate and cognitively costly goal-directed choice mechanisms on the one hand and simpler, less flexible habitual choice mechanisms, on the other, have emerged as highly relevant. Imbalances between these two broad systems are present across a range of disorders [32,34,36,39, Huys et al. 2021; 55,86], and initial evidence points towards an impact of TYR [51].

A central problem in RL is the tradeoff between exploration (sampling novel options for information gain) and exploitation (selecting known options for reward maximization) [2,15,56,98]. Dynamically changing environments require a balance of these strategies [28,36,98], since both overreliance on exploitation and excessive exploration can lead to performance decrements [24,29,73]. Exploration can be further subdivided into (at least) two component processes: random vs. directed exploration [Gershman, 2018; 2019; 98,96]. While random exploration is characterized by choice randomization, directed exploration is linked to a strategic exploration of uncertain options for information gain [26,29,36,96]. This can be formalized by including an uncertainty-based “exploration bonus” in formal RL models [74,73,95,12,96,19,20,84].

Several mental disorders and subclinical variations thereof have been reliably associated with aberrant explore-exploit behaviour (e.g., [2,5,24, Gershman 2018,32,95]. Likewise, manipulations of the DA system have been linked to changes in exploration [12,16,23,24,43] and conceptually closely-related model-based control [12,51]. Another factor potentially contributing to dysregulated choice behaviour is perseveration [27,57,92]. Low or high levels of perseveration may be associated with exploitation and exploration, but choice repetition may also occur independent of value and information gain [53]. Such value-free perseveration is more closely related to habitual responding, which is linked to various mental disorders [35, Waller et al, 2012,86]. Many computational models account for perseveration by including a term reflecting repetition of the previous choice (trial t-1). More recent approaches have extended this to also account for value-free perseveration beyond trial t-1 (i.e., accounting for the impact of longer choice histories) [9,10, Gershman, 2020,53].

However, the specific role of TYR in regulating exploration is currently unclear.

To address this issue, we preregistered a counterbalanced, placebo-controlled double-blind within-subject study to assess the impact of a single 2g dose of tyrosine (vs. placebo) in a gender-balanced sample. We opted for a comparatively large sample size (n = 63 in total), considering mostly small samples sizes prevalent in the supplementation field. The preregistration, model files, and data sets can be found at https://osf.io/4z39r/files/osfstorage.

Based on previous research by Mathar et al [51], we formulated the following hypotheses. The first hypothesis (H1) states that we expect a reduction in physiological arousal, namely heart rate (HR) and variance of pupil dilation (PDV) following TYR supplementation. H2 regards the results of another task not presented here. The third hypothesis (H3) includes a) an expected reduction in directed exploration and b) a reduction in RTs under TYR, as well as c) a reduction of DDM parameter α (threshold) under TYR.

Results

Model-free results

Descriptive statistics of model-agnostic performance indices separated by factors gender (female vs. male) and condition (PLC vs. TYR) are shown in Fig 1 (see S1 Table). Generally, participants performed above chance, choosing the option associated with the highest reward in 66.57% (SD = 6.39; Fig 1A) of the trials, which earned them an average payout of 19,000 points per session (SD = 1033; i.e., around 63.36 points per trial). Choice data also contained evidence for exploration behaviour, as participants switched to a different option in around 30% of trials (Fig 1B). Median RTs were around.3 seconds (Fig 1C).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Model-Agnostic Performance Measures Separated by Condition and Gender Group.

Supplementation conditions PLC vs TYR are shown in turquoise and magenta, respectively. Dots depict individual subjects. Shading in the lower panel plots mark gender groups (light = female; dark = male). A: percent of choices for the highest-value option. B: proportion of choice alterations (vs. repetitions). C: median reaction times in sec. * p < 0.05 ** p < 0.01 *** p < 0.001.

https://doi.org/10.1371/journal.pcbi.1014356.g001

Trial-wise optimal choices (i.e., choice of the highest rewarded bandit) were increased under TYR vs. PLC) (Table 1; Fig 1A, main effect of condition p = .019), whereas the condition x gender interaction was not significant (Table 1, p > .5).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Results from Regression Analyses of Model-Free Performance Measures.

https://doi.org/10.1371/journal.pcbi.1014356.t001

Switching was significantly reduced under TYR (Table 1, Fig 1B, main effect of condition p < .001), and lower in men vs. women (Table 1, main effect of gender p = .029). The regression on trial-wise RTs also revealed a small increase in RTs under TYR (Table 1) that was in the opposite direction compared to previous RT findings in two other tasks (see [51]).

To gain a basic insight into the influence of past choices on current ones (i.e., perseveration behaviour), we set up a simple regression model, testing the extent to which present choices can be predicted based on choices on the previous four trials [S1 Fig, c.f. Lau & Glimcher, 2005; 53]. All lags indeed showed significant effects on current choices, illustrating a model-agnostic counterpart to the above introduced model-based HOP (i.e., value-free perseveration). Previous choices further remained significant predictors for current choices when additionally including previous rewards in this model (S2 Fig).

Step 1: Model comparison & selection

Following our preregistration, model comparison was performed separately for each supplementation condition. This was done to clarify whether the manipulation has an influence on model ranking, which was not the case (Table 2). The BL+Bandit model provided the best fit to the data across both conditions (Table 2) and was therefore used for the in-depth analysis of model performance and condition effects on parameters.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Model Comparison Results Using Leave-One-Out Cross-Validation. BL = Bayesian Learner; QL = Q-Learner; turquoise = PLC; magenta = TYR; elpddiff = difference in estimated log pointwise predictive density; sediff = standard error of the difference.

https://doi.org/10.1371/journal.pcbi.1014356.t002

The posterior distributions of group-level parameters derived from fitting data from the PLC and TYR condition and corresponding summary statistics of the difference distributions (TYR-PLC) are provided in the supplement (S5 Fig, S4 Table).

To further validate the winning model, we performed a parameter recovery. All parameters showed high correlations between the true and fitted estimates (subject-level parameters: S6 Fig, S5 Table; group-level parameters:S7 Fig).

Step 2: Modelling of supplementation effects

We next examined a combined model that directly included condition-dependent changes in each parameter due to the supplementation. Here, for each parameter, the PLC condition was modelled as the baseline, and changes in each parameter from PLC to TYR were modelled via additional parameters modelling condition effects (“shift” parameters). Thus, resulting posteriors reflect the shift in each parameter from PLC to TYR and directly reflect the effect size in units of that parameter.

Parameters reflecting directed exploration (Fig 2B) and perseveration (Fig 2C) exhibited an estimated change largely centered around 0, reflecting the fact that these choice mechanisms were largely unaffected by TYR supplementation. In contrast, the beta parameter reflecting choice stochasticity (i.e., exploitation) was markedly increased under TYR (Fig 2A; 95%HDI ≥ 0, see Table 3). This dovetails with the model-agnostic results presented earlier: The increase in choices for the highest value bandit and the decrease in switching behaviour can both be attributed to an increase in choice consistency (i.e., an increase in exploitation) under TYR. In contrast, there was no credible evidence for a TYR modulation of directed exploration or perseveration.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Summary of TYR-Effects on Model Parameters.

https://doi.org/10.1371/journal.pcbi.1014356.t003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Posterior Distributions of Shift-Parameters Depicting Supplementation Effects. ß: shift (i.e., change) parameter of the SoftMax parameter for value-based choices (i.e., exploitation).

Φ: shift of the parameter depicting directed exploration; ρ: shift of the parameter depicting perseveration (choice repetition from t-1). Dashed lines mark a value of 0; percentages show the proportion of the posterior estimates above 0.

https://doi.org/10.1371/journal.pcbi.1014356.g002

To complete the current modelling step, we examined the association of TYR-related changes reported in the section on model-agnostic results with the “exploitation” parameter ß. Correlation analyses (S8 Fig) confirmed that the TYR-induced increase in exploitation (ß) showed the predicted associations with model-agnostic behavioural measures. Higher exploitation was associated with a lower switch rate, and a greater proportion of optimal choices, in both genders.

Step 3: Investigation of gender differences

We next leveraged our comparatively large and gender-balanced sample to examine potential gender-differences in TYR effects. Due to the sparse research on gender-specific behavioural and neurocomputational changes under TYR, we did not formulate specific a priori assumptions on these effects. Here, we applied a shift version of the best-fitting model (c.f. Step 2) separately to data from self-identified male and female participants. To examine overall gender effects, we first examined the baseline (PLC condition) parameters per gender (Fig 3). There was no credible evidence for gender effects on any parameter (Fig 3A, 3B & 3C, Table 4). This was further confirmed by a paralleled lack of gender differences in TYR-effects on these parameters (Fig 3D, 3E & 3F).

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Summary of The Posterior Difference-Distribution Between Gender Groups. Parameters denote the group-level means’ difference distribution (fem-mal) of corresponding SoftMax parameters.

https://doi.org/10.1371/journal.pcbi.1014356.t004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Posterior Distributions for the PLC Condition Separated by Self-Identified Gender Group.

A-C: ß: parameter for value-based choices (i.e., exploitation). Φ: directed exploration; ρ: (first-order) perseveration; D-E: TYR-effects on respective model parameters. light and dark gray:results from female and male subsample respectively; bars below the distributions depict the 95% HDI for each group. X-Axis: value estimate of parameter; y-axis: density of the posterior distribution.

https://doi.org/10.1371/journal.pcbi.1014356.g003

Nonetheless, some patterns in the data deserve mention. Numerically, compared to the male sample, female participants showed less exploitation (stickiness) and more exploration. These differences may in combination have contributed to the significant gender effect we observed in the model-free analysis of switching behaviour (c.f. section Model-Free Results; Table 1; Fig 1).

Step 4: Reliability of parameter estimates (Multivariate priors)

Leveraging the comparatively large sample size, we next re-formalized the best-fitting model to obtain a Bayesian estimate of test-retest reliabilities on all model parameters. The hierarchical model was adjusted such that parameters were drawn from bivariate Gaussian distributions across the two supplementation conditions, allowing us to obtain Bayesian estimates of parameter covariance matrices to compute test-retest reliabilities of model parameters. Fig 4 shows the posterior estimates of test-retest correlations for exploitation (β), directed exploration (Φ), and perseveration (ρ). Group-level means for Φ and ρ exhibited high consistency across conditions (r(Φ)=0.59, r(ρ)=0.65), in line with the finding that these parameters were largely unaffected by supplementation. For exploitation (β), on the other hand, the test-retest reliability posterior distribution was symmetrically centred around 0 (r(β)=-0.02), consistent with a considerable change due to the TYR supplementation. More generally, these findings are in line with the idea that exploitation (β) predominantly reflects state-dependent effects, whereas directed exploration (Φ) and perseveration (ρ) also show considerable within-subject stability. These reliability analyses suggest some degree of consistency of parameter estimates across the two supplementation conditions, at least for rho and phi. However, in the present supplementation design, reliability estimates are jointly affected by psychometric properties, and potential condition effects (state-dependency). These estimates should therefore not be taken as “pure” estimates of test-retest reliability.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Posterior Distributions of Bayesian Test-Retest Reliabilities.

Group-level posterior of the correlation matrices of multivariate model variant (Step 4), showing the test-retest reliabilities (PLC vs. TYR) A: β_corr, value-based choices (i.e., exploitation); B: Φ_corr, directed exploration; C: ρ_corr, perseveration; dashed lines mark x = 0; bars below the distributions depict the 95% HDI; x-axis: value estimate of parameter; y-axis: density of the posterior distribution.

https://doi.org/10.1371/journal.pcbi.1014356.g004

Step 5: Posterior predictive checks

In a final step, we examined performance of the best-fitting model using posterior predictive checks by simulating choice data based on draws from each participant’s posterior distribution. We deviated slightly from the preregistered approach (which outlined simulating 500 artificial data sets based on the joint posterior group-level parameters) and examined 8000 simulated data sets per subject using the best-fitting model BL + Bandit.

Each simulated data set was based on a subject-specific parameter combination drawn from the participants’ posterior distributions. Each simulated data set was compared to the respective observed data. Median correct choice predictions separated by supplementation condition are shown in Fig 5A. Prediction accuracy was at a similar level for both conditions and well above chance level (PLC: Mdn = 66.31%; TYR: Mdn = 67.54%). We next evaluated, for each participant, each of the 8000 full simulated data sets using model-agnostic measures (% best choice, %switches) and averaged these predicted scores across simulations. These largely reproduced the observed behavioural data in both conditions and both for % best choices (Fig 5B) and switch rate (Fig 5C), although the latter was slightly overestimated by the model. Taken together, posterior predictive checks confirmed that the best-fitting model reproduced key patterns in the data.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Analysis of Simulated vs. Observed Choice Data.

Turquoise = PLC, Magenta = TYR, Points represent participants. A: predictive accuracy calculated as the proportion of correct choice predictions by the winning model compared to observed choices. Panel B & C: Proportion of highest-value choices (%best) and choice alterations (%switches) separated by condition. Observed = empirical data (c.f. model-agnostic results); simulated = predicted choice data based on the winning model.

https://doi.org/10.1371/journal.pcbi.1014356.g005

Step 6: Exploratory modelling

6.A. Higher order perseveration.

We ran additional, non-preregistered modelling analyses, based on previous results that perseveration behaviour may extend across longer choice sequences [higher-order perseveration; 53,10,84,62]. As mentioned above, we consequently set up another refined version of model BL + Bandit in which the indicator function depicting FOP was replaced by the mechanisms suggested by Miller et al [53], i.e., a habit vector which tracks the choice history for each bandit (Gershman, 2020; [53]) (see Methods section, S1, S2 & S9 Figs).

This extended model (Habit; Table 5) was only applied to the previous BL model variants, which outperformed all QL models; (c.f. Table 1), and we focused on the base model and exploration models BL+Bandit & BL+Sigma. Model comparison favored model variants accounting for HOP, with the extended BL+Bandit model showing the best fit across both conditions.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Model Comparison Results via Leave-One-Out Cross-Validation. BL = Bayesian Learner; Habit = HOP extension (vs. FOP); turquoise = PLC; magenta = TYR; elpd_diff = difference in estimated log pointwise predictive density; se_diff = standard error of the difference.

https://doi.org/10.1371/journal.pcbi.1014356.t005

As done previously for the FOP version, we also performed a parameter recovery analysis to validate this model version, following the same procedure as before. This yielded satisfactory correlations between true and recovered model parameters (S6 Table, S10 and S11 Figs).

Model recovery showed that for the FOP data sets, both models’ fits were indistinguishable, the HOP model provided a superior fit across all five simulated data sets based on this model (S7 Table).

However, regarding the interpretation of model recovery results, please note that the FOP model is nested in the HOP model (i.e., setting α_HOP to 1 yields the FOP formalism). Due to this nested relationship between the models, the HOP model will always be able to fully account for data patterns generated by a FOP mechanism, while the reverse is not the case.

Fig 6 depicts posterior estimates of supplementation effects for the four main parameters of interest (c.f. Results on the BL+Bandit model with the addition of HOP step-size α_HOP). Modelling results of supplementation effects accounting for HOP reproduced the previously observed increase in exploitation (β) under TYR (Fig 6A and Table 6). However, the extended model also revealed credibly reduced parameters for directed exploration as well as perseveration (Φ and ρ, respectively, 95% HDI < 0; Table 6, Figs 6C & 6D) under TYR. Accounting for HOP therefore led to a more fine-grained decomposition of choice-mechanisms, and now revealed an attenuation in directed exploration and perseveration under TYR as well as substantiating the previously moderate effect on exploitation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Posterior Estimates From Step 2 (Shift Model) of the Adapted BL + Bandit + Habit Model; β_shift = effect on value-based decision making; Φ_shift = effect on directed exploration; ρ_shift = effect on perseveration (HOP).

https://doi.org/10.1371/journal.pcbi.1014356.t006

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Posterior Distributions of TYR Effects on Parameters from the “Habit” Model-Extension.

Depicted are the posterior distributions from Modelling Procedure Step 2 based on the BL+Bandit+Habit Model. “Shift”-Parameters indicate the supplementation effect (PLC vs. TYR) on corresponding parameters: A: higher-order perseveration step-size ; B: exploitation ; C: exploration ; B: perseveration . Dashed lines mark a value of 0; shaded areas mark values <0.

https://doi.org/10.1371/journal.pcbi.1014356.g006

Results from fitting the extended model to each condition (S8 Table) and gender group separately (S12 Fig, S9 Table) are provided in the supplement.

6.B. DDM: Stay vs. switch choices.

As preregistered we also conducted a model-based analysis of RT-distributions on the bandit task using a Drift Diffusion Model (DDM). In this model each decision is modelled as a noisy evidence accumulation process toward one of two boundaries. These mark the criterion for a decision, so that crossing either decision boundary is equivalent to the observable point at which a choice is made [Meyers et al., 2022; 68]. In order to dichotomize the four possible responses on the task, we coded each choice on trial t as either a stay (0) or switch (1) relative to the previous choice on trial t-1. This FOP-categorization yielded the decision boundaries for the DDM-analysis [c.f. 37].

Graphic and tabular depictions of posterior parameter estimates per condition and the resulting difference distributions can be found in the supplement (S13 Fig, S10 Table). Modelling of TYR effects on DDM parameters in a combined model indicates that non-decision time () and boundary separation () were largely unaffected. HDIs of the a priori bias (; 95% HDI=[-0.01, 0.05]) and the rate of evidence accumulation (δ_shift; 95% HDI=[-0.45, 0.04]), show more noteworthy deviation from 0 while, however, still not providing credible evidence for an effect (Fig 7, Table 7).

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Posterior Parameter Estimates Derived from a DDM With Stay-Switch Decision-Boundaries. α = boundary separation; z = bias; δ = drift rate; τ = non-decision time.

https://doi.org/10.1371/journal.pcbi.1014356.t007

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Posterior Distributions of DDM-Parameters Depicting Changes Due To Condition.

Group-level shifts in parameter estimates from condition PLC to TYR;A: α = boundary separation; B: τ = non-decision time; C: δ = drift rate; D: z = bias. Decision Boundaries were defined as 0 = stay (i.e., repetition of preceding choice); 1 = switch (choice alteration).

https://doi.org/10.1371/journal.pcbi.1014356.g007

The above reported model-free slowing effect of TYR on median RTs thus, may be driven by contrasting effects: a small shift of the starting point in favor of a choice switch (increased , Fig 7D) paired with a simultaneous increase in the drift toward a choice repetition (decreased , Fig 7C). Recall, however, that this analysis was based on the dichotomization into stay and switch choices, while the referred to main effect of RT-slowing under TYR did not make this distinction.

Thus, overall examination of the combined model including condition effects on DDM parameters (Fig 7, Table 7) revealed no credible evidence for TYR-related effects.

With the aim to provide further insight into the relation between DDM parameters and model-agnostic RT measures, we conducted correlation analyses. Here, we looked at the relationship between the supplementation change in parameters and difference in mean RTs for stay and switch trials (S14 and S15 Figs). The tyrosine-related shift in the drift parameter (Mdn = -0.20;Table 7) showed significant positive association with the difference (TYR-PLC) in mean RTs for switch trials (r = 0.45, p < .001) as well as the difference in proportion of switch choices (r = 0.95, p < .001). These findings are consistent with the overall model specification, such that lower drift rates indicated a trend toward stay decisions (i.e., lower switch rates). Model-agnostic findings of lower switch rates under tyrosine also fall in line with this logic (i.e., stronger drift toward stay-decision under tyrosine is mirrored in behavioural readout).

Discussion

Here we show that a single dose of the catecholamine precursor TYR modulates reinforcement learning and decision-making in a commonly applied RL task with a dynamic reward schedule. Most notably, we found an overall increase in choice consistency and optimal choices under TYR, and both effects were largely independent of gender. Contrary to our preregistered hypotheses, we found no changes in physiological arousal (H1) and slightly increased (vs. decreased) RTs (H3b) following TYR supplementation. Comprehensive computational modelling further revealed that TYR supplementation improved performance by increasing value-based exploitation, in line with the observed increase in optimal choices and a reduction in the switch rate. In this preregistered model, the predicted reduction in directed exploration (H2) was not confirmed. Extending this model variant by also accounting for higher-order perseveration (HOP) substantiated the increase in value-based exploitation. In this model, TYR furthermore reduced directed exploration and value-independent perseveration, effects that were potentially masked in the previous, simpler model variant. This was accompanied by a change in HOP step-size, indicating a reduced integration window of prior choices under TYR. The reduction of directed exploration in the extended model is in line with H3a. In an exploratory DDM analysis of stay vs. switch decisions, we found TYR supplementation to influence subjects’ starting points and drift rates, rather than lead to a reduction in thresholds, as stated in hypothesis H3c.

These results provide insights into the neurocomputational mechanisms underlying TYR supplementation effects on RL performance. Model comparison revealed that behaviour was best accounted for by a model that assumes an uncertainty-dependent learning process (i.e., Bayesian Learner via Kalman Filter). The best-fitting model further included a heuristic-based directed exploration mechanism (i.e., “Bandit Heuristic”, where participants track how many other options were sampled since a specific option was last selected), as in related work [10]. These modelling results are in line with previous studies that also favored Kalman Filter over less flexible and uncertainty-naïve Q-learning variants [see, e.g., 12; Daw et al., 2006; 95]. However, in the present study, directed exploration was better accounted for by a heuristic process, as opposed to the uncertainty estimate of the Kalman Filter model [c.f. e.g., 12; Daw et al., 2006, 29].

Analysis of this preliminary best-fitting model revealed that the performance improvement under TYR was due to an increase in value-driven exploitation (beta parameter of the softmax function) [72,44,1]. Parameters reflecting (heuristic-based) directed exploration and perseveration (FOP) did not exhibit meaningful variation across conditions based on this model variant, although numerical trends pointed toward a reduction in these two components under TYR. Extending this model by accounting for HOP in an additional exploratory modelling step, however, revealed a substantial improvement in model fit. It also substantiated the previously observed increase in value-based exploitation under TYR, whereas TYR-related reductions in directed exploration and value-independent perseveration became considerably more substantial. Our pre-registered hypothesis of a reduction in directed exploration under Tyrosine was thus confirmed only in this extended model, which, however, showed a substantial improvement in model fit. It remains an open question if Tyrosine effects on these different computational processes result from a shared underlying mechanism. In the current study however, Tyrosine-related shifts in parameters depicting HOP step-size, exploitation, and directed exploration exhibited no significant correlations between each other (S11 Table), pointing toward separate mechanisms.

Both the preregistered model and the extended model exhibited good parameter recovery [97]. Model recovery showed that data generated using a higher-order perseveration process where substantially better accounted for by the respective model. In contrast, for data generated by a first-order perseveration process, model fit was similar for both models. This is to be expected in the case of nested models, as the HOP model will always be able to account for a FOP process by setting α_HOP to 1, resulting in the simpler nested FOP model.

This effect of model refinement echoes previous findings of improved granularity upon defining and formalizing related processes [54,97]. For example, Chakroun et al. [12] observed an increase in directed exploration estimates after accounting for FOP, compared to a model variant without a perseveration term [Daw et al., 2006], which dovetails with the effects of the HOP extension in the current study. Specifying previously unaccounted-for processes can thus aid the identifiability and interpretability of other model components by accounting for regularity in the data that would otherwise reflect unexplained variation. Finally, it should be noted that not all parameter changes under Tyrosine necessarily reflect the performance improvement observed for the model-agnostic analyses. For example, both excessive and reduced directed exploration may impair performance, depending on task demands [20]. The model-agnostic performance improvement therefore reflects the net result of overall Tyrosine effects across model parameters.

The work by Chakroun et al. also relates to the present results with respect to the role of DA, as it involved a more direct manipulation of DA levels via a pharmacological intervention with either Haloperidol or Levodopa (LDOPA; vs. PLC). Based on their computational model, L-DOPA reduced directed exploration compared to both placebo and the D2 antagonist Haloperidol. Although a direct comparison is complicated by the different approaches to catecholamine modulation (pharmacological vs. supplementation-based), these results converge with our observation of reduced directed exploration under TYR in the best-fitting model variant. TYR supplementation presumably elicits a more indirect manipulation of DA-levels compared to pharmacological approaches [40,41,49,8,33]. Nonetheless, the observed reduction in directed exploration observed here dovetails with the results by Chakroun et al. [12], while the robust increases in exploitation under Tyrosine across models appear somewhat distinct from the Chakroun et al. [12] results.

Mathar and colleagues [51] followed the same supplementation regime as done here but examined a different RL-task (two-step task; [21]). Although this complicates a direct comparison of results, it may explain contrasting RT effects of TYR in Mathar et al. and the present study. Mathar et al. observed RT reductions under TYR for the two-step task, whereas we observed RT increases for the bandit task, and this might be related to the specific decision problem. The two-step task is more complex due to its sequential, probabilistic architecture, and involves binary decisions on all stages, whereas participants decide between four options on the bandit task. Demands therefore clearly differ between tasks. Across both studies and tasks, however, participants exhibited a higher tendency to repeat vs. alternate preceding actions following the ingestion of TYR, consistent with the idea of choice stabilization. Generally, TYR effects may manifest differently depending on the task demands and performance-related processes.

In contrast to our earlier study [51], we did not observe effects of tyrosine supplementation on any of the physiological measures recorded. Note that this was also the case when analyzing male participants only (S12 Table), which rules out that differences are due to the different gender distributions across studies [51, tested only males]. At the same time, we replicated the findings of Mathar et al. [51] with respect to test-retest reliability of physiological measures (assessed at t0, prior to ingestion of tyrosine or placebo). Reliability was > .70 for all measures, showing that the failure to replicate the physiological effects is unlikely to be due to poor data quality or poor reliability. Since the overall experimental procedures were highly consistent across the two studies, the failure to replicate the previous physiological effects might be due to a false positive effect and/or sample specific effects in the earlier smaller study.

In order to provide more complete perspective on the observed data, we also modelled RT-distributions for choice switches vs. repetitions using a variant of the drift diffusion model (DDM).This analysis, independent of further assumptions regarding learning and valuation processes, revealed a supplementation effect on the drift rate parameter, reflecting a tendency for an increased rate of evidence accumulation (in favour of choice-repetitions under TYR. However, an opposing effect was found for the bias (starting point), which was shifted towards the “switch”- boundary. These contrasting effects may partially explain the small yet significant increase in median RTs under TYR (vs. PLC) observed in our model-agnostic analyses.

The present study focused on effects of a single TYR dose, and did not investigate long-term effects of supplementation. While there are first reports indicating positive long-term effects of TYR on domains such as working memory, and fluid intelligence [45], more exhaustive investigations are clearly required. Such future work would preferably also include more diverse samples. Another aspect that would be of interest in the context of longitudinal studies in light of our results (exploratory modelling using HOP) concerns perseveration and habitual responding. Because a key differentiating factor between perseveration and habits concerns their temporal extension, long-term investigations might shed light on TYR effects on perseveration and/or habitual responding over time. While perseveration can to some extent be measured in a cross-sectional laboratory-based studies, habits are much harder to capture as they have developed over and may persist across more extended time periods [9; see also 11,61].

There are several potential modes of action of a low dosage supplementation regime as implemented in the present study [see, e.g., 41,Jongkees et al., 2020; 8,33]. A common interpretation of TYR effects is based on the assumption that increasing precursor availability (e.g., via supplementation) can increase neurotransmitter synthesis under demanding conditions [Attiepoe et al., 2015;40,41,47], However, individual supplementation effects may depend on individual differences in the DA system [66], which may contribute to mixed effects in the literature. Furthermore, the degree to which catecholamine neurotransmitters beyond DA may be affected by TYR remains unclear. Even when focusing on the role of DA in RL, as probed by, e.g., the restless bandit task applied here, supplementation may affect phasic and/or tonic DA levels [7,30,66] and effects may additionally depend on individual factors such as baseline availability and synthesis capacity [see, e.g., 40,33].

Overall, current findings are in line with recent theories on the role of DA in reinforcement learning and decision-making (Niv et al., 2007; Friston et al., 2012; Mikahel et al., 2021). The theory of Rational Inattention for example, posits that precision in learning and reliance of actions on learned representations (e.g., reward magnitudes) heavily rely on individuals’ dopaminergic functioning. Mikahel and colleagues (2021) state that high (vs. low) tonic DA levels foster learning from rewards (vs. punishment or low reward) and influence action control in favour of exploitation (vs. exploration). Constituent mechanisms here may include a more pronounced contrast in subjective values (due to more precise estimates) and a resulting preference for higher value options (i.e., exploitation over exploration) [see also 89]. Thus, under the assumption that TYR supplementation in the present study modulated DA-tone, our results largely align with these ideas.

Limitations & future directions

Despite the relatively large and gender-balanced sample and comprehensive modelling procedures, several limitations of our approach need to be addressed. Conclusions are limited to western, educated, white and cis-gendered, mostly young and able-bodied volunteers, limiting generalizability. We also did not examine potential dosage effects, but used a widely-used fixed dosage of 2g [see, e.g., 41,81,22,69,51]. Recommendations and applications with regard to TYR-doses vary across previous investigations (e.g., Deijen 2005: 14mg/kg), which precludes direct comparison of findings and may contribute to mixed effects. However, control analyses of model-agnostic effects that included weight as an additional random effect confirmed our results (S13 Table). Nonetheless, even a more direct and rigorous tracking of catecholamine levels would not eliminate variability of TYR ingestion via other exogenous sources, such as dietary choices. While for the current study, we did ask participants to abstain from large, protein-rich meals in advance of testing sessions, adherence to this request could not be directly controlled. It should be noted though, that despite their potential influence on present findings, TYR-related effects found were stable across gender groups as well as age and weight ranges (as narrow as these may have been).

Due to the experimental design and nature of the supplementation procedure, potential dopaminergic and noradrenergic effects cannot be clearly disentangled. However, based on previous research, DA appears to be critically involved in directed exploration. Research using the D2/D3 receptor antagonist amisulpride shows that DA modulates sensitivity to choice-relevant features such as reward values, depletion rates, and opportunity costs [16]. Further studies revealed that DA mediates exploration by changing the precision of value-based choice selection, such that increased DA activity is associated with decreased exploration and vice versa [14]. In line with these findings, Chakroun et al [12] found DA to attenuate directed exploration and modulate neural representations of uncertainty in the insula and dorsal ACC. Noradrenaline, while also regulating exploratory behaviour, seems to be more closely associated with random exploration [91]. When participants receive ß-adrenergic antagonist propranolol, they showed reduced tendency to use value information and switched more while rewards were still high [16]. Additional research confirms that random exploration is attenuated under propranolol but not under DA blockade, demonstrating that this form of exploration is more closely linked to noradrenergic influences [24]. NA activity seems to influence exploration by changing outcome sensitivity, and this effect seems to be sex-dependent [14]. The lack of differences in physiological arousal between conditions PLC and TYR further points to a dopaminergic over noradrenergic effect. While the cardiovascular system may also be influenced by DA, it is primarily under autonomic control and thus, more closely associated with NA effects [31]. Pupil dilation is even more closely related to LC-NA functioning and sympathetic activity, and not primarily DA [3, Joshi et al., 2015; 58]. While the SEBR is commonly used as a proxy for DA function [Jongkees & Colzato, 2016], previous findings are mixed and include reports of missing associations [18,75]. Together, these results may be more compatible with a dopaminergic as opposed to a noradrenergic effect.

Although our results do provide new insights into latent effects of TYR on cognitive subprocesses, our study design only allowed for a cross-sectional snapshot of such effects. Investigations into long-term effects of TYR (self-) administration on (among others) learning and decision-making are called for (see above). This seems especially important as the practical application of such supplements is not limited to a one-time low-dose ingestion. Indeed, between 40–60% of customers in India (and around 45% in the United States) reported daily use of dietary supplements in a recent survey [Rakuthen 67,80].

As outlined previously, the exact neurochemical effects of TYR are still only partially understood, but are thought to be modulated by baseline DA levels (see, e.g., prominent related work by [41] on a potential inverted-U shaped association of DA and executive functions). Consequently, effects of additional exogenous manipulation of catecholamine synthesis (e.g., via supplementation) may elicit different effects depending on such endogenous factors.

Another limitation may lie in the analysis approach applied. Despite their clear advantages and benefits compared to model-free compound measures, cognitive computational models have their own inherent limitations. Model variants considered in this project cover a considerable range of established accounts, but all conclusions remain restricted to the examined model space. For example, even the best-fitting model only accounted for around 67% of choices, on average, such that there is clearly room for further improvements. We wish to address and tackle this issue head-on by making our analysis procedure, including computational details along with explicit model codes available. Other interested researchers are invited to test and tweak modelling accounts presented here, and report on their results obtained in this endeavor.

Conclusion

We investigated the effect of a single dose of the catecholamine precursor tyrosine (2g) on RL and decision-making in a young and healthy gender-balanced sample. Tyrosine improved performance in terms of the percentage of optimal choices, and extensive computational analyses linked this effect to an increase in value-based exploitation. Further model extensions that accounted for higher-order choice perseveration effects [53,10,84,62] substantiated this effect, and further clarified underlying mechanisms by revealing a reduction in directed exploration (resonating with previous pharmacological studies, [12]) and value-independent perseveration under TYR. In a comparatively large and gender-balanced sample, our results reveal robust effects of TYR on neurocomputational processes thought to be under dopaminergic control.

Methods

Procedure

Participants participated in testing at the University of Cologne Biopsychology lab on two separate days. Sessions were spaced 3–5 days apart and took place in the same time slot each for every participant. Prior to participation all subjects gave their written, informed consent. On each testing day we first obtained baseline measures of physiological measures such as skin conductance response, heart rate and pupillometry. Individuals then consumed 200ml of orange juice containing either 2g of dissolved TYR (TheHut.com Ltd.) or PLC (microcrystalline cellulose), followed by a 60-minute waiting period in a separate room (with minimal external stimulating or distracting features; i.e., empty office room). Participants were instructed to abstain from drinking alcohol or consuming protein-rich food from the evening prior and up to each testing day in order to minimize external influences on TYR levels [Deijen, 2005].

Following the 60min waiting period, physiological measures were obtained again, and participants performed a short temporal discounting task (10–15min). Temporal discounting and physiological data were obtained following the procedures of Mathar et al. [51] and were included as a direct replication, which will be reported elsewhere. This was followed by 300 trials of a four-armed restless bandit task [12,95, Daw et al., 2006] with changing option reward values over time (Fig 8). Values changed independently following Gaussian random walks. Participants could gain an additional payout of up to 4€ depending on task performance. Task and supplementation order were randomized and counterbalanced within each gender group and thus, also across the entire sample. At the end of the second testing day participants also completed a number of self-reports and sociodemographic questionnaires not relevant for the present report (see preregistration, S14 Table).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Schematic Depiction of the 4-Armed Restless Bandit Task.

On each trial, participants selected between four choice options marked by different colors. Following each choice, the reward associated with that option was displayed. Rewards varied independently according to Gaussian random walks [95; Daw et al., 2006; 12] and were displayed in the form of hypothetical monetary amounts between 0 and 100€.

https://doi.org/10.1371/journal.pcbi.1014356.g008

Supporting information