Adolescent and adult mice use both incremental reinforcement learning and short term memory when learning concurrent stimulus-action associations
Fig 6
Model comparison and validation.
(A) We examined 4 parameters that isolated different ways mice might use 1-trial back information (S1-S4). Later in the best-fitting model, S2 and S4 were collapsed to one parameter to reduce complexity. Each left cartoon indicates the stimulus and the mouse’s action at time t and the cartoon on the right reflects the choice options at time t+1 that a mouse has when presented with either stimulus. The plus/minus sign reflect the positive/negative direction of the specific strategy parameter and the name of each strategy parameters describes the positive value of the parameter at time t+1. (B) The winning model a0bs1232 (where a indicates a free α+ parameter, 0 indicates that α− = 0 is fixed, and s1232 indicates three free S parameters with S4 = S2) was compared to other a0b models with various combinations of strategy parameters (s) or no strategy parameters (nos) as well as a model leaving the negative learning rate free (aabs1232). All competing models built from basic RL model described in Materials and methods. The winning model had an average BIC of 859.4, not significantly different from the full model a0bs1234 (BIC = 857.6191). However, a Bayesian model comparison analysis showed that a0bs1232 was significantly more frequent across sessions (C; posterior frequency 39.74% vs. 26.11% for a0bs1234), with a protected exceedance probability (pxp, [12]) of 0.991 (D), confirming that it was the best model at the group level. (E-F) Example logistic regression. To validate our winning model we analyzed simulated behavior from parameters fit on each individual and session with the same logistic regression approach. We found that the simulated behavior captured well the impact of repeat, reward, their interaction, and the effect of reward history on a session by session basis, as shown by the group level effect (E-F) and tight correlations between mouse behavior and simulated behavior using fit parameters for regression #3 (sim; G-K).