Effects of subclinical depression on prefrontal–striatal model-based and model-free learning
Fig 1
Markov decision task structure.
We use the two-stage Markov decision task proposed by Lee et al. (2014). (a) Two-stage Markov decision task. In each stage, participants make a binary choice (left or right). After the first choice in the initial state (S1), they were moved forward to one of four states in the second stage (S2 or S3 when making the ‘left’ choice, S4 or S5 when making the ‘right’ choice) with certain state-action-state transition probability p. The transition probability (0.5, 0.5) and (0.9, 0.1) corresponds to a high-uncertainty and a low-uncertainty environment, respectively. The task consists of the two goal conditions: a specific-goal and a flexible-goal condition. In the specific goal condition, subjects can collect coins (redeemable for monetary reward) only if the coin color matches with the color of the token box (red, blue, yellow). In the flexible goal condition indicated by the white token box, all types of coins are redeemable. (b) Illustration of the task. Each state is associated with a specific fractal image. When the initial state is shown on the screen, participants make a binary choice, and by doing so they proceed to the next state; the state transition follows the state-transition probability. In the next state, the participants make a binary choice to get to an outcome state. Each outcome state is associated with a specific coin (red, blue, yellow). In the specific-goal condition, participants collect the coin only when the color of the coin matches with the color of the given box, whereas in the flexible-goal condition, they collect coins regardless of its color.