When Does Model-Based Control Pay Off?
Fig 9
The influence of reducing the number of second-stage action.
Because of the deterministic transitions, model-based choices in the Doll two-step task always result in the desired state outcome. Combined with increased distinguishability and increased drift rate in the reward probabilities, this task results in a substantial increase in the relationship between planning and reward.