Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making
Fig 3
Novelty in episode 1 of block 1.
A. The number of state visits (left panel) and novelty (right panel) as a function of time for one representative participant: The number of visits increases rapidly for the trap states and remains 0 for a long time for the states closer to the goal. Novelty of each state is defined as the negative log-probability of observing that state (see Eqs 1 and 2) and, hence, increases for states which are not observed as time passes. The first time participants encounter state 7 (the state before the goal state) is denoted by t*. B. Average (over participants) novelty (color coded) at t*: Novelty of each state is a decreasing function of its distance from the goal state.