Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning
Figure 3
Geodesic representation required for learning when value function has sharp discontinuities in Euclidean space.
(A) Each column displays the gridworld configuration whereby individual squares are discrete states, thick black lines are walls, and the star indicates the goal state with reward of 1. (B) Each column shows performance measured as the mean number of steps to goal, over 10,000 runs for the environment in the corresponding column in A. The width of each line occupies at least the 95% confidence interval on the means (range 3.2–4.5 steps). Notice that the collapse of learning, present in the Euclidean grid cells (labeled euc) and place cells (blue and red), is recovered by their geodesic counterparts (labeled geo, yellow and green, respectively).