Figure 1.
Euclidean spatial generalization benefits learning in simple navigation tasks.
(A) Each column displays the gridworld configuration whereby individual squares are discrete states, thick black lines are walls, and the star indicates the goal state with reward of 1. (B) Each column shows performance measured as the mean number of steps to goal over 10,000 runs for the environment in the corresponding column in A. The width of each line occupies at least the 95% confidence intervals on the means (range 3.9–4.4 steps). Within a given gridworld the different colored lines represent different basis sets with black for tabular, blue for grid cells, and red for place cells.
Figure 2.
Qualitative comparison of learned value functions using tabular, Euclidean grid cell, and Euclidean place cell bases.
In each figure A–C, the column titles indicate the representation used to learn the value functions for a given gridworld configuration, and each row corresponds to an environment. White lines are walls, discrete squares indicate states, and the gray scale from dark to light indicates low to high value, respectively. To ease comparison between spatial representations within a given gridworld, the image brightness was normalized with respect to the optimal value function. (A) Snapshot of value representation after 15 learning trials. (B) Snapshot of value representation after 25 learning trials. (C) Snapshot of value representation after 50 learning trials. Notice that for both grid cells and place cells, the value representation bleeds across walls, indicated by red arrows where the estimated value is too low (relative to ground truth) on the side of a wall nearer a reward or too high on the far side.
Figure 3.
Geodesic representation required for learning when value function has sharp discontinuities in Euclidean space.
(A) Each column displays the gridworld configuration whereby individual squares are discrete states, thick black lines are walls, and the star indicates the goal state with reward of 1. (B) Each column shows performance measured as the mean number of steps to goal, over 10,000 runs for the environment in the corresponding column in A. The width of each line occupies at least the 95% confidence interval on the means (range 3.2–4.5 steps). Notice that the collapse of learning, present in the Euclidean grid cells (labeled euc) and place cells (blue and red), is recovered by their geodesic counterparts (labeled geo, yellow and green, respectively).
Figure 4.
Example geodesic transformations of grid cells and place cells.
(A) Geodesic coordinates for different environments. (B) Single grid-cell using respective geodesic coordinates. Each grid cell generated using the same spacing, orientation, and relative spatial phase. (C) Single place-cell using respective geodesic coordinates. Each place cell generated using the same mean and variance.
Figure 5.
Qualitative comparison of learned value functions using tabular, geodesic grid cell, and geodesic place cell bases.
In each figure A–C, the column titles indicate the representation used to learn the value functions for a given gridworld configuration (denoted by row). White lines are walls, discrete squares indicate states, and the gray scale from dark to light indicates low to high value, respectively. To ease comparison between spatial representations within a given gridworld, the image brightness was normalized with respect to the optimal value function. (A) Snapshot of value representation after 25 learning trials. (B) Snapshot of value representation after 25 learning trials. (C) Snapshot of value representation after 50 learning trials. In contrast to Euclidean bases, the geodesic representation does not smear value across walls but instead tracks around them.
Figure 6.
Example of geodesic place cell model qualitatively capturing recorded place cell data.
(A) Data adapted and replotted from [64]. Light blue shows presence of rat, red & yellow indicate action potentials of a two hippocampal place cells, and dark blue are areas rat did not visit. (B) Simulated geodesic place cell firing fields roughly resemble data in A (left and middle). The black to white color scale represents low to high firing rates.
Figure 7.
Geodesic spatial representation models can also account for the disappearance of place fields when a wall bisects the firing field.
Muller & Kubie [65] observed that place fields disappeared when bisected with a wall. (A) Graphical intuition of the effect adding a wall has on the coordinates and hence the place cell firing properties. In the left panel are 20 by 20 evenly sampled points in an open square environment. Shown in the right panel are the geodesic transformed coordinates for a 20 by 20 state environment when a single vertical barrier bisects the middle section of the gridworld. Underlying each of the coordinates is a model place cell's firing field in Euclidean space (low to high firing represented by dark to light grayscale). (B) Left panel shows an open field place field, while right panel shows a geodesic place field for coordinate shown in A. Both simulated firing fields used the geodesic place cell model. Compare to Figures 8 and 9 in [65].
Figure 8.
Example of geodesic place cell model qualitatively capturing recorded place cell data.
(A) Two example environments used in [66]. Maze on the left was used for training & exploration and maze on the right was used for testing whether the rat learned to take the shortcut route. (B) Geodesic embedding of mazes shown in A. Underlying each of the coordinates is a place field. (C) Example place field computed using coordinates shown in B; the place field center and half-width was the same in each condition. The geometric distortion in the coordinates introduced the wall can lead to increased activity in the geodesic place cell model.
Figure 9.
Example of geodesic grid cell model qualitatively capturing recorded grid cell data.
Derdikman et al. [24] recorded while a rat explored a hairpin maze and observed fractionated grid cell firing fields that were phase locked to alternating arms of the maze. Shown is an example geodesic grid cell firing field for a similar hairpin maze that resembles that used in [24]. The black to white color scale represents low to high firing rates.