Fig 1.
Tasks and navigation strategies.
A, Schematic of the guidance task. The goal is unmarked, but fixed in space and can be identified in relation to stable landmarks. B, Schematic of the aiming task. The goal position changes frequently and is marked by a visual cue. C, Navigation strategies. Following the same sequence of actions using an allocentric navigation strategy leads to the same final location irrespective of the starting heading direction. However, when using an egocentric navigation strategy the same action sequence will lead to different final locations depending on the initial heading direction.
Fig 2.
Schematic of the Deep-Q network.
A, Visual inputs from the environment are processed by the CNN layers and then to a fully connected layer with 50 units (green circles) from where the spatial representations are analysed and classified. The output layer consists of allocentric (grey circles) and egocentric (red circles) action units in the full model, and only ego- or allocentric units in the constrained models. B, Schematic of the allocentric action space. C, Schematic of the egocentric action space.
Fig 3.
Task demands drive choice of navigation strategy and emerging spatial representations in the full model.
A Example trajectories before and after learning in the full model for guidance and aiming. At the end of training, agents have learned to take direct paths to the goal. Trajectories are smoothed for visualization. B, A clear preference for allo- and egocentric actions emerge for guidance and aiming in the full model, respectively. Dark lines represent means over 15 simulations. Shaded regions represent the standard deviations.
Fig 4.
Stereotypical spatial representations emerging in the guidance and aiming tasks.
A, Sample place cell-like responses in guidance using an allocentric strategy. Larger firing field maps show head-direction averaged responses of the unit, and the six adjacent smaller maps show the corresponding response for individual head directions spaced 60 degrees apart. B, Place fields cover the entire arena, but tend to cluster around the goal location. Left, location of place field centers in the arena. Right, corresponding normalized sum of firing rate maps of all place-like cells. The entropy of the distribution corresponding to the place cell coverage is H = 9.035 (for a uniform distribution H = 9.288) C, Sample vector-like representations in the aiming task using an egocentric strategy from our model, showing tuning to both the egocentric direction (ECD) and distance to the cue that marks the goal. D, Left, Two sample vector-like representations in the aiming task using an egocentric strategy from our model. Right, experimentally observed representations of ECD in mice navigating to cue lights in the environment (reproduced with permission from [18]).
Fig 5.
Classification of units in the network.
Spatial representations emerging in guidance and aiming show clear differences commensurate with the preferred navigation strategy. Bars represent means over 15 simulations, the outcome of each is represented by a circle. Error bars represent the standard deviation.
Fig 6.
Evolution of learning and spatial representations.
A, Learning curves for guidance and aiming. Curves show average reward at each learning episode over n = 10 runs. The shaded area represents the standard deviation. In guidance, the speed of learning for both strategies is similar, but the asymptotic performance is slightly better for the allocentric strategy. In aiming, the learning is accelerated when using an egocentric strategy as compared to using the allocentric strategy. B, Evolution of spatial representations with learning for all four task-strategy combinations.
Fig 7.
Preferred strategy use leads to superior task generalization.
A, Learning curves for small, medium and large environments in guidance. B, Generalization performance of agent in guidance (successful trials during test after being trained from different start locations) for small, medium and large environments. Egocentric strategy shows worse generalization performance in all environment sizes. C Learning curves for aiming with different amounts of spatial information in the environment using allocentric and egocentric strategies. D, Generalization performance of agent in aiming (successful trials during test after being trained to navigate to different cued goal locations) for environments with different amounts of spatial information. E, Effect of regularization in the form of dropout for guidance task, which encourages the network to look for more general solutions. In guidance, the number of allocentric place-cell-like responses increases with the degree of dropout applied to the network. F, Effect of dropout for aiming task. An increase of vector representations with dropout is not observed, which could be due to the presence of a ceiling effect.
Fig 8.
Different types of spatial representations reveal certain types play a special role.
A, Effect of injecting noise into single units of different types. B, Effect of injecting noise into populations of units of the same type. Horizontal axes represent the standard deviation of the normal distribution from which noise signals are drawn, while the vertical axes indicate the proportion of successful trials in a test phase.