Fig 1.
Task Context and model architecture.
(A) 9-dimensional arm movement data acquired from motion capture suit is represented by Euler angles between body segments, following the ISB Euler angle extractions [29] in a ZXY coordinate frame. (B) Illustrative example of movements carried out in the natural scenario (cf. S1 Fig for further examples). (C) Illustrative example of planar centre-out reaching movements. (D) General architecture of a VAE: Inputs are encoded as a latent distribution, from which samples are decoded to reconstruct the original input. (E) Topo-VAE architecture. Our cortical neural representation is modelled by an 80x80 cortical grid of artificial neurons in the latent layer q(z). In contrast to conventional VAEs which model these latent neurons as multi-dimensional Gaussian random variables, we used Poisson random variables. Input stimuli drive these model neurons via a multi-layer perception (MLP) encoder network (hidden layer of 500 neurons). A linear decoder is applied to the spike counts emitted at a given timepoint (corresponding to a single input sample), to reconstruct the sensory input stimuli. (F) To embed the model neurons in a cortex-like topography we assign each neuron a 2D position and define their lateral interactions as a Mexican-hat function of the distance between them. (G) This interaction ψ(p,q) is also characterised by a length scale σ. Nearby neurons are excited, intermediate-range neurons are inhibited and there is no effect on distant neurons. See methods for further details. (H) Flow of natural and planar movement data (joint angular velocities) in this work.
Fig 2.
Comparing tuning of modelled and recorded neurons.
(A) Tuning maps from example neurons observed in the latent layer of the topo-VAE under planar movements (B) Tuning maps of example area 2 neurons during planar movements, compared to their closest matches in A. Top panels are examples of the highest quality matches, bottom panels are examples from the 50th percentile of match quality. (C) Circular density histograms of preferred directions in our topo-VAE model trained on natural data (n = 6400) (blue) and neurons recorded from area 2 of 3 monkeys (n = 383) (orange). Note, that the neural models were not fitted to monkey neural data, but directly predicted from the statistics of natural human body kinematics. (D) Frequency histogram of mode bias in PD distributions for topo-VAE models trained on natural data (blue) and stereotyped centre-out reach data (n = 10 for each condition) vs mode bias of PD distribution in recorded neurons (Orange line). (E) Circular density histograms of preferred directions in our topo-VAE trained on stereotyped centre-out reach data (n = 6400) (green) and neurons recorded from area 2 of 3 monkeys (n = 383) (orange).
Fig 3.
Comparing tuning surface features and firing rate distributions between modelled and recorded neurons.
(A) Tuning surface from a single modelled neuron, illustrating two measures we computed to summarise neuronal modulation with velocity: half-peak width of spatial tuning (at maximum end-point speed) and velocity gradient. (B) Jensen-Shannon divergence between the distributions of half-peak widths in modelled neurons trained under different conditions (with (+) and without (-) lateral effects, trained on natural data, trained on planar data) (6400 neurons, 10 models with different weight initialisations per condition) and recorded neurons (383 neurons). The shuffled control is the same as the natural trained model with lateral effects, except the neural activity is shuffled prior to training the Poisson-GLM in the tuning surface calculation step. (C) Equivalent plot to (B) for velocity gradient distributions. (D) Firing rate modulation (peak rate minus mean rate) by endpoint-velocity across all reach directions in modelled (6400 neurons) aI(E) in recorded neurons (383 neurons). Histograms represent actual distributions, whereas lines represent fitted gamma distributions (R2>0.95 for all; Kolmogorov-Smirnov test).
Fig 4.
Predicting the topography of proprioceptive cortex.
(A) Illustration of pairwise comparisons of preferred directions recorded on same (blue) and different (orange) electrodes, performed in recorded and modelled neurons. Neuron A and B are recorded from the same electrode, Neuron C is recorded from a distant electrode. (B) Distribution of PD difference for same electrode (blue histograms) and different electrode (orange histograms) comparisons in modelled neurons and (C) recorded neurons (length scale σ = 2); error bars are standard deviation. (D) The PD map in a topo-VAE with σ = 2, the hyperparameter value which well approximates the recorded neuron topography (cf. S4 Fig for other values of length scale σ).
Fig 5.
Mexican hat lateral effects are essential for reproducing spatial organisation.
(A) Alternative lateral effect functions tested in control experiments, compared to the Mexican hat (blue line). (B) resulting PD maps for each function (colour matched to 5D).
Fig 6.
Sensitivity of latent layer neurons in topo-VAE to different input dimensions.
(A) Joint angle axes, 3 per joint. (B) 3D plot showing the relative sensitivity of each neuron to joint inputs. Sensitivity is defined as the sum of correlations across X/Z/Y axes of all joints for a given neuron (colour normalised by maximum values per joint) (n = 6400). (C) Correlations between each dimension of input joint motion and neural response. (D) Joint sensitivity map where each pixel represents a neuron and the pixels RGB values (colour wheel inset) are reflecting the correlation with shoulder (“S”), elbow (“E”), and wrist (“W”) angular velocity, respectively (colours normalised by maximum value across joints).