Figure 1.
Network inputs consisted of retinal disparity maps (67 units each) for hand and target positions, retinal position maps (253 units each) for hand and target positions as well as 3D eye and head orientations signals (6 push-pull units each) and a vergence input (1 unit). Example population activations (color surfaces above maps of units) are shown for two different retinal disparities and retinal positions (hand and target). All inputs were fully connected to the 2nd (hidden) layer composed of 200 units through weight matrix win. All hidden layer units (HLUs) were fully connected to the 125 population output units (POUs) through weight matrix wout. To train the network, we designed an optimal linear estimator (OLE, weights fixed) read-out layer in which each of the 3 units represented one dimension of the decoded motor vector. See text for more details.
Figure 2.
Schematic showing the relationship between relative distance, absolute distance, fixation distance and movement depth.
Figure 3.
Typical retinal position and disparity receptive fields and depth gain modulation.
A. Position receptive fields have 90deg limits. Black and white bars indicate eye and head movement sensitivity vectors respectively. HLU: hidden layer units. POU: population output units. B. Retinal disparity fields have 30deg horizontal and 15deg vertical limits. Back bars indicate vergence sensitivity vectors. Each receptive field and retinal disparity field is characterized by a maximum (blue cross), minimum (magenta circle) and center of mass (magenta square) of activity. Different preferred distance codings of each HLU are analogous to Gnadt & Mays [21].
Figure 4.
Vergence modulation of visual (position) receptive fields.
A. Typical example of vergence modulation of a receptive field for a HLU. Black bars indicate the strength of vergence sensitivity. Otherwise the same conventions as in Figure 2 apply. B. Typical example of vergence modulation of a receptive field for a POU. Whereas HLUs are only gain modulated by vergence, POU receptive fields tend to also shift with vergence changes, as can be observed by the shift in the center of mass (magenta square). C. Indices of horizontal and vertical receptive field shifts due to vergence for HLUs (left) and POUS (right). Normalized histograms show proportion of data points in bin of size 0.1. Most HLUs do not have shifting receptive fields (histogram narrowly centered around 0). Indices of vergence-induced receptive field shifts for POU units show a wide distribution. Red dots indicate examples shown in panels A and B.
Figure 5.
Receptive field modulations with hand/target disparity.
Same conventions as in Figure 3, but now showing hand/target disparity-related effects. A. Receptive field modulations with horizontal target disparity for a typical HLU and POU unit. B. Relationship between horizontal hand and target disparity-induced receptive field shift indices for HLUs. C. Same relationship for vertical shift indices. D. Relationship between horizontal hand and target disparity-induced receptive field shift indices for POUs. E. Same relationship for vertical shift indices shows that POU receptive fields are broadly shifting with changes in hand/target disparity (depth).
Figure 6.
Vergence, hand and target depth gain modulation.
A. Summary of gain modulation analysis for HLUs. Vergence (black), hand depth (cyan) and target depth (red) gain modulation indices are shown as a histogram for all HLUs. We used an arbitrary threshold of 0.2 to determine the percentage of “significantly” modulated units. Note that vergence modulated HLUs less than hand/target depth. See Methods section for calculation of the gain modulation index. B. Gain modulation summary for POUs. Vergence, hand and target depth had similar effects.
Figure 7.
Simulation of experiments performed by Ferraina et al. [20].
A. Original data from Ferraina et al. [20]. Left panel shows a schematic of the setup with 5 different target positions (T), 3 different initial hand positions (H) and 3 different fixation distances (vergences, V). Center and right panels show the modulation of neuronal activity across target distance with vergence and initial hand position distance respectively. B. Typical hidden layer unit activity under similar simulated conditions. Same representation as in panel A. C. Typical population output unit activation under the same conditions. Note that while HLUs were strongly modulated both by vergence and hand distance, POUs were generally not modulated by vergence but only by initial hand position.
Figure 8.
Vergence modulation of retinal disparity receptive fields.
A. Typical HLU and POU unit activity modulation with ocular vergence angle. Same representation as in Figure 2. B. Horizontal and vertical receptive field shift index with vergence angle for all HLUs. Histograms show that shift indices center tightly around zero. Vergence-related activity modulation in HLUs is mainly due to gain-like mechanisms. This is analogous to Genovesio & Ferraina [23]. C. Same receptive field shift indices for all POU units displays a much broader distribution of indices.
Figure 9.
Relative versus absolute distance coding.
A. Changes of typical hand RD fields with target depth (coded in degrees of disparity). The HLU shows some gain modulation but no RD field shift, while the POU's RD field shifts with target position, as evidenced by the shifting centre of mass (magenta square). B–G. RD shift indices of relative hand/target depth (panels B and E), movement depth (panels C and F) and vergence (panels D and G) for HLUs (panels B–D) and POUs (panels E-G). This confirms the observation from the typical trials in panel A. RD fields of HLUs do not shift, while large shifts are observed for POUs.
Figure 10.
A. Hand-target depth modulation for two typical HLUs. Color coding shows unit activity levels (same color scale as in Figure 7) for different combinations of hand and target distance. B. Hand-target depth modulation for two typical POUs. HLUs' activity is modulated either in the hand or in the target direction but not both, while POUs' activity shows maximal activity for a specific combination of hand-target distance. C–H. Separability plots for target-vergence depth dependencies (panels C and F), hand-vergence depth dependencies (panels D and G) and hand-target depth dependencies (panels E and H). HLUs code hand and target distance separately (panel E); POUs mainly code for the difference between hand and target distance (panel H).
Figure 11.
Depth from eye/head rotations.
A. Schematic illustrating how eye/head rotations of the same visual hand-target vector lead to different motor depths. B. Modulation of RD fields with horizontal eye position for typical HLU and typical POU. The HLU is gain-modulated by eye position, but it's preferred RD does not shift. The preferred RD of the typical POU shifts significantly (magenta square). C, F. Indices of RD shift due to horizontal and vertical eye position changes for HLUs (no shifts, panel C) and POUs (wide distribution of shifts, panel F). D, E, G, H. RD shift indices for horizontal (panels D, G) and vertical (panels E, H) eye versus head rotations for HLUs (panels D, E) and POUS (panels G, H). There was a significant correlation between the eye and head indices both horizontally and vertically throughout the network.