Figure 1.
Proposal for a minimal network in SC for an inter-subjective mind.
In comparison to normal stimuli, we propose that faces are particular patterns because the visual and somatic maps in the superior colliculus are perfectly aligned topologically in the intermediate layer. We suggest that the spatial distribution of the neurons in the somatotopic map is preserved in the intermediate map, which makes the multimodal neurons salient to visual patterns with a similar spatial configuration of eyes and mouth. We hypothesize that this feature potentially influence the social skills in neonates, for detecting faces and reproducing facial movements.
Figure 2.
The distorsion of the facial tissue is simulated as a mass-spring network of tactile points and
springs. Stress and displacement of the facial tissue are rendered by the actions of group muscles around the mouth and the eyes. In A, the front view of the face, the warm colors indicate the position of the segments in depth. The plot in B, the profile view, indicate the action limits of the face mesh in Z axis.
Figure 3.
Strain/stress evolution of the facial tissue during the opening and the closing of the mouth.
The figures highlight the propagation of the strain/stress lines on the facial tissue around the mouth during its opening. The color intensity indicates the variation on each edge of the relative stress, which is propagated from neighbouring points to neighbouring points. The tension lines permits to draw the functional connectivity of each region on the facial tissue.
Figure 4.
Stress intensity profile observed in one node.
We can observe the very dynamic stress intensity level during facial movements on one node, normalized between . Its complex activity is due to the intermingled topology of the mesh network on which it resides. Some features from the spatial topology of the whole mesh can be extracted however from its temporal structure.
Figure 5.
Evolution of the neural growth and synaptic plasticity during map formation.
The plots describe the global variation of the synaptic weights and the number of units in each map, over time. The colors correspond respectively to the somatic map (in blue) and to the visual map (in red). Over time, the unisensory layers converge to stable neural populations through the mechanism of reinforcement learning (hebbian synaptic plasticity) as goes to zero and neurogenesis, as the maps reach their maximum number of units allowed; one hundred units. The density distribution of the neural populations depends on the sensory activity probability distribution.
Figure 6.
Visuotopic reconstruction using the Fruchterman-Reingold layout algorithm.
This graphic layout (right) displays spatially in a 2D map the distance between neurons computed in the weights space on the principle of attraction/repulsion forces. The layout models grossely the molecular mechanisms of map formation. The graph shows that the visual neural network represents well the fovea-centered distribution of its visual input represented on the left with the same color code.
Figure 7.
Somatopic reconstruction using the Fruchterman-Reingold layout algorithm.
As in the previous figure, the Fruchterman-Reingold graphic layout (right) displays spatially in a 2D map the distance between neurons computed in the weights space for the tactile neurons, based on the principle of attracting and repelling forces. In accordance with the previous figure, the graph shows that the tactile neural network respects quite well the topology of the face (left) with the same color code for the neurons connected to their respective somatic area: the neural clusters respects the vertical and horizontal symmetries of the face with the orange-red-pink regions corresponding to the lower part of the face, the green-cyan-blue regions to the higher part of the face, the green and orange regions to left-side of the face and the blue-pink regions to the right-side of the face.
Figure 8.
Multimodal integration schema in SC between vision and tactile information.
Integration is done as follows, the visual signals in the superfical layer and the somatosensory signals in the deep layer converge to the intermediate multimodal map (no reentrance) in which bimodal neurons align pair-wise visuo-tactile associations. In certain cases, the synaptic links from different neurons in the unisensory maps converge to the same bimodal neurons whereas in other cases the synaptic links from the same neurons in the unisensory maps diverge to different bimodal neurons.
Figure 9.
Raster plots from the visual, the tactile and the bimodal maps, during visuo-tactual stimulation when the hand skims over the face.
The activity of the visual, tactile and bimodal maps is drawn respectively at the bottom, the middle and at the top frame. At a given time, the spikes contingency across the neurons in the three different maps creates the conditions for reinforcing their synaptic links from the neurons of the unisensory maps to the neurons of the bimodal map. The difference of spiking rates between the maps show that there is not a bijective connection between the neurons and that some bimodal neurons may associate groups of visual neurons to groups of tactile neurons.
Figure 10.
Networks analysis of visuo-tactile integration and connectivity.
A Connectivity circle linking the visual and tactile maps (resp. green and red) to the bimodal map (blue). The graph describes the dense connectivity of synaptic links starting from the visual and tactile maps and converging to the multimodal map. The colored links correspond to localized visuo-tactile stimuli on the nose (green/red links) and on the right eye (cyan/magenta links), see the patterns on the upper figure. The links show the correct spatial correspondance between the neurons of the two maps. B Weights density distribution from the visual and tactile maps to the bimodal map relative to their strength. These histograms show that the neurons from both modalities have only few strong connections from each others. This suggest a bijection between the neurons of each map. C Normalized distance error between linked visual and tactile neurons. When looking at the pairwise neurons of the two maps (red histogram in B for weights ), the spatial distortion between the neurons from the two maps is weak: vision neurons coding one location on the eyes receptive fields are strongly linked to the tactile neurons coding the same region on the face.
Figure 11.
Neural arrangement and synaptic alignment.
Spatial topology of the neurons in the visual and tactile maps, with their respective pairwise connections to the bimodal neurons, the darker the link, the more aligned are the neurons. In accordance with the results found in Fig. 9, the spatial error between the neurons of each map is weak, which is seen in the alignment of synapses that are mostly parallel; e.g., the dark links. At reverse, the few spatial errors present big spatial distortion (light grey).
Figure 12.
Sensitivity to face-like patterns for certain orientations.
This plot presents the sensitivity of the neural network to face-like patterns, with an experimental setup similar to the three-dots test done in newborns [29]. When rotating the three dots pattern centered on the eye, the neural activity within the visual map and the bimodal map gets higher only to certain orientations, and
, when the three dots align correctly to the caricatural eyes and mouth configurational topology.
Figure 13.
Performance Tests for different configurational patterns.
We perform several experiments around the three dots test, the results on the sensitivity of the bimodal neurons are averaged on twenty experiments. In A the performance of the network on the black background and the three white dots, in B on the eyes only, in C on the mouth only, in D on a pitch black pattern, in E on a random pattern and in F on the reverse pattern. Bimodal neurons show a maximum intensity for the pattern A, where the three dots match the spatial location of the eyes and of the mouth. In comparison, its constitutive patterns presented separately to the network in B and in C generate a much lower activity, whereas The full back pattern in D and the random pattern in E reach an averaged activity level inside the network and the reversed pattern in F, its lowest level. This last performance is due to the contrast polarity sensitivity of the rank-order coding neurons, which is a characteristic comparable with the capacities of the visual system [65], but here the system learns light components against dark background but not dark components against light background as observed in infants [84].
Figure 14.
Neural activity taken from the intermediate visuo-tactile map during observation of a facial expression: surprise (red frame) and stare (green frame).
We present a sequence of facial expressions from surprise to stare and vice-versa. The selected bimodal neuron taken from the intermediate map triggers to the characteristic visual configurational patterns of the face during rapid changes, which permits to detect the mouth and eyes movements. this behavior is due to the sensory alignment and of the high correlation with the tactile distribution of its own face. Note: the subject has given written informed consent to publication of his photograph.