2.1 Model overview

We start with a pre-existing model of the compass system in the insect brain [27]. This circuit in the CX (specifically in the EB) is composed of three neuron subsets called EPGs, PEGs and PENs (section 2.4) constituting a so-called ring attractor [16]. We assume here a visual input to the EPG neurons, and proprioceptive input to PENs, which creates an activity ‘bump’ that tracks the animal’s heading [15, 29]. We extend this model to control steering by introducing two sets of PFL3 neurons [30] (one controlling each turning direction, left and right) that have heterogeneous connection strengths from the EPG (section 2.5); this sets a steering direction relative to the bump. We then suggest a mechanism by which these heterogeneous weights can be adapted during behaviour (section 2.6), effectively associating self-motion in a certain direction to a positive (or negative) reward (operant conditioning). This allows steering in a goal direction relative to the stimulus for any arbitrary offset of the bump relative to the stimulus. Finally, we consider how a MB output, signalling familiarity of a visual panorama, could be used as the reward input to the CX and hence establish steering in relation to a route or a visual homing memory (section 2.7). We show this model can account for ablation experiments [13] in which the loss of one MB results in ants exhibiting innate instead of learned attraction (section 2.8).

2.2 Simulations

The model was implemented in python 3.6. Each neuron is represented by a simple firing rate model [31]. The activation rule is either a linear function of the input (linear simple perceptron) capped between 0 and 1/-1 (excitatory/inhibitory) or a logic function (active or inactive based on a threshold). The input function is the sum of the activity rate of the pre-synaptic neurons.

The model simulations are conducted in a simple 3D environment containing a single black object in a white background world, comparable with an laboratory experimental paradigm used frequently in different insects and specifically in the experiments we seek to replicate [13]. The 3D environment is a custom implementation in python 3.6 using the pyopenGL library. The agent is placed at the beginning of the simulation at the center of the virtual environment and moves freely until it either reaches the border of a circular area of 100 length unit (l.u.) radius or exceeds 5000 time units (steps). The agent has a constant speed of 0.25 l.u./step and moves in a straight line following the heading orientation updated every step. During each simulation, a single object is placed randomly in the environment at 150 l.u. from its center, thus beyond the border of the agent’s arena. This single object, also referred as a cue or landmark, is randomly chosen between a vertical cylinder, a cube or an inverted cone during every individual simulation run. An innate exploratory behaviour is obtained by the addition of a gaussian noise (sigma = 10°) in the model steering behaviour, providing a basal exploratory behaviour independent of the CX model output.

2.3 Visual circuit

The first layer of the model is a simple visual processing stage. The visual processing is not based on biology, but just provides a simple edge detection process to detect the single landmark used in the simulations. Vision is considered fully panoramic around the azimuthal dimension and from 0° to 80° in elevation (horizon line to the top, note the simulated world contains no information below the horizon). The visual system is segmented into 1296 visual units [48 × 27, inherited from the simulation resolution and the division of the azimuth (48 units) into 8 quadrants (6 units each)], homogeneously distributed on both dimensions. Each unit acts as an edge detector (Fig 2B). For each unit, the summed intensity of the left (S_L) and right (S_R) half of its visual field is compared (the difference is divided by the sum for normalization) to obtain an index ranging from 0 (no edge) to 1 (vertical edge). The same process is used to compare the top (S_T) and bottom (S_B) half of the visual unit. Then, the two indexes for the vertical and horizontal edge are averaged to obtain the final output of the unit, a orientation-free edge index, ranging from 0 to 1: (1) Where S_left, S_right, S_top and S_bottom are the 4 halves summed light intensity values, Iv_i and Ih_i are respectively the vertical and hoizontal edge indexes, and I_i the combined edge index for that unit. I_i is close to 1 if the unit faces a corner (I_i = 1 if only a full corner of the visual unit is obstructed), close to 0.75 if the unit faces only a vertical or a horizontal edge, and 0 if it does not face any edge.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Model visual system.

A. Map of the 1296 (48x27) elementary units composing the visual system. The visual span is fully panoramic (360°) on the azimuth and 81° upward on the elevation. The red framed visual unit is the one used as example in B. B. Elementary visual unit process. Each unit acts as a orientation-free edge detector. The comparison of the summed intensity of the left vs the right of the unit and of the top vs the bottom gives an index from 0 (no difference ↔ no edge) to 1 (corner edge) for each unit.

https://doi.org/10.1371/journal.pcbi.1009383.g002

2.4 Compass: Ellipsoid body model

The main connectivity pattern in the Ellipsoid Body (EB) model is taken from Pisokas et al. [27], who modeled the ring attractor function of the CX that has been highlighted in recent neurophysiological studies in insects [15, 32, 33]. Four neuron types interact to create a ring attractor, three forming the excitatory part of the circuit, EPGs, PEGs and PENs, and one contributing inhibition, Delta7 [27]. The connective scheme between all these neuron types are organised in set of columns in two main parts of the CX, the EB and the Protocerebral Bridge (PB) but functionally form a ring [15, 27], as shown in Fig 3A. The functionality that results from this ring attractor circuit is the creation of a bump of activity in the EB that follow the rotation of the visual environment, creating an allocentric representation of the insects’ orientation similar to a compass.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Ellipsoid body compass model.

A. Connectivity diagram between the main neurons of the EB-PB model, EPGs, PEGs and PENs. EPGs have inhibitory connections with each other as indicated, via Delta7 neurons (black lines; one example is shown from the EPG₄ in the central part of the diagram). EPGs (light blue) form a recurrent circuit with PEGs (green) while forming connection with neighbouring EPGs through PENs (pink) on each side. B. Connection diagram from the visual circuit to the EPGs. Each angular segment connects in a retinotopic manner to the EPGs. Intrinsically this results in an activity bump forming in the circuit that corresponds to the direction of highest visual contrast. C. Proprioceptive inputs to the PENs. When the agent is engaged in a left turn, PEN₁₋₈ are stimulated, while during a right turn PEN₉₋₁₆ are stimulated. This will trigger a counter-motion of the bump (posterior view) so that it still indicates the relative direction of the external cue(s). D. Example of the function of the EB model in an arena surrounded by a single cue (here a cylinder represented by the black circle). The blue line shows the path of the agent without any influence of the CX model, i.e. only produced by the steering noise. E. Activity rate of the three neuron types constituting the EB model. On the first line, the activity of EPGs (blue for no activity to yellow for active) shows a perfect following of the cue orientation in the agent’s visual field (red line, scale on the right). Even during a darkness episode (from 1500^th to 2500^th steps), the model keeps track of the cue position with a relatively low error thanks to the PENs (second line) and PEGs (third line) combined actions.

https://doi.org/10.1371/journal.pcbi.1009383.g003

The visual input described above is segmented into 8 equal regions around the azimuth, which (in our model) connect in a retinotopic pattern with the first neuron type, the EPGs (Fig 3B). This retinotopic mapping has been made to simplify in the model the compass function derived directly from a single conspicuous bar, and allow orientation tracking in Drosophila during a bar fixation paradigm [15, 34]. Note that in fact, during bar following experiments, the bump of activity in the EB expresses a constant, arbitrary offset with the bar’s real position [15]. The real circuitry from vision to the EB in insects’ brain is known to be more complex and involves the Ring neurons, which we will return to in the Discussion. We will also show that the steering mechanism we propose is not dependent on the retinotopic mapping but works for any arbitrary offset (section 2.6; Fig C in S1 Text).

The inhibitory connections between EPG neurons are considered global, as described in studies of Drosophila brains [27]. The Delta7 inhibitory circuit is represented as a direct inhibition from each EPG to all the other EPGs. This creates a winner-take-all circuit, resulting in a single bump in neural activity that will follow the main contrast of the visual scenery. In the simulations here, this is a single object present in the otherwise homogeneous surroundings of the agent.

Inside the same column, EPGs and PEGs form a recurrent circuit, which allows the bump to persist in the absence of visual input (Fig 3E). EPGs also give presynaptic excitation to PENs from the same column, but PENs form presynaptic connections with EPGs from a neighbouring column. In addition, we assume the PENs receive input from the proprioceptive sensory system (self-motion) through the Nodulus [15, 16]. The rotational speed of the agent is measured and injected as a binary input, 0 or 1 modulated by a fixed gain (K = 0.75), to all the PENs on one side of the PB (PEN₁₋₈ for a right turn or PEN₉₋₁₆ for a left turn, Fig 3C). This causes the bump to move rightwards or leftwards around the ring, maintaining coherence of the bump position to the external cues even in darkness, as observed in the Drosophila brain [15]. The activity rate of each neuron group of the compass model is set by the following set of equations: (2) Where VU_i is the input from an octant of visual units (each octant connecting to a single EPG, see Fig 3B), EPG_j the vector of EPGs, excluding the current EPG_i, acting through the inhibitory pathway (Delta7), and Nod_L(t) and Nod_R(t) the rotational motion input from the noduli encoding left turns (Nod_L(t) = 1) and right turns (Nod_R(t) = 1) respectively. The different weights used to modulate the influence of each neuron type have been set to optimize the tracking of the heading even in darkness (Fig 3E).

2.5 Steering: EPGs to PFL3s heterogeneous connection scheme

A neurophysiological bridge from the compass to the control of motor behaviour has recently been identified in Drosophila [30]. This consists of a subset of neurons, the PFL3s, that are downstream to the EPGs, and upstream to the DNa2 neurons, which are bilateral descending neurons responsible for motor control and steering maneuvers [30]. Moreover, the PFL3s are reported to represent the second largest input to these DNa02 neurons [37], supporting their important role in steering control. We therefore modelled the PFL3 layer to transform the compass bump into a steering command. We used Drosophila databases [35–38] to elaborate the connection scheme from EPG to PFL3 neurons (Fig 4), forming synapses at the PB level. This approach has already been used by Rayshubskyi et al. to model the steering behaviour of Drosophila [30]. Each brain hemisphere possess 7 PFL3s, which each form synapses mainly with a single EPG wedge (Fig 4A). We assumed that the DNa2 of each hemisphere integrate the inputs from the whole set of PFL3s of the ipsilateral hemisphere, as each DNa02 has been shown to be acting independently [30]. Thus, the influence of each EPG depends only on its synaptic connection to one or more PFL3 on each side. We therefore represent two PFL3s (one on each side) for each EPG (Fig 5A), and set the synaptic weight between them to the sum of synapses from each EPG tile (Fig 4B) to a PFL3. Note this means a PFL3 cell in the model could represent from 0 to 3 actual PFL3 cells, with a corresponding weights from zero to one which we obtain from the Drosophila database [35–38](i.e., the weights are normalized). The synapses between EPGs and PFL3s are represented as excitatory (Fig 5A) as EPGs are reported to form cholinergic synapses with PFL3 [35–39]. The output of the PFL3 neurons are determined by the following equation: (3) With the synapse strength between an EPG neuron and the corresponding PFL3 neuron. The unilateral sum of the PFL3 outputs is therefore considered to represent the motor signal received by each DNa02 neuron [30] and their comparison directly gives a steering command, calculated as follows: (4) With Δ_steer the change in direction taken by the agent at time t, and the gaussian steering noise with a standard deviation of 10°. The steering signal produced by the PFL3s populations comparison is constrained to a [-2.5 2.5]°.step⁻¹ range to avoid unrealistic turning speed and instability. Note that this constraint on the model output is smaller than the steering noise applied and, therefore, preserves stochastic exploratory behaviour (see next section 2.6) while guiding the general direction of motion. Please note that we did not consider here the polarity (excitation/inhibition) of the PFL3 to DNa2 synapses but rather use the creation of a left-right difference to drive the steering. However, it has been reported PFL3 neurons from the left (respectively right) PB connect directly onto DNa02 on the right (left) [40]. In addition, considering a greater activation on the right DNa02 (compared with the left) produces a leftward turn [30], our PFL3 synapses to the DNa02 can be considered inhibitory (left activation promote left turns), as it has been assumed by Hulse et al. [40] whereas Rayshubskiy et al. [30] assumed the opposite.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The EPG to PFL3 connectome at the interface between the compass and the steering control.

The full connectome is assessed from online Drosophila brain databases (https://v2.virtualflybrain.org, [35–38]). The full list of the synaptic connection between neurons annoted PFL3 and EPGs is given in S1 Table. A. (a) Division of the EB in wedges 15. (b) Quantity of synapses expressed as a function of the EB wedges and the PFL3 columns. B. (a) Relationship between EB wedges and the EPGs modeled here (Fig 3A). (b) The connectome is transformed into EPG-PFL3 synapse weights to create the steering model (see section 2.5). Weights are obtained by summing for each EPG the synapses on the right or on the left part of the PB. The gains are then normalized to fit between 0 and 1.

https://doi.org/10.1371/journal.pcbi.1009383.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. EPG to PFL3 connectivity scheme.

A. Diagram of the EPGs to PFL3s connectivity that drives the steering. Each EPG makes inhibitory connection to two PFL3 neurons, one on each side. The strength of the synapse depends on the index of the EPG_i as shown in B. The summed activity rates of the right and left PFL3 neurons are compared to obtain the CX steering signal. Finally, gaussian noise (ν = 0;σ = 10) is added to obtain the final steering command. B. (a) Synapse strength as a function of the index of EPG_i, obtained from the connectomic approach (Fig 4B; [30]). Synapses are inhibitory and therefore multiplied by -1. Red lines represent the connection with the subset of PFL3s from the right part of the PB and green from the left part. Synapse weights are presented in a circular representation to show the correspondence with the EPG input geometry. Index 1 corresponds to the synapse between ellipsoid body tile EPG₁, connected to the rear right FOV (-157.5°), and PFL3₁, and so on, through to index 8, the synapse between EPG₈, connected to the rear left (157.5°), and PFL3₈. (b) Heatmap in the arena from 100 simulations in total. The data are normalized to the landmark direction (indicated on the right). (c) Final direction/position of each simulation (n = 100) with the cue placed in different positions around the arena, expressed in the cue reference frames. C. (a) Simulations (n = 100 for each shift condition) where EPG-PFL3 synapse weights were shifted for 1 cell to the right/clockwise (compared to B.). Results show the heatmap relative to the cue (top) of the 100 simulations. (b) Simulations (n = 100 for each shift condition) where EPG-PFL3 synapse weights were shifted for 1 cell to the left/anti-clockwise. Results show the heatmap relative to the cue (top) of the 100 simulations. (c)-(d) Equivalent to (a) and (b), respectively, with a 2 cells shift.

https://doi.org/10.1371/journal.pcbi.1009383.g005

To generate a steering signal in the PFL3 layer, we propose here that the observed heterogeneity in the distribution of EPG-PFL3 synapse weights is ideally suited to give rise to a Left-Right inequality and thus to a turn. More specifically, the specific weights/connections pattern (Fig 5B) that frames the EB bump to the front of the visual field would suffice to generate an innate attraction to the conspicuous landmark. The detection of the cue in the right part of the visual field (as schematised in Fig 5A) would lead to an unbalanced excitation of the PFL3s and thus a increased signal to turn right and ultimately centering the cue in the frontal field of view (equally the detection of the cue in the left part of the visual field would have the opposite effect). The general direction taken depends upon the intersection between the left and right EPG-PFL3 synaptic weights, the steering noise only creating some stochasticity in the behaviour and a convoluted path as a consequence. When tested in the virtual environment, mimicking indoor arena experiments with a single landmark classically used in wood ants [13, 41], we observed accordingly an innate attraction behaviour toward the cue (Fig 5B).

Crucially, assuming heterogeneous weights (or connections) occur between EPG and PFL3 neurons provides the flexibility to consider the adaptative outcome if the weight distribution is altered. For example, it could be shifted to the right or the left (Fig 5C) to maintain the bump in a different position other than the front of the agent. The subsequent oriented behaviour is opposite in direction to the shift of the EPG-PFL3 synapse weights, because they define the position in which the bump will be maintained during behaviour. Thus, by setting the weights accordingly, the model is capable of producing either direct attraction to a cue or menotaxis at any offset from the cue direction. We now go on to explore how the weights, instead of being pre-determined, could be plastically altered through the agent’s own experience to produce appropriately a flexibility of the oriented behaviour in different contexts.

2.6 A memory to modulate the EPG-PFL3 connection weights

In order to allow insects to deal with different goals at different times, the mechanism that orients them with respect to landmarks, visual memories or other external information should allow some kind of plasticity or adjustment over time. Here we propose a mechanism that adopts some characteristics of the model proposed by Stone et al. for path integration [21], based on neuroanatomical evidence in bees, that accumulates a homing vector from the optic flow speed estimation and the sky compass. Specifically, we add a set of units (similar to the CPU4 units in [21] model) that get both compass and self-motion input, and implement a one column (left or right) shift [42] before connecting to the steering neurons (PFL3 in our model—CPU1a in [21] model). In the present model we limited the self-motion input to the perception of rotations (in a bi-modal, left or right, way), which is used to inhibit ipsilaterally the update of the synaptic weights between the EPG and PFL3 neurons, rather than directly contributing to steering (Fig 6). The existence of such input, potentially provided by the noduli, could be supported by recent observation in Drosophila at the PB level [42]. Stone et al. [21] proposed that CPU4 neurons (PF-LC in Drosophila), arborizing between the FB and the PB, could encode the memory for the path integrator. In contrast, here we propose that the modulation that would drive the use of landmarks could be represented rather by a direct plasticity between EPGs and PFL3s. Several studies have linked the FB to the encoding of desired direction in insects brain [43–45]. In addition, PFL3s receive upstream inputs from synapses in the FB [38], we therefore consider it as a target for potential modulatory neurons and refer to the neurons in our model as FB neurons (FBn). In addition, we set the formation of this memory under the control of a global reward signal representing the sensory pathway(s) influenced by the desired heading [18]. The aim of the weight modulation is to enable the formation of a similar spatial weight pattern as presented in the previous section. We specifically decided to apply the memory by the mean of a modification of the synaptic weight here for consistency with the connectomic approach presented in the previous section (2.5). However, we recognize that there is no evidence so far of EPG-PFL3 synapse plasticity and that the memory could, for example, be accumulated at the FB level, as it is proposed by a CPU4 accumulation in Stone et al. PI model [21]. In fact, for the level of abstraction used here, such an implementation would be computationally equivalent (as demonstrated in Fig C in S1 Text). We rather focus on the potential of the functional columnar shift [21] and of the convergence of the three following components: the self-motion perception, the compass information and the influence of goodness-badness encoded sensory pathways. The FBns activity is set by the following equation: (5) With Nod the inhibitory self-motion signal, from right (NodR(t) = 1) and left (NodL(t) = 1) turns respectively. Therefore during right turns and during left turns. Then, the EPG to PFL3 synapse weights are updated as follows: (6) Where is the synapse weight between EPG_i and PFL3_i on the right (R) or on the left (L) FB. The synapses are initialized in the model with a value of 0.5 and restricted between 0.2 and 0.8 during the simulations. α is a free parameter set equal to 0.001. The reward signal (Rew(t)) is set (as we describe below) to act as modulator for the synapse update by the FBns of the neighbouring functional column on the right (for the left PFL3s) or the left (for the right PFL3s). This learning rule is used as an efficient way to implement the key conceptual function, that the compass shift associated with rewarded actions should bias the steering weights, rather than being intended as a biologically justified plasticity mechanism; as already discussed, and shown Fig C in S1 Text, there are alternative mechanisms that could serve the same purpose.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Steering CX model with external control.

The central part of the diagram is similar to the model of EPG to PFL3 connection presented in Fig 5A except the synapse weights are initially set homogeneously (). On each side the model copies the [21] model architecture such that the connection between FBns [red (right FBns) and green (left FBns) cells] and steering neurons (PFL3s, yellow cells) are shifted by one column, in opposite directions on each side. FBns also receive rotatory self-motion information inhibiting inputs [light red (right turns) and green (left turns) lines] for the ipsilateral side, therefore the right FBns are activated during left turns and viceversa. The reward signal (pink) is provided by the sensory pathway indicating goodness-badness of the current sensory input. The modulation of the EPG-PLF3 synapses is under the control of this reward signal and the FBns as shown in the bottom-right inset. The steering command is obtained by comparing the right and left summed activity of the PFL3s after the addition of a gaussian noise (ν = 0;σ = 10) as previously (Fig 5A).

https://doi.org/10.1371/journal.pcbi.1009383.g006

The effect of Eqs 5 and 6 is as follows. Say the bump is centred on EPG₃, as shown in Fig 6, and the agent is turning right and observes a view which elicits a reward above threshold from any sensory system. The right FBns being inhibited by the right turn, the left FBns will induce an increase, under the control of the high reward level, in the EPG-PFL3 synapse shifted by one column to the left (counter-clockwise) from the EPG bump, on the EPG₄. This leads to the increase of the tendency to turn right if the cue was to appear on the left of its current rewarded orientation. It therefore induces a memory of the motor action that led to a rewarded orientation, and thus corresponds perfectly to an operant behaviour. By exploring the environment, this mechanism creates two weight distributions that frame the position of the EPG activity bump at its maximally attractive orientation in the local environment.

To reproduce the innate attraction to a simple cue, widely observed in insects [46, 47], we created a reward signal provided by the visual processing used in this paper. The aim is to have reinforcement of the signal from the frontal ommatidias (i.e. when facing the cue). This can be done using a mask on the visual units as shown in (Fig 7, top panels). The mask is multiplied with the visual input and the output summed across all units to provide the reward signal: (7) With Rew_CX being the CX reward signal used to update the EPG-PFL3 synapses, I the matrix of the edge detection indexes (Eq 1) estimated by the visual units (Fig 2) and M the mask matrix (same size as I). We tested alternative masks (M) with either a sharp or graded preference (Fig B in S1 Text). for a frontal orientation of the strongest contrasts, and either with or without negative responses for contrasts at the rear of the visual field (Fig 7A and 7B for sharp masks and Fig B in S1 Text for graded ones). The aim of these masks, rewarding contrasts in the frontal part of the visual field, is to reinforce the weights so as to stabilize the landmark in the center of the visual field and therefore produce attraction to it.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Innate behaviour under the control of a visual reward signal.

Simulations of the FB steering model (Fig 6) using a reward signal provided by the visual processes to modify the EPG-PFL3 synapse weights. We created the visual reward signal to the CX using different masks. Results for each panel include (a) the final path directions (n = 50 simulations), (b) the probability density function of the final direction vector relative to the cue orientation, (c) the averaged EPG-PFL3s synapse weights and (d) examples of 5 simulation paths. See also Fig B in S1 Text for results with continuous masks. A. Visual reward to the FBs is equal to the sum of the visual units signal through a discrete mask equal to 0 outside the 30° frontal area or 1 inside. B. Visual reward to the FBs is equal to the sum of the visual units signal through a discrete mask equal to 0.5 inside the 30° frontal area, -0.5 inside the 30° rear area, or 0 otherwise. C. The mask is shifted by 45° to the left of the visual field. The reward area extends from 30° to 60°. D. The mask is shifted by 45° to the left of the visual field. The reward area extends from -30° to -60°.

https://doi.org/10.1371/journal.pcbi.1009383.g007

Using the architecture presented in Fig 6, we show its capability to generate a memory able to drive an innate attraction behaviour toward a single cue (Fig 7A). As observed with the synaptic weights inherited from the native connectomic (Fig 5B), the weights produced by the model define the general direction taken by the agent at the intersection of the left and right sets of weights. In addition to adding stochasticity to the path, the noise added to the steering behaviour is crucial here to generate the initial exploration necessary for the synaptic memory acquisition. Even using a discrete filter, which does not carry any information about the left or right displacement of the cue from the current heading, the model generates a consistent behaviour and coherent weight distribution for the EPG to PFL3 synapses (Fig 7A). The use of a negative sensory input, when the cue appears at the rear of the visual field, to the model improves the performance (10% more path ending at the cue orientation, Fig 7B). The negative input is more essential when a graded preference function is used, otherwise experience of some positive reward in all directions leads to random preferred orientations (Fig B in S1 Text). In both cases (continuous or discrete masks), the advantage of positive and negative sensory inputs is of interest considering that insects are known to associate (innately or learned) positive or negative values to olfactory [48] and/or visual cues [2]. Furthermore, the modification of the mask, to reward the cue in a shifted position, produces an oppositely shifted behaviour (Fig 7C and 7D), showing the ability to use the sensory information to drive oriented behaviour relative to a landmark, similar to menotaxis [14, 19].

The formation of a memory in the CX also permits a sustained behaviour when directly attractive cues disappear in the environment. To illustrate this we tested the model in static simulations, where the agent is maintained in the center of the arena and can only rotate, and we made the cue disappear after a given time, necessary to form the EPG-PFL3 synaptic memory (Fig 8A). The persistence of the behaviour depends on the capability of the EB compass to maintain its function in darkness [15]. The accumulation of error along time in absence of the landmark produces a gradual shift in the behaviour (Fig 8C.a). Alternatively, if we complement the compass with an absolute reference in the environment, which could be provided in insects by sky cues ([49, 50], Fig 8B), the agent can maintain a consistent straight direction for an indefinite period after the disappearance of the attractive landmark (Fig 8C.b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Persistence of direction in the absence of the landmark.

A. (a) Simulations are done in a static mode, i.e. the agent never leaves the center of the arena and only rotates around its vertical axis. (b) After the 2000^th step the cue disappears. B. (a) We tested the model in the same configuration as before, where the landmark is used both as the main cue for the compass and to generate the directed behaviour (Fig 7C). Results are shown in C.a. (b) We tested another configuration where the compass was provided with cues from the absolute orientation in the environment (such as sun position or sky polarization could provide). The attraction behaviour is kept based on the landmark as before. Results are shown in C.b. C. EPG activity rate (blue to yellow) and relative landmark orientation (red line, right scale) during simulation of the cue disappearance paradigm showed in A. (a) The compass and the attraction are both based on the landmark (B.a). The direction can be maintained but slowly drifts as there is no external reference. (b) The compass is based on an absolute orientation perception (potentially from sky cues) while the attraction is based the landmark (B.b). Note this creates a potential offset in the position of the bump and the landmark, but the agent still moves towards the landmark. The heading can be accurately maintained when the landmark disappears.

https://doi.org/10.1371/journal.pcbi.1009383.g008

Note that for these results (excepting Fig 8C.b) the bump and the landmark which generates the attraction were directly attached, i.e., the ring-attractor compass represents a truly allocentric compass based on the external landmark cue. In contrast, in the insect brain during bar-following paradigms it is known that while the movement of the EPG bump follows the bar movement, the actual position of the bump has an apparently arbitrary offset from the bar azimuth [15]. To explore this, we reproduced such an offset between the EB bump and the visual cue’s position in the visual field by applying a random shift in the connectivity between the visual units and the EPGs (Fig D in S1 Text). This means each simulation expresses a unique offset between the bump and the landmark. However, the innate attraction behaviour to the cue produced by the model is unchanged as the model defines the correct set of weights to drive the behaviour toward the landmark in any situation. Thus, the model does not use the bump as an absolute compass but rather as a reference point to interpret another signal (here the valence signal from a sensory pathway) to define the orientation to take.

2.7 Incorporating a visual familiarity memory from the mushroom bodies

In addition to using a single visual input to generate an innate attraction to single object, as described above, we also tested how the output of the Mushroom Bodies (MB), a bilateral structure tightly associated with memory in the insect brain [51], could be used to generate a local memory in the CX to control appropriate orientated behaviour. To do so, we adapted the model of familiarity encoding for route memories that has been hypothesised to occur in the social insects MB [26], and use its output as the reward input in our model. Recently, experimental results have supported the importance of MBs to view-based memory and navigation in ants [13, 52, 53]. The direct influence of a MB memory is also supported by the existence of several MBONs largely projecting into the FB in Drosophila [54].

The MB is modeled as a network (Fig 9A) composed of 10000 Kenyon cells (KCs) each sparsely connected to 3 to 5 visual Projection Neurons (vPNs). vPNs are directly taken as the outputs of the visual units calculating the edge indexes from the panoramic view (Fig 2). The long-term memory is formed by a simple associative learning rule at the interface between the KCs and the output of the MB (MBON, Mushroom Body Output Neuron). The MBON signal is calculated as the sum of the KCs activity [KC_i—binary, either 0 (active) or 1 (inactive)] weighted by their associated synapse weight () and normalized by the number of KCs active, thus maintaining the MBON value between 0 and 1: (8)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Orientation to a goal direction supported by the MB long-term visual memory.

A. Mushroom bodies model diagram. The visual Projection Neurons (vPN) correspond to the visual units (Fig 2) extracting edge from the panoramic view. Each KC has a random post-synaptic connection pattern with 2 to 5 vPNs and a pre-synaptic connection to the MB output neuron (MBON). During the learning procedure, the memory reward signal (DAN) is activated and induces the decrease of the synaptic strength between the activated KCs and the MBON. The MBON activity is then used as the reward signal to adapt the CX circuit. B. Schematic of the learning phase before the simulation. The agent faces the feeder during 100 steps from a position in between the feeder and the center of the arena (learning pose), with an orientation error from −15 to 15°. The MB model memory is updated during this phase (DAN active). C. Example paths from the retrieval experiments using the MB model as an input for the CX model. D. Final direction vectors for 50 simulations with the feeder at 0° and the cue oriented at 45°. The red arc shows the median (dot) 95% C.I. obtained by bootstrapping (rep = 10000). E. Averaged right (red) and left (green) EPG-PFL3 synapses weights (shaded area: ±s.d.) obtained from 50 simulations.

https://doi.org/10.1371/journal.pcbi.1009383.g009

The KC-MBON synapses weight are initially set as 1 and updated under the control of a rewarding signal (DAN, Dopaminergic Neuron) following the learning rule (inherited from classic MB associative memory models [26, 51]): (9) Where (∈[01]); and is given in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Mushroom bodies visual long-term memory learning rules.

KC to MBON synapses are updated depending on the KC activity level and a reward signal coming from dopaminergic neuron (DAN), considered on (1) during the learning phase and off (0) otherwise (during the test phase). Only the combination of an active KC with an active DAN induce the reduction of the corresponding KC-MBON synapse weight.

https://doi.org/10.1371/journal.pcbi.1009383.t001

The learning phase is made to mimic learning walks/flight observed in insects during which they face a targeted location (nest, food) and acquire some visual memory [55–57]. Therefore, the DAN activity is kept to 1 during the whole learning phase to generate the visual memory. The agent is placed at the mid-distance between the center of the arena and a virtual feeder (always at 0° at the edge of the arena) and scans the environment during 100 time steps (0.3°/step) over a range from −15 to 15° centred on the feeder (Fig 9B). This range is chosen arbitrarily to match the span of the discrete visual mask (Fig 7A). At the end of the learning phase the DAN activity is set to 0 during the test phase to avoid the development of new MB memories.

Following the MB learning rule, which decreases the synapse strength from KC to MBON during learning episodes, the weaker the signal from the MBON, the stronger the familiarity of the current visual scene. Therefore, the MBON output is translated into a positive reward for greater familiarity, (Rew_CX(t) = 1 − MBON(t)), and used to influence the EPG to PFL3 weight updates as before. To avoid the influence of the noise at low familiarity, we used a threshold on the MBON value so as any value lower than 0.25 is equal to 0.0.

Using the MB familiarity of the visual scene as the reward input to the CX model enables the agent to be directed towards the feeder location (Fig 9C and 9D). Despite the restriction of the memory to a single episode at a single location, and the noisy familiarity observed during the simulations, the model is able to induce direct oriented behaviour toward the goal location albeit sometimes after expressing an initial erratic behaviour, due to the steering noise. The averaged EPG-PFL3 synapses weights clearly show a pattern to keep the landmark/bump in the correct orientation [≈45° on the left (counter-clockwise)] to reach the virtual feeder (Fig 9E).

2.8 Lesion experiments

In order to link our model to biological experiments [13], we set up simulations where the MB could be fully silenced (i.e., output equal to 0). The mushroom bodies are present as a pair of structures in insect brain. Therefore, the first step was to duplicate the MB used in previous simulation to obtain two identical structures, one for each hemisphere. Both MB follow the same vPN-KC connectivity parameters (as previously shown, Fig 9A), but their connectivity is independent. In addition, each MB receives homogeneous inputs from the entire set of vPNs from the panoramic view of the agent, i.e., the connectivity from both eyes is presupposed identical (Fig 10A). It is not known to what extent the mushroom bodies receive inputs from the contralateral compound eyes, however, anatomical [58] and behavioural [13] observation supports the existence of cross-over in ants. The MB output consists on each side of a single MBON neuron which connects to the FBns of the same side only. Therefore, the ablation of one MB deprives the ipsilateral FBns of any memory related inputs. To keep the model to the simplest case scenario we linearly added both innate visual (Vin) and MB inputs to generate a common reward signal to control the memory as follows: (10) With Vin the signal provided by the visual mask (see Eq 7), ω_Vin the gain modulating the visual innate attraction pathway and ω_MB the gain modulating the MB pathway. For each simulation, a pair of gains is randomly selected from a uniform distribution in the range indicated in Eq 10. The ranges of the pair of gains are set to advantage the MB pathway (approximately with a factor 4 here) and obtain a behaviour biased towards a learned visual heading (Fig 10B), mimicking the real behaviour of wood ants in similar experimental paradigm [13]. The unilateral “lesion”/silencing of a MB occurs between the learning phase and the test phase (Fig 9B) and, therefore, blocks the influence of the visual long-term MB memory on the formation of the CX memory (EPG-PFL3 synapses) on the side of the lesion.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Replication of unilateral mushroom body lesion experiments from [13].

A. Schematic of the model including both MB and innate visual attraction. Both elements are linearly summed to form a single reward signal, using individual weights (ω_Vin & ω_MB) to modulate the influence of each component. Lesions are indicated by the black scissors and the results are reported in C and D (Left and right lesion respectively). B. Simulation with the combination of innate attraction (ω_Vin) and the mushroom bodies (ω_MB) pathway. The learning procedure is similar as presented in Fig 9B. From top to bottom, final directions (blue arrows, n_sim = 100) taken during the simulations [blue arc indicate the median (dot) 95% C.I. obtained by bootstrap (n_rep = 10000)], examples of path obtained during simulations, and averaged right (red) and left (green) EPG-PFL3 synapse weights (shaded area: ±s.d.) obtained during 50 simulations. C. Unilateral lesion of the left mushroom body. Final direction vectors and example paths. D. Unilateral lesion of the right mushroom body. Final direction vectors and example paths.

https://doi.org/10.1371/journal.pcbi.1009383.g010

When a MB is unilaterally silenced and the agent presented with a landmark at ±45° from the feeder, the model reverts to a strong attraction to the landmark in all cases (Fig 10C and 10D). This is due to the importance of both EPG-PFL3 distributions (i.e., one for each side of the CX model) in defining the orientation in which to keep the EB bump. With only one of the two, the MB memory does not influence the position of the bump and therefore only the innate attraction to the landmark actually influences the behaviour.

3.1 Assumptions in the model about CX inputs

Our model has abstracted from neuroanatomical reality to simplify the connections from the visual system to the EB, by using a direct retinal input. Indeed, it is known that the visual inputs to the EPGs, conveyed by ‘ring’ neurons [67], are not fully retinotopic. Ring neurons connect from the Lateral Triangle (LTR) to the EB [34]. LTR glomeruli conserve localized visual receptive fields but form connections with the entire EB tile set [67], potentially modulated by neuronal plasticity [63]. The effect of this connection scheme is the preservation of a visual scenery representation in the EB [63] whereas an arbitrary offset can arise between the external location of visual information and the EB activity ‘bump’ in bar following experiments [15]. Although we did not explicitly include ring neurons, the model could reproduce the effect of this variable offset through a retinotopic shift in the connection from the visual inputs to the EB, and can function with such offset (Fig D in S1 Text). The representation of innate attraction as a modulatory signal and not as an intrinsic hardwired system allows this flexibility. Alternatively, the apparent randomness of some menotactic behaviour in insects could be explained by a hardwired architecture, where some arbitrary bump offset fixes the preferred direction. The different behaviours (direct attraction or menotaxis) observed in different species and in different contexts [wood ants [7, 9, 13] and flying Drosophila [68] attracted by landmark versus walking Drosophila, innately [18], and ants, through learning [69], both keeping a landmark in a specific orientation of the visual field] argues for some flexibility in insect oriented behaviour.

The second input we assume is available to modulate the memory is the perception of rotational self-motion. The interaction of optic-flow and compass-mediated signals has already been proposed in the form of a speed measurement for the path integrator [21]. However, in our model, the implementation of a pathway carrying self-rotation estimation is required. The ability of insect visual systems to untangle of the optic flow generated by translation and rotation [70] supports our choice. In addition, self-rotation might also be perceived by a mechanosensory pathway from the legs [71], or be estimated from efference copy from the motor system [72]. Nevertheless, the implementation here is highly inferential; but could be supported by the existence of pathways between the EB and the FB involving the noduli inputs (hAB neurons [42], P-F3N2d and P-F3N2v [39]). In our model, we found that the main advantage of selecting, specifically, ipsilateral inhibition (i.e., right turns inhibit right FBns and left turns inhibit left FBns) was to facilitate the exploration and the integration of several cues (innate and memorized). Contralateral inhibition, for example, would lead to an attraction to the stronger cue and neglect of sparsely accessible cues such as the MB long-term memory (Fig E in S1 Text).

Finally, a key assumption of the model is that the CX receives a reward signal based on sensory or memory pathways. The FB represents an ideal target to model the influence of parallel sensory pathways on the CX [44] because it receives inputs from several other brain areas, including the MB [73–75], while sharing the columnar organisation observed in the EB and the PB [16]. For example, Drosophila FB has been shown to be necessary to the control of the steering in a visual pattern recognition paradigm [76], involving the EB [77]. We did not target a specific neuron type so used a generic FBn denomination. However, the parallel with CPU4a/b identified in bees as part of the path integrator makes the corresponding neurons P-F3N2d and P-F3N2v [39] plausible targets in the FB as they also make mixed synaptic connections to the noduli [39]. In addition, the CX is one of the main structures in the insect’s brain, along with the MB, receiving dopaminergic innervation, notably in the FB [78]. The role of dopaminergic input in synaptic modulation and learning processes in the MB [79] suggests a similar role in the CX to generate a synaptic memory. This is consistent with observations that link the dopaminergic circuit in the CX to the regulation to wake/sleep arousal or stimulus specific arousal [80] and more generally its proposed function to modulate the arousal threshold in Drosophila [81].

3.2 An operant strategy to find where to go

The main principle of our model is the formation of a memory during the exploration of the environment surrounding the insect under the influence of goodness/badness signals derived from innate and/or learned preferences. The exploratory behaviour here is crucial, supporting the scan of the surroundings and integrating the proprioception information correctly (Fig E in S1 Text) to ensure an optimal behaviour. Note that, although our model does not need any systematic scanning of the visual environment to decide the course to take [26], it is not able to provide a steering command to an orientation that has not been previously seen (some other proposed navigation models do have this capacity, e.g., by using the frequency transformation of the visual scenery [22, 82], or a biased memory [44]). However, a large number of insect species express systematic scanning procedures before determining a route/direction to take [e.g, ants [83, 84]; dungbeetles [85], sandhoppers [86]], supporting our model assumptions. To create an exploratory behaviour in our simulation we add noise to the steering output from the PFL3, which is needed to generate correct attraction to the cue or the feeder (Fig F in S1 Text). Adding a stochastic component as noise to the behaviour is intended to account for the effect of locomotor mechanics and/or external disturbances. Alternatively, the oscillatory behaviour observed in the path of insects (wood ants [9], cockroachs [87], locusts [61], wasps [88], moths [89]) could provide a systematic and controlled exploration of their surroundings that would serve the same function. In addition, head movements and initial systematic scanning behaviour [83] offer ways to increase the exploration of the environment while maintaining the course.

3.3 Why use the CX for innate attraction?

Our model focuses on the potential of the specific CX architecture to support different oriented behaviours (attraction, menotaxis, visual memory guided) based on inputs in the form of (undirected) attractiveness and/or repulsion. As such, the EB is assumed to play a role even in the innate attraction to a visual landmark, which we assume induces a reward signal when appearing in the frontal area of the visual field, modulating the EPG to PFL3 synaptic weights to cause steering towards it. In walking Drosophila, disrupting the EB abolishes menotaxis but not direct attraction to a cue [18]. Indeed, it seems highly likely that alternative direct sensory pathways for innate attraction exist, such as observed connections from visual lobes to steering neurons downstream from the CX [30], and are able to control the behaviour without requiring the orientation representation provided by the CX [90]. However, although CX memory may not be necessary for attraction to conspicuous cues, it might still provide several advantages in enhancing innate behaviours [43].

One is the creation of a stable heading that can persist even with a disappearance of the cue(s) (Fig 8), as long as the EB bump motion is supported by other cues (for example the proprioceptive cues maintaining heading tracking in darkness, Fig 3D). The existence of a persistent influence of a cue after its disappearance has been shown in Drosophila [28, 91], as well as the relative reliability of the EB compass in darkness [15]. Therefore, the CX could act as a smoothing filter, in addition to direct taxis mechanisms, to generate a short-term memory of the sensory world when this appears to be unsteady. For example, the experiments by Neuser et al. [28] on the short-term memory of landmark guidance with disappearing bars support the interplay between sensory taxis pathways and the formation of a short-term orientation memory. In their paradigm, Drosophila show a sequence of behaviour while the environment is modified: (i) a patrol behaviour between two bars at 0 and 180°; (ii) attraction to a new bar appearing at 90° while the others disappear; and (iii) a reversion to the patrol behaviour in the original direction, in the absence of any cue, when the newly appeared bar disappears. This would be consistent with our model if it was assumed that the CX memory formed during (i) is over-ridden, but not altered, during the brief presentation in stage (ii), and will guide the behaviour in (iii) when all landmarks disappear (c.f. Fig 8).

Note that these advantages of smoothing and persistence should also apply to enhance either menotaxis or memory-guided orientation over a direct taxis system, e.g., smoothing the integration of the MB signals that can be unsteady [44]. The integration of a sparsely accessible sensory signals could be particularly interesting considering the cluttered and dynamic environments within which insects forage. The ability of desert ants to perform homing while walking backwards would also be supported by a short term storage of the direction indicated by visual memory, which can be set during intermittent ‘peeks’ to check the direction [92]. In addition, the temporal dynamics properties of neurobiological circuits involved in the smoothing/filtering of this signal probably should include some forgetting mechanisms allowing the memory to be reversible. For the paradigm we used, with a distant landmark for an agent confined to an arena, forgetting was not necessary, but in a more complex navigation scenario where the insect may pass the landmark, forgetting, and therefore the creation of a real working memory, would be crucial. We plan to investigate in the future how this model could be applied to complex route learning/following using an integration/decay dynamics [44], or whether the integration of the path could erase the memory along a route [21].

The temporal integration at the CX level also makes it possible to explore several innately attractive features of the environment and build-up a global direction vector combining them. This individual consideration of landmarks rather than the deciphering of the global layout of the environment would match some observations in insects [93, 94]. It is also noteworthy that attraction to vertical lines in wood ants has been proposed [95] to be sustained by their functional potential to resemble tree trunks (a key source of food) and therefore could be determined by earlier life experience. That is, the nature of the attractive target may become refined as particular visual stimuli are discovered to be rewarding. This can be considered as a more general principle, in which a ‘map’ of the visual environment’s attractiveness [96] generates a heading vector, for any location, based on the EB bump. The same architecture could also be generalised to other sensory inputs, such as olfaction or wind sensing (see Fig G in S1 Text for gradient ascent capability), to generate such a combined heading vector (or a set of vectors) and enhance the robustness of the behaviour (see section 3.4).

3.4 Optimal sensory integration

In addition to its spatio-temporal properties, the convergence of several sensory pathways onto a single compass-driven scheme offers an interesting substrate to look further into optimization of the navigation behaviours. Optimal integration in biological systems has been considered to conform to Bayesian-like principles [23, 24, 97–99], as a principled way to take noise and uncertainty into account. Social insects and central-place foragers have evolved under high pressure to be able to return reliably to their nests, and hence make use of multiple redundant mechanisms, and a wide range of sensory cues including visual cues (polarised light, landmarks, sun, light gradients), magnetic fields, optic flow, odours, wind direction, tactile cues and/or proprioception [100–103]. The problem of how these are combined for robust navigation has been considered in mathematical models [23] but how it might be realised in the brain of the animal is still unknown [22]. Here, the modulation over time of the weight distribution during the exploration of the sensory environment offers a way to generate directly a combined probablility distribution from multiple inputs. An example that we observe from our simulation is the impact of negative rewards in addition to positive ones in creating the landmark attraction (Fig 7). It has been recently shown in homing ants that memorizing anti-views or repellent memories, in addition to an attractive memory, could help in determining the course to take [2, 104]. The combination of these two opposed pathways on our synaptic memory model would automatically create a combined distribution (Fig H in S1 Text), improving the homing performance as proposed in [105].

Importantly, this use of two different reward signals could be directly extended to two (or more) rewards that come from different sensory systems. For example, if the agent was equipped with mechanosensors similar to insect antennae, we could define the equal rearward deflection of both antennnae as rewarding, corresponding to an upwind heading direction. This could be combined with the existing reward when the visual landmark is centred, so that the two inputs simultaneously affect the adaptation of the weights. The relative strength of the two cues might then allow smooth transition between upwind flight when the visual stimulus is weak (far away from a learned location) to visual attraction as this cue becomes stronger or more reliable (from close range). In the model presented here, there is one set of weights between the EPG and PFL3, which can be influenced by multiple reward signals as just described. However, an alternative implementation (with similar computational effect) would be to assume the FB connections provide pre-synaptic modulation of the EPG-PFL3 connections, and the extent of this modulation is under adaptation by the reward signal. This alternative opens up the interesting possibility of having multiple sources of compass information integrated as parallel streams, i.e., several sets of FBns that, instead of copying the EPG compass activity, have an activity bump driven by independent directional cues, such as wind [106–108], or celestial information [101]. Indeed, the summation of several sensory compasses offer a good basis for theoretically optimal cue integration [22]. The combination of different compasses under a single reward provided by sensory and/or memory pathways might allow their flexible re-use depending on their availability and their reliability in the local environment.

Here we combined two different systems for visual reward, one innate and one learned. The interaction between innate and learned modalities is important for insect navigation [7, 102]. We considered the simplest case where the influence of each pathway is added linearly. The exploration of different integration strategies could help us better understand the interaction between innate and learned behaviours. We therefore believe that this model will help in exploring the ontogeny of the transformation from innate to learned landmark use [7, 109].