Bodily motion fluctuation improves reaching success rate in a neurophysical agent via geometric-stochastic resonance

Organisms generate a variety of noise types, including neural noise, sensory noise, and noise resulting from fluctuations associated with movement. Sensory and neural noises are known to induce stochastic resonance (SR), which improves information transfer to the subjects control systems, including the brain. As a consequence, sensory and neural noise provide behavioral benefits, such as stabilization of posture and enhancement of feeding efficiency. In contrast, the benefits of fluctuations in the movements of a biological system remain largely unclear. Here, we describe a novel type of noise-induced order (NIO) that is realized by actively exploiting the motion fluctuations of an embodied system. In particular, we describe the theoretical analysis of a feedback-controlled embodied agent system that has a geometric end-effector. Furthermore, through several numerical simulations we demonstrate that the ratio of successful reaches to goal positions and capture of moving targets are improved by the exploitation of motion fluctuations. We report that reaching success rate improvement (RSRI) is based on the interaction of the geometric size of an end-effector, the agents motion fluctuations, and the desired motion frequency. Therefore, RSRI is a geometrically induced SR-like phenomenon. We also report an interesting result obtained through numerical simulations indicating that the agents neural and motion noise must be optimized to match the prey’s motion noise in order to maximize the capture rate. Our study provides a new understanding of body motion fluctuations, as they were found to be the active noise sources for a behavioral NIO.


Introduction
The bodily movements of a biological system are noisy because of the stochastic nature of biological sensory, neural, and actuation systems. Sensory and neural noise induce stochastic resonance (SR), which provides a variety of benefits to an organism. These benefits include improvements in information transmission to and through the neural system. Furthermore, sensory and neural noise enhance cognitive performance [1,2], reflexes [3], feeding [4][5][6], stochastic action selection [7], and memory-perception balance [8,9] (for other benefits of SR, see the following reviews [10][11][12]). In contrast to the tremendous known benefits of neural a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 and sensory noise, the benefits of the noise inherent in motion fluctuations are largely unclear. In fact, only a few SR-like benefits have been proposed for this type of noise (e.g., improvements in visual acuity due to eye tremor [13]).
The major reason that motion fluctuation is not considered to be a source of behavioral SR or other noise-induced order (NIO) is that it exhibits a Lorentz-type spectrum and long-term correlation. In fact, it has been shown that colored noise in general degrades the SR effect in a nonlinear system [14,15]. Because the motion fluctuations of relatively large-bodied animals, such as mammals, reptiles, and fish mostly have long correlation times, they cannot be used directly as sources of noise to induce SR in a nonlinear system. However, these motion fluctuations can still be helpful for a neurophysical agent. We find that by using the measure of reaching success rate, we can observe a novel kind of NIO. Furthermore, we report that the bodily motion fluctuations of a neurophysical agent provide aperiodic and stochastic input signals to a feedback motion controller consisting of neurons. This leads to the emergence of neural aperiodic SR [16][17][18].
In the following article, we report the results of our theoretical analysis of the reaching success rate improvement (RSRI) of a Brownian particle that is controlled to reach a periodicallymoving target. Next, we demonstrate that the RSRI ratio is dependent on the geometric size of an end-effector used to catch a target. We further show the RSRI is a novel NIO based on the mechanism of geometric-stochastic resonance (GSR), wherein the geometric size of the endeffector, the frequency of the target movement, and the motion noise intensities interact with each other and improve reaching success ratio. As an applicative and more general experimental framework of GSR, we consider a numerically-simulated neurophysical agent with a twodimensional body and a neural motion controller consisting of two arrays of FitzHugh-Nagumo neurons. Furthermore, we consider two experimental setups implemented using numerical simulations: a static reaching task, wherein the agent tracks along a predesigned path, and a dynamic capturing task, wherein the agent captures randomly moving objects.

Theoretical basis of RSRI via GSR
We consider an overdamped Brownian particle driven by a feedback controller as where x is the position of the particle, x g is the pre-designed goal position at time t 0 and x g (t 0 ) = 0 cos(f 0 t 0 ), K is the feedback gain, D 0 is the noise intensity, and ξ is the Gaussian noise of the unit standard deviation [Fig 1(A)]. By introducing a new timescale t = (λ + K)t 0 , we can eliminate the prefactor of the term (λ + K)x which appears on the right hand side, and Eq (1) is transformed to the standard form of the Langevin equation The probability density function of the particle position P(x, t) is fully described by the following Fokker-Planck equation: where @ x denotes the operator @/@ x . The exact solution of Eq (3) is provided by the Ref. [19] as , α = arccos(g). The analytical form for the reaching success rate P R (t), which is the probability that the agent reaches the range of [x g (t) − θ, x g (t) + θ] is computed by where s(t) = A cos(ft), A is the effective amplitude of x g , and is 0 /(λ + K), B = c − gc α , c = cos (ft), = 0 K(λ + K), c α = cos(ft + α). Note that the analytical form of the ensemble average of where hzi denotes the ensemble average of z. The peak P R (t) is calculated using D = R e (θB/log(±Q)), where Q ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðÀ y À BÞ=ðy À BÞ p , and R e () denotes taking the real part. This indicates that the optimal noise intensity needed to maximize reaching probability is dependent on the interplay between the geometrical size of the end-effector and the drive frequency.
The probability hP R i = hP R (t = 1/f)i exhibits two distinct modes based on the balance between θ and B, as shown in Fig 1(F). Surprisingly, lim D!0 hP R i is limited to either 1 or 0, as lim D!0 The condition θ B corresponds to an unreachable regime, where the agent cannot reach the target position without the help of noise, and hP R i is maximized by the optimal noise intensity, as shown in Fig 1(B), 1(C) and 1(F). In contrast, θ > B corresponds to a reachable regime and hP R i = 1 by D = 0, as shown in Fig 1(D) and 1(F). It may be counterintuitive that hP R i increases following increases in f (i.e., a higher frequency leads to a more reachable condition) (Fig 1(B) and 1(C)). This is because an increase in f results in a decrease in B, as shown in Fig 1(E), and θ ! B leads to hP R i > 0.

Design of simulated neurophysical agent
As an applicative and more general experimental framework for RSRI via GSR, we consider a two-dimensional particle system driven by a nonlinear feedback controller consisting of two arrays of FitzHugh-Nagumo neurons. The motion dynamics of the particle system are described as where v is the velocity vector of the agent, f is the force generated by a neural motion controller, γ is the friction coefficient and γ = 0.6 throughout this article, ξ m (t) is a Gaussian noise vector of unit intensity, and D m is the motion-noise intensity. The motion controller is designed to receive a feedback signal s(t) and outputs the force f(t) based on the neuronal firing rate of two ensembles of FitzHugh-Nagumo (FHN) neurons, one for each dimension, as where K is the feedback gain, R 0 is a offset variable, and R(t) is the firing rate of the neuron ensemble. The dynamics of the ith FHN neuron of the jth ensemble is expressed as where = 0.005, b is the bias signal, ξ b is a Gaussian noise of unit intensity, D b is the bias variability and D b = 0 unless otherwise stated, s j (t) is the input feedback signal to the jth neuron ensemble, V is the fast variable, and W is the slow recovery variable. Independently of the motion additive noise D m ξ m (t), a neuron ensemble receives additive noise D s ξ(t) where ξ(t) is the Gaussian noise of the unit standard deviation and D s is the noise intensity. The firing event R j i ðtÞ of a neuron is computed by The mean firing rate of the jth ensemble is computed as R j ðtÞ ¼ 1=N P N i¼1 R j i ðtÞ (N = 500 unless otherwise stated). The ensemble is in an excitable regime for b 0.274, and is in an oscillatory regime (i.e., neurons spontaneously fire without any signal input) for 0.274 The input signal s(t) encapsulates the computation performed by the neuron ensemble, namely that of a positional PI (proportional-integral) feedback controller, as where K I is gain for the error-integral control and g is the input gain. Furthermore, x(t) and x g (t) are the current and desired agent positions, respectively. Note that x g (t) is predesigned in the path-tracking experiment in Fig

Numerical simulation of static reaching task
We consider two different kinds of tasks. First, we consider a static goal-reaching task. This task consists of static path planning and path tracking. A static reaching task is the standard motion control task for a biological system, e.g., arm-reaching or eye movement [20][21][22], and for robot navigation. In this standard goal-reaching task, we assume that the target is fixed. Agent motion fluctuations are influenced by the two noise sources, (1) the motion additive noise D m ξ m (t), and (2) the neural additive noise D s ξ(t).
In the numerical simulation shown in Fig 2(A), the agent is controlled to visit the four goal positions (X i , Y i ) = (cos(1/4π + π/2i), sin(1/4π + π/2i)) (where i = 0, 1, . . ., 3) sequentially, switching to a new goal every T [s] ((i = 0, 1, . . ., 3)). When the agent goal is switched to the next one, the line from the agent's current position x(t = kT) to the next goal position t = (k + 1)T is equally partitioned into T/Δt sub-goals x g . The agent is controlled to track this pre-

Fig 2. Neurophysical agent design and task setup. (A, B)
Two different numerical simulation setups for studying behavioral NIO. In setup (A), we study the NIO when a neurophysical agent tracks along a static predesigned path. In setup (B), we study the NIO that occurs when the agent captures randomly moving (i.e., noisy) targets. In the second paradigm, we consider not only the additive neural and force noises internal to the subject agent, but also the motion noise of the moving target. https://doi.org/10.1371/journal.pone.0188298.g002 designed path x g (t). The offset R 0 is computed as the time-average of R of the initial 100s in every task simulation and this initial period is excluded from the performance analysis.

Performance measures of motion accuracy
Because the agent is scheduled to reach the kth goal at t = kT [s], we compute the distance d e of the agent position from the kth goal every T [s], and use d e as a linear measure for the static goal-reaching task. The ensemble average of d e over different simulation trials, hd e i is computed as Furthermore, we use the average motion error he m i as another linear sensorimotor performance index.
where T e is a sufficiently long time. That is, he m i is computed by averaging the error across all simulation time steps.
A nonlinear performance measure: Goal-reaching success rate In addition to the measures d e and e m , we use a measure that is obtained by applying a nonlinear function to d e . A straightforward example of such a nonlinear measure involving using a threshold function to digitize the distance d e is where d e is the distance of the agent from the goal position. Note that the measure S R (d e ) is applicable to many biological tasks, such as capturing prey, and to reaching tasks, where a system's physical body is required to be within a certain range of an object within a certain time period. We use the ensemble average of the goal-reaching success rate hP R i as a task evaluation measure. S k R ðd e Þ is calculated every at t = kT [s], and the ensemble average of P R , hP R i is computed as Numerical simulation of dynamic capturing task In the numerical simulation shown in Fig 2(B), we consider a dynamic reaching task where the goal position (i.e., the position of the target objects) moves. In this setup, we use Brownian particles as target objects. Therefore, the motion of the target object provides an additional source of agent motion fluctuation. Note that in the framework (B), there is no pre-designed path to track and the agent goal position is updated at every time step based on the movement of the target object.
In the simulation environment, N p moving target objects are located randomly within range [−L, L] (we use N p = 100 and L = 7.5 in this paper), as shown in Fig 2. The ith target object moves randomly based on the dynamics where m g = 0.1, v g,i is the velocity, z i (t) is the Gaussian noise of unit variance, and D p is the noise intensity. The target object is subjected to a force F w from a virtual wall when it moves beyond the region [−L, L].
The agent is controlled to pursue the position of a moving target. That is, x g (t) = x p (t)+v p (t) Δt, where x p (t) and v p (t) are the position and the velocity of the current moving target, respectively. When the distance d p between the agent and the target satisfies d p < θ, the moving target is "captured" and is removed from the simulation. After the agent captures a certain target, the target is switched to the nearest moving object. The offset R 0 is computed as the time-average of R of the initial 300s in every task simulation and this initial period is excluded from the performance analysis.
The capturing rate per unit time is C r , which provides a nonlinear measure of the agent's sensorimotor performance on this task. This is computed as where Θ(z) is the Heaviside step function.

Benefits of noise in the behavior of a neurophysical agent
For D s = 0 and D m = 0, the system exhibits fully deterministic behavior, although it may exhibit jittering due to the overshooting characteristics of a simple feedback controller. In this deterministic regime, the neural system exhibits a totally synchronized firing pattern across different initial neuronal conditions (see R T e 0 ðk sðtÞ kk RðtÞ kÞ dt. We can see that there exist two kinds of SR-based motion accuracy improvements: a subthreshold SR corresponding to a small input gain g [Fig 4(E)], and a suprathreshold SR corresponding to a large input gain g [Fig 4(F)]. With the small input gain g * 0, the agent motion tends to overshoot the desired path due to the poor information transmission to the motion controller [Fig 4(E), bold gray line for D s = 0 and solid red line with D s = 0.06]. In contrast, with the large input gain g ) 0, the agent motion tends to oscillate around the desired path due to the hard synchronization of neuronal firing [Fig 4(F), bold gray line for D s = 0]. This synchronized neuronal firing is due to the poor pooling ability of the motor controlling neurons (the panel (F) is obtained using D b = 0.1). The large bias variability D b ) 0 can improve the pooling ability in the noiseless neural controller D s = 0. However, it must be noted that D b ) 0 obscures the SR effect for D s > 0. Furthermore, the motion accuracy provided by SR is higher than the motion accuracy realized using noiseless pooling with D b ) 0 and D s = 0, as shown in panel (G). Note that g ) 0 is exactly the case wherein SR growth [18] occurs. Furthermore, we could not find any motion accuracy improvements due to the presence of force noise (Fig 5).

Improvement in static reaching success rate
The distance d e , which was used in the previous study, represents the "linear" difference between the agent and the position of the goal. Because this difference is linear, d e exhibits a monotonic increase in response to the intensity of the additive force noises. This implies that if we use a certain measure with a nonlinear dependence on the agent and goal positions, we may observe benefits of motion noise in the sensorimotor task. In fact, as expected from the theory of GSR, we observe the benefits of the noise when we use reaching success rate as a measure, as shown in Fig 6.  Fig 6 shows the goal-reaching success rate as a function of the neural and motion additive noises D s and D m in the experimental setup shown in Fig 2(A). With the parameter sets shown in Fig 6(A), 6(B), 6(D) and 6(E), the agent cannot achieve a good reaching success rate using the default deterministic feedback control because force feedback gain K is not sufficient. In this "deterministically unreachable" parameter region, the combination of nonzero neural noise and nonzero motion noise leads to an improvement in the goal-reaching success rate. This realization of RSRI can be interpreted as a result of the interplay between the motion noise and the neural noise: motion noise generates an aperiodic input signal in the neural system, as shown in Fig 3(S1), 3(R1), 3(S2) and 3(R2), and neural noise generates an aperiodic neural SR [16][17][18]. Note: in the deterministically reachable region in > 0 for D s = 0 and D m = 0), the P R improvement effect due to motion noise is in principle unobservable (because P R = 1 by default).

Improvement in the capturing rate due to environment-agent noise balancing
In the dynamic reaching experiment, we consider a task where the agent is controlled to capture moving prey. Furthermore, we investigate how the capture number per unit of time depends on the motion fluctuations of the agent and those of the moving targets. The dependence of capture rate hC r i on D m and D p is shown in Fig 7(A) and 7(B), with K = 0.5 and θ = 1. Clearly, hC r i is improved with the presence of motion additive noise D m . Interestingly, hC r i is also improved with the presence of D p , which indicates the prey's motion noise. Furthermore, hC r i is a function of D p , D m , and D s . These interesting results imply that the ability to capture is a function not only of agent motion and neural noise, but also of the prey's motion noise. From the point of view of the capturing agent, the neural and motion noises must be adjusted to match the intensity of the prey's motion noise. On the other hand, for the prey to avoid being captured, it must adjust the intensity of its motion noise away from that of the agents. This experimental result implies that biological systems in the context of survival competition will control their neural and motion noise intensities based on their environmental noise levels.
The improvement in the capturing rate hC r i due to D m is dependent on the size of the geometric threshold θ, as shown in Fig 8. It may be reasonable to presume that a larger biological agent can more efficiently exploit the GSR that is induced by motion fluctuations when capturing small targets.
Ref. [4][5][6] report that the feeding rate of paddlefish (capturing rate of planktons per minute) is improved in the presence of electrical sensory noise. Conventionally, this feeding behavior improvement has been thought to be the result of sensitivity improvement due to the presence of electrical sensory noise. Our results may imply that, in addition, the electrical signal noise induces motion fluctuations in the capturing agent, which then help to improve the capture rate.
The effect of motion noise on the capturing task is summarized as follows: motion noise enables reaching that is not obtainable deterministically, and the optimal motion noise intensity is determined by a balance with the target motion noise.

Dynamic capturing rate improvement using a simple PI feedback controller
To determine whether our results have general applicability, we obtained experimental data using a simple PI motion controller that did not have any neurons. The force output of this  005 for (B). The error bars indicate standard deviations. Note that we could not find any significant improvements due to the presence of force noise D m . Numerical he m i is computed from 500 trials of a 500 s numerical simulation, and the error bars correspond to the standard error.
simple PI motion controller is described as where e(t) = x g (t) − x.
Although several parameter adjustments are required, it is possible to reproduce the results of Figs 7 and 8. show the improvement in the capture rate. These results support the idea that behavioral SR in reaching and capturing tasks is a general property of feedback-controlled physical agents.

Discussion
We investigated NIO in the context of sensorimotor coordination in a neurophysical 2D particle agent. The motion controller of the agent consisted of an FHN neuron ensemble. The addition of neural noise to the controller led to an improvement in the agent's motion accuracy, as shown in Fig 4(A)-4(C). The motion accuracy improvement by the addition of neural noise would be primarily a consequence of the neural SR that optimizes the controller feedback output, i.e., the maximization of ρ as shown in Fig 4(D). It must be noted that the addition of neural noise decreases the variance of a neural output [i.e., hN o i shown in Fig 4(D) decreases by the addition of neural noise]. Because the addition of force noise to the agent body Although motion fluctuations per se degrade motion accuracy, we found that a nonlinear performance measure such as goal-reaching success rate can exhibit the emergence of an NIO induced by the the motion fluctuations. Particularly, we found that motion fluctuations improved the goal-reaching success rate hP R i, as shown in Figs 6-9.
Interestingly, for a neurophysical agent hP R i was a function of force additive noise intensity D m and neural additive noise D s as shown in Fig 6. Furthermore, in a capturing task where not only the neurophysical agent but also the prey's motion was noisy, the capturing success rate hC r i was a function of agent force noise intensity D m , neural noise intensity D s , and prey's motion noise intensity D p as shown in Figs 7 and 9. These results imply that biological systems may handle the balancing of motion and neural noise dependently on the environmental noise. Task success ratio as a marker of NIO It should be noted that we did not find any benefit of motion fluctuation based on measures of the relationship between predesigned paths and the actual motion trajectories. Likewise, we could not find SR-like benefits using measures of the distances among the goal positions and the final agent positions. Only when using measures of the discretized state probability, such as goal-reaching success rate and capturing rate, did we observe SR phenomena induced by motion fluctuation.
It is worth noting that most of the conventional systems for studying SR require a discretized dynamical representation or a digitized output. These systems include traditional double-well potential systems, threshold systems, the spiking neuron, and two-state dynamical systems (e.g., the FitzHugh-Nagumo neuron). These systems are capable of digitizing the input signals or generating digitized output (for reviews of the conventional SR-capable systems and frameworks, see Refs. [11,[23][24][25][26]).
Furthermore, the conventional framework for detecting a weak signal requires a discretized measure (i.e., detection or no-detection). In behavioral frameworks, responding to weak and subtle sensory signals also requires a digitized response (i.e., response or no-response). A recent concept for studying SR, called entropic SR [27] and GSR [28,29], posits that Brownian particles move between two rooms connected by a narrow aperture. In this case, the setup of the two rooms provides the state digitization. (Note that the mechanism of GSR described in Eq (5) would be very different from the conventional mechanism of GSR reported in Refs. [28,29], although both studies share the common characteristic that the interaction of noise and geometric constraints induce NIO). In this manner, a digitized measure, or digitizing dynamics, may be implicit requisites for observation of the SR. At present, this idea is only an inference from analogy and requires theoretical analysis.
Measures of discretized-state probabilities, such as the goal-reaching success rate and capturing rate can be generalized as task success ratios. It should be noted that the measure "task success ratio" is a highly nonlinear function of a variety of arguments, such as the appropriateness of feedback gain, neural noise intensity, neuron size, input signal gain, input-output information, and the distance from the goal position. Therefore, the task success rate may be able to implicitly represent the extent to which calculations of these arguments are improved by some intervention.

Conclusion
In this paper, we investigated the prospective benefits of the bodily motion fluctuations of an embodied physical agent. We considered a static path-tracking task and a dynamic capturing task for moving prey agents. We found that motion fluctuations degrade motion accuracy, but improve the reaching success rate and capturing rate. These results imply that a biological agent may exploit bodily motion fluctuation in several behavioral tasks, such as reaching, capturing, and navigation, by adjusting the intensity of the motion noise.