Weak-Periodic Stochastic Resonance in a Parallel Array of Static Nonlinearities

This paper studies the output-input signal-to-noise ratio (SNR) gain of an uncoupled parallel array of static, yet arbitrary, nonlinear elements for transmitting a weak periodic signal in additive white noise. In the small-signal limit, an explicit expression for the SNR gain is derived. It serves to prove that the SNR gain is always a monotonically increasing function of the array size for any given nonlinearity and noisy environment. It also determines the SNR gain maximized by the locally optimal nonlinearity as the upper bound of the SNR gain achieved by an array of static nonlinear elements. With locally optimal nonlinearity, it is demonstrated that stochastic resonance cannot occur, i.e. adding internal noise into the array never improves the SNR gain. However, in an array of suboptimal but easily implemented threshold nonlinearities, we show the feasibility of situations where stochastic resonance occurs, and also the possibility of the SNR gain exceeding unity for a wide range of input noise distributions.

There has been considerable interest in the amplification of the signal-to-noise ratio (SNR) of a periodic signal by exploiting the SR effect [2,8,[53][54][55][56][57]. From the viewpoint of the gain behavior, i.e., the SNR at the output divided by that at the input, the primary issue of a gain exceeding unity has been found for suprathreshold input signals [54][55][56][57]. However, most previous SR studies involved a fixed nonlinearity. When we consider an arbitrary adjustable static nonlinearity, the maximum SNR gain is achieved by a locally optimal nonlinearity for a weak periodic signal in additive white noise [29,58]. Since the SNR gain of a locally optimal nonlinearity is given by the Fisher information of the noise distribution, we demonstrated that the SNR gain of a locally optimal nonlinearity certainly exceeds unity for a weak periodic signal in additive non-Gaussian noise, and SR does not exist in an updated locally optimal nonlinearity [58]. However, the structure of the locally optimal nonlinearity is determined by the noise probability density function (PDF) and also the noise level [58,59]. Then, in some practical signal processing tasks, the locally optimal nonlinearity may be too complex to be implemented, and also can not be established for an unknown noise distribution [59]. Therefore, this provides an opportunity for the suboptimal nonlinearity to improve the SNR gain by the SR effect [3,4,7,8,[25][26][27][28][29]58].
In this paper, we focus on amplifying the output-input SNR gain of an uncoupled parallel array of static nonlinearities for transmitting a weak periodic signal in additive white noise. For an array of arbitrary static nonlinearities, the asymptotic expression of the SNR gain is first developed. Then, for a given nonlinearity and fixed noise levels, we prove that the SNR gain of an array is a monotonically increasing function of the array size. It is shown that the SNR gain maximized by the locally optimal nonlinearity is the upper bound of the performance of an array of static nonlinearities. Furthermore, it is demonstrated that the internal array noise components are incapable of further improving the SNR gain for locally optimal processing. This result extends the study of SR in a single static nonlinearity [58,60] to an array of static nonlinearities. The establishment of a locally optimal nonlinearity needs the complete descriptions of the noise PDF and the noise level. Therefore, when this is not feasible, we propose instead a parallel array of suboptimal but easily implemented threshold nonlinearities for transmitting a weak periodic signal, in order to improve the SNR gain via the SR phenomenon. It is shown that such an array of threshold nonlinearities exhibits the SR effect by increasing the array noise level and the array size. Moreover, with a sufficiently large array size, the fact of the SNR gain exceeding unity is shown for a wide range of underlying noise distributions. These interesting results demonstrate that a parallel array of threshold nonlinearities can be practically exploited, and is useful for nonlinear signal processing.

Model
Consider the observation of a process x(t)~s(t)zj(t), where the component s(t) is a weak periodic signal with a maximal amplitude A (Ds(t)DƒA) and period T, and zero-mean additive white noise j(t), independent of s(t), having a PDF f j and variance s 2 j~E j ½x 2 ~Ð ?
{? x 2 f j (x)dx. Next, the input x(t) is applied to an uncoupled parallel array of N identical static nonlinearities. In these nonlinearities, the noise terms g n (t), independent of x(t), are the internal noise components for each static nonlinearity g, so as to yield the outputs [26]. y n (t)~g(x(t)zg n (t)), n~1,2, Á Á Á , N: Here, assume that the derivative g'(z)~dg(z)=dz exists for almost all z, and g has zero mean under f z , i.e. E z ½g(x)~0, which is not restrictive since any arbitrary g can always include a constant bias to cancel this average [59]. The internal noise components g n (t) are mutually independent and identically distributed (i.i.d.) with the same PDF f g and variance s 2 g . The noise components j(t) and g n (t) are all assumed to be stationary random variables. Since j(t) and g n (t) are independent, Eq. (1) can be rewritten as y n (t)~g(s(t)zz(t)), where the composite noise components z(t)~j(t)zg n (t) are with the same convolved PDF f z (x)~Ð ?
Then, the array output y(t) is given by The input SNR for x(t) can be defined as the power contained in the spectral line at 1=T divided by the power contained in the noise background in a small frequency bin DB around 1=T, this is with Dt indicating the time resolution or the sampling period in a discrete-time implementation and the temporal average defined as . Since s(t) is periodic, y(t) is in general a cyclostationary random signal with period T [3]. Similarly, the output SNR at y(t) is expressed as where the nonstationary expectation E z ½y(t)~E z ½y n (t)~Ð ? {? y n (t)f z (x)dx and nonstationary variance var½y(t)~E z ½y 2 (t){E 2 z ½y(t) are also temporal functions of time t [3]. Then, the SNR gain, G N , is defined as the ratio of the output SNR over the input SNR [3,26,57] for an array of static nonlinearities with array size N.

SNR Gain of an Array for Weak Signals
For a weak signal s(t) (A?0 and Ds(t)DƒA) and at a fixed time t, we make a Taylor expansion of g around z and have the asymptotic form where the outputs y n are i.i.d. for n~1,2, Á Á Á , N. The output degree-two moment is given by where E z ½ : ~E j fE g ½ : g and E g ½y n ~E g ½y m . Therefore, based on Eq. (6), we have where the approximations are up to first order in the small signal s(t). Substituting the asymptotic forms of E z ½y(t) of Eq. (6) and var½y(t) of Eq. (8) into Eq. (5), we obtain the asymptotic expression of the SNR gain of a parallel array of static nonlinearities as where terms 2s(t)E z ½g(z)g'(z) and 2s(t)E j fE 2 g ½g'(z)g(z)g, compared with primary terms E z ½g 2 (z) and E j fE 2 g ½g(z)g, are neglected as A?0 (Ds(t)DƒA). It is interesting to note that the SNR gain G N in Eq. (9) is applicable for an arbitrary weakperiodic signal s(t) throughout an array of static (yet arbitrary) nonlinearities.
For the random variable g(jzg) and the convex function x 2 , by the Jensen inequality [61], we have for any fixed variable j [61]. Therefore, we have From Eq. (11) and for any integers K §N §1, we have Thus, for the given nonlinearity g and fixed noise components j(t) and g n (t), the SNR gain G N in Eq. (9) is a monotonically increasing function of the array size N. From Eq. (9), we have the minimum.
for N~1, and the maximum for N~?. Naturally, Eq. (12) inspires us to consider the increase of array size N for the further improvement of the SNR gain obtained by a single nonlinearity. We will demonstrate in Eq. (17) that this thought is infeasible for locally optimal processing.
Without the internal noise g n (t), Eq. (9) becomes where the array size N does not work, and the equality occurs as g becomes a locally optimal nonlinearity for the derivative f ' j (x)~df j (x)=dx (without loss of generality C~{1) [29,59]. Here, I(f )~E½f ' 2 =f 2 is the Fisher information of the noise distribution f [61]. We add the extra noise g(t) to the observation data X , aiming to improve the performance of g opt . However, based on the Fisher information in inequality I(f z )ƒ min (I(f j ),I(f g )) [61], we have where the later inequality indicates that the addition of extra noise cannot improve the performance of a single locally optimal nonlinearity g opt [58]. Based on Eq. (12), the SNR gain of an array of arbitrary static nonlinearities attains its maximum G ? in Eq. (16). Using the Cauchy-Schwarz inequality, we have where the nonlinearity E g ½g(jzg) is a function of j. The equality occurs as the nonlinearity E g ½g opt (jzg)~Cf ' j (j)=f j (j), i.e. the nonlinearity g opt (j)~Cf ' j (j)=f j (j) and the PDF f g (x)~d(x).
Here, d(x) is the Dirac delta function, and this means there is no internal noise in the nonlinearity. From Eqs. (17), (19) and (20), this result indicates that the upper bound of the SNR gain G ? is achieved by g opt in Eq. (18) without the internal noise g n (t).
Therefore, the addition of internal noise components g n (t) to the signal is never helpful for improving the SNR gain that is obtained by the locally optimal nonlinearity of Eq. (18). Thus, Eq. (17) extends our previous result of the incapability of SR in the SNR gain improvement of a single locally optimal nonlinearity [58] to the configuration of the array of static nonlinearities. For instance, consider the Gaussian noise components j(t) and g n (t) with PDFs f j (x)~exp ({x 2 =(2s 2 j ))= ffiffiffiffiffiffiffiffiffiffi 2ps 2 j q and f g (x)~exp ({x 2 =(2s 2 g ))= ffiffiffiffiffiffiffiffiffiffi 2ps 2 g q , respectively. Then, the composite noise z(t) is also Gaussian distributed with PDF f z (x)~exp ({x 2 =(2s 2 z ))= ffiffiffiffiffiffiffiffiffiffi 2ps 2 z p and variance s 2 z~s 2 j zs 2 g . The Gaussian distribution corresponds to the locally optimal nonlinearity g opt (x)~x [58]. Substituting g opt into Eq. (9), we have where E j fE 2 g ½g opt (jzg)g~s 2 j and the Fisher information of f j is I(f j )~1=s 2 j [59]. In Eq. (21), it is seen that the increase of noise variance s 2 g only degrades the SNR gain. The upper bound of unity can be only achieved for the infinite array size N~? or s 2 g~0 by the locally optimal nonlinearity of Eq. (18).

Noise-enhanced Signal Transmission in Arrays
It is seen in Eq. (17) that, for transmitting a weak periodic signal in additive white noise, the addition of internal array noise to an uncoupled parallel array of nonlinearities is incapable of improving the SNR gain of the locally optimal nonlinearity g opt . However, the structure of g opt in Eq. (18) depends on the complete description of the noise PDF and the noise level, and in practice it may be difficult to obtain an explicit analytical expression of g opt in the unknown noisy environment [59]. Moreover, the presence of internal noise g n (t) is unavoidable in some practical signal processing cases [5,6,[38][39][40][41]59]. Thus, we place suboptimal but easily implemented nonlinearities in a parallel array to transmit a weak periodic signal, and then show the feasibility of the SR phenomenon [1,3,26].
In the observation model of Eq. (1), the external noise j(t) is considered as zero-mean generalized Gaussian noise, which is a flexible family containing some common important cases (e.g. Gaussian noise and Laplacian noise) [27,59,61]. The generalized Gaussian noise j(t) has PDF where c 1~a 2 C for a decay exponent aw0 [59]. The array noise terms g n (t) are assumed to be i.i.d. uniform noise with the same PDF for {bƒxƒb (b~ffi ffi ffi 3 p s g w0) and zero otherwise. When the exponent a~2, Eq. (22) represents the PDF of Gaussian noise j(t). In this case, the signal s(t) is buried in the composite noise z(t)~j(t)zg n (t). Then, the corresponding locally optimum nonlinearity g opt of Eq. (18) needs to be updated as with Q(x)~Ð ?
x exp ({t 2 =2)= ffiffiffiffiffi ffi 2p p dt. It is seen in Eq. (24) that the structure of g opt is rather complicated and depends closely on the noise root-mean-square (RMS) amplitudes s j and s g . An illustrative plot of the locally optimum nonlinearity is shown in Fig. 1 for s j~sg~1 .
A suboptimal but easily implemented nonlinearity that we consider is the three-level threshold nonlinearity [3,55] with the sign or signum function sign( : ) and the response threshold h. Furthermore, the SNR gain G N of a parallel array of threshold elements is plotted in Fig. 2 as a function of the RMS amplitude s g of the array noise g n (t) and the array size N. Here, the Gaussian noise j(t) is with RMS amplitude s j~1 , and the response threshold of g th takes h~1. From the bottom up, the SNR gain G N is shown for N~1,2,5,100,500 and ? in Fig. 2 (solid lines). It is seen in Fig. 2 that, for an isolated static nonlinearity g th (N~1), the SR effect does not appear, and the SNR gain decreases monotonically as s g increases. The SNR gain G opt 1 of a single locally optimum nonlinearity g opt is also plotted in Fig. 2 (dashed line). It is seen in Fig. 2 that the SNR gain G opt 1 of g opt is always better than G 1 of a single threshold nonlinearity g th (N~1). However, as the array size N §2 and the array noise RMS amplitude s g increases, G N of the array of threshold nonlinearities gradually catches up, and finally exceeds G opt 1 of the isolated locally optimum nonlinearity g opt , as shown in Fig. 2. Additionally, upon increasing the array size N, the bell-shape behavior of G N of a parallel array of threshold elements versus s g and N is clearly visible, this is the array SR effect. It is also noted in Fig. 2 that, for a sufficiently large array size Nw500, the SNR gain G N tends to its upper limit of G ? for N~?. Of course, based on Eq. (17), the upper limit of G ? is less than the quantity of s 2 j I(f j )~1 achieved by the locally optimal nonlinearity g opt in Eq. (18) (without the internal noise g n (t)), as shown in Fig. 2.
Next, an interesting question is, for transmitting a weak periodic signal, whether the SNR gain G N of an array of threshold nonlinearities can exceed unity or not. This possibility, for the case of SNR gain exceeding unity, is shown for Laplacian noise j(t) with a~1 in Eq. (22). In this case, when the array noise components g n (t) are i.i.d. uniform random variables, the locally optimum nonlinearity should be updated as where F j (x)~½1zsign(x)(1{ exp ({ ffiffi ffi 2 p DxD)=s j )=2 is the cumulative distribution function of Laplacian noise j(t). For the noise RMS amplitudes s j~sg~1 , an illustrative example of the structure of g opt is plotted in Fig. 3. Furthermore, when we fix s j~1 and tune s g , the SNR gains G N of an array of threshold elements with the threshold h~1 are presented in Fig. 4. It is seen in Fig. 4 that, for a sufficiently larger array size N §5, the fact of the SNR gain G N exceeding unity is clearly demonstrated in this case. It is also interesting to note in Fig. 4 that SR effect survives for a single threshold nonlinearity g th with N~1.
In Eq. (12), it is known that the performance of an array of nonlinearities increases monotonically with the array size. As indicated in Figs. 2 and 4, we advocate the significance of a parallel array of nonlinearities with large array size N: The region of the noise level that improves the SNR gain of an array is gradually expanded as the array size N increases. Thus, increasing the array size N provides a simple alternative means of improving the performance of nonlinearities, especially when the optimal noise associated with a single nonlinearity [28,30] is not known or accessible.
We also emphasize that the fact of the SNR gain exceeding unity is not exceptive. Here, we employ an array of g th threshold elements with the threshold h~0 and the array size N~100. The external generalized Gaussian noise j(t) is with the RMS amplitude s j~1 . The array noise is uniform noise with its RMS amplitude s g~1 . It is shown in Fig. 5 that, for a sufficiently large array size N~100, the SNR gain G 100 (red line) can be larger than    . Output-input SNR gain G N . Output-input SNR gain G N as a function of the RMS amplitude s g of the array uniform noises g n (t) in the array of threshold nonlinearities of Eq. (25). The external noise j(t) is with the RMS amplitude s j~1 and the decay parameter a~1 (Laplacian noise). The threshold of g th takes h~1. The SNR gain G N of Eq. (9) is plotted by black lines for N~1,2,5,100,500 and ? (from the bottom up). For comparison, the SNR gains G opt 1 (blue line) of the locally optimum nonlinearities g opt in Eq. (26) and the quantity of s 2 j I(f j )~2 (red line) achieved by the locally optimal nonlinearity g opt in Eq. (18) (without the internal noise g n (t)) are also illustrated. doi:10.1371/journal.pone.0058507.g004 unity for the decay exponent av1:83, which represents a wide range of generalized Gaussian noise distributions. Here, the black line indicates the upper limit of [58,59] that the SNR gain G ? cannot exceed, as Eq. (20) indicated.

Discussion
In this paper, for a weak periodic signal in additive white noise, we study the characteristics of the SNR gain of an uncoupled parallel array of arbitrary static nonlinearities. Under the assumption of weak signal, an explicit expression of the SNR gain of an array is developed. Then, it is proven that, for a given nonlinearity and fixed noise levels, the SNR gain of an array is a monotonically increasing function of the array size. Furthermore, it is demonstrated that the internal array noise components are incapable of further improving the SNR gain of locally optimal processing. However, since the locally optimal nonlinearity requires a complete knowledge of the underlying noise statics, the structure of the locally optimal nonlinearity may have no analytical expression or be intractable. Therefore, a parallel array of suboptimal but easily implemented threshold nonlinearities becomes an optional approach. It is shown that such an array of threshold nonlinearities can exhibit the SR effect by increasing the array noise level and the array size. For a sufficiently large array size, we also show that the SNR gain of an array of threshold nonlinearities can exceed unity for a wide range of noise distributions, e.g. the exponent aƒ1:83 in Fig. 5.
Some interesting open questions arise. For example, we only considered the array of threshold nonlinearities for processing a weak noisy signal. Therefore, can other tractable nonlinearities be connected in parallel for achieving improved output-input SNR gain via the array SR effect? As indicated in Fig. 2 and 4, we can operate an array of nonlinearities with large array size at a feasible level of noise. Therefore, given an acceptance criterion of the performance of nonlinearities, how large the array size is and which level the noise takes are interesting questions. These questions will be of interest for further studies of nonlinear signal processing in the context of array SR, especially in the ensemble of neurons. Often quite a number of neurons have similar properties and respond to the same stimuli [5,6,15], thus the condition of all neurons in parallel having the same pattern of input and output connections will be considered. It is of interest to explore how the external (internal) noise components assist the information transfer through the neural network. For the static nonlinearity considered in Eq. (1), Eq. (20) provides the upper bound of the performance of the array of static nonlinearities. While many neuron models, such as the leaky integrate-and-fire model and the Hodgkin-Huxley model [6,8], represent the neurodynamics with the time evolution nonlinear process (not a static nonlinearity), then whether the transmission efficiency of a parallel array of neurons has an upper bound for the neural signal propagate or not deserves to be studied.

Methods
Under the assumption of weak signals, the Taylor expansion of the noise PDF is utilized in Eqs. (6), (7), (8) and (9). The Jensen inequality is applied to Eq. (11). The Cauchy-Schwarz inequality is extensively used in Eqs. (17), (20) and (21). Figure 5. Output-input SNR gain G 100 . Output-input SNR gain G 100 (red line) of an array of threshold nonlinearities with array size N~100 as a function of the decay exponent a for the response threshold h~0. The external noise j(t) is the generalized Gaussian noise with PDF in Eq. (22) and RMS amplitude s j~1 . The uniform array noise components g n (t) are with the RMS amplitude s g~1 . The black line indicates the SNR gain s 2 j I(f j )~a 2 C(3a {1 )C(2{a {1 )=C 2 (a {1 ) achieved by the locally optimal nonlinearity g opt in Eq. (18) (without the internal noise g n (t)), which is also the upper limit that the SNR gain G ? cannot exceed. The blue line is the benchmark of unity. doi:10.1371/journal.pone.0058507.g005