Pattern Recognition in Neural Networks with Competing Dynamics: Coexistence of Fixed-Point and Cyclic Attractors

We study the properties of the dynamical phase transition occurring in neural network models in which a competition between associative memory and sequential pattern recognition exists. This competition occurs through a weighted mixture of the symmetric and asymmetric parts of the synaptic matrix. Through a generating functional formalism, we determine the structure of the parameter space at non-zero temperature and near saturation (i.e., when the number of stored patterns scales with the size of the network), identifying the regions of high and weak pattern correlations, the spin-glass solutions, and the order-disorder transition between these regions. This analysis reveals that, when associative memory is dominant, smooth transitions appear between high correlated regions and spurious states. In contrast when sequential pattern recognition is stronger than associative memory, the transitions are always discontinuous. Additionally, when the symmetric and asymmetric parts of the synaptic matrix are defined in terms of the same set of patterns, there is a discontinuous transition between associative memory and sequential pattern recognition. In contrast, when the symmetric and asymmetric parts of the synaptic matrix are defined in terms of independent sets of patterns, the network is able to perform both associative memory and sequential pattern recognition for a wide range of parameter values.


Introduction
Neural networks were originally developed to model the behavior of the brain. However, due to the great complexity of the brain's neural circuitry and of the synaptic interactions, it was necessary to propose simplified models, such as the McCulloch-Pitts model [1] and the Hopfield model [2] which, although simple, still capture some important characteristics of the neuronal dynamics. One important question in this field is how and when a neural network is able to memorize a given set of patterns. Two main different mechanisms to store information in a neural network have been identified, to wit, the Associative Memory (AM) on the one hand, and the Sequential Pattern Recognition (SPR) on the other hand. The neural network performs AM when its dynamical attractors are fixed points, each corresponding to one of the patterns that we want to store in the network. This type of dynamic behavior is characterized by a symmetric interaction matrix that contains the connection strength between the neurons. Some examples of this dynamics are the Hopfield model and the Little model [2], [3], [4], [5]. Contrary to the above, in SPR the network memorizes a fixed set of patterns which are retrieved in certain order in time. From a dynamical point of view, this corresponds to a cyclic attractor consisting of the sequence of patterns stored in the network in a given order. A necessary condition for SPR to occur is that the matrix of neuron-neuron interactions has to be asymmetric. A well known example of this type of dynamics is the asymmetric Hopfield model [2], [6].
Several dynamical phases have been identified in both the AM and SPR models. Broadly speaking, these phases characterize how well the network can recognize its set of patterns, and the type of memory, i.e., whether AM or SPR. Both AM and SPR have been widely studied separately. Nonetheless, there is evidence showing that in real neural networks the synaptic connections are neither fully symmetric nor fully asymmetric [7], [8]. Rather, they can be considered as a mixture of these two cases, generating an interaction network with a complex topology. Additionally, there is evidence that the brain is capable to perform both AM and SPR [8], [9]. For instance, recalling the color of a simple object would be an example of AM, whereas recalling the digits in a phone number in the proper order would constitute an example of SPR. Since these two types of pattern retreival coexist in the brain, several authors have introduced modifications to the Hopfield model in order to obtain both types of pattern retrieval within the same network [10], [11], [12]. One approach to this problem was proposed by Coolen and Sherrington in Ref. [10]: They introduced a model in which the interaction matrix has two parts, one symmetric and one asymmetric. These two parts are weighted by a mixture parameter l, in such a way that the interaction matrix W, (also called the synaptic matrix), can be written as where W s and W a are symmetric and asymmetric matrices, respectively. For l~1 only the symmetric part is present (the classical Hopfield Model) and therefore the network performs AM, while for l~0 only the asymmetric part survives (the asymmetric Hopfield Model) and the network performs SPR. For intermediate values of l, there is a competition between the symmetric and asymmetric parts of the synaptic matrix. One of the main questions in this model, which we will refer to as the Coolen-Sherrington model, (or the CS model for short), is how the network dynamics transit from AM to SPR as l varies from 1 to 0. The point is that for l~1 all the patterns are stored as independent attracting fixed points, whereas for l~0 the patterns are stored as part of a single large cyclic attractor. Is this transition from AM to SPR continuous or discontinuous? Can some of the patterns be stored in a cyclic attractor whereas some other patterns are stored as fixed point attractors? As we will see, the answer to these questions depends on the definition of the symmetric and asymmetric parts of the synaptic matrix.
In the original CS model the symmetric part W s and the asymmetric part W a are correlated because they are defined in terms of the same set of patterns (see the definition of the model in the next section). Additionally, Coolen and Sherrington studied this model for the particular case where the number of patterns stored in the network is smaller that the number of neurons (p% ffiffiffiffi ffi N p ). In terms of the load parameter a~p=N, the above condition corresponds to a~0 in the thermodynamic limit N?? (this regime is termed non saturated.) Within this regime, Coolen and Sherrington found that for parallel updating and for l&1, the network dynamics exhibit only fixed point attractors, i.e. the network performs AM. However, when l is decreased, a first order phase transition appears: Below a certain critical value l c (T) that depends on the temperature T, the dynamical trajectories end up either in cyclic attractors (the networks exhibits SPR), or in stable mixed states that consist of combinations of the desired patterns. However, no coexistence of AM and SPR was found.
Afterwards, in Ref. [13] the authors studied the CS model using correlated patterns. They found that it is possible to have SPR (the stable cycle limit is still present) when the correlation between the patterns is small. In the case of AM the network goes to a fixed point attractor but this attractor does not coincide with any of the desired patterns. Metz and Theumann [14], [15] presented a full study of the stability of the patterns in a multi-layered neural network with competition between AM and SPR, finding the phase space regions where the network performs AM, SPR and the region for the spin-glass solutions (SGS), but no coexistence between AM and SPR was found either. By ''coexistence'' of AM and SPR we mean that some of the patters are stored as fixed point attractors while other patterns are stored in larger cyclic attractors. When such a coexistence does not exist, then all patterns are stored either as independent fixed point attractors or as a single large cyclic attractor. Recently, the same authors in [16] studied a model similar to the CS model by means of the generating functional technique. They present a study of the stationary states and the different regions on the phase space where either fixed points or cyclic attractors are attained.
It is important to note that the lack of coexistence of AM and SPR in the original CS model is not a trivial result. The synaptic weights are a weighted mixture of symmetric and asymmetric matrices. Therefore, especially for intermediate values of the mixture parameter l, it could have happened that some of the patterns were recognized as fixed-point attractors whereas some other patterns were recognized in sequential order. But this was not the case: either all the patterns are fixed point attractors or all of them form a huge cyclic attractor (remember that we are working in the saturated regime where the number of stored patters is a finite fraction of the number of neurons: p~aN, which becomes infinite in the thermodynamic limit N??). This all-or-none behavior was not expected for the original CS model and deserved a careful study carried out by several authors.
Using the generating functional formalism developed in Refs. [17], [18], [19], [20][21] we investigate how the network transits from pure AM to pure SPR for the CS model, first in the case in which the symmetric and asymmetric parts of the synaptic matrix are correlated (defined in terms of the same set of patterns) and then when they are independent (defined each in terms of different sets of patterns). Both cases are studied for systems in which the number of patterns p is a finite fraction of the total number of neurons N, namely, in the saturated regime for which a~p=Nw0 even in the thermodynamic limit N??. We compute the phase space over the parameters l, a and the temperature T, finding the regions where the networks performs AM and/or SPR, as well as the spin-glass region, for different values of the mixing parameter l. As might have been expected, we find that when W s and W a are correlated, the network either performs AM or SPR, but it is incapable to perform both for the same set of parameter values. In contrast, when W s and W a are independent of each other, AM and SPR coexist within a large region of the parameter space. We present a complete explicit characterization of the different phases, as well as the transition between AM and SPR when these behaviors coexist.
In the next section we present the two versions of the CS model we study in this work. In Sec. we compute the dynamical equations that determine the temporal evolution of the network using the generating functional formalism developed in Refs. [17], [18], [19], [20]. In Sec. we present the results for the original CS model and determine the structure of the parameter space identifying the regions of highly correlated, weakly correlated and spin-glass solutions. We do this for the AM and SPR dynamics and show that these two types of pattern retrieval do not coexist. In Sec. we present analogous results but for the modified version of the CS model, and we show that in this case AM and SPR dynamics do coexist. Finally, in Sec. we summarize our results.

Model Definition
The network under consideration consists of N binary neurons, fs 1 ,s 2 , . . . ,s N g, each acquiring the values s i~+ 1. We will denote ass s(t)~(s 1 (t),s 2 (t), . . . ,s N (t)) the dynamical state of the entire network at time t. The interaction between the neurons is determined by the function where the W ij , the components of the synaptic matrix W, are defined in terms of the patterns that we want to store in the network. The variables h i (t) represent external fields. We will come to the precise definition of the synaptic weights W ij in a moment. For the time being, let us assume that these synaptic weights have already been defined. Then, the network dynamics are given by the synchronous updating of all the network elements in such a way that where T~b {1 is the statistical ''temperature''. The above updating rule is equivalent to saying that the transition probability W(s s(tz1)js s(t)) for the entire network to change from the statẽ s s(t) at time t to the states s(tz1) at time tz1 is In order to define the synaptic weights W ij , we will assume that they consist of a symmetric part W s ij and an asymmetric part W a ij as where l[½0,1 is the mixture parameter. In the original CS model, each of these parts are defined in terms of a unique set of p uncorrelated patterns, X~fj j 1 ,j j 2 , . . . ,j j p g, as where each patternj j m is a distinct vector of N binary digits; j j m~( j m 1 ,j m 2 , . . . ,j m N ), and each entry can take the values j m i~+ 1. For the purpose of this work, these binary sequences are generated randomly. In the last expression, the sum over the patterns is modulo p, i.e.j j pz1~j j 1 .
In order to make the two types of pattern retrieval AM and SPR coexist in the same network, in this work we introduce a variation of the CS model by using two different sets of uncorrelated patterns, X~fj j 1 ,j j 2 , . . . ,j j p g and Z~ff f 1 ,f f 2 , . . . ,f f p g. Each pattern is uncorrelated with all the other patterns in its own set and also with all the patterns in the other set. We can use these two independent sets of patterns to define W s ij and W a ij independently of each other as Use of independent patterns in Eq. (5) has been studied previously for the special case of 2p|2p matrices [21], [20]. Here, we extend these calculations for general matrices and compute the whole phase space.
The quantity that determines how close the states s(t) is from a given pattern, sayj j m , is the overlap function m t m , defined as Thus, if m t m &0, thens s(t) andj j m are very different and uncorrelated, whereas if m t m &1 thens s(t) andj j m are almost identical. Finally, if m t m &{1 thens s(t) andj j m are specular copies of each other (i.e. they are fully anticorrelated).
In what follows, we analyze the models given in Eqs.
(2)-(5) both numerically and by the generating functional approach, from which we derive the equations that rule the dynamical evolution of each system and determine the structure of their respective phase spaces.

Numerical simulations
Before introducing the mathematical formalism to analyze the different models presented in the previous section, we illustrate here the coexistence of the two types of dynamics, SPR and AM, with a numerical simulation. For this, we constructed a neural network that can store 10 patterns X~j j 1 ,j j 2 , . . . ,j j 10 n o as independent fixed points (for the AM recognition), and 10 patterns in a single cyclic attractor (for the SPR). Each pattern is a grayscale digitalized image 200|200 pixels in size, and each pixel has a depth of 8 bits in order to encode 256 shades of gray needed for the black and white image. Fig. 1 shows the 20 patterns used in our numerical simulations. The neural network thus consists of N~8|200|200~320000 binary variables. Furthermore, although in theory one assumes that each neuron is connected to each other neuron in the network, in our case this would give a very large connectivity matrix with N 2 *10 11 independent entries. Such a large synaptic matrix is not necessary to store 20 patterns. Hence, in our simulations we worked with networks where each neuron receives inputs only from K~200 other neurons randomly chosen with uniform probability from the entire network. Once the K~200 input connections of each neuron have been randomly assigned, they do not change throughout the dynamics of the network. Thus, the synaptic matrix W is a sparse matrix constructed according to Eq. m~1 m(f f m ). Fig. 3a shows the results of the simulation for the case when X and Z are different sets of patterns (X consists of the patterns 1 to 10 whereas Z contains the patterns 11 to 20 of Fig.1). It is clear for this case that in the interval l[(0:45,0:55) the network can perform both associative memory and sequential pattern recognition almost perfectly (m am &m spr &1). Depending on the initial condition, the network will retrieve one of the fixed point patterns j j m in X, or it will retrieve all the patternsf f 1 , . . . ,f f 10 in Z in a cyclic order. Thus in this case AM and SPR coexist for l[(0:45,0:55). By contrast, Fig. 3b shows that when the sets X and Z contain the same set of patterns (patterns 1 to 10 of Fig. 1), namely, wheñ j j m~f f m for m~1,2, . . . ,10, then there is no value of l for which m am &1 and m spr &1 at the same time. Therefore, in this case the network cannot perform AM and SPR and these two types of dynamics do not coexist.
Finally, it is worth mentioning that the relatively high values of the overlap observed in the interval l[(0:35,0:65) in Fig. 3b are due to the fact that in this region the network invariably falls into a ''frustrated'' state consisting of a random superposition of different patterns, as the one shown in the inset. Therefore, in this region the network always has a non negligible overlap with any of the stored patterns, producing a relatively high value of the overlap.

Generating functional approach
We use the standard generating functional approach [17], [18], [19], [20], [21] to derive the dynamical mapping that determines the temporal evolution of the average of the overlap m m (s t ). This formalism allows us to derive state equations for the macroscopic variables of the system at finite temperature T §0. For the sake of completeness, we outline the procedure following [17]. Thus, we define the generating functional Z½y as: whereỹ y(t)~(y 1 (t),y 2 (t), . . . ,y N (t)) is a set of auxiliary variables used to derive the macroscopic parameters that characterize the system, and p½s s(0),s s (1)  Each pattern is a digitalized image with 200|200 pixels, each with 8 bits of depth. The binary strings directly reproducing these images were stored in the synaptic matrix W without any previous randomization. In the case when X and Z are two independent sets, patters 1 to 10 were used for X and patterns 11 to 20 for Z. When X and Z are the same set, patterns 1 to 10 were used for both. doi: 10 where the conjugate fieldsĥ h(s) arise from the integral representation of the delta functions that enforce the values of the h i (s).
To continue we must assume that one stored pattern is condensed at each time-step, that is, only one pattern can be highly correlated with the network. Thus, the overlap between this condensed pattern and the network state should be of order O(1), whereas for the non-condensed patterns, which play the role of quenched disorder, the overlap must be of order O(1= ffiffiffiffi ffi N p ). In the thermodynamic limit N?? one can use Coolen's mean-field approach to average Eq. (7) over the non-condensed patterns.
Eq. (7) contains all the dynamical properties of the system. In particular, one can obtain all the relevant average quantities from the generating functional by differentiation (see Ref. [17]): G(s,s 0 )~i lim C(s,s 0 )~{ lim where m(s) is the overlap, G(s,s 0 ) is the response function and C(s,s 0 ) the correlation function.

Analytic solutions for one set of patterns
As reference, we start by considering the case given in Eqs. (4), which corresponds to the original CS model where the symmetric and asymmetric parts of the synaptic matrix are correlated due to the fact that the same set of patterns X~fj j 1 ,j j 2 , . . . ,j j p g is used to define these two parts. Using the generating functional formalism, we determined the dynamical equations of the system and found  the conditions for the existence of fixed points (associative memory) and limit cycles (sequential pattern recognition) for different values of the load parameter a, the mixing parameter l, and the ''temperature'' T~b {1 . The detailed computations are shown at the Appendix S1.
Associative memory solutions. In order to find the regions of the parameter space where the patternsj j n (and the corresponding anti-patterns) are fixed point attractors of the network, we look for solutions in which the final state of the network is strongly correlated with one of the patterns, sayj j 1 . Following the procedure presented in Ref. [10], leads to the following equations for the observables: In the above equations, m t 1~m 1 (t) is the overlap with the first pattern, q t the variance of the overlap, and r t the correlation between two network states.
AM spin-glass solutions. We start the analysis of Eqs. (9) by computing the spin-glass solutions, which are characterized by m t 1~0 , q t =0, and r t =0. These are easily obtained at zero temperature (b??), and then one can investigate the existence of spin-glass solutions at non-zero temperature by a series-expansion technique around the zero temperature solution. Since m t 1~0 , we can disregard Eq. (9a) and focus our attention only on the other equations (putting m t 1~0 ). For non-zero temperature (finite b), it is known that in the Hopfield model, which is obtained here for l~1, the spin-glass solutions disappear continuously as b decreases via a second order phase transition. A similar behaviour is observed for l=1, as is shown in Fig. 4, where a vertical line indicates the continuous transition from the spin-glass region SG to another region TS where only the trivial solution m t 1~q t~rt~0 exists. Note that close to this transition r t and q t are very small. Therefore, in order to find the critical temperature at which this transition occurs, we can expand Eqs. (9c) and (9b) up to the first order in r t and q t , keeping m t 1~0 . Solving the resulting equation for T~b {1 in terms of a and l, we obtain where we have used T sg to denote that this value corresponds to the spin-glass transition temperature. The above equation is well defined for every a and l values and therefore, there is always a critical temperature at which the spin-glass phase appears. AM regions. The regions in which associative memory exists (AM-regions hereafter) are characterized by m t 1 &1. Note that the transition between the AM-region and the spin-glass region (SPregion hereafter) must be discontinuous, since in the latter m t 1 &0. This occurs for some values of l and a, as is apparent from Fig. 4. However, for other values of the parameters the spin-glass region disappears and m t 1 , q t , r t all vanish continuously, as shown in Fig. 5. It is still possible to define a region of high correlation  (region HC) and another region of weak correlation (region WC) by defining thresholds to the values of the overlap m t 1 . We choose the highly correlated patterns as those for which m t 1 w0:75, whereas for the weakly correlated patterns 0ƒm t 1 ƒ0:75. The whole structure of the parameter space for the AM dynamics, shown in Fig. 6 for different values of l, can be found by numerically solving the system of Eqs. (9). Four regions are found, which will be referred to as regions HC (high correlations), WC (weak correlations), SG (spin glass), and TS (trivial solution). Region HC is characterized by the existence of non-trivial highly correlated solutions (m t n w0:75). Although in this region spin-glass solutions might also exist, the network performs AM since the highly correlated solutions are always preferred. In region SG the network is not capable to perform AM. Rather, it always falls into spin-glass solutions. In region TS only the trivial solution exists. Finally, region WC is a transition region where the highly correlated patterns continuously become weakly correlated. Even when there are non-trivial solutions in region WC, it is not possible to have AM because the final pattern has at most 75% of its neurons correct with respect to the desired final state. It is worth emphasizing that in the transition from region HC to SG the overlap m t n varies discontinuously from & 1 to &0, see Fig. 4, whereas in the transition from region HC to WC to TS the overlap varies continuously from &1 to &0, see Fig. 5).

Sequential pattern recognition solutions
Now we find the regions of the parameter space where sequential pattern recognition exists. In these regions, the patterns X~fj j 1 ,j j 2 , . . . ,j j p g, form a cyclic attractor with period p, i.e. j j pz1~j j 1 . We will look for solutions of Eq. (7) in which the state of the network at time t is strongly correlated only withj j m , at time tz1 it is strongly correlated only withj j mz1 , and so on until the time step tzp, at which the state of the network is strongly correlated only withj j mzp , and the cycle starts over again. The condition that at each time the network state is correlated with only one pattern can be written as where d t m is the Kronecker delta function and e m m t is a quantity such that e m m t *O(1). The calculation again follows along the lines presented in Ref. [10]. The final set of equations for e q q t , e r r t , and e m m t is e m m tz1 e r r t~1 q g: The system of equations above is always satisfied by the trivial solution e m m tz1~0 , e q q t~0 and e r r t~0 . However, depending on the values of the parameters a, l and T, the trivial solution can be stable or unstable. We are of course interested in the regions of the  [22], [23], [24]. Note that as l decreases, region HC of highly correlated patterns decreases, whereas region WC of weakly correlated patterns increases. doi:10.1371/journal.pone.0042348.g006 parameter space where the set of patterns X~fj j 1 ,j j 2 , . . . ,j j p g is retrieved in the specified order. This corresponds to solutions for which m t m~d t m e m m t *O(1) with no restrictions on e q q t and e r r t . The spin-glass solutions are also important in the phase diagram. They are obtained from the system of Eqs. (12) by imposing the conditions e m m t~0 , e q q t =0. Before presenting the different regions of the phase space, in the next section we focus on the calculation of the critical value of the capacity a in the deterministic case of zero temperature (T~0, b??), and for purely asymmetric synaptic weights, i.e. l~0.
SPR dynamics at zero temperature. For l~0 the system of Eqs. The last three equations coincide with the ones found in Ref. [6]. Let us define e C C~b(1{e q q t ). Note from Eq. (13a) that in the limit b??, the e C C remains finite and is different from zero. Thus, using the saddle point approximation in the limit b??, the system of equations (13) reduces to By defining y~e m m t = ffiffiffiffiffiffiffiffi ffi 2ae r r t p , the above equations can be solved for y, which gives erf 2 (y)~2y 2 which is equivalent to The preceding equation has non trivial solutions as long as a is smaller than a critical value a c (see Fig. 7). This critical value gives the maximum storage capacity of the network. Beyond that value, SPR cannot occur. By solving numerically Eq. (15) one finds a c &0:26909, which is in agreement with the value found in Ref. [17].
SPR spin-glass solutions. An analysis similar to that in Sec., but now using Eqs. (12), leads to the following expression for the spin-glass transition temperature: Again this transition temperature is well defined for all values of the parameters, and therefore there is always a finite temperature at which the spin-glass phase appears. SPR regions. Fig. 8 shows the regions in the parameter space where different types of solutions are obtained in the SPR case. There are only three regions, which are referred to as HC, SG and TS. Region HC is where highly correlated solutions, characterized by e m m t &1, coexist with Spin-Glass solutions. In region SG the highly correlated states disappear and only the Spin-Glass solutions exist. Finally, in region TS only the trivial solutions exist. Note that in this case there is no region of weakly correlated solutions, (as for AM). Therefore, the highly correlated solutions always disappear discontinuously through a first-order transition from region HC to region SG. In contrast, the spin-glass solutions vanish continuously from region SG to region TS.

Coexistence of AM and SPR dynamics
The analysis presented so far shows that, unsurprisingly, the original CS model is unable to perform both AM and SPR dynamics for the same value of l. This is illustrated in Fig. 9 for the case of zero temperature, but the same happens for T=0. This is obviously due to the fact that the same set of patterns is used to define both the symmetric and asymmetric parts of the synaptic matrix. But the symmetric part W s ij is responsible for the AM dynamics, whereas the asymmetric part W a ij is involved in the SPR dynamics, and it is impossible for a given pattern j m to be a onestate attractor (AM), and at the same time to belong to a cyclic attractor (SPR). Thus both dynamics cannot coexist in the original

Analytic solutions for two independent sets of patterns
In this section we present the results of the modified CS model in which the symmetric part W s ij of the synaptic matrix is defined in terms of a set of patterns X~fj j 1 ,j j 2 , . . . ,j j p g, whereas the asymmetric part W a ij is defined using a different set of patterns Z~ff f 1 ,f f 2 , . . . ,f f p g, as in Eq. (5). This gives the network the possibility to perform AM and SPR dynamics independently of each other. The mathematical formalism is completely analogous to the one used in the previous section. As for the original CS model, here we present first the AM solutions and afterwards the SPR ones.
AM solutions. To obtain the AM solutions we again demand that the network state is highly correlated with only one of the patterns X~fj j 1 ,j j 2 , . . . ,j j p g, sayj j 1 . Thus, the overlap m t 1 between the network state and j 1 must satisfy that m t 1 &1, whereas all the other overlaps m t m must be of order 1= ffiffiffiffi ffi N p . Under such circumstances, the equations that determine the stationary state of the network, equivalent to Eqs. (9), are:    9) and (12). Note that there is no overlap between the shaded region corresponding to AM and the one corresponding to SPR. doi:10.1371/journal.pone.0042348.g009 X~fj j 1 ,j j 2 , . . . ,j j p g will be of order 1= ffiffiffiffi ffi N p . Taking these considerations into account, the equations that determine the cyclic behaviour of the network are q t~ð dz ffiffiffiffiffi ffi 2p p e { z 2 2 tanh 2 fb½(1{l)m t 1 zz ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ar t (l 2 z(1{l) 2 ) q g ð20aÞ Through a calculation similar to the one used in Sec., it can be shown that in the limit T?0 the system of Eqs. (20) reduces to erf 2 (e y y){2e y y 2 2 p exp ({2e y y 2 )za 1z l where e y y~(1{l) e m m t = ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2ae r r t l 2 z(1{l) 2 Â Ã q . This equation has nontrivial solutions as long as a is smaller than a critical value a c (l) that depends on l. By numerically solving Eq. (21) we obtain the curve a c (l) plotted in the bottom graphs in each panel of Fig. 10, in which the shaded area corresponds to the region of highly correlated solutions. Note from this figure that the shaded regions corresponding to AM and SPR do intersect over a large region of parameter values. This indicates that in the modified CS model, AM dynamics do coexist with SPR dynamics for a wide range of values of the mixture parameter l.
On the other hand, the spin-glass solutions at zero temperature are given by e r r t~1 z 2(1{l) 2 pa½l 2 z(1{l) 2 , e q q t~1 , e m m t~0 , which shows that the spin-glass solutions at zero temperature always exist for any aw0 and l[½0,1. For non-zero temperature, the first order series expansion of Eqs. (20) with e m m t~0 gives e q q t &b 2 ae r r t l 2 z(1{l) 2 Â Ã e r r t & e q q t 1{(1{l) 2 b 2 : From the above equations it follows that the spin-glass transition temperature e T T sg is Figure 12 summarizes the structure of the a-T space in the SPR case for different values of l. The usual three regions HC (high correlations), SG (spin-glass solutions only), and TS (trivial solution) are indicated. The boundary between regions SG and TS is given by Eq. (22), whereas the curve separating regions HC Figure 11. Structure of the AM phase space for different values of l in the modified CS model. The first panel for l~1 corresponds to the well known symmetric Hopfield model. Note that as l increases, region HC of highly correlated patterns decreases. However, it remains of considerable size even for l~0:5, where the AM and SPR dynamics equally compete. Note also that there is no region WC of weakly correlated solution. doi:10.1371/journal.pone.0042348.g011 and SG was obtained numerically from Eqs. (20). Note that region HC remains of considerable size for a wide range of values of l.

Discussion
We have obtained the complete phase diagram and analyzed the transitions from AM to SPR in a neural network model proposed by Coolen and Sherrington in which both types of pattern retrieval compete, as well as in a simple modification of the model in which both types of pattern retrieval may coexist. In these systems, the AM and SPR dynamics are encoded in the symmetric and asymmetric parts of the synaptic matrix, respectively, and the contribution of each of these parts is weighted by a parameter l in such a way that when l~1 only the symmetric part survives, whereas when l~0 only the asymmetric part is present. In the original Coolen and Sherrington model, the same set of patterns are used to define the symmetric and asymmetric parts of the synaptic matrix. Using the standard functional generating formalism, we obtained the phase diagram of the system which shows that in the original Coolen-Sherrington model AM and SPR dynamics cannot coexist. This is simply due to the fact that a given pattern j m cannot be a fixed point and part of a larger cyclic attractor at the same time. Therefore, the original CS model can retrieve patterns only for the limiting cases l&1 or l&0. For intermediate values of l the network is ''frustrated'' and can perform neither AM nor SPR.
To prevent the system from falling into a ''frustrated'' state as mentioned above, we modified the CS model by using two independent sets of patterns in order to define separately the symmetric and asymmetric parts of the synaptic matrix. In doing so we allow the possibility for the network to have fixed points belonging to one set of patterns, and simultaneously cyclic attractors constructed with the patterns that belong to the other set. Our goal was to determine how the network transits from AM to SPR as l varies from 1 to 0 in this new case where the two sets of patters were independent. As expected, in this case the AM and SPR dynamics coexist for a wide range of values of l. However, some other aspects of the model can be analyzed. For instance, quasi periodic states are known to occur in the original CS model (with only one set of patterns) and it would be interesting to determine to what extent these quasi periodic states exist in the modified CS model (with two independent sets of patterns). Also, it is possible to have an intermediate situation in which the two sets fj j 1 ,j j 2 , . . . ,j j p g and ff f 1 ,f f 2 , . . . ,f f p g share some of the patterns. In this case the two sets would not be fully independent and the transition from AM to SPR dynamics could be more complicated.
Finally, the generating functional approach that we used to determine the structure of the phase space works very well when the patterns are uncorrelated and the network is fully connected. It would also be interesting to extend the analysis to networks with more realistic topologies, such as the small-world and scale-free topologies, in order to determine how the network topology affects the dynamics.