^{1}

^{*}

^{1}

^{2}

^{2}

^{3}

Conceived and designed the experiments: MR VA. Performed the experiments: RH. Analyzed the data: RH PV. Contributed reagents/materials/analysis tools: RH PV. Wrote the paper: MR RH PV.

The authors have declared that no competing interests exist.

The idea that cognitive activity can be understood using nonlinear dynamics has been intensively discussed at length for the last 15 years. One of the popular points of view is that metastable states play a key role in the execution of cognitive functions. Experimental and modeling studies suggest that most of these functions are the result of transient activity of large-scale brain networks in the presence of noise. Such transients may consist of a sequential switching between different metastable cognitive states. The main problem faced when using dynamical theory to describe transient cognitive processes is the fundamental contradiction between reproducibility and flexibility of transient behavior. In this paper, we propose a theoretical description of transient cognitive dynamics based on the interaction of functionally dependent metastable cognitive states. The mathematical image of such transient activity is a stable heteroclinic channel, i.e., a set of trajectories in the vicinity of a heteroclinic skeleton that consists of saddles and unstable separatrices that connect their surroundings. We suggest a basic mathematical model, a strongly dissipative dynamical system, and formulate the conditions for the robustness and reproducibility of cognitive transients that satisfy the competing requirements for stability and flexibility. Based on this approach, we describe here an effective solution for the problem of sequential decision making, represented as a fixed time game: a player takes sequential actions in a changing noisy environment so as to maximize a cumulative reward. As we predict and verify in computer simulations, noise plays an important role in optimizing the gain.

The modeling of the temporal structure of cognitive processes is a key step for understanding cognition. Cognitive functions such as sequential learning, short-term memory, and decision making in a changing environment cannot be understood using only the traditional view based on classical concepts of nonlinear dynamics, which describe static or rhythmic brain activity. The execution of many cognitive functions is a transient dynamical process. Any dynamical mechanism underlying cognitive processes has to be reproducible from experiment to experiment in similar environmental conditions and, at the same time, it has to be sensitive to changing internal and external information. We propose here a new dynamical object that can represent robust and reproducible transient brain dynamics. We also propose a new class of models for the analysis of transient dynamics that can be applied for sequential decision making.

The dynamical approach for studying brain activity has a long history and is currently one of strong interest

The execution of cognitive functions is based on fundamental asymmetries of time – often metaphorically described as the arrow of time. This is inseparably connected to the temporal ordering of cause-effect pairs. The correspondence between causal relations and temporal directions requires specific features in the organization of cognitive system interactions, and on the microscopic level, specific network interconnections. A key requirement for this organization is the presence of nonsymmetrical interactions because, even in brain resting states, the interaction between different subsystems of cognitive modes also produces nonstationary activity that has to be reproducible. One plausible mechanism of mode interaction that supports temporal order is nonreciprocal competition. Competition in the brain is a widespread phenomenon (see

Recently functional magnetic-resonance imaging (fMRI) and EEG have opened new possibilities for understanding and modeling cognition

Common features of many cognitive processes are: (i) incoming sensory information is coded both in space and time coordinates, (ii) cognitive modes sensitively depend on the stimulus and the executed function, (iii) in the same environment cognitive behavior is deterministic and highly reproducible, and (iv) cognitive modes are robust against noise. These observations suggest (a) that a dynamical model which possesses these characteristics should be strongly dissipative so that its orbits rapidly “forget” the initial state of the cognitive network when the stimulus is present, and (b) that the dynamical system executes cognitive functions through transient trajectories, rather than attractors following the arrow of time. In this paper we suggest a mathematical theory of transient cognitive activity that considers metastable states as the basic elements.

This paper is organized as follows. In the Results section we first provide a framework for the formal description of metastable states and their transients. We introduce a mathematical image of robust and reproducible transient cognition, and present a basic dynamical model for the analysis of such transient behavior. Then, we generalize this model taking into account uncertainty and use it for the analysis of decision making. In the Discussion, we focus on some open questions and possible applications of our theory to different cognitive problems. In the

A dynamical model of cognitive processes can use as variables the activation level _{i}_{i}

Metastable states of brain activity can be represented in a high-dimensional phase space of a dynamical model (that depends on the cognitive function) by saddle sets, i.e., saddle fixed points or saddle limit cycles.

In turn, reproducible transients can be represented by a stable heteroclinic channel (SHC), which is a set of trajectories in the vicinity of a heteroclinic skeleton that consists of saddles and unstable separatrices that connect their surroundings (see

The SHC concept is able to solve the fundamental contradiction between robustness against noise and sensitivity to the informational input. Even close informational inputs induce the generation of different modes in the brain. Thus, the topology of the corresponding stable heteroclinic channels sensitively depends on the stimuli, but the heteroclinic channel itself, as an object in the phase space (similar to traditional attractors), is structurally stable and robust against noise.

The SHC is built with trajectories that condense in the vicinity of the saddle chain and their unstable separatrices (dashed lines) connecting the surrounding saddles (circles). The thick line represents an example of a trajectory in the SHC. The interval _{k}_{+1}−_{k}

Based on these ideas we model the temporal evolution of alternating cognitive states by equations of competitive metastable modes. The structure of these modes can be reflected in functional neuroimage experiments. Experimental evidence suggests that for the execution of specific cognitive functions the mind recruits the activity from different brain regions

We suggest here that the mathematical image of reproducible cognitive activity is a stable heteroclinic channel including metastable states that are represented in the phase space of the corresponding dynamical model by saddle sets connected via unstable separatrices (see

To make our modeling more transparent let us use as an example the popular dynamical image of rhythmic neuronal activity, i.e., a limit cycle. At each level of complexity of a neural system, its description and analysis can be done in the framework of some basic model like a phase equation. The questions that can be answered in this framework are very diverse: synchronization in small neuronal ensembles like CPGs, generation of brain rhythms _{i}

Before we introduce the basic model for the analysis of reproducible transient cognitive dynamics, it is important to discuss two general features of the SHC that do not depend on the model. These are: (i) the origin of the structural stability of the SHC, and (ii) the long passage time in the vicinity of saddles in the presence of moderate noise.

To understand the conditions of the stability of SHC we have to take into account that an elementary phase volume in the neighborhood of a saddle is compressed along the stable separatrices and it is stretched along an unstable separatrix. Let us to order the eigenvalues of a saddle as_{i}

The problem of the temporal characteristics of the transients is related to the “exit problem” for small random perturbations of dynamical systems with saddle sets. This problem was first solved by Kifer

A biologically reasonable model that is able to generate stable and reproducible behavior represented in the phase space by the SHC has to (i) be convenient for the interpretation of the results and for its comparison with experimental data, (ii) be computationally feasible, (iii) have enough control parameters to address a changing environment and the interaction between different cognitive functions (e.g., learning and memory). We have argued that the dynamical system that we are looking for has to be strongly dissipative and nonlinear. For simplicity, we chose as dynamical variables the activation level of neuronal clusters that consist of correlated/synchronized neurons. The key dynamical feature of such models is the competition between different metastable states. Thus, in the phase space of this basic model there must be several (in general many) saddle states connected by unstable separatrices. Such chain represents the process of sequential switching of activity from one cognitive mode to the next one. This process can be finite, i.e., ending on a simple attractor or repetitive. If we choose the variables _{j}

We will use two types of models that satisfy the above conditions: (i) the Wilson-Cowan model for excitatory and inhibitory neural clusters

Here _{j}_{ji}_{j}

To illustrate the existence of a stable heteroclinic channel in the phase space of Equation 2, let us consider a simple network that consists of three competitive neural clusters. This network can be described by the Wilson-Cowan type model as_{jj}<0, _{j≠i}≥0,

The network can also be described by a Lotka-Volterra model of the form:_{ji}≥0. In all our examples below we will suppose that the connection matrix is non symmetric, i.e., _{ji}_{ij}

(A) Wilson-Cowan clusters. (B) Lotka-Volterra clusters.

Both models demonstrate robust transient (sequential) activity even for many interacting modes. An example of this dynamics is presented in

(A) The activation level of three cognitive modes are shown (E14, E11, E35), (B) Time series illustrating sequential switching between modes: 10 different modes out of the total 200 interacting modes are shown.

_{ji}_{ij}_{i}

The figure shows the time series of 10 trials. Simulations of each trial were initiated at a different random initial condition. The initial conditions influence the trajectory only at the beginning due to the dissipativeness of the saddles (for details see also

Because of the complexity of System 4 with large

It is important to emphasize that the SHC may consist of saddles with more than one unstable manifold. These sequences can also be feasible because, according to

As we mentioned above, the variables _{i}

Decisions have to be reproducible to allow for memory and learning. On the other hand, a decision making (DM) system also has to be sensitive to new information from the environment. These requirements are fundamentally contradictory, and current approaches

A key finding in Decision Theory

To illustrate how the SHC concept can be applied to the execution of a specific cognitive function, let us consider a simple fixed time (_{j}

It is difficult to estimate analytically which strategy is the best to solve the first problem. It can be done in a computer simulation, but we can make a prediction for the second problem. Let us assume that we have a successful game and, for the sake of simplicity, that the reward on each state is identical (as our computer simulations indicate, the results do not qualitatively change if the rewards for each step are different). Thus, the game dynamics in the phase space can be described by the system_{j}_{k}_{j}_{k}_{k}_{k}_{j}_{k}_{k}_{k}_{k}_{k}_{k}

The parameters of the model were selected according to a uniform distribution in the range

When the trajectory reaches the vicinity of a saddle point within some radius _{j}_{(q)i} = _{j}_{(q)}−_{j}_{(q)i}_{j}_{(q)} with _{k}_{j}_{(q)i} at each saddle. In other words, we choose the maximal increment, which corresponds to the fastest motion away from the saddle _{i}

To evaluate the model, we analyzed the effect of the strength of uncorrelated multiplicative noise 〈_{j}_{j}

(A) Cumulative reward calculated as the number of cognitive states that the system travels through until the final time of the game

Concerning the formation of a habit it is important to note that the memorized sequence is subjected to the external stimulation that can change the direction at any given time. This fact is reflected in the results shown in

This simple game illustrates a type of transient cognitive dynamics with multiple metastable states. We suggest that other types of sequential decision making could be represented by similar dynamical mechanisms.

We have provided in this paper a theoretical description of the dynamical mechanisms that may underlie some cognitive functions. Any theoretical model of a very complex process such as a cognitive task should emphasize those features that are most important and should downplay the inessential details. The main difficulty is to separate one from another. To build our theory we have chosen two key experimental observations: the existence of metastable cognitive states and the transitivity of reproducible cognitive processes. We have not separated the different parts of the brain that form the cognitive modes for the execution of a specific function. The main goal of such coarse grain theory is to create a general framework of transient cognitive dynamics that is based on a new type of model that includes uncertainty in a natural way. The reproducible transient dynamics based on SHC that we have discussed contains two different time scales, i.e., a slow time scale in the vicinity of the saddles and a fast time scale in the transitions between them (see

Winnerless competitive dynamics (represented by a number of saddle states whose vicinities are connected by their unstable manifolds to form a heteroclinic sequence) is a natural dynamical image for many transient cognitive activities. In particular we wish to mention transient synchronization in the brain

Cognitive functions can strongly influence each other. For example, when we model decision making we have to take into account attention, working memory and different information sources. In particular, the dynamic association of various contextual cues with actions and rewards is critical to make effective decisions

The dynamical mechanisms discussed in this paper can contribute to the interpretation of experimental data obtained from brain imaging techniques, and also to design new experiments that will help us better understand high level cognitive processes. In particular, we think that the reconstruction of the cognitive phase space based on principal component analysis of fMRI data will allow finding the values of the dynamical model parameters for specific cognitive functions. To establish a direct relation between model variables and fMRI data will be extremely useful to implement novel protocols of assisted neurofeedback

We consider a system of ordinary differential equations^{2}-smooth. We assume that the system M1 has _{1}_{2}_{N}_{i}_{i}_{i}_{+1}.

The set

We denote by

We will use below the saddle value (see Equation 1)

For readers who are interested in understanding the details of these results we recommend, as a first step, to read references

It was shown in _{1} remains in a neighborhood of Γ until it comes to a neighborhood of _{N}_{i}_{ = 1, 2,…,N}

Of course, the condition

We consider now another system, say,_{1}_{2}_{N}_{i}_{i}_{i}_{i}_{i}_{i}_{i}^{d}.

_{0} ⊂ _{0}), 0≤

_{0}) = _{0}

_{0}) ∈

_{i}

Thus, if _{0}) can be treated as a sequence of switchings along the pieces _{i}

It follows that the property to possess a SHC is structurally stable: if a System M3 has a SHC then a ^{1}- close to System M3 also has it.

We prove this fact here under additional conditions. Denote by _{i}_{i}_{i}_{γ}(^{d}. The boundary ∂_{i}_{i}^{d}^{−2}×^{d}^{−2} is the (_{i}

_{0}<1 _{0} _{i}_{0} _{i}_{i} for which the following statement holds^{d}, _{i}_{i}_{0}) _{0},

_{i} and the other one_{0}.

_{i}_{i}

_{0}),

The lemma is a direct corollary of the theorem of continuous dependence of a solution of ODE on initial conditions on a finite interval of time.

Now, fix the numbers _{i}_{i}

The lemma M2 implies that there exits

Again, there exits

Continuing we come to

(

We choose

The following theorem is a direct corollary of Lemmas 1 and 2, the assumptions _{N}−_{2} and the choice of numbers

Under

The proof of Corollary is based:

on the fact that the local stable and unstable manifolds of a saddle point for an original and a perturbed system are ^{1}-close to each other;

on the theorem of smooth dependence of a solution of ODE on parameters and

on the open nature of all assumptions of Theorem M2.

The conditions

^{1}-

The proof can be made by a rather standard construction. Since _{1}∈_{2}∈^{d}^{−1}, _{1},_{2}), and the inequality _{1}>0 determines the side of ^{1}-smooth function ℜ^{d}→ℜ^{+} such that _{i}_{i}_{i}