Condition 1 (Drift-Diffusivity Condition).

For a positive vector , there exist positive constants and a nonnegative constant such that for all (5a)(5b)

Here denotes the standard inner product on . If we consider the process , then its dynamics can be seen to have two components drift and diffusion which have the form and respectively when (see page 2 in the Supplementary Material S1). Condition 1 gives upper-bounds for the magnitude of these two components and hence we call it the drift-diffusivity condition (abbreviated to Condition DD from now on; the abbreviations DD1 and DD2 stand for the first and second inequality, respectively). Observe that when the process goes above then it experiences a negative drift, suggesting that it will move downwards. This fact will be crucial for our analysis.

For now, we assume that a vector satisfying Condition DD has been found. In later sections we demonstrate how can be determined for a large class of networks by solving suitably constructed optimization problems.

For any positive integer , let denote the -th moment of defined by(6)Similarly let denote the -th moment of at time . Then is a tensor of rank whose entry at index is given by(7)where and is the distribution of .

Suppose that for some positive constants and we have(8)For any , let be the compact (finite) set defined by and let denote its complement. Markov's inequality (see [52]) implies that for any we can choose large enough to satisfyHence Prohorov's theorem (see Chapter 3 in [51]) ensures that condition SC1 holds. Similarly we can prove that condition SC2 will hold if for some there exists a constant such that(9)Relations (8) and (9) give uniform and asymptotic upper-bounds for . Using these relations we can also obtain uniform and asymptotic upper-bounds for the entries of . Such moment bound results have applications in queuing theory and control theory (see [53]). In Theorem 2 we show that under certain conditions, (8) and (9) hold and the upper-bounds can be easily computed.

Instead of the -th moment of the process , one can ask if the exponential moment of this process is uniformly bounded from above. This will happen if for some we have(10)If (10) holds, then the distribution is light-tailed (a distribution is called light-tailed if its tails are majorized by an exponential decay) uniformly in . This shows that all the cumulants of the distribution exist, which is an important result for the following reason. There is a considerable body of research dedicated to estimating the moments of the process directly without computing the distribution functions . For any integer , one can easily write the differential equations for the dynamics of the first moments. However when the reaction network has nonlinear interactions, this system of equations is not closed for any . Various moment closure methods (see [54], [55]) exist that specify ways to close these equations artificially and estimate the moments approximately. A popular moment closure method is the cumulant-neglect method which ignores the higher order cumulants of the distribution for all . Of course this method is only valid when the higher order cumulants exist. This is guaranteed if (10) holds. In Theorem 3 we give conditions for verifying (10).

We now come to the question of checking condition SC3 which says that the process is ergodic. This can only happen if the state space is irreducible, which means that all the states are accessible from each other. Recall the definition of from (2). Mathematically, we say that is irreducible if for all , we have and for some . In order to check the irreducibility of , one has to verify that there is no proper subset , such that once the process reaches a state in , it stays in forever. For reaction networks with mass-action kinetics, methods for checking irreducibility have recently been reported in [56] and [57]. These methods can be extended to situations where the propensity functions are positive in the positive orthant. When the propensity functions vanish inside the positive orthant, the problem of checking irreducibility can become much more complicated, and to the best of our knowledge no methods exist in the literature for this purpose.

We mentioned before that the vector is chosen so that the process has a negative drift at large values. Assuming irreducibility, this is sufficient to verify ergodicity of (see Proposition 4).

Suppose that condition SC3 is satisfied and the process is ergodic with stationary distribution . For any positive integer , let denote the -th moment of the stationary distribution . Then is a tensor of rank defined in the same way as (see (7)), with replaced by . Using Theorem 2 we can determine the values of for which is finite (componentwise) and as (see Theorem 5). We can also identify functions for which(11)holds for any . If is such a function, then the ergodic theorem for Markov processes (see [58]) says that(12)for any . Lastly, we also obtain conditions to check if the stationary distribution is light-tailed (see Theorem 6).

Moment bounds.

Our first result establishes that for certain values of , we can obtain uniform and asymptotic moment bounds for the -th moment of the process .

Theorem 2 Assume that Condition DD holds. Let be given by(13)For any positive integer , if then there exist positive constants and such that (8) and (9) hold.

The values of the constants and can be explicitly computed using a recursive relationship (see the Supplementary Material S1). Note that if , then for any we have for any . Hence for any we have Therefore using Theorem 2, we can obtain uniform and asymptotic moment bounds for the reaction dynamics (see the Supplementary Material S1).

Observe that if then . In this case, Theorem 2 says that for each positive integer and there exists a constant such that (8) holds. By showing that we have a such that for all positive integers , we obtain our next result, which gives sufficient conditions to check (10).

Theorem 3 (Uniform Light-Tailedness) Suppose that Condition DD holds with . Given an initial state there exists a such that

Ergodicity and Moment Convergence.

The next result verifies the ergodicity of a reaction network satisfying Condition DD. It follows from Theorem 7.1 in Meyn and Tweedie [59].

Proposition 4 (Ergodicity) Assume that the state space of the Markov process is irreducible and Condition DD1 holds. Then this process is exponentially ergodic in the sense that there exists a unique distribution along with constants such that for any

This result says that as , the distribution converges to exponentially fast. Henceforth we assume that the process is ergodic with stationary distribution .

Let be a function such that for some positive integer , there exists a satisfying for all . Using Theorem 2 we can prove that for such a , the relations (11) and (12) hold. As a consequence we obtain the following result about the convergence of moments with time.

Theorem 5 (Moment Convergence) Assume that Condition DD holds. Let be any positive integer satisfying . Then is finite (componentwise) and as .

If then Theorem 2 and (11) imply that for any positive integer there exists a positive constant such that(14)In particular, if then and (14) holds for each . By proving the existence of a constant such that for all positive integers we get our last result which shows that the stationary distribution is light-tailed.

Theorem 6 (Light-Tailedness at stationarity) Suppose that Condition DD holds with . Then there exists a such that

The framework described above is very general and can be applied to any network that satisfies Condition DD. In what follows, we specialize the results for two wide classes of networks with mass-action kinetics, namely reaction networks with monomolecular and bimolecular reactions. It will be, however, pointed out in the examples that the scope of our approach is much broader since more general propensities, such as those involving Hill functions or more general mass-action kinetics, can be considered.

Motivations.

The unimolecular case may seem quite restrictive at first sight and not of particular practical interest. We demonstrate below that, on the contrary, the proposed results on unimolecular reaction networks complete existing ones and are, therefore, of practical and theoretical interests. Although some explicit solutions for the CME are indeed known for some particular unimolecular reactions [16], it is still unknown whether the CME admits an closed-form solution for all possible type of unimolecular reactions. Note that no simplification nor assumption is made on the problem in our work. Hence, we are dealing with the very general unimolecular case.

The results developed of this section are useful in several ways. First of all, all types of unimolecular reactions can be handled with the proposed approach, making it more general than existing ones in this regard. Moreover, given a specific reaction network, the method allows one to establish whether a unique stationary distribution exists without solving the CME. This is particularly important since unimolecular networks may not be ergodic. In this case, the network can exhibit unstable behaviour which may suggest a flaw in the model if the considered real-world system exhibits stable trajectories. Moreover, in certain design applications such as those in synthetic biology, it seems natural to design networks that have well-behaved dynamics. Checking ergodicity provides a convenient way to determine if the network dynamics is well-behaved. Note, furthermore, that it is, in general, difficult to infer ergodicity directly from the solution of the CME (when it is known) since proving the existence of a unique globally attractive stationary distribution amounts to checking convergence of the solution to the CME to the same distribution for all possible initial distributions, which are in infinite number in our setup. This fact is even more true when large networks are considered since the explicit form of the solution to the CME is, in this case, very intricate [16]. The proposed results allow one to circumvent this difficulty and demonstrate that ergodicity can be assessed by very simple means, i.e. using basic notions of linear algebra. The results can be furthermore used to assess the structural ergodicity of a reaction network, that is, the ergodicity of a network for any combination of the rate parameters, by very simple means. This very strong and practically relevant notion is extremely difficult, again, to check from the solution of the CME since it would require to check the convergence of the solution of the CME to the same stationary distribution for all initial conditions and all positive values of the rate parameters, a very cumbersome task, even for small networks. Finally, the results pertaining to unimolecular networks will also turn out to play an important role in the ergodicity analysis of bimolecular reaction networks.

Theoretical results.

Let us start with several theoretical results that characterize the long-term behavior of unimolecular networks of the form (15).

Proposition 7 (Ergodicity of unimolecular networks) Let us consider the general unimolecular reaction network (15) and assume that the state-space of the underlying Markov process is irreducible. Let the matrices and , , be further defined as(16)Then, the following statements are equivalent:

1. The matrix is Hurwitz-stable, i.e. all its eigenvalues lie in the open left half-plane.
2. There exists a vector such that .

Moreover, when one of the above statements holds, the Markov process describing the reaction network is exponentially ergodic and all the moments are bounded and globally converging.

The above result shows that, for unimolecular networks, ergodicity and the existence of moment bounds can be directly inferred from the properties of the matrix defined in (16). The second statement, which characterizes the Hurwitz-stability of in an implicit way, will turn out to play a key role in the analysis of unimolecular and bimolecular reaction networks since checking whether for some is a linear programming problem.

It is important to stress that, in the result above, if we simply demand that the moments be bounded and converging, then may be allowed to have zero eigenvalues in certain cases. Note, however, that the moments will converge to values that may depend on the initial conditions.

In the case that the structure of the network (the reactions and the stoichiometries) is exactly known, but that the reaction rates are subject to uncertainties, the above theorem can be robustified to account for these uncertainties. To this aim, suppose that the matrix depends on a vector where is the number of distinct uncertain parameters. We write this matrix as and assume that there exists a matrix satisfying the following properties:

1. (in the componentwise sense) for all
2. There exists a such that .

Note that such a matrix may not exist, especially when some entries are not independent. However, when exists we have the following result.

Proposition 8 (Robust ergodicity) Let us consider the general unimolecular reaction network (15) described by some uncertain matrices and , . Assume further the matrix admits the upper-bound defined above and that the state-space of the underlying Markov process is irreducible for all uncertain parameter values . Then, the following statements are equivalent:

1. The matrix is Hurwitz-stable for all .
2. The matrix is Hurwitz-stable.
3. There exists a positive vector such that .

Moreover, when one of the above statements holds, the Markov process describing the reaction network is robustly exponentially ergodic and all the moments are bounded and globally converging.

Observe that checking the Hurwitz-stability property of each is equivalent to checking it for only . Hence we can conclude that, in this case, checking ergodicity of a family of networks is not more complicated than checking ergodicity of a single network. The case when the matrix is not defined is more complicated and is discussed in the supplementary material S1.

Computational results.

We now present several computational results that accompany the theoretical results of the previous section. It is possible to extract many computational results from our general framework, but for simplicity we only address the problems of checking ergodicity and computing the first-order moment bounds. The asymptotic first-order moment bound, defined in Theorem 2, is given by . So the question arises: what is the smallest value for such a ratio? Or, in other words, what is the smallest attractive compact set for the first-order moment of ? Several numerical methods, solving exactly or approximately this problem, are discussed in the supplementary material S1. One of them is the following optimization problem which is fully equivalent to Proposition 7:

Optimization problem 9 Let us consider the general unimolecular reaction network (15) and assume that the state-space of the underlying Markov process is irreducible. Assume further that the optimization problem(17)is feasible with as minimizer. Then, we have and Proposition 7 holds.

A striking feature about the above optimization program is that the numbers of variables and constraints are given by and , respectively. This means that the optimization problem scales linearly with respect to the number of species () in the network, and is independent of the number of reactions . Therefore, from the point of view of this optimization problem, the size of a unimolecular network can be identified with the number of species, and not the number of reactions. The above optimization problem can be efficiently solved using a bisection algorithm over that is globally and geometrically converging to . Each iteration consists of solving a linear program, a class of optimization problems known to be very tractable, and for which numerous advanced solvers exist [60]. These properties, altogether, make the overall approach highly scalable, which is necessary for dealing with very large networks.

Theoretical results for bimolecular networks.

When bimolecular reaction networks of the form (15)–(18) are considered, the left-hand side of condition (5a) can be expressed as(19)where is symmetric, and . Let be the stoichiometry matrix of the bimolecular reaction network (15)–(18), and let be the restriction of to bimolecular reactions, only. Further define a setWhen , the quadratic term in (19) vanishes, and equation (19) reduces towhich is exactly the same expression as in the case of unimolecular networks. This means that, with the additional constraint that , all the results derived for unimolecular networks directly apply to bimolecular networks as well. This allows us to obtain the following result.

Proposition 10 (Ergodicity of networks) Let us consider the bimolecular reaction network of the form (15)–(18) such that in (19) and assume that the state-space of the underlying Markov process is irreducible. Assume further that the network admits a non-empty .

If there exists a vector such that the inequality holds, then the stochastic bimolecular reaction network (15)–(18) is ergodic and all the moments are bounded and globally converging.

It is important to mention that the existence of a non-empty set is a prerequisite for utilizing the above result. Non-emptiness of is equivalent to the existence of a conservation relation for all the bimolecular reactions, i.e. the value of (at least) a positive linear combination of the species populations remains unchanged when any of the bimolecular reactions fires. Note that this definition extends to more general mass-action kinetics as well. A necessary condition for the non-emptiness of is that is not full-row rank. This non-emptiness condition may seem restrictive at first sight, but it will be shown that several important reaction networks from the literature satisfy this condition.

Whenever is empty or there is no such that holds, the next result can be used.

Proposition 11 (Ergodicity of bimolecular networks) Let us consider the bimolecular reaction network of the form (15)–(18) such that in (19) and assume that the state-space of the underlying Markov process is irreducible. Assume further that one of the following statements holds:

1. There exists such that and hold.
2. There exists such that is negative definite.

Then, the stochastic bimolecular reaction network (15)–(18) is ergodic and all the moments up to order are bounded and globally converging.

In the above result, the first statement can be checked using a linear program since the inequalities are componentwise. Checking the second statement, however, requires a semidefinite program, which is a more general convex program, that can be solved using solvers such as SeDuMi [61] and SDPT3 [62]. More details on the above result can be found in the supplementary material S1.

Computational results for bimolecular networks.

It is shown here that, once again, the theoretical results can be easily turned into linear programs that can be checked in a very efficient way. The following result is the numerical translation of Proposition 10.

Optimization problem 12 Let us consider a bimolecular reaction network (15)–(18) and assume that the state-space of the underlying Markov process is irreducible. Assume further that and that the optimization problem(20)is feasible with as minimizer. Then, we have and Proposition 10 holds.

The computational complexity of this optimization problem scales linearly with the number of species and can therefore be solved for large networks. The following optimization problem is the computational counterpart of the first statement of Proposition 11.

Optimization problem 13 Let us consider a bimolecular reaction network of the form (15)–(18) and assume that the state-space of the underlying Markov process is irreducible. Assume further that the nonlinear optimization problem(21)is feasible with as minimizer. Then, we have and Proposition 11 holds.

The above optimization problem does not scale as nicely as (20) since, in the worst case, the number of constraints related to is quadratic in the number of species. The problem, however, remains tractable due to the linear programming structure.

Jumping potential wells.

Our first example shows that stochastic dynamics can jump potential wells and leave the stability regions of the deterministic dynamics, resulting in an unstable behavior. Consider the following reaction network:(22)where . The deterministic dynamics for this network is given by(23)where denotes the concentration of . The fixed points for the dynamics are and , respectively. From the graph , it is immediate that the fixed point is locally asymptotically stable with the region of attraction as while the other fixed point is unstable.

We now consider the stochastic version of this network and let be the generator of the corresponding Markov process. For the identity function we have(24)The polynomial on the right-hand side has two positive roots that are(25)This means that for all satisfying , we have , for some , implying that the drift is positive. So if the state of the state of the network reaches a value that is greater than , then there is a possibility that the trajectories become unbounded with time.

To demonstrate this, we pick and . In such a case, the largest root of the polynomial on the right-hand side of (24) is . We can see that the region where the drift is negative is actually larger than the region of attraction of the locally asymptotically stable fixed point for the deterministic dynamics. This is due to the fact that the propensity function of the bimolecular reaction differs from whether we are in the deterministic or in the stochastic setting.

Let us now set the initial condition for the deterministic model and for the stochastic one. Note that they both lie within the region of attraction of the fixed point of the deterministic dynamics and in the region of negative drift for the stochastic dynamics. We then perform 1000 SSA runs over 100 seconds and stop the simulation when the propensity function of the bimolecular reaction exceeds the value corresponding to molecules (approx. ). At this rate value, the bimolecular reaction fires, on average, every seconds, leading to an explosion of the state of the system and to unbounded trajectories. Out of 1000 SSA runs, all were stopped before the end of the simulation time-period (100 seconds). This behavior strongly indicates that the system is not ergodic despite the fact that the deterministic model has a locally asymptotically stable fixed point. Figure 1 illustrates the above discussion.

Globally stable deterministic dynamics does not imply moments stability.

In the previous example, the stochastic and deterministic behaviors were different, but one can still understand stochastic instability through the deterministic model. The deterministic dynamics possesses a region in which the solutions explode and the randomness in the stochastic dynamics allows it to enter this region in finite time and grow unbounded thereafter. We now present an example which is more striking in the sense that the deterministic model cannot be used in any way to infer the instability of the stochastic model. In this example, the deterministic dynamics has a unique fixed point which is exponentially stable, while the stochastic dynamics is not ergodic with all its moments growing unboundedly with time.

Consider the reaction network given by(26)Let be the vector of concentrations. The state of the deterministic model evolves according to(27)Assume that the initial conditions satisfy , for some . Then we have the following result.

Theorem 14 The unique equilibrium point of the dynamics (27) given by(28)is globally exponentially stable.

In the stochastic setting, the picture is completely different as the next result indicates.

Theorem 15 The Markov process corresponding to the stochastic model of network (26) is not ergodic and all its moments grow unboundedly with time. Moreover, if for some , we have that for all .

To illustrate this result, we simulate the deterministic and the stochastic process (10000 SSA runs) for , , , and . The results are shown in Figure 2.