1 Introduction

Ever since the dawn of cellular biology, the central analogy that we employ to describe cells is that of miniature machines that transform the information about its environment to appropriate responses. The responses take the form of increased or decreased gene expression, protein activation or deactivation, or regulation of transport between cellular compartments and exterior of the cell. There is only a short step from viewing cells as little machines to the desire of controlling them, repairing them, and then building new cellular functions. This is the starting point of synthetic biology [1–4]; for recent review of progress and challenges see [5]. The success of engineered mechanical or electronic systems, crucially depends on (i) modularity of their designs and (ii) ability to model complicated assemblies of parts before they are built. In synthetic biology both of these steps present significant challenges. The focus of this contribution is on a novel mathematical approach to addressing the second challenge.

The strength of our approach that we call Dynamic Signatures Generated by Regulatory Networks (DSGRN) is that we are agnostic to the specific biochemical or biophysical design of the elements of the circuits that we analyze. The input consists of a mathematical abstraction of a gene regulatory network, e.g. Fig 1a, that consists of nodes and annotated directed edges indicating activation or repression. The user is required to provide a means of scoring the behavior of the network from the information about dynamics that is computed by DSGRN. The DSGRN software [6] then allows a ranking of the networks in question based on this score.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1.

(a) Regulatory network 107. There are three nodes labelled 0, 1, and 2. Edges → indicates up regulation and ⊣ indicates down regulation. (b) Conceptual image of ascending hysteresis. Off/low output for low values of input signal s. On/high output for high values of s. Bistability for intermediate values of s allowing for hysteresis. (c) Descending hysteresis. Results, analogous to those for ascending hysteresis, are discussed in Section 5.

https://doi.org/10.1371/journal.pcbi.1009189.g001

To demonstrate the applicability of DSGRN we focus on the question of design of a hysteretic switch. There are three reasons that make this a natural choice. First, it is conceptually simple. The same ideas can be applied to the design or analysis of more complicated logic circuits, but this naturally entails a corresponding increase in complexity of computation and analysis. Second, it is one of the early successes of synthetic biology [7]. Third, its resolution requires a global understanding of the dynamics of the design over multiple phenotypes, e.g. monostability versus multistability, and therefore is a nontrivial mathematical problem. While experimental implementation of a design consisting of two mutually repressing transcription factors was a great triumph of predictive modeling, the design of [7] seems to be fragile and follow-up attempts [8] have been made to make it more robust. A natural question arises if more complex networks are able to exhibit a more robust switching behavior. This paper provides an efficient algorithmic approach towards addressing this question.

We begin with the well established observation that in vivo gene regulatory networks operate under noisy conditions [9, 10]. Rather than attempting to provide a specific model for the noise, we adopt the perspective that noise is significant enough to impact the initial conditions of the dynamics and the parameter values at which the network operates, but not so significant that it overwhelms the underlying nonlinear dynamics. Thus, for this paper we adopt the following design principle: a synthetic network should attempt to maximize the range of the phase space and parameter space where it exhibits the desired function.

Fig 1b summarizes the minimal structure and functionality of an (ascending) hysteretic switch. As values of an input signal are increased from a low level, the output signal is off. Once the input signal achieves a given threshold, S₁, the output signal changes to on and remains at on for high values of the signal. As the input signal is lowered the output value remains on until the input signal reaches the threshold S₀, that is lower than S₁, at which point the output signal switches to off.

This property of “remembering” past states is called hysteresis. Since on and off are determined by dynamics, they must be represented by stable states. To obtain hysteresis requires that both stable states be present for the range of signal between S₀ and S₁, e.g. that the system exhibits bistability. For this reason a system of this type is often referred to as a bistable switch.

In order to have a well defined problem we ask and provide answers to the following question: Which three-node networks exhibit the functionality of a bistable switch over the largest range of parameter values? The reader is no doubt aware that as of yet we have not described our model for the network dynamics nor indicated what signals indicate on and off.

This is discussed in varying detail in later sections. We have adopted this approach in an attempt to emphasize that DSGRN can be used with minimal knowledge of the rather substantial mathematical theory and machinery that justifies the software [11–17]. While dynamics is expressed as an action on a phase space, the specific action very much depends on parameters. With this in mind the DSGRN model provides a combinatorial representation of a decomposition of parameter space and combinatorial representations of dynamics. We attempt in Section 2 I1-I3 to provide a minimal description of these combinatorial representations that allows us to describe our results. We do not expect that this description is sufficient for the typical reader to understand how DSGRN works. Thus, we provide more detailed descriptions of various aspects of the DSGRN machinery in Section 4.

We remark that it is the fact that DSGRN is a combinatorial model that allows us to perform extremely efficient computations, and as indicated above, allows DSGRN to be agnostic to the biochemical or biophysical details. Of course, it is precisely these details that play essential roles in the actual construction of components of a synthetic network. With this in mind, the true novelty of DSGRN is that it employs ideas from computational algebraic geometry to provide an explicit decomposition of parameter space [15, 17] on which dynamics is understood. It is unreasonable to expect that these precise bounds on parameters should be valid for more traditional models involving explicit nonlinearities. Nevertheless, as we demonstrate in the context of ordinary differential equation (ODE) models using Hill function nonlinearities with more than 20 dimensional parameter spaces, DSGRN provides considerable insight into parameter values at which bistable switching occurs.

2 Results

Our goal is to identify three-node networks that act as bistable switches over large regions of parameter space. As indicated in Fig 1a we label the nodes in our network by 0, 1, and 2, and assume that node 0 is directly affected by the input and the output of the network is expressed via node 2. Since each node can influence any other node (itself included) in three ways—activation, repression, or no impact—there are 3⁹ = 19, 683 distinct three node networks. The stipulation that 0 is an input node and 2 is the output node precludes any reduction in the number of networks due to symmetries. We exclude 5, 103 trivial networks as defined in Section 4.6.

The number of regions into which DSGRN decomposes parameter space grows rapidly with the number of edges in the network. For example, three-node networks with 8 edges can have up to 823, 011, 840 distinct parameter regions, while for 9 edges this number increases to 93, 329, 542, 656. Because of this size, we only consider one network with 9 edges, that where each edge is an activator. Therefore, we consider 14, 068 networks. These networks are analyzed using the DSGRN software described in Section 4. However, for the purpose of reporting the results we include the following information about the combinatorial structure of DSGRN. I1 provides information about the combinatorial dynamics. I2 and I3 discuss the decomposition of parameter space.

1. I1. For three node networks the phase space is {(x₀, x₁, x₂) ∣ x_n > 0} where the variable x_n is associated with node n. For a given parameter value DSGRN decomposes phase space into cubes defined by the hyperplanes x_n = θ_m,n where θ_m,n is the threshold parameter associated with an edge from node n to node m. The global dynamics at the given parameter value is determined by a state transition graph (STG) defined on these cubes. A cube C that has a self edge under the STG is labeled as an FP(i₀, i₁, i₂). This should be interpreted as a stable state under the associated dynamics. The i_k ∈ {0, 1, 2, 3} indicates that if x ∈ C, then x_k is greater than i_k of the thresholds θ_*,k, and thus provides information about the location in phase space of the stable state.
2. I2. The parameter space for the DSGRN model consists of multiple positive real numbers associated with each node (1 for the node, 2 for each incoming edge, and 1 for each outgoing edge). For node n the DSGRN software produces a finite decomposition of parameter space and encodes this decomposition via a factor graph, denoted by PG(n). Two vertices in the factor graph are connected by an edge if they represent regions of the continuous parameter space whose closures intersect on a codimension-one face.
   Details about the parameters are presented in Section 4.1. For the moment we remark that if at a vertex in the factor graph parameters associated with the in-edges do not align properly (i.e. the parameters corresponding to the in-edges are consistently too high, or consistently too low) with the parameters associated with the out-edges, then one can remove edges associated with the node. This in turn implies that the dynamics is captured by a simpler regulatory network. A node in the factor graph is defined to be essential if every in-edge and every out-edge are relevant for the dynamics [16]. The essential factor graph PG_e(n) is the subgraph of the factor graph PG(n) consisting of the essential nodes.
3. I3. The full parameter space of a regulatory network is a product of the parameter spaces associated with each node. The decomposition of the full parameter space is indexed by the parameter graph PG. Since each region of this decomposition is made up of the product of the region from the decomposition of the parameter space of each node, the parameter graph is the product of the factor graphs, i.e. . Of fundamental importance is the fact that for each node in the parameter graph the state transition graph is constant over all parameters in the associate region. For each node in the parameter graph the DSGRN output includes the FP(i₀, i₁, i₂) that arise from the associated state transition graph.

Fix a regulatory network with a fixed set of parameter values. In particular, this identifies a unique vertex (v₁, v₂) in the graph PG(1) × PG(2). Since we are interested in a direct correspondence between network topology and bistable switching we restrict our attention to the essential nodes PG_e(1) × PG_e(2).

We view the continuous change of the input signal s to node 0 as a curve through the parameter space associated with node 0, e.g. monotone change in inducer concentration induces a monotone change in abundance of protein produced by gene 0 (cf. IPTG in [7]).

The DSGRN analogue to a continuous change in the inducer is a discrete path within the factor graph PG(0), which is realized in the entire parameter graph as a path within the graph PG(0) × (v₁, v₂). Each vertex on this path is an element of the parameter graph PG and hence for each vertex DSGRN can determine the global dynamics.

We say the path exhibits ascending hysteresis if at the initial vertex of the path there is a FP(i₀, i₁, j₁), at the final vertex of the path there is a FP(i₀, i₁, j₃) where j₃ > j₁, and at some intermediate vertex of the path there are two stable states FP(i₀, i₁, j₁) and FP(i₀, i₁, j₂) with j₂ > j₁. For the purposes of this paper we set j₁ = 0 and require that j₂ > 0 and j₃ > 0. Note that since we need to observe at least three distinct forms of global dynamics we insist that our paths be of length at least three.

Since such a path need not traverse all of PG(0) we refer to it as a partial path. We focus on partial paths because we do not presume to know the parameter values associated to node 0 at which the regulatory network is acting in the absence of the input signal (cf. in the context of construction of the toggle switch [7] we do not presume to know the level of protein production in the absence of the added IPTG).

We define the hysteresis score of a regulatory network to be the percentage of paths that exhibit ascending hysteresis among all paths. The total number of the paths is given by the number of paths of length at least 3 in PG(0) times the total number of vertices in PG_e(1) × PG_e(2).

The ranking according to the hysteresis score is presented in Fig 2 (left). Observe that the typical three-node network is incapable of exhibiting hysteresis and less than 1% of networks are capable of producing hysteresis for the majority of parameter values. However, fourteen networks are capable of producing hysteresis for more than 60% of the paths. Based on our design principle we now restrict (for the most part) our attention to these fourteen three-node regulatory networks shown in Fig 3.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2.

Left: 14,098 three node networks ranked by (ascending) hysteresis score analyzed at the essential parameters. This is the percentage of partial paths in the essential parameter sub-graph which exhibit switch-like behavior. The rapid decrease in score indicates that most networks are unlikely to exhibit hysteresis at most parameters. Right: The top scoring networks are further discriminated by scoring hysteresis in a neighborhood of the essential parameters. The scores after perturbation (red) are significantly lower than the scores at essential parameters (blue) for the top 4 networks. This is an indication that these networks are fragile i.e. will fail to perform well if any component of the network is removed.

https://doi.org/10.1371/journal.pcbi.1009189.g002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. The top fourteen regulatory networks by hysteresis score.

The six regulatory networks in dotted boxes are fragile, while the eight outlined via the dashed lines are robust. Observe that the three networks outlined by the solid lines are also consistent and each node acts only as an activator.

https://doi.org/10.1371/journal.pcbi.1009189.g003

Returning to the motivation of our design principle that in vivo gene regulatory networks operate under noisy conditions, we remark that by restricting our analysis to essential nodes we are assuming that even under noisy conditions each edge of the regulatory network operates effectively. For example, networks 1-4 in Fig 3 have a partial path hysteresis score of 100% since they exhibit partial path hysteresis in all of their essential parameter nodes. However, if any one of the edges is removed, then the remaining network will not be bistable and not capable of hysteresis (we return to this point in greater detail in Section 4.4). This observation motivates a search for a measure of the robustness of hysteresis with respect to network perturbations.

A more reasonable assumption might be that not all edges in the regulatory network function effectively at all times. To capture this, for the top fourteen regulatory networks we consider the set of parameter nodes in PG(1) × PG(2) that are within one edge of PG_e(1) × PG_e(2) and repeat the computation of the partial path hysteresis score. The results—we call this the perturbed hysteresis score—are shown in red in Fig 2 (right).

As expected, the perturbed hysteresis score is less than the hysteresis score. However, this loss of functionality varies widely across networks and is difficult to predict from the topology alone. We define the robustness score of a regulatory network to be its perturbed hysteresis score divided by its hysteresis score. We refer to each network by its position in the list ordered by decreasing hysteresis score (see [18]). Networks 1-4, 13, and 14 (boxed with dotted lines in Fig 3) have robustness scores under 0.5. This suggests that under ideal conditions these networks will perform well as a switch. However, they are easy to break in the sense that small perturbations from the essential parameters largely destroy their ability to act as a switch. With this in mind we call regulatory networks with robustness score less than or equal to 0.5 fragile, while those scoring above 0.5 are referred to as robust (and are boxed by dashed lines in Fig 3).

Invoking the hysteresis rank and robustness allows us to reduce our attention to eight regulatory networks at which point we can focus on the actual topology of the design. The implementation of a gene regulatory network is constrained by available control mechanisms which must be considered when comparing networks. For instance, Network 5 in Fig 3 requires that node 0 act both as an activator and repressor. While this is biologically possible, e.g. the dimer CI acting in phage λ lysogenic/lytic switch, simpler design features may be desired. We call a node consistent if it acts as an activator or a repressor, but not both. As is indicated by the solid boxes in Fig 3 there are three high ranked and robust regulatory networks in which all nodes are consistent: 6, 11, and 12.

Note that in networks 13 and 14, node 1 provides a constant input to node 0. Therefore node 1 does not affect the existence of ascending hysteresis. Removing this ineffectual node transforms networks 13 and 14 into a two node toggle switch with mutually activating edges, and with positive self-regulation on node 0. The fact that these networks are fragile in our analysis recapitulates the observation from [7, 8] that the two-node design of the toggle switch is fragile.

It is interesting to observe that the top three consistent regulatory networks that provide ascending hysteresis are based on nodes that are activators. The consistent regulatory network based on repressing nodes that has the highest hysteresis score (33.33%) is shown in Fig 4. This is a fragile network with the perturbed hysteresis score of 6.66%.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4.

(a) The best network with only repressing edges, network 66, has a hysteresis score of 33.33%. (b) Continuation of equilibria in the Hill model (1) for regulatory network 12 using n = 4. (c) Regulatory network 33. (d) Regulatory network 3839.

https://doi.org/10.1371/journal.pcbi.1009189.g004

Thus, simple robust design of ascending hysteresis seems to require the use of activators. Interestingly, the role of activators in ascending hysteresis is not mirrored by the role of repressors in the descending hysteresis. First, no 3-node network that only consists of activators is capable of producing descending hysteresis. Second, in contrast with Fig 3, there is no network among the top 14 networks for descending hysteresis, with only repressing edges (see Fig 9).

Based on three criteria—hysteresis score, robustness score, and consistency of nodes—Network 12 is the most desirable design. Returning to the question posed in the introduction—are more complex networks capable of exhibiting more robust switching behavior—the answer is a qualified yes. However, complexity alone is not sufficient. This is evidenced by the fact that if Network 12 is modified by adding an additional activating edge from node 1 to node 2, a self activation for either 1 or 2, or any combination of these edges the resulting network has a smaller hysteresis score, often dramatically so. For instance if we attempt to maximize complexity by adding every single edge as an activator, the resulting network has a hysteresis score of 0%.

The results discussed up to this point have all been obtained from the combinatorial computations of DSGRN. More traditional modeling of regulatory networks is based on ODEs. As is discussed in Section 4.1 and in Section 6, there is a direct translation from DSGRN parameters to nonlinearities based on Hill functions in the limit when the exponents in the Hill function are very large. We now demonstrate that information from DSGRN has implications for the ODE models. Two important observations are that (i) trustworthy ODE computations are many orders of magnitude more expensive than DSGRN computations, and (ii) it is unreasonable to expect the explicit DSGRN decomposition of parameter space to apply precisely to any specific ODE.

To expand on this we consider Network 12 and a corresponding ODE. Assumptions need to be made on how multiple in-edges to a node impact the rate of change of the associated variable. These assumptions are discussed in detail in Section 4.1, but for the moment it suffices to state that since all the arrows in Network 12 have the form → leads to a summation of the nonlinear terms affecting the growth rate. Hence we consider (1) where for simplicity have made two modeling assumptions. First, the effect of the external signal on the growth rate of x₀ is given by a simple linear additive term s. Second, the exponents n of the Hill functions are the same. In addition, there are 21 other parameters that lie in (0, ∞)²¹ (see Section 4.1). There are 707 vertices in PG(0) (see [15]), and 24 vertices in PG_e(1) × PG_e(2) (see Section 4.2).

The most computationally efficient means of identifying the desired hysteresis curve in Eq (1) is to fix a parameter value in (0, ∞)²², choose an initial value s₀ for s, find a stable fixed point with low x₂ value, perform continuation with respect to arc-length of a fixed length, and check that two saddle-node bifurcations have occurred. An example of this computation is shown in Fig 4b (see Section 6 for details). With the goal of quantifying how robustly this system exhibits hysteresis the obvious question is how many parameter values should be chosen and what is the appropriate choice of arc-length. Based on the number of vertices in PG(0) and PG_e(1) × PG_e(2), the number of partial paths computed by DSGRN is on the order of 10⁵. Furthermore, since each region of parameter space is an unbounded open set in (0, ∞)²² even sampling each region is non-trivial. This suggests that performing sufficiently many continuation computations to compare with the DSGRN hysteresis score is prohibitively expensive.

With this in mind we greatly simplify the DSGRN computations being performed. We remark that a partial order can be placed on the vertices of PG(0) (see Section 4.1) such that there is a unique minimal vertex and unique maximal vertex . A path within the factor graph PG(0) is full if and . This leads to two new scores obtained as follows.

For each vertex in (v₁, v₂) ∈ PG_e(1) × PG_e(2) we consider all full paths and mark those paths that exhibits hysteresis as hysteretic. The full path hysteresis score is the percentage of hysteretic paths among all full paths. The perturbed full path hysteresis score is the same but based on the one edge neighborhood of PG_e(1) × PG_e(2).

To compare the predictions of DSGRN against ODE models we chose four regulatory networks, Network 12 from Fig 3, Network 107 from Fig 1a, and Network 33 and 3839 shown in Fig 4c and 4d. These latter three networks were chosen because they exhibit different hysteresis scores: 42.46%, 18.95%, and 0%, respectively. For each network we performed two sets of experiments. For the first we randomly chose 1000 parameter values that lay in , and for the second we chose 1000 parameter values in the one edge neighborhood with respect to PG_e(1) × PG_e(2). In each case for each parameter choice we performed the above mentioned procedure to identify whether or not one obtains a hysteresis curve. The results are indicated in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. The regulatory network number comes from the ranking of the hysteresis score [18].

The bottom two rows indicate the full path and partial path hysteresis scores obtained from DSGRN. The first column indicates the exponent of the Hill function used in the ODE model for the regulatory network, e.g. (1). The two columns under each regulatory network number indicate the percentage of continuation computations that result in a hysteresis curve. The first column assumes the parameter value is in a region defined by and the second column assume the parameter value is in region defined by a one edge neighborhood. 1000 curves were computed for each entry.

https://doi.org/10.1371/journal.pcbi.1009189.t001

We highlight three observations from Table 1.

• For large Hill exponent n the DSGRN full path scores and the ODE scores are quite similar. This is not surprising. DSGRN is based on a mathematical approach to nonlinear dynamics that captures features that persist under perturbation [11–14, 19].
• For more biologically realistic levels of n the quantitative agreement between the scores disappears. Again, this is not surprising. It has long been known that in order for nonlinearities with gentle sigmoidal shape to intersect at multiple points, their parameters must be carefully adjusted. As a result, for low n, bistability is rare.
• The relative ranking by DSGRN of the capability of regulatory networks to achieve robust switching is predictive of the observations from the ODE models. Moving from left to right along the rows, DSGRN predicts that the corresponding networks are progressively less capable of acting as a robust switch. For the most part the ODE simulations agree with this prediction. Most importantly, Network 12 is the best at all values of n. We include Networks 107 and 3839 to emphasize that the predictive power of DSGRN is not perfect. However, for these networks the realization of ascending hysteresis is consistently low, again suggesting that DSGRN is capable of identifying regulatory networks of interest.

3 Discussion

DSGRN provides a modeling framework and associated computational tool that is capable of analyzing all 3-node regulatory networks for prevalence over a large range of parameter values of a particular phenotype. Our investigation into the identification of the robust expression of the phenotype of hysteresis demonstrates DSGRN’s practical value—in synthetic biology hysteresis forms a basis for a design of a bistable switch. It also demonstrates the power of DSGRN to capture complex dynamics—hysteresis arises from global organization of multiple phenotypes (monostability, bistability, monostability) as a function of increasing external input. Furthermore, the publicly available searchable database of all 3-node networks allows synthetic biologist to select robust designs that meets additional implementation criteria [18].

It is important to note that the complexity of hysteresis phenotype makes it challenging to succinctly describe the network features i.e. number, sign and position of edges, that characterize high scoring networks. While it is known that presence of positive edges generally leads to bistability, our computations show that there is no simple relationship between the hysteresis score and the number of positive edges. Furthermore, we believe that as size and complexity of networks increase, simple network features are even less likely to predict presence or absence of specific dynamics. Thus, a direct evaluation of the prevalence of such dynamics across parameters by DSGRN becomes a crucial tool in understanding of behavior of complex networks.

Obviously, DSGRN can be used to search for simpler phenotypes. In particular, it has been used to catalog types and number of intermediate steady states in epithelial-mesenchymal transition network [20], as well as to characterize START network controlling G1/S transition in human cell cycle [16]. In principle it can be applied to the analysis of more complicated control circuits.

DSGRN occupies a novel niche in the collection of modeling tools for regulatory networks.

On one hand it is similar to Boolean models, where in the simplest setting gene expression is either on or off, i.e. 0 or 1, and the update rule that encodes the dynamics is a Boolean function. The dynamics of Boolean models is thus efficiently computable. Conceptually, the closest analogue to DSGRN is the work pioneered by L. Glass and S. Kauffman [21, 22] involving switching systems where the logic of the Boolean system is embedded into continuous differential equations with the goal of predicting qualitative features of differential equation dynamics by the dynamics of the asynchronously updated embedded Boolean system. The state transition graphs used by DGSRN extend the embedded Boolean systems and allow for modeling a broader class of dynamics, while preserving the efficiency of computations. DSGRN also combinatorializes parameter space to understand how dynamics changes under the change in parameters. Again, similar to the Boolean models the goal of DSGRN is not to precisely match and reproduce carefully measured expression data of genes over a wide variety of growth conditions.

However, in the setting of systems biology more often than not such measurements are not available, especially for networks involving more than a few genes. In such situations, DSGRN can be a first step in understanding of network dynamics. DSGRN can search through many proposed networks over a wide range of parameter values, and eliminate those that do not support the desired dynamical behavior, coarsely defined e.g. equilibria or oscillations with particular patterns of high and low expression values. Elimination of networks or reduction of potential functional parameter values for a given network provides significant reduction of hypotheses space.

In contexts where one has carefully measured expression data of genes, modeling tools of choice often involve ODEs with experimentally determined parameters. In contrast to the Boolean approach, the mathematical foundations of DSGRN—a continuous phase space and parameter space—allows for direct comparison with an extremely broad class of ODE models [19]. As is demonstrated in this paper, DSGRN provides a means to compare systems of ODEs. In particular, it can rank the relative ability of ODE models to produce particular dynamics over large ranges of parameter values. At the same time, DSGRN provides a priori bounds on parameter regions where sampling of parameters and fitting the expression data is feasible. Finally, DSGRN provides, at low computational cost, the ability to describe relationships between any simultaneous change in many parameters and the changes in network dynamics. This facilitates generation of hypothesis of behavior of a system under different conditions and leads to prioritization of experiments.

To add additional emphasis on the importance of the computational efficacy of DSGRN we note that the comparisons in Table 1 are based on full path and perturbed full path hysteresis scores. We expect that in many applications it is more likely that external control will not lead to a path that extends across the entire parameter domain of the input node. In this case the partial path statistics are more relevant. However, carrying out such computations in the setting of tradition ODE models appears to be computationally prohibitive.

Due to its ability to describe complex relationship between network parameters and network dynamics, albeit on a coarse level, and the associated systematic reduction of the hypothesis space for experimental examination of this dynamics, DSGRN should become a part of an essential toolbox in systems and synthetic biology.

4 Methods

We provide a brief description of how DSGRN combinatorializes both phase space and parameter space of regulatory networks. For more details the reader is referred to [15, 16, 23].

4.2 Parameter graph

As indicated above, given a regulatory network with N nodes and E edges, the DSGRN parameter space is (0, ∞)^N+3E. The parameter graph provides combinatorial representation of a finite decomposition of this parameter space. Each node of the parameter graph corresponds to an explicit open semi-algebraic set [15, 17] with the property that the STG (see Section 4.3) is constant for all parameters in that set.

As is discussed in I2 and I3 the parameter graph is the product of the factor graphs and each factor graph is determined by the in-edges and out-edges of its corresponding node in the regulatory network. Fig 5a shows the factor graphs for nodes whose number of in and out-edges are (from left to right) (2, 1), (1, 2), and (1, 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5.

(a) For fixed ordering of the thresholds the parameter graph for the network in Fig 4a where the nodes 0, 1, and 2 have 2 in-edges and 1 out-edge,1 in-edge and 2 out-edges, and 1 in-edge and 1 out-edge, respectively, written as a Cartesian product of the three factor graphs. For i = 1, 2, PG_e(i) is the subgraph consisting of only the blue vertices. (b) Phase space decomposition for Regulatory Network shown in Fig 4a under the assumption that θ_2,1 < θ_0,1. The cell identified by ⋆ is labeled (1, 0, 0).

https://doi.org/10.1371/journal.pcbi.1009189.g005

General descriptions of how parameters are identified with nodes of the factor graph can be found in [15, 17]. To provide intuition we focus on the simplest factor graph corresponding to (1, 1), i.e. one in-edge and one out-edge.

Because there is a single in-edge and a single out-edge, as indicated in (2) there is a unique γ, θ, ℓ and δ. Furthermore, the sign of the expression (2) is constant over the subsets of parameter space defined by the inequalities (3) where θ_k,n is the threshold associated with the edge from node n to node k. These three regions are represented by the nodes in the rightmost factor graph PG(2) in Fig 5a.

Observe that if the parameters satisfy the leftmost or rightmost sets of inequalities in (3), then the sign of (2) is independent of the value of x_m. Since we are assuming that n has a unique in- and out-edge, this implies that we can remove node n from the network without losing information about the potential dynamics (the constant growth rate on x_k due to x_n will be compensated for by the parameter values). Thus only the node associated with the middle set of inequalities in (3) is essential, as is indicated by the blue node in Fig 5a.

We remark that in the context of switching systems the analogue of an essential parameter node is the notion of effective regulator [25]. However, the restriction in the DSGRN setting is in the choice of parameter, i.e. a node in the parameter graph, as opposed to the choice of a Boolean function defined on the regulatory network.

Observe that the order of the set of inequalities of (3) can be obtained by associating it with increasing values of γ. This same approach applies in general and we use it to induce a partial order on any form of factor graph that PG(0) may assume. All full and partial paths discussed in Section 2 are chosen to be strictly monotone with respect this partial order.

4.4 Intuition into robustness and fragility

A focus of this paper is on identifying regulatory networks that, if they can be built, will perform as desired under a variety of settings. This led to a measure of robustness and fragility. We do not claim to have a sharp characterization of the quantities, but we can provide a posteriori intuition.

To understand fragility consider Network 1 in Fig 3. Because there is one out-edge for each node, there is one hyperplane x_n = θ_m,n associated to each coordinate of phase space (0, ∞)³. Thus phase space is divided into eight three-dimensional cubes indexed by {0, 1}³ that is represented by the graph shown in Fig 6 where each node represents a cube and edges indicate that the two associated cubes intersect along a hyperplane. For this simple example, the direction of the arrow is determined by the sign of (2) evaluated at the hyperplane x_n = θ_m,n (see Section 7 and [15] for the general procedure).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Hysteresis representation in DSGRN.

State transition graph for Network 1 in Fig 3 over a factor graph PG(0) at the unique essential node of PG(1) × PG(2). () represents a low input to node 0 and exhibits FP(0, 0, 0); () represents a medium input to node 0 and exhibits bistability between FP(0, 0, 0) and FP(1, 1, 1); () represents a high input to node 0 and exhibits FP(1, 1, 1).

https://doi.org/10.1371/journal.pcbi.1009189.g006

There are 27 nodes in the parameter graph for Network 1 in Fig 3. The same argument as presented in Section 4.2 shows that there is a single essential node in PG_e(1) × PG_e(2) given by the inequalities (4) Again, as discussed in Section 4.2, the factor graph for PG(0) consists of three nodes and thus for Network 1 there is a unique full and unique partial path . The STG associated to each node in the path through parameter the parameter graph are indicated in Fig 6.

The three STGs shown in Fig 6 indicate the existence of ascending hysteresis. The blue nodes in the STGs indicate the attracting states that, as indicated in I1, DSGRN labels as an FP. Thus moving from left to right we have monostability (FP(0, 0, 0)), bistability (FP(0, 0, 0) and FP(1, 1, 1)), and monostability (FP(1, 1, 1)). Observe that the x₂ values at these attracting states (again moving from left to right) are 0, 0 and 1, and 1. Since this is the unique partial path for Network 1, the hysteresis score is 100%, in agreement with Fig 2. However, we leave it to the reader to check that if one chooses a node in PG(1) × PG(2) that differs from the essential node by a single inequality (there are four such nodes), then along the associated path one will not achieve the desired bistability state. Thus, the perturbed hysteresis score is 20% and Network 1 is labeled as fragile.

To provide intuition into robustness consider Network 6 in Fig 3. Both node 1 and node 2 have a single in and out-edge, and thus there is a single essential node in PG(1) × PG(2). Having fixed this parameter value, we need to consider the STGs associated with paths over the factor graph PG(0). Observe that phase space is partitioned into regions bounded by the hyperplanes defined by x₀ = θ_1,0, x₀ = θ_2,0, x₀ = θ_3,0, x₁ = θ_0,1, and x₂ = θ_0,2. Thus the nodes of the desired STGs are as shown in Fig 7.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. STG for network 6 from Fig 3.

Since node 0 has three output edges there are three thresholds of the variable corresponding to 0 and four states 0,1,2,3. The arrows are valid for any essential parameter nodes in PG. The direction of other, un-oriented edges, depends on a choice of a particular essential node in PG(0). Since the bistabilty between (000) and (311) is assured for any such choice resulting in lack of fragility of the bistablty and hysteresis.

https://doi.org/10.1371/journal.pcbi.1009189.g007

We begin by focusing on identifying bistability. High values of variables x₁ and x₂ are represented by the four cubes labeled (0, 1, 1), (1, 1, 1), (2, 1, 1), and (3, 1, 1) where x₁ > θ_0,1 and x₂ > θ_0,2. Observe that for these cubes the variable x₀ will increase (arrows pointing to the right). Similarly, low values of x₁ and x₂ are represented by (0, 0, 0), (1, 0, 0), (2, 0, 0), and (3, 0, 0) and there the variable x₀ will decrease (arrows pointing to the left). Let us now restrict our attention to essential nodes in PG(0). We leave it to the reader to check that for any essential node in PG(0) the direction of arrows on the left and right squares are as depicted in Fig 7. The directions of the other, unoriented edges, are dependent upon the specific essential node. However, observe that the bistabilty between (000) and (311) is assured for any such choice. As a consequence of this multitude of means of maintaining bistability, perturbing away from the essential node of PG_e(1) × PG_e(2) does not necessarily destroy bistability. Finally, consider any full path over the factor graph PG(0). We claim that at the endpoints of this graph the STG gives rise to monostability. However, any such path goes through an essential node of PG(0) and thus experiences bistability. Similarly, there are full paths over the factor graph PG(0) based at nodes obtained from perturbing away from the essential node of PG(1) × PG(2). Therefore, it is not surprising that this network exhibits robust hysteresis. Again, we emphasize that this is an a posteriori computation; we can explain the results found from the DSGRN computations, but we cannot predict them.

We remark that there are similarities and differences in the concept of robustness used in this paper from those that are explored in the context of switching systems [25, 26] or Boolean models [27]. The overwhelming similarity is that we are concerned with whether the dynamics observed at one parameter value is equivalent to that at nearby parameter values. In our case parameter space is continuous and partitioned into a finite set of regions. Thus, nearby parameters either lie in the same region or a region that differs by a co-dimension one hypersurface. In the case of the Boolean models [27], a nearby parameter value is a Boolean function that differs by a single entry. A subtle difference is that our primary focus is not on matching the existence of individual trajectories arising as solutions to a differential equation that to the temporal sequence of the Boolean updates, but rather with the existence of a global dynamical structures, e.g. monostability and bistability. A more important difference is that we are interested in tracking and organizing these global structures over large ranges of parameter space, e.g. hysteresis consists of a prescribed combinatorial sequence of monostability, bistability, monostability.

5 Results for descending hysteresis

We consider descending hysteresis in which the switch-like behavior transitions from a high steady state to a low steady state with bistability in between. A schematic for this case is shown in Fig 1c.

We carried out the same analysis as for the ascending hysteresis. The combinatorial definition of descending hysteresis is analogous to that of ascending hysteresis. Specifically, a path exhibits descending hysteresis if at the initial vertex of the path there is a FP(i₀, i₁, j₁), at the final vertex of the path there is a FP(i₀, i₁, j₃) with j₃ < j₁, and at some intermediate vertex of the path there are two stable states FP(i₀, i₁, j₁) and FP(i₀, i₁, j₂) with j₂ < j₁. For the purposes of this paper we set j₃ = 0 and require that j₁ > 0 and j₂ > 0. As in the study of the ascending hysteresis, we only consider paths of length at least three.

The ranking according to the hysteresis score is presented in Fig 8 (left) along with the scores after perturbation for the top 14 networks in Fig 8 (right). These 14 networks are shown in Fig 9. Observe the striking similarity with Fig 3. In both cases there are exactly 4 networks that have a 100% hysteresis score and contain only 3 edges that cyclically connect the nodes. All of these networks are fragile and their corresponding robustness scores are also similar. Comparing Fig 3 with Fig 8 we observe that the distribution of ascending and descending hysteresis scores are very similar and indicate that robust switching is relatively rare in either case. The close similarity between analysis of the entire collection of 3 node networks with respect to ascending and descending hysteresis suggests that there exists some relation, e.g. symmetry, where a network’s ascending hysteresis and robustness scores are comparable to its partner’s descending scores. This relation does not appear to be obvious as it must be compatible with observations about asymmetry made in the main text. For example, even though there is a network consisting of only repressors that exhibits ascending hysteresis ranked networks, there is no network with only activators that exhibits descending hysteresis. Exploring this relationship and its implications is the subject of current research and remains an open problem.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8.

Left: 14,098 three node networks ranked by (descending) hysteresis score analyzed at the essential parameters. Right: The top scoring networks are further discriminated by scoring hysteresis in a neighborhood of the essential parameters. The scores after perturbation are shown in (red) and the scores at essential parameters in (blue).

https://doi.org/10.1371/journal.pcbi.1009189.g008

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. The top fourteen regulatory networks by descending hysteresis score.

The six regulatory networks in dotted boxes are fragile, while the eight outlined via the dashed lines are robust. Observe that none of the networks are consistent. Networks 13 and 14 are analogous to networks 13 and 14 of the main paper. The fact that they are fragile recovers observation from [7] that 2-node toggle switch is fragile (see comment in the main text).

https://doi.org/10.1371/journal.pcbi.1009189.g009

6 Hill model continuation computations

The dynamics of a regulatory network as modelled by DSGRN is obtained by assuming that the rate of change of x can be approximated by (5) where x = (x₀, …, x_N−1) represent the state variables of the N nodes of the network, Γ is a diagonal matrix where γ_i > 0 is the decay rate of x_i, and Λ(x) = (Λ₀(x), …, Λ_N−1(x)) takes the form described below. Let and

Assume that node i has k in-edges from the nodes j₁, …, j_k in the regulatory network. Furthermore, assume that of these edges are activating and are repressing. For the computations of this paper we set (6) As an example, for network 12 in Fig 3, Eq (5) takes the form since all the in-edges are activating.

For the numerical computations to obtain the results in Tables 1 and 2 we consider Hill function models where H_i is obtained from the regulatory network by replacing the step functions σ⁻ and σ⁺ in Λ_i by the decreasing and increasing Hill functions and respectively. We represent the input signal to node 0 by the additive parameter s in the first equation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. (Hysteresis and perturbed hysteresis scores) Results for comparisons with four networks.

Numbers for Regulatory network come from ranking in Fig 8. Bottom rows are the full and partial path hysteresis scores obtained from DSGRN for Regulatory networks 12, 33, 108, and 4346. The first column indicates the exponent of the Hill function used in the ODE model for the regulatory network. The two columns under each regulatory network number indicate the percentage of continuation computations that result in a hysteresis curve. The first column assumes the parameter value is in a region defined by and the second column assume the parameter value is in region defined by a one edge neighborhood. 1000 curves were computed for each entry.

https://doi.org/10.1371/journal.pcbi.1009189.t002

Returning to network 12 in Fig 3, the Hill model is given by Notice that in (1) we combined the ℓ parameters as L₀ = ℓ_0,0 + ℓ_0,1 + ℓ_0,2, L₁ = ℓ_1,0 + ℓ_1,2, and L₂ = ℓ_2,0.

Using the Hill models we compute curves of equilibria using a pseudo arclength continuation method [29] to detect fold bifurcation points. A sample continuation curve for (1) is presented in Fig 4b, where the following values of parameters were used:: γ₀ = 1, γ₁ = 1, γ₂ = 1, L₀ = 0.508736659464953, L₁ = 0.823149364604282, L₂ = 0.129562882298977, θ_0,0 = 2.742699202456864, θ_1,0 = 3.176067131107269, θ_2,0 = 3.406985767928092, θ_0,1 = 1.753260803421655, θ_0,2 = 0.724695975751957, θ_1,2 = 1.566020932246446, δ_0,0 = 1.172412555847297, δ_1,0 = 2.862607698545040, δ_2,0 = 4.947150771599252, δ_0,1 = 0.946904335902318, δ_0,2 = 0.077108238106769, and δ_1,2 = 1.624416688203425.

For the numerical continuation computations DSGRN provides sample parameter values from parameter regions and for each sampled parameter point we search for hysteresis using the following procedure.

For ascending (descending) hysteresis we randomly sample 10 initial guesses satisfying the conditions , , and ( for descending hysteresis). For each initial guess we perform the following computations (where the successive steps are dependent on the successful completion of the previous ones):

1. Run Newton’s method with as initial guess to find an equilibrium solutions to the Hill model with s = 0.
2. If Newton’s methods converges to an equilibrium solution (x₀, x₁, x₂) check if it satisfies the condition 0 < x₂ ≤ min θ_*,2 (max θ_*,2 < x₂ for descending hysteresis).
3. If the above condition is satisfied we use the solution (x₀, x₁, x₂) and s = 0 as the initial point for a pseudo arclength continuation method to compute a curve of equilibria from s = 0 up to s = 4.
4. During the continuation of the equilibria we identify saddle-node bifurcations by monitoring the sign of the determinant.
5. If during the continuation of the equilibria we get an even number of saddle-node bifurcations and for each bifurcation the value of x₂ just before the bifurcation point is smaller (larger for descending hysteresis) than the value of x₂ just after the bifurcation point, then we declare this a hysteretic curve.

If we get a hysteretic curve for at least one of the random initial guesses we declare the sampled parameter point used for the computations as a hysteretic parameter point.

The hysteresis score of a set of sampled parameter points is the percentage of hysteretic parameter points in the given set of parameter points. For the computations in Tables 1 and 2 we used 1, 000 sampled parameter points (1, 000 curves) for each of the scores. Networks 33, 108, and 4346 used in Table 2 are shown in Fig 10.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 10.

(a) Regulatory network 33. (b) Regulatory network 108. (d) Regulatory network 4346.

https://doi.org/10.1371/journal.pcbi.1009189.g010

7 Extending DSGRN capabilities

To be processed by the original DSGRN software [15] a regulatory network was required to satisfy the following conditions:

1. Every node must have an in edge.
2. No repressing self-edges.
3. Every node must have an out edge.

These assumptions are too restrictive as they remove a tremendous number of potentially interesting regulatory networks. The current version of DSGRN [6] overcomes these constraints as indicated below.

7.2 Node with a self repressing edge

We begin by quickly surveying how DSGRN produces a state transition graph (STG) that is a representative for the dynamics. For more details see [15]. Consider a regulatory network with N vertices. Let E(n) denote the number of out edges from node n. Thus, the set of threshold values associated with the n-th node is where we assume that . The complement of the set of hyperplanes , n = 0, …, N − 1 defines a collection of open cubical subsets of (0, ∞)^N. We refer to these sets as top cells. We index these cells by where (j₀, …, j_N−1) is the cell containing points x satisfying with the convention that and . The boundaries of the top cells are called walls and each interior wall, i.e. a wall contained in (0, ∞)^N, is a subset of a hyperplane . Furthermore, each such wall is the boundary element of exactly two top cells and we use this fact to index the walls by where the pair (κ₀, κ₁) indicates the wall whose two top cells are indexed by κ₀ = (i₀, …, i_N−1) and κ₁ = (j₀, …, j_N−1). We refer to i₀, …, i_N−1 as the coordinates of κ₀. Observe that this allows us to adopt the following convention: the wall indexed by (κ₀, κ₁) is a subset of the hyperplane if and only if i_n = k − 1, j_n = k, and i_ℓ = j_ℓ for ℓ ≠ n. If we wish to emphasize this information we write (κ₀, κ₁)_k,n Consider κ₀ and (κ₀, κ₁)_k,n. Referring to (5) we define (κ₀, κ₁) to be repelling or absorbing (in [15] they are referred to as incoming and outgoing, respectively) with respect to κ₀ if (7) respectively, for x in the top cell indexed by κ₀. The opposite set of inequalities are used to define repelling and absorbing with respect to κ₁.

Observe that the indexing of top cells only depends on the ordering of the thresholds, not their numerical values. Thus, is a purely combinatorial object. Nevertheless, based on the motivating geometry we say that κ and κ′ are adjacent if all their coordinate values are the same except for one coordinate and in that coordinate they differ by exactly one. It is only in (7) that the value of the thresholds plays a role. The decomposition of parameter space is chosen such that for each region of the decomposition the inequalities of (7) are preserved.

As is described in [15] if there are no repressive self-edges in the regulatory network, then given (κ₀, κ₁) the options are:

1. A1. (κ₀, κ₁) is absorbing with respect to κ₀ and repelling with respect to κ₁;
2. A2. (κ₀, κ₁) is repelling with respect to κ₀ and absorbing with respect to κ₁;
3. A3. (κ₀, κ₁) is repelling with respect to both κ₀ and κ₁.

According to these three options the classical DSGRN defines an edge κ₀ → κ₁ in case A1, an edge κ₁ → κ₀ in case A2, and no edge between κ₀ and κ₁ in case A3 (see Fig 11). We can view this as suggesting that the absorbing direction dictates how one top cell is mapped to a neighboring top cell. Performing this computation over all of produces the STG. Observe that is the set of vertices of the STG and that edges only exist between adjacent elements of .

Download:

• PNG
  larger image
• TIFF
  original image

Fig 11.

(a) A potential original DSGRN complex where the vertical threshhold is associated with a self-repressing edge. An arrow going from a top cell κ_i to a wall indicates an absorbing wall and an arrow from a wall to a top cell indicate a repelling wall. (b) Portion of the refined DSGRN complex . The cells κ₀ and κ₁ are the additional cells in the refined complex. The wall labelling in (b) induces the edges , , , κ₀ → κ₁, , and in the STG on .

https://doi.org/10.1371/journal.pcbi.1009189.g011

If there is a repressive self-edge in the regulatory network, then it is possible that (κ₀, κ₁) is absorbing with respect to both κ₀ and κ₁. The naive response is to introduce edges κ₀ → κ₁ and κ₁ → κ₀, but this suggests recurrent dynamics where it may not exist. Thus, this case was not considered in the classical DSGRN.

However, based on [15] (see in particular Section 4.2) we claim the results:

1. R1. Consider (κ₀, κ₁) indexing a wall contained in x_n = θ_n,n. Then, for all i ≠ n and for any x ∈ κ₀ and .
2. R2. In addition, consider indexing a wall contained in x_n = θ_n,n, such that and are indices for walls, i.e. κ₀ and are adjacent as are κ₁ and . If is repelling (absorbing) with respect to κ₀ or , then is repelling (absorbing) with respect to κ₁ or .

We resolve the issue of a self-edge by expanding . Fix a region of parameter space. This implies that the inequalities (7) are fixed. Define where E*(n) = E(n) + 1 if there is a self-repressing edge to node n and E*(n) = E(n) otherwise. If there is a self-repressing to node n then the threshold corresponding to this edge is and we indicate this by denoting and . In this case it is possible to have a wall (κ₀, κ₁)_k,n such that (κ₀, κ₁) is absorbing with respect to both κ₀ and κ₁. Again based on [15] this is only possible for the k corresponding to the above. To define the STG we need to consider adjacent cells in , i.e., the set where again it is assumed that a single coordinate of κ₁ is larger than the coordinate in κ₀. Let . If neither κ₀ nor κ₁ contain a as a coordinate, then we use the classical DSGRN rules based on A1–A3. Thus, we only need to consider where either κ₀ or κ₁ contains as a coordinate. Consider . Then the classical DSGRN rules apply to determine whether (κ₀, κ₁) is absorbing or repelling with respect to κ₀. If (κ₀, κ₁) is absorbing (repelling) with respect to κ₀, define (κ₀, κ₁) is repelling (absorbing) with respect to κ₁. Consider . Then the classical DSGRN rules apply to determine whether (κ₀, κ₁) is absorbing or repelling with respect to κ₁. If (κ₀, κ₁) is absorbing (repelling) with respect to κ₁, define (κ₀, κ₁) is repelling (absorbing) with respect to κ₀. Now assume that a is a coordinate of both κ₀ and κ₁ and hence we need to consider (κ₀, κ₁)_ℓ,n′. Once again there are three cases to consider , , or the ℓ-th threshold is not associated with a repressive self-edge. Classical DSGRN does not apply for determing absorbing and repelling in any of these cases. To determine this consider such that (see Fig 11)

Note that while or must exist, it is possible that only one pair exists. Also observe that classical DSGRN applies to the pairs and and thus absorbing and repelling of and is determined. If both and exist, then by R2 is absorbing/repelling with respect to () if and only if is absorbing/repelling with respect to (). We define (κ₀, κ₁) to be absorbing/repelling with respect to κ₀ (κ₁) in accordance with or .

Acknowledgments

The authors thank Bree Cummins for helpful discussions.