Framework for deriving microscopic resolution networks from experimentally inferred Boolean network topologies

In this Section we outline the steps for constructing higher-resolution regulatory networks with explicit promoter architecture, gene states, and transcription factor copy numbers, starting from experimentally-inferred Boolean network topologies. To illustrate the potential of our modeling framework we have chosen the most comprehensive Boolean network to date which describes the regulation of pluripotency factors of mouse embryonic stem cells [16]. In the next subsection we describe how Boolean logic is converted into the molecular logic of promoter states via a set of simplifying assumptions. Once promoter logic is defined, in the second subsections we go on to define the resolution of protein to promoter interactions.

These assumptions lead to a set of chemical reactions describing transitions between gene states. In the third subsection we introduce the rules defining the production and degradation of transcription factors. In the fourth subsection we give an overview of the multi-scale simulation techniques appropriate for the simulation of stochastic switching of genetic states with a single-molecule-level resolution. In particular we contrast the rigorous yet computationally inefficient individual-based model (IB) with a more efficient piecewise-deterministic Markov Process (PDMP) which we adopt and extensively validate in the present work.

Extending the Boolean logic to molecular logic

The Boolean model of Dunn et al. specifies twelve genes in the network. We use G_i to denote a gene and P_i to denote the corresponding functional TFs (i ∈ {1, 2, …, 12}) in the following framework. The network topology (Fig 1) contains static information about the types of interactions between pairs of genes. The interactions are classified as being either repressing or activating. To study the dynamics of complex genetic networks, one has to extend the Boolean-level description to account for the molecular logic of gene regulation. This molecular logic specifies the precise relation between the binding of transcription factors to the promoters and the regulatory outcome in terms of gene activation or repression. In the case where the same sites can be bound to different transcription factors, the combinatorial nature of regulation can give rise to ambiguity in molecular logic. Indeed, even on the level of Boolean networks, the logic is ambiguous and many possible truth tables have to be enumerated in order to select an appropriate picture for the dependence of a gene on its regulating signals. Ideally, such ambiguities should be resolved by directly inferring regulatory logic. Alternatively, one could simulate different combinatorial possibilities until sufficient agreement with experiments is reached.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Network topology and molecular logic.

The left panel shows a schematic diagram of the network topology, reproduced from Dunn et al. [16]. Each node corresponds to a given gene and their placement from left to right is chosen to indicate a trend of downstreamness from three external inputs. In our mechanistic model, each gene produces a unique transcription factor at a rate which depending on the binding state of its promoter site. These transcription factors then go on to bind and activate (black arrow) or repress (red bar) other genes. The three nodes on left correspond to extra-cellular signals, which are either absent or present. The right panel shows our assumed molecular logic of transcriptional regulation when there are N = 2 promoter sites per gene. Each circle is a binding site: it can be either empty (white), bound by an activator (green), or bound by a repressor (red). The right panel lists possible combinations of the promoter sites. Depending on the configuration of the promoter site, transcription factors are produced with rates 0, Ωα_m, and Ωα_max, modeling the effects of cooperative binding.

https://doi.org/10.1371/journal.pcbi.1006000.g001

In our case, we assume a set of rules which applies to every gene in the network. This is a simplifying assumption, the justification for which is obtained a posteriori; networks with optimized parameters yield gene expression patterns consistent with the experiments. The fact that these simplifying assumptions work surprisingly well implies a dominance of the global network topology over the local details of molecular regulatory logic of gene promoters.

For each gene, we assume a fixed number N of promoter sites. The genes which are only regulated by activators are always “OFF” unless N promoter sites are bound to activators. Similarly, the genes which are only regulated by repressors are always “ON” unless N promoter sites are bound to repressors. For those genes which are regulated by both activators and repressors, their activity is up-regulated to “ON” (resp. down-regulated to “OFF”) only when N promoter sites are bound to activators (resp. repressors), otherwise they have a “MEDIUM” activity. These rules are illustrated quantitatively in the right panel of Fig 1.

Binding and unbinding of transcription factors to promoter sites

From the topology, we know that each gene is regulated (activated or repressed) by a subset of TFs; we denote the subset of the TFs which regulates gene i by S_i ∈ {P₁, P₂, …, P₁₂}. For example, the subset of the TFs which regulates the core gene Nanog is the set of transcription factors Klf2,Sox2, MEKERK (see Fig 1). The binding and unbinding of the transcription factors to the promoter sites of gene i can be summarized by the following binding and unbinding reactions: (1) where the rates are written per the reactants, i.e., a given P_j binds to a given unbound promoter G_i with rate Nk_onΩ⁻¹ etc. The “system size” parameter Ω has been introduced which sets the scale for a typical number of TFs in the system [4, 17] without loss of generality. The scaling of these switching reactions is chosen such that the time scale of gene switching is independent of Ω [18]. Notice that we have adopted the parallel binding mechanism [19] which assumes the transcription factors bind to any of the unbound promoter sites independently with a rate constant k_on.

Production and degradation of transcription factors

Each gene G_i “produces” its own transcription factor P_i with a rate which depends on the promoter state of G_i. We use a one-step model of transcription [20] whereby transcription factors are produced via a unimolecular step. This approximation coarse grains several sequential intermediate reactions (polymerase recruitment, mRNA production, RNA splicing, etc.) into one effective step [21, 22]. This approximation has become popular in models of protein production [18, 23, 24]. All of the transcription factors are assumed to have finite lifetimes set by the rate of degradation, thus ensuring the existence of stable steady states with a finite number of molecules. The absolute time is set to be ∼ 1 hr which is consistent with experimental measurements reported on the lifetimes of pluripotency transcription factors [25–28].

The reactions are (2) Here, the rate constant of the production of TFs (Ωα) depends on the promoter’s configuration and is determined by the molecular logic defined previously. We further assume the production rate when the gene is “ON”, “MEDIUM”, and “OFF” to be Ωα_max, Ωα_m, and 0 (see Fig 1). These rates are taken to be identical among all the genes.

We note that one can adopt a view with a higher resolution of the network depending on the available experimental data and the nature of the question posed. The computational framework can readily incorporate in an explicit manner more steps, for instance to model the effects of cell cycle regulation, different epigenetic states, and binding of non-coding RNAs. Herein we consider the most simplified dynamical model which describes only the promoter configurations and the population dynamics of the transcription factors. This model aims to use the optimal resolution for capturing trends in gene expression while remaining feasible for efficient stochastic simulations. After converting the Boolean topology into a higher resolution network of biochemical reactions, our next goal is to exhaustively sample a vast space of parameter space in order to identify the optimal parameter regimes with which the model best reproduces the experimental results.

Multi-scale simulation of complex genetic networks

Biochemical reactions in gene networks are of a fundamentally probabilistic nature; this demands a stochastic description of the kinetics. The conventional mean field mass-action-based kinetics only describe the dynamical behavior in the thermodynamic limit, and ignore the stochastic effects which arise from the discreteness of the system. In our model, the system is highly discrete, as there is only a single copy of each gene, and their discrete promoter states dictate the dynamics of the transcription factors. The most rigorous way to simulate such a reaction system is by numerically solving the chemical master equation which accounts for all possible states of the network down to the level of single molecules [4, 20]. In high dimensions, i.e., when the number of species is large, this approach is not computationally efficient. Instead, kinetic Monte Carlo algorithms are the most straightforward way to generate sample paths of the stochastic process. We refer to the stochastic process modeling the reactions down to the single-molecule level with the well-mixed assumption as the individual-based model (IB).

The state of our individual-based model is characterized by the state of each gene’s promoter sites—how many sites are bound to specific kinds of transcription factors—and the integer populations of the transcription factors. The rates at which the process stochastically transitions from one state to another are specified by the reactions Eqs (1) and (2). Fully individual-based models, however, still suffer from a steep scaling of computational time with the number of discrete system states. This fact renders them inefficient for simulating large gene networks, especially when it comes to scanning or exploring the parameter space.

A wide range of approximate schemes have been employed to simulate complex gene regulatory networks [29–31]. Most conventional approximations so far have been the size-expansion methods [32, 33] which are known for being problematic when the molecular noise induced by discrete genetic switching becomes non-negligible [18, 34, 35]. On the other hand, for embryonic stem cell networks it is essential to account for the stochastic nature of genetic switching events which give rise to multimodality in the probability distributions of transcription factors populations. These local maxima in the multidimensional probability distributions of a stochastic genetic network correspond to metastable states and are also known as local attractors which corresponding to distinct promoter configurations and hence likely also to distinct phenotypic states [36–38]. Theoretically a network with M independently functioning genes can generate up to ∼ 2^M distinct phenotypic states [39].

Thus, even if populations of all the other species are present in large quantities, the stochastic fluctuations caused by the genetic switches (due to stochastic binding-unbinding events of the transcriptional factors to a discrete number of promoter sites) between ON, MEDIUM, and OFF states cannot be ignored, unless the switching is operating in the extremely fast limit compared to any other reactions, known as the “adiabatic regime” [35, 39, 40]. In the other cases—the non-adiabatic regime—gene switching can completely dominate the dynamics in the network [35, 41]. Eukaryotic gene regulatory networks are often found in the intermediate regime where gene-switching events are dynamically interwoven with the rest of the reactions in the network and cannot be ignored [5, 42, 43]. Single-cell studies of mESC in particular have shown bursty behavior in gene expression with sudden jumps in the levels of proteins resulting in multi-modal distributions of core transcription factors [7].

The sheer volume and complexity of information that is emerging from experiments on embryonic stem cells have motivated the application of a wide variety of modeling strategies for confronting the observed patterns of gene expression in ESCs [44, 45]. Some of the computational techniques used so far include Boolean networks [16, 46–48], Hopfield neural networks [49–51], systems of coupled ordinary [52, 53] and stochastic differential equations [10, 54, 55] with Hill coefficients, agent-based models [56], individual-based models [43, 57, 58] and small noise approximations to individual-based models [59–61].

Many of the early computational models of embryonic stem cells focused on small fragments of pluripotency networks, typically involving bistable switches [10, 52]. These early pioneering studies yielded many insights on stochastic decision making in regards to fate determination and self-renewal [44]. Relatively few studies have also looked at larger portions of regulatory networks while including a stochastic treatment of genetic switching dynamics by either carrying out individual-based simulations [43, 57] or small noise approximations [59–62].

Because of the computationally demanding nature of individual-based models and the restricted validity of small noise approximations to the near-adiabatic gene switching, the full range of stochastic and dynamical regimes displayed by different gene switching time-scales has remained unexplored. Additionally with very few exceptions [43], the kinetic parameters in many of the previous models have been not thoroughly explored or informed by data and have had to be selected from physical intuition alone. These limitations have now been overcome in the present work. Thanks to a series of recent developments in modeling of gene expression dynamics [18, 23, 24, 35, 63–65], a novel computational framework utilizing piecewise-deterministic Markov processes (PDMPs) [66] has emerged as a rigorous approximation to the fully individual-based model.

The idea behind PDMP is simple: reactions with large number of molecules are evolved deterministically, while reactions with smaller numbers of molecules are propagated as discrete, random switching events. This approach treats discrete genetic switching events exactly, while assuming noise due to the finite nature of populations of transcription factors to be small in comparison. As shown in the subsequent sections, the assumptions underlying the PDMP approach turn out to be sound as we go on to obtain a nearly perfect quantitative agreement with full blown kinetic Monte Carlo schemes even for the case of the complex networks of mESC operating in the intermediate gene-switching regime. The mathematical details of the PDMP are given in the next subsection.

The PDMP simulations carried out in the present study show nearly fold faster generating stochastic trajectories compared to conventional individual-based kinetic Monte Carlo techniques (the algorithm for generating PDMP sample paths is provided in the Method section). This rigorous and rapid sampling of gene-switching events has not only allowed us to investigate the stochastic dynamics of the regulatory network at a longer time scale compared to conventional kinetic Monte Carlo methods, but also enabled us to explore a vast parameter space efficiently. We have used the obtained information to derive microscopic resolution models of ESCs. The mathematical details of the PDMP described in greater detail can be found in the next few subsections.

In the pluripotent state the mean levels of gene expression are robust with respect to changes of gene state switching rates

After converting the Boolean topology to a higher resolution genetic network we use a computationally efficient stochastic dynamics method (PDMP in the present case) to explore the kinetic parameter regimes. This is done in order to: i) Identify all the unique steady states ii) Obtain a set of rate coefficients that best reflects the experimental constrains iii) Study the response of the network to gene switching dynamics the rate of which is taken as a proxy for epigenetic states. The experimental data used for constraining rate coefficients are the binarized gene expression levels of pluripotency transcription factors under well-defined culture conditions consisting of different combinations of leukemia inhibitory factor (LIF), glycogen synthase kinase 3 (CH), and mitogen-activated protein kinase (PD). Identical culture conditions were used by Dunn et al. [16] to infer the original Boolean topology.

The optimization of rate coefficients is done by defining a uniform threshold η among all the TFs and with different external conditions; TFs whose population densities are higher than η are classified as expressed, while those below the threshold are not expressed. Then, we employ the Hamming distance (the number of discrepancies between the simulated and experimental profiles) as a cost-function for the optimization. The Hamming distance is minimized through multiple rounds of simulations where we vary the uniform threshold η and four free and non-dimensional model parameters (see Method section): the number of promoter sites per gene N, the rate constant of TFs binding to the promoter k_on, rate constant for a bound TF to dissociate from the promoter k_off, and the production rate α_m when the gene is in the “MEDIUM” state.

Through this procedure, we find that the binary expression patterns can be closely reproduced (minimal Hamming distance = 3) by the model when the number of the promoter sties N ≥ 2. We remark that, given the small number of free parameters, this is a very small amount of deviation from the experimental results; the remaining deviations are likely to have been caused by the simplifications to the logic and dynamics in the construction of our individual-based model. The qualitative features for N = 2, 3, … 5 cases are similar (see S2–S7 Figs in the Supporting Information). We chose to present the data for the simplest case N = 2 in the manuscript. We note that various other forms of cooperativity, including cases when bound TF at the promoter recruits other TFs (either of its own kind or other type), can be readily incorporated into the current computational framework by changing the association rates k_on as promoter-state dependent. For simplicity, we only illustrate the most basic form of cooperativity in this pilot paper. We find for any given N there exists a “valley” in the remaining model parameter space k_on, k_off, α_m, as illustrated in Fig 2(A). The inferred parameter α_m is consistently low (≲ 0.02), meaning that the “MEDIUM” production rate is almost zero. This implies that the negative regulation of those genes which are regulated both by activators and repressors (Tfcp2l1, Esrrb, Nanog, and Oct4) in the model are technically fulfilled by inhibition (i.e., regulating TFs by blocking the promoter sites), instead of actually down-regulating the production activity. In addition, the valley suggests a relationship between k_on and k_off, where k_on ≈ 10k_off. This implies a certain asymmetric time scale between the binding and unbinding processes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. PDMP stochastic simulations identify three genetic switching regimes that are consistent with experimental data.

When switching is slow, intermediate, or fast we find certain parameters which closely match the experimental results obtained by Dunn et al. [16]. The consistency of our model with experimental results is measured using a Hamming distance—a measure where one counts the number of discrepancies between the binary expression of each TF for both cases. (A) Shown are the identified regions in parameter regimes that minimize Hamming distance. There are three free parameters: the binding rate k_on, unbinding rate k_off, and basal transcription rate α_m. For slow switching, the parameters are k_on = 3.2, k_off = 0.2, α_m = 0.02; for intermediate switching, k_on = 16, k_off = 1.5, α_m = 0.01; for fast switching, k_on = 102, k_off = 10, α_m = 0.005. The selected parameter sets are presented as red dots in the landscapes in the upper panel, hereby referred to as the slow, fast and intermediate parameter regimes. (B) Comparison of computed and discretized gene expression profiles with those of the experiments (Benchmark panel) [16].

https://doi.org/10.1371/journal.pcbi.1006000.g002

We chose three parameter sets in this valley of cost function, corresponding to three distinct dynamical regimes of binding and unbinding reactions between the TFs and the promoter sites: slow (k_on = 3.2, k_off = 0.2, α_m = 0.02), intermediate (k_on = 16, k_off = 1.5, α_m = 0.01), and fast (k_on = 102, k_off = 10, α_m = 0.005) compared to the time scale of the TF dynamics. We remark that as we non-dimensionalize the model using the protein degradation rate γ (see Method section), the time scale of the binding and unbinding are fast, intermediate, or slow relative relative to the time scale of the protein dynamics, also set by γ. See Eq 3. All of the parameter sets successfully reproduce the experimental gene expression patterns (Fig 2) corresponding to pluripotent and lineage committed cells.

The fact that the rate parameters of the network occupy finite regions and cover different regimes suggests that distinct gene expression profiles can tolerate fluctuations in reaction rates. Such rate fluctuations, reflecting the effect of extrinsic noise, are inevitable in dynamic cellular environments of embryonic cells which experience frequent epigenetic and extracellular perturbations [15, 26, 67]. In a way, changes in gene-switching rates can be seen as a proxy for how global epigenetic changes govern the rates of transcription factor binding to target genomic regions.

From the methodology point of view, the absence of a unique regime of rates implies the following: inferring networks using only mean levels of gene expression (as is done for Boolean networks) may lead to the loss of valuable information contained in higher moments of distribution. Thus new approaches of inference need to be developed in order to account for broad distributions of transcription factors. For this reason, we look beyond the comparisons of mean expression levels and turn to comparing stationary distributions of transcription factors observed in experiments with the computationally generated distributions in three chosen parameter regimes identified in Fig 2. We generate the stationary distributions of the TF concentrations in the model, with eight different environmental conditions, in the specified three parameter sets in Fig 3. We observe qualitatively that in the “fast” parameter set the distribution of each TF is unimodal, and in the “slow” parameter set, the distribution of each TF is peaked near “0” and “1” in non-dimensional units. In the intermediate regime, on the other hand, with some environmental conditions, broader distributions of the gene expression are observed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Gene expression profiles of pluripotency factors predicted by PDMP simulations.

Each column corresponds to different external inputs, and each row corresponds to regimes of slow, intermediate and fast gene switching. More than 10⁵ sample paths were used for generating each condition.

https://doi.org/10.1371/journal.pcbi.1006000.g003

The pluripotent state of mESCs has been the subject of intense investigations by nucleic acid-based single-cell techniques such as RNA-seq, sm-FISH, qPCR and there is now extensive data on the steady-state distributions of RNA and transcription factors maintained under pluripotency-favoring culture conditions [7, 8, 68]. These experiments have revealed a heterogeneous nature of gene expression with many core pluripotency factors, such as Nanog and Esrrb, having long-tailed or bimodal distributions (see for example Fig. 5a in Ref. [7]). Additionally, the single-cell stochastic trajectories of TFs have shown sharp, bursty transitions implying infrequent genetic switching events [7]. Qualitatively, these experimental observations are more consistent with the intermediate regime of genetic switching as seen from Fig 3, as in the case of slow switching nearly all of the transcription factors express bimodality and in the case of fast switching the expressions of all the factors are narrow and unimodal.

In Fig 3, consistent with experiments, we also find that under LIF gene expression is more heterogeneous than under 2i, but also that the overall levels of expression are higher [7, 8]. In all three regimes of genetic switching supporting pluripotency (LIF+2i,LIF+PD, LIF+CH, LIF and 2i), core transcriptions factors such as Nanog, Oct4, Sox2 are highly expressed. The same factors are also repressed in conditions favoring differentiation (CH, PD and none) irrespective of gene-switching regime. This shows that the pluripotent and differentiated states, as determined by the pattern of gene expression, are hardwired in the architecture of the genetic network independent of genetic switching rates. Nevertheless, as we will show in the next section, the routes and dynamics of lineage commitment from pluripotent states strongly depend on the level of molecular noise generated in different gene-switching regimes.

Although the PDMP approach accurately captures the effects of genetic switching, it assumes demographic noise arising from finite populations to be negligible. To test the validity of this assumption and assess the contribution of different sources of noise in establishing the steady-state distribution of the pluripotent state, we carry out individual-based simulations for the intermediate switching regime of the network, Fig 4. In the individual-based model all reactions are treated stochastically thereby accounting for all of the sources of noise in the system. The resulting gene expression profiles follow closely those obtained by PDMP simulations, showing that the noise arising from stochastic switching events of promoter configuration accounts for the significant part of overall variability in the network. Trajectories of individual transcription factors show that indeed most of the variance in the molecular distributions are generated by genetic switching events which appear as abrupt stochastic jumps.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Gene expression profiles of pluripotency factors predicted by individual-based simulations.

The intermediate switching regime is chosen to be presented as it is the regime which best captures the experimentally measured distributions [7, 8]. (A) Shown are two representative trajectories and full distributions of select few transcription factors under three different conditions generated by individual-based simulations. Typical lifetime for pluripotency transcription factors [25–28] (∼ 1 hr) is used for setting the absolute time-scale of simulations (B) Gene expression profile showing the near quantitative agreement with results of PDMP simulations shown in Fig 3. (C) Gene expression profile of individual-based model, for comparison with the PDMP and experimental benchmark shown in Fig 2.

https://doi.org/10.1371/journal.pcbi.1006000.g004

Dynamics of lineage commitment is significantly affected by the dynamical changes of individual gene-switching rates

We next ask how the steady-state gene expression patterns displayed by the gene network respond to extracellular perturbations in the form of the initiation or termination of pluripotency signals. Both dual inhibitor 2i (PD+CH) and Leukemia factor LIF based signaling have been shown to provide a stable environment for maintaining pluripotency of stem cells in vitro [3, 16]. Conversely, withdrawal of either LIF or 2i leads to irreversible lineage commitment after a 24-hour period. Despite a similar ability to guard pluripotent cells against lineage commitment, these factors deploy different regulatory mechanisms reflected in distinct distributions of pluripotency factors. As a result, stem cell differentiation by withdrawal of different signals proceeds via different routes. To gain a mechanistic understanding of how the interplay of signaling, molecular noise and network architecture gives rise to the steady-state expression profiles, we study the dynamics of transitioning between the pluripotent and lineage committed states induced by rapid initiation and withdrawal of signaling conditions (LIF, CH, PD).

The temporal evolution of distributions of the TFs exiting (LIF/2i withdrawal) and entering (LIF/2i immersion) pluripotent states reveals rich, dynamical signatures of these transitions (Fig 5). To reveal the role of stochasticity in these transitions, we compare the intermediate regime—which is dominated by genetic switching—to the fast regime—in which transitions are largely governed by the network topology and the fluctuation of promoter configuration is almost-completely ignored. The irreversible nature of transitions manifests clearly in different routes exiting and entering the pluripotent state (transitions to and from the None state in Fig 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The dynamical behavior of the distributions of transcription factor densities for the intermediate and fast switching regimes.

At time t = 0, the external inputs are changed. The plots show the evolution in probability density for a large ensemble (10⁵) of PDMP sample paths. The times and populations are respectively rescaled by 1/γ and α_max/γ (see Method section). We assume the proteins are stable and so the their estimated half-life is of an order 8 hour [7]; in other words γ ∼ 1/8 hr⁻¹, and the entire course of simulation (t ∈ (0, 30)) corresponds to physically 240 hours. A validation of the PDMP prediction by individual-based simulations can be found in S8 Fig in the Supporting Information.

https://doi.org/10.1371/journal.pcbi.1006000.g005

In the intermediate regime of genetic switching rates, expression noise greatly facilitates transitions out of a pluripotent state by making the network more responsive to changes in environmental signaling. In contrast, in the fast-switching regime, upon withdrawal of pluripotency signals the downstream regulation happens on a much slower time scale with some factors remaining virtually unresponsive to changes of signaling. This signaling enhancement in the intermediate regime reveals the importance of molecular noise in making pluripotent states more sensitive to environmental conditions. There are qualitatively different patterns of re-entrance into pluripotent states upon LIF vs 2i addition, with LIF being much more efficient at reversing pluripotency compared to the 2i. The different potentials of signaling cultures for pluripotency reversal has been established in experiments [3] which have shown that in the later stages of commitment only the LIF is able to reverse lineage-primed cells back to their naive pluripotent states. The exit and re-entrance from pluripotency upon withdrawal and addition of LIF shows complex signatures of hysteresis and bifurcations. This suggests that there can be multiple pathways of entering or exiting pluripotency. The transition times for all signaling-induced changes of the steady states of the network are visualized on a kinetic diagram, Fig 6. This figure shows an underlying structure to these transitions where conversion among pluripotent states takes place with lower “activation barriers” compared to transitions accompanying loss of pluripotency. In the intermediate switching regime there is a clear time scale separation between transitions which keep cells in pluripotent states and transitions out of pluripotent states. One may argue that such a time scale separation between signals inducing differentiation and pluripotency allows embryonic cells to execute developmental decisions more faithfully. In the fast-switching regime this clear time scale separation is partially lost where only the loss of all three signals is separated from the rest of the transitions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Transition times between the stationary distributions of different external conditions.

A larger, darker arrow indicates that a given transition takes a longer time to converge to its stationary state. This time scale is measured by simulating a large ensemble (10⁵) of PDMP sample paths to provide a simulated probability density and finding the Jensen–Shannon divergence [69, 70] between the instantaneous distribution of each TF and its final stationary distribution. The time for the each divergence to fall below a threshold (≔ 0.3) is recorded, and we choose the largest of these as a quantification of the transition time. The numerical values can be found in S1 and S2 Tables in the Supporting Information.

https://doi.org/10.1371/journal.pcbi.1006000.g006

Detailed analysis of individual distributions and trajectories of transcription factors can be very informative due to their high information content. It is, however, not immediately clear how changes in the expression of individual genes contribute to global changes corresponding to different phenotypic transitions. To reveal such global changes, we project stochastic trajectories of all transcription factors onto the first two eigenvectors obtained by principal component analysis (PCA) of the reference pluripotent steady state (LIF+2i). Most of the variance of transcription factors is well captured by the few principal components. The high-dimensional steady state of the cellular network can thus be conveniently projected onto a 2-dimensional subspace, allowing us to visualize the attractor states of the network as probability landscapes π(PC₁, PC₂) which are often masked by heterogeneous distributions.

Comparing these probability landscapes with different gene-switching regimes reveals the distinct roles played by gene switching-induced molecular noise and the deterministic network topology in guiding the transition out of the pluripotent states (Fig 7). The intermediate gene-switching regime, once again, appears to be the more viable regime underlying pluripotent states since the probability landscape shows up as a broad attractor with interconnected states. Going towards the limit of slow switching results in the fragmentation of the landscape into states separated by high barriers. This gene switching-induced remodeling of attractors shows the potential for regulation via global epigenetic changes which are purported to act via silencing or activating entire sets of genes at once [15]. Thus one may view gene-switching rates as a proxy for genome wide acetylation/methylation patterns which can dramatically alter the access of transcription factors to key target genomic sites. The sequential removal of pluripotency-inducing signals reveals a consistent change in the size of the attractor towards occupying smaller regions on the landscape. This argues for the physical state of the network corresponding to pluripotent states to be the one with maximal variance of regulatory transcription factors where lineage commitment is accompanied by their gradual constraining and repression. A similar idea which views pluripotency as a macrostate emerging from an ensemble of cells which try to maximizes the information entropy with respect to regulatory transcription factors has been postulated before [12, 71]. The analysis of the steady-state stochastic dynamics of the pluripotency network in this work appears in agreement with this view. Furthermore, we are able to suggest a microscopic origin of this entropic paradigm. By analyzing pairwise correlations among different transcription factors we find that signals like LIF/2i create greater independence between the expression of core transcription factors, leading them to explore a larger range of values. Hence, removal of these signals leads to more constrained and interdependent patterns of gene expression for the same transcription factors which greatly diminishes overall variance (see S1 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Mapping the cellular attractors of the genetic network under different switching and signaling conditions.

We project PDMP-simulated gene expression distributions onto the first two principal components. The reference state for principal components was chosen to be the LIF+2i/intermediate switching.

https://doi.org/10.1371/journal.pcbi.1006000.g007

Constructing the piecewise-deterministic Markov process (PDMP)

In the individual-based description of complex genetic networks studied in the present work, one models each individual reactive events as a Markov jump processes. The underlying master equation governing the Markvoian evolution of the entire network is analytically intractable and in general even numerical simulations quickly become computational inefficient once dimensionality of the system becomes too high [77]. Specifically what contributed to this inefficiency is the population scale of transcription factors for which it is common to have values on the order of Ω = 10⁴ as is characteristic for biological cells. Thus the use of standard continuous-time Monte Carlo [17, 78] sampling techniques becomes unfeasible especially if one wants to sample the kinetic parameter regimes for finding optimal set of rate coefficients.

Fortunately the latest efforts of modeling gene expression dynamics [18, 23, 24, 35, 63, 64, 79] have lead to the emergence of a new class of techniques which are broadly based on using a piecewise-deterministic Markov process (PDMP) to approximate the individual-based model with a switching property. In this section, we briefly recapitulate the construction of the PDMP. A more thorough analysis can be found in the literature cited [18, 23, 24, 35, 63, 64, 79].

A PDMP is a process such that, in between discrete random switching events, the evolution of the process is deterministic. To construct the deterministic evolution of the TF populations, starting from the chemical master equations, we performed Kramers–Moyal expansion [77, 80] in the population of TFs while maintaining the discreteness of the genetic state; we keep only the first order of the expansion. The result is a standard Liouville equation governing the deterministic flow of the distribution. The joint probability distribution of our model converges to the deterministic flow in a given genetic state and in the thermodynamic limit Ω → ∞ [80]. With the PDMP approach, the demographic noise originating from random birth-death events are neglected, so that the population density x_i(t) of each TF evolves according to (3) where α_i ∈ {0, α_m, α_max} is the production rate of the ith TF dependent on the ith gene’s configuration of promoter sites. While the evolution of the TF population density is deterministic, the binding and unbinding events of the regulating TFs to their target genes are still stochastic and formulated according to Eq 1 in the main text.

We finally emphasize that the PDMP only retains the contribution of switching noise which arise from the discrete and stochastic binding and unbinding events between the TFs and the promoter sites, and ignores demographic stochasticity from the discrete production and degradation processes of the TFs. The PDMP is the limiting process when the population scale Ω → ∞ [18], and the error bound of the description can be rigorously derived to be [81].

Generating exact sample paths of the PDMP

To simulate the stochastic binding and unbinding statistics of the promoter sites, accurate waiting times must be generated. A waiting time exists for each possible stochastic transition; the smallest of these times tells us how long the system stays in the current configuration of promoter sites, and to which promoter configuration it transitions. In general, waiting times can be generated by mapping a uniform random variable to a random time using the survival function. Since in our case the transition rates are functions of dynamical state variables, this involves the numerical integration of survival functions describing each potential transition [63].

In our case, the simple form of Eq 3 (and thus of the transition rates) allows us to improve upon this by generating waiting times without numerical integration, detailed in the Supplementary Information.

Non-dimensionalization of model parameters

Under the assumptions we proposed, there are initially six free model parameters: Ωα_max and Ωα_m as the production rates when each of the genes has an “ON” or “MEDIUM” activity, γ as the protein degradation rate, N as the number of promoter sites, and lastly k_onΩ⁻¹ and k_off as the binding and unbinding rates between the TFs and the promoter sties. We remark that the population scale Ω is fixed at 10⁴.

Through suitable non-dimensionalization of the physical time and concentrations of the TFs, we reduce the number of parameters. As can be seen from the above formulation (Eq 3), the time scale of the TF dynamics is set by the degradation rate γ. For stable proteins, the time scale of degradation is of the order of the times of the cell cycle. We therefore choose the unit of physical time such that γ is 1. Similarly, the maximum concentration in the TF can achieve in Eq 3 is α_max/γ. We can choose a unit for the concentrations of the chemical species such that α_max = 1, so the concentration of the TFs are always bounded in (0, 1). After non-dimensionalization, the model ends up with four free parameters: α_m ∈ {0, 1} as the intermediate production rate of those genes which are regulated by both activators and repressors, k_on, k_off as the binding and unbinding rate of the TF to the promoter sites, and N as the number of promoter sites per gene. We use these non-dimensionalized parameters to report our results in the manuscript.

Using the checkerboard diagram to infer the parameter regime

To narrow down the parameter regime, we match our model predictions to the experimental findings of Dunn et al. [16] in which the authors measured the TF expression under various combinations of external signals, i.e., LIF, CH, and PD. We aim to match the model prediction to a twelve-by-five “checkerboard diagram” which records the experimentally measured expression pattern presented in Fig 2 in the main text. To achieve this goal, we performed a sweep in a vast parameter space: α_m ∈ [0, 1], k_on, k_off ∈ [0, 110], and N = 1, 2 … 5. For each parameter set, we simulated 10³ PDMP sample paths for a time to sufficiently reflect the stationary state, and the average TF expression levels were recorded. Because of the non-dimensionalization, the expression level (the population density) of each TF is a real number in between 0 and 1. This results in a twelve-by-five real-valued matrix, which is binarized by a threshold. To find the optimal threshold, we use the number of discrepancies between the model prediction and the target matrix—the Hamming distance—as a quantitative measure. For each parameter set, an optimal threshold which minimizes the Hamming distance was then found computationally, and the minimal Hamming distance was recorded and plotted in Fig 2 in the main text as a “landscape” of how good the model captures the experimental results. We found that for N = 1 and N ≥ 2, the global minimal Hamming distance is 5 and 3 respectively. We chose N = 2 to present our follow-up analysis, as it incorporates the capacity of modeling cooperative binding which is often modeled phenomenologically. We find the Hamming distance can be constantly as small as 3 in a vast region in the space of binding/unbinding rates when α_m is small (≲ 0.02). Therefore, in the manuscript we present the landscape of a fast switching regime k_on ≈ 100, an intermediate regime k_on ≈ 15 and a slow switching regime k_on ≈ 3.

Validating the PDMP against the individual-based model

For the three selected parameter sets, 10⁴ sample paths of a fully individual-based model were generated by standard kinetic Monte Carlo simulations—namely Gillespie’s stochastic simulation algorithm (SSA) [17, 78]. The population scale Ω for each TF is set to be 10⁴. A parallel analysis is carried out and the results are consistent with the predictions from using the PDMP. We report the results of the intermediate switching regime in Fig 4 of the main text.

Visualizing stochatic fluctuations in gene expression on low dimensional manifolds using principal component analysis (PCA)

While the joint probability distributions are measured by kinetic Monte Carlo sampling, the dimensionality of the dynamical system is very high: each TF has a real-valued density, so that even if we marginalize over the genetic states the probability density is a 12-dimensional object. Although Fig 4 in the main text summarizes the marginal distributions of the real-valued TF density and contains rich information, it is desirable to visualize the results in a lower dimensional space to draw qualitative conclusions. To achieve this goal, we perform the standard principal component analysis [82]. We chose a baseline external condition to be LIF+2i; the first two principal components were computed. For the rest of the external conditions, the joint probability distributions are projected onto the plane spanned by these principal components; the results are presented in Fig 7 in the main text.

Dynamical transitions between different external signals

To investigate dynamical transitions when the external driving conditions (whether LIF, CH, and PD are present) change, we prepare 10⁵ independent sample paths with an initial external condition until the joint probability distribution converges to the stationary distribution. Then, the external condition is switched instantaneously to the second condition. We further evolve the dynamical system until stationarity for the second conditions is reached. The results are summarized in Fig 5 of the main text. To estimate the transition times between the stationary distributions with different external conditions, we measure the Jensen–Shannon distance of the marginal distribution of each TF density, at any given time during the transition to the final marginal distribution. We measure and report the first time when all 12 distances are below a threshold value of 0.3, presented in Fig 6 of the main text.

Numerical implementation

Both the PDMP and the IB models have been implemented in c++, using the algorithm presented in the Supporting Information and the standard kinetic Monte Carlo algorithm [17, 78] respectively. The network topology was hard-coded in the simulators to improve the simulation efficiency. The simulators are provided in the Supporting information. To facilitate the use of our computational work and ensure full reproducibility, the IB model is translated into standardized BioNetGen language [83, 84] and provided in Supporting Information, along with the standard SBML format of the model exported by BNGL.