Information-theoretic analysis of the directional influence between cellular processes

Inferring the directionality of interactions between cellular processes is a major challenge in systems biology. Time-lagged correlations allow to discriminate between alternative models, but they still rely on assumed underlying interactions. Here, we use the transfer entropy (TE), an information-theoretic quantity that quantifies the directional influence between fluctuating variables in a model-free way. We present a theoretical approach to compute the transfer entropy, even when the noise has an extrinsic component or in the presence of feedback. We re-analyze the experimental data from Kiviet et al. (2014) where fluctuations in gene expression of metabolic enzymes and growth rate have been measured in single cells of E. coli. We confirm the formerly detected modes between growth and gene expression, while prescribing more stringent conditions on the structure of noise sources. We furthermore point out practical requirements in terms of length of time series and sampling time which must be satisfied in order to infer optimally transfer entropy from times series of fluctuations.


Introduction
Quantifying information exchange between variables is a general goal in many studies of biological systems because the complexity of such systems prohibits mechanistic bottom-up approaches. Several statistical methods have been proposed to exploit either the specific dependence of the covariances between input and output variables with respect to a perturbation applied to the network [1], or the information contained in 3-point correlations [2]. These methods are potentially well suited for datasets obtained from destructive measurements, such as RNA sequencing or immunohistochemistry.
However, none of these methods exploits the information contained in time-lagged statistics, which is provided for instance by non-destructive measurements obtained from time- lapse microscopy of single cells. Such experimental data should be quite relevant to understand functional relationships since they merely reflect the time delays present in the dynamics of the system. Time-delayed cross-correlations between gene expression fluctuations have indeed been shown to discriminate between several mechanistic models of well characterized genetic networks [3]. However, such methods become difficult to interpret in the presence of feedback. This situation is illustrated in reference [4] where the fluctuations in the growth rate and in the expression level of metabolic enzymes have been measured as a function of time by tracking single cells of E. coli with time-lapse microscopy. The interplay between these variables has been characterized using cross-correlations as proposed in [3]. To circumvent the difficulty of discriminating between many complex and poorly parametrized metabolic models, the authors reduced functional relations to effective linear responses with a postulated form of effective couplings.
In the present work, we instead use a time-lagged and information-based method to analyze the interplay between the two fluctuating variables. A crucial feature in this method is that it is model-free and it is able to disentangle the two directions of influence between the two variables, unlike the cross-correlations discussed above. This type of approach was first proposed by Granger [5] in the field of econometrics and found applications in a broader area. More recently, transfer entropy [6], which is a non-linear extension of Granger causality, has become a popular information-theoretic measure to infer directional relationships between jointly dependent processes [7]. It has been successfully applied to various biomedical time series (see for instance [8]) and used extensively in the field of neurobiology, as shown in Ref. [9] and in references therein. This is the tool that will be used in this work.
The plan of this paper is as follows. We first introduce two measures of information dynamics, transfer entropy (TE) and information flow (IF). We then illustrate our numerical method on a well controlled case, namely a simple linear Langevin model, and show that we can properly estimate these quantities from the generated time series. We then analyze experimental data on the fluctuations of metabolism of E. coli taken from Ref. [4]. We provide analytical expressions for the transfer entropy and information flow rates for the model proposed in that reference. After identifying a divergence in one TE rate as the sampling time goes to zero, we introduce a simplified model which is free of divergences while still being compatible with the experimental data. We conclude that the inference of information-theoretic dynamical quantities can be helpful to build physically sound models of the various noise components present in chemical networks.

Information theoretic measures
Unlike the mutual information I(X : Y) that only quantifies the amount of information exchanged between two random variables X and Y as defined in the section on Methods, the transfer entropy (TE) is an asymmetric measure that can discriminate between a source and a target [6]. Consider two sampled time series {‥x i−1 , x i , x i+1 ‥} and {‥y i−1 , y i , y i+1 ‥}, where i is the discrete time index, generated by a source process X and a target process Y. The transfer entropy T X!Y from X to Y is a conditional, history-dependent mutual information defined as i ; x ðlÞ i and over all values of the time index i. To put it in simple terms, T X!Y quantifies the information contained from the past of X about the future of Y, which the past of Y did not already provide [7,8]. Therefore, it should be regarded as a measure of predictability rather than a measure of causality between two timeseries [10]. For instance, when x ðlÞ i does not bring new information on y i+1 , then Pðy iþ1 jy ðkÞ i ; x ðlÞ i Þ ¼ Pðy iþ1 jy ðkÞ i Þ and the transfer entropy vanishes because the prediction on y i+1 is not improved. With a similar definition for T Y!X , one can define the net variation of transfer entropy from X to Y as ΔT X!Y T X!Y − T Y!X . The sign of ΔT X!Y informs on the directionality of the information transfer.
The statistics required for properly evaluating the transfer entropy rapidly increases with k and l, which in practice prohibits the use of large values of k and l. The most accessible case thus corresponds to k = l = 1, which we denote hereafter as " T X!Y . This quantity is then simply defined as When the dynamics of the joint process {X, Y} is Markovian, one has Pðy iþ1 jy ðkÞ i ; x ðlÞ i Þ ¼ Pðy iþ1 jy i ; x i Þ and since Hðy iþ1 jy ðkÞ i Þ Hðy iþ1 jy i Þ one has " T X!Y ! T X!Y (see Ref. [11]). Therefore, " T X!Y represents an upper bound on the transfer entropy. In the case of stationary time series, which is the regime we consider in this work, it is natural to also introduce the TE rate Hðy tþt jy t Þ À Hðy tþt jx t ; y t Þ t Iðy tþt : y t ; x t Þ À Iðy tþt : where the continuous time variable t replaces the discrete index i. In practice T X!Y ' " T X!Y =t, but only for sufficiently small time step τ.
The most direct strategy to evaluate Eq (1) would be to construct empirical estimators of the probabilities from histograms of the data. Although this procedure works well for evaluating other quantities, for instance the entropy production in small stochastic systems [12], it completely fails in the case of transfer entropy. Indeed, such a method leads to a non-zero TE even between uncorrelated signals, due to strong biases in standard estimators based on data binning. In order to overcome this problem, we used the Kraskov-Stögbauer-Grassberger (KSG) estimator which does not rely on binning, as implemented in the software package JIDT (Java Information Dynamics Toolkit) [13]. Using estimators of this kind is particularly important for variables that take continuous values.
In the following, the inference method will be applied to time series generated by diffusion processes. It will then be interesting to compare the TE rate T X!Y to another measure of information dynamics, the so-called information flow [14][15][16] (also dubbed learning rate in the context of sensory systems [11,17]), which is defined as the time-shifted mutual information [18] Iðy t : x t Þ À Iðy t : In the special case where the two processes X and Y experience independent noises (the system is then called bipartite) [15], one has the inequality I flow [17], which in turn implies that when the joint process is Markovian. Observing a violation of this inequality is thus a strong indication that the noises on X and Y are correlated. As will be seen later, this is indeed the situation in biochemical networks, due the presence of the so-called extrinsic noise generated by the stochasticity in the cell and in the cell environment [19] which acts on all chemical reactions within the cell, and thus induces correlations.

Test of the inference method on a Langevin model
In order to benchmark our inference method and perform a rigorous test in a controlled setting, we first applied it on times series generated by a simple model for which the transfer entropy and the information flow can be computed analytically. The data were obtained by simulating the two coupled Langevin equations that describe the dynamics of a particle of mass m subjected to a velocity-dependent feedback that damps thermal fluctuation [16,20,21] (in these equations, the dependence of the variables on the time t is implicit). Here, ξ(t) is the noise generated by the thermal environment with viscous damping γ and temperature T, while η(t) is the noise associated with the measurement of the particle's velocity v(t). The two noises are independent and Gaussian with zero-mean and variances hξ(t)ξ(t 0 )i = 2γk B Tδ(t − t 0 ) and hη(t)η(t 0 )i = σ 2 δ(t − t 0 ). a is the feedback gain and τ r is a time constant. The two Langevin equations were numerically integrated with the standard Heun's method [22] using a time step Δt = 10 −3 , and the transfer entropy in the steady state was estimated from 100 time series of duration t = 2000 with a sampling time (i.e., the time between two consecutive data points) τ = Δt. We first checked that the TE in the direction Y!V does vanish in the absence of feedback, i.e. for a = 0, whereas it is non-zero as soon as a > 0. We then tested the influence of the measurement error σ 2 for a fixed value of the gain a. As can be seen in Fig 1, T V!Y diverges as σ 2 ! 0, a feature that will play an important role in our discussion of the model for the metabolic network. In the figure, the color of the symbols correspond to three different values of the parameter k which represents the history length in the definition of the transfer entropy (see Eq (1)). One can see that the estimates of T V!Y for k = 1 are in very good agreement with the theoretical prediction for " T V!Y (upper solid line). Moreover, the estimates decrease as k is increased from 1 to 5, and one can reasonably expect that the theoretical value of T V!Y (lower solid line) computed in Ref. [16] and given by Eq (21) in the section on Methods would be reached in the limit k ! 1.
Finally, by estimating the information flow and the transfer entropy, we checked that inequality (5) holds, as a result of the independence of the two noises ξ and η (see section on Methods).

Analysis of stochasticity in a metabolic network
Experimental time series. We are now in position to analyze the fluctuations in the metabolism of E. coli at the single cell level obtained in Ref. [4] using the information-theoretic notions introduced and tested in the previous section. Since there are a multitude of reactions and interactions involved in the metabolism of E. coli, a complete mechanistic description is not feasible, and our model-free inference method has a crucial advantage. In Ref. [4], the length of the cells was recorded as a function of time using image analysis, and the growth rate was then obtained by fitting this data over subparts of the cell cycle. In the same experiment, the fluorescence level of GFP, which is co-expressed with growth enzymes LacY and LacZ was The two time series have a branching structure due to the various lineages, which all start from a single mother cell as shown in Fig 2. The experimental data thus come in the form of a large ensemble of short times series which represent a record of all the cell cycles. There are about *3000 time series, with 2 to 8 measurement points in each of them which are represented as colored points in Fig 2. In order to correctly estimate the transfer entropy from such data, we have analyzed the multiple time series as independent realizations of the same underlying stochastic process. For the present analysis, we fix the history length parameters k and l to the value k = l = 1, which means that we focus on " T rather than T. We infer the values of " T in the two directions, from growth (denoted μ) to gene expression (denoted E) and vice versa. The results obtained for the three concentrations of IPTG are represented in Table 1. The negative value of " T m!E which is found in the intermediate case is due to the numerical inference method and should be regarded as a value which cannot be distinguished from zero.
Based on this analysis, we conclude that the influence between the variables is directed primarily from enzyme expression to growth in the low and intermediate IPTG experiments, while it mainly proceeds in the reverse direction in the high IPTG experiment. Such results are https://doi.org/10.1371/journal.pone.0187431.g002 Table 1

. Inferred values of the transfer entropies in the directions E!μ and μ!E, and the difference
T m!E for low, medium and high concentrations of IPTG based on the data of ref. [4]. The TE are given in nats. in line with the conclusions of Ref. [4] based on the measured asymmetry of the time-lagged cross-correlations. Moreover, the present analysis provides an estimate of the influence between the two variables separately in the two directions from E to μ and from μ to E. In particular, we observe for the low experiment that the values of TE in the two directions are of same order of magnitude, whereas in the intermediate experiment the TE from E to μ is larger, a feature which could not have been guessed from measured time delays. Theoretical models. We now turn to the analysis of the model proposed in Ref. [4] to account for the experimental data. The question we ask is whether the model correctly reproduces the above results for the transfer entropies, in particular the change in the sign of D "

Conc. of IPTG
T E!m for the high concentration of IPTG.
The central equation of the model describes the production of the enzyme as where E is the enzyme concentration, p its production rate, and μ the rate of increase in cell volume. Although the function p is typically non-linear, its precise expression is irrelevant because (7) is linearized around the stationary point defined by the mean values E = E 0 and μ = μ 0 . This linearization then yields in terms of perturbed variables δX(t) = X(t) − X 0 , where X 0 denotes the mean of X.
The model of Ref. [4] is essentially phenomenological in nature because it approximates the noises as Gaussian processes. Although this approximation is often done in this field, it may not always hold since fluctuations due to low copy numbers are generally not Gaussian [23]. In any case, the model contains three Gaussian noises: N G is a common component while N E and N μ are component specific to E and μ. These noises are assumed to be independent Ornstein-Uhlenbeck noises with zero mean and autocorrelation functions hN i ðtÞN i ðt 0 Þi ¼ As commonly done, the three Ornstein-Uhlenbeck noises are generated by the auxiliary equations where the x 0 i s are zero-mean Gaussian white noises satisfying Introducing the constant logarithmic gains T XY that represent how a variable X responds to the fluctuations of a source Y, the equations of the model read [4] where specifically T Eμ = −1 and T μG = 1. Then, eliminating δp from Eqs (8) and (10), one obtains the coupled equations where we have defined the reduced variables x = δE/E 0 , y = δμ/μ 0 . We stress that N G is an extrinsic noise that affects both the enzyme concentration and the growth rate, whereas N E (resp. N μ ) is an intrinsic noise that only affects E (resp. μ). Note that the two effective noises T EG N G + N E and T μG N G + N μ acting on _ x and y are colored and correlated, which makes the present model more complicated than most stochastic models studied in the current literature. In fact, since we are mainly interested in the information exchanged between x and y, it is convenient to replace one of the noises, say N G , by the dynamical variable y. Differentiating the second equation in Eq (11), using Eq (9) and performing some simple manipulations, one then obtains a new set of equations for the four random variables x, y, u N E , v N μ : where the coefficients a j and b j (j = 1. . .4) are defined by Eq (24) in the section on Methods and ξ y = ξ μ + ξ G is a new white noise satisfying hx y ðtÞx y ðt 0 Þi ¼ ðy 2 m þ y 2 G Þdðt À t 0 Þ and hx y ðtÞx m ðt 0 Þi ¼ y 2 m dðt À t 0 Þ. The calculation of the transfer entropy rate T X!Y (which coincides with T E!m since the TE is invariant under the change of variables from E to x and μ to y) is detailed in the section on Methods, together with the calculation of the information flows. The final expression reads where p(x, y) is the steady state probability distribution and the functions " g y and g y are defined in Eqs (40) and (43), respectively. This result agrees with that obtained in Refs. [11,18] and in [24] in special cases.
In Table 2, we show the results of the analysis of the time series generated by Eq (12) using our numerical inference method with a sampling time τ = 1min (equal to the time step Δt used to numerically integrate the model). One can see that the estimates of T E!m are in good agreement with the predictions of Eq (13), with the values of the model parameters taken from Table S1 in Ref. [4]. Note that the negative number given by the inference method in the high IPTG experiment signals that the actual value of T E!m cannot be distinguished from zero, which is indeed the theoretical prediction. In contrast, the estimated and theoretical results for T m!E do not agree, as the inference method yields finite values in all cases whereas the theoretical values diverge. This behavior is due to the absence of a white noise source directly affecting the dynamical evolution of x in the set of Eq (12). Indeed, as pointed out in Ref. [6] and also observed above in Fig 1, a TE rate diverges when the coupling between the variables is deterministic. In the model of Ref. [4], this feature can be traced back to the fact that the noise N E affecting the enzyme concentration is colored with a finite relaxation time b À 1 E . Therefore, when taking the limit τ ! 0 in Eq (3), one explores a time interval t < b À 1 E where N E is not really random. This is illustrated in Fig 3a that corresponds to the low IPTG experiment: we see that the estimate of T m!E with the inference method is indeed diverging when the sampling time τ approaches zero. On the other hand, as expected, T E!m remains finite and the points nicely lie on the plateau determined by Eq (13).
The obvious and simplest way to cure this undesirable feature of the original model is to treat N E as a purely white noise, which amounts to taking the limit b À 1 E ! 0. In fact, it is noticeable that the values of b À 1 E extracted from the fit of the correlation functions in Ref. [4] (resp. b À 1 E ¼ 10:7; 9:9 and 8.15 min for the low, intermediate, and high IPTG concentrations) are significantly smaller than the time steps τ exp used for collecting the data (resp. τ exp = 28, 20 and 15.8 min). Therefore, it is clear that the experimental data are not precise enough to decide whether N E is colored or not. This issue does not arise for the other relaxation times in the We thus propose to modify the model of Ref. [4] by describing N E as a Gaussian white noise with variance hN E (t)N E (t 0 )i = 2D E δ(t − t 0 ) and the same intensity as the colored noise in the original model, i.e. D E ¼ Z 2 E =b E (which yields D E % 0.188h, 0.100h, 0.031h for the three IPTG concentrations). Unsurprisingly, this modification does not affect the auto and crosscorrelation functions used to fit the data, as shown in  Tables 2   and 3) and, more importantly, T m!E , given by Eq (60) is now finite. As a result, the model predicts that the difference DT E!m ¼ T E!m À T m!E is positive at low and intermediate IPTG concentrations and becomes negative at high concentration, which is in agreement with the direct analysis of the experimental data in Table 1. In contrast, DT E!m was always negative in the original model as T m!E is infinite.
This new behavior of the TE rates is also manifest when the inference method is applied to the time series generated by the model and the sampling time τ is varied. As observed in Fig  3b, the inferred value of T m!E no longer diverges as τ ! 0 (compare the vertical scale with that in Fig 3a). The estimates of T E!m and T m!E are also in good agreement with the theoretical predictions, except for the shortest value of τ which is equal to the time step Δt = 1 min used to numerically integrate the equations. It worth mentioning, however, that the error bars increase as τ is decreased.  Information-theoretic analysis of the directional influence between cellular processes While the change in the sign of DT E!m is now confirmed by the model, which is the main outcome of our analysis, one may also wonder whether the numerical values in Table 1 are recovered. This requires to multiply the rates in Table 3 by the experimental sampling times τ exp which are different in each experiment, as indicated above. One then observes significant discrepancies for the low and intermediate IPTG experiments. We believe that the problem arises from the presence of many short time series in the set of experimental data. This is a important issue that needs to be examined in more detail since it may be difficult to obtain long time series in practice.
To this aim, we have studied the convergence of the estimates of DT E!m to the exact asymptotic value as a function of N, the length of the time series generated by the model in the stationary regime. As shown in Fig 5, the convergence with N is slow, which means that one can make significant errors in the estimation of DT E!m if N is small. On the other hand, the convergence can be greatly facilitated by choosing a value of the sampling time which is not too short (but of course shorter than the equilibration time of the system), for instance τ = 6min Information-theoretic analysis of the directional influence between cellular processes instead of 1 min in the case considered in Fig 5. The important observation is that the sign of DT E!m is then correctly inferred even with N % 1000. In contrast, with τ = 1min, this is only possible for much longer series, typically N % 50000. This is an encouraging indication for experimental studies, as the overall acquisition time of the data can be significantly reduced.
Finally, we briefly comment on the results for the information flows I flow E!m and I flow m!E . As already pointed out, the fact that the noises acting on the two random variables are correlated invalidates inequality (5). This is indeed what is observed in Table 4. It is also noticeable that I flow E!m 6 ¼ À I flow m!E , except in the high IPTG experiment where T μE = 0.

Discussion and conclusion
A challenge when studying any biochemical network is to properly identify the direction of information. In this work, using the notion of transfer entropy, we have characterized the directed flow of information between the single cell growth rate and the gene expression, using a method that goes beyond what could be obtained from correlation functions, or from other inference techniques which do not exploit dynamical information. Another crucial challenge in the field is to properly model the various noise components. It turns out that biological systems are generally non-bipartite due the presence of an extrinsic component in the noise. The present work provides on the one hand analytical expressions for the magnitude of the transfer entropy (or at least an upper bound on it) and of the information flow when the system is not bipartite, and, on the other hand a numerical method to infer the TE in all cases. Furthermore, we have shown that one can correctly infer the sign of the TE difference even with short time series by properly choosing the sampling time (see Ref. [25] for more details on the dependence of TE on the sampling time).
To conclude, we would like to emphasize that the transfer entropy is a general tool to identify variables which are relevant for time series prediction [26]. As such, the method has a lot of potential beyond the particular application covered in this paper: Predicting the current or future state of the environment by sensing it is an adaptation strategy followed by biological systems which can be understood using information-theoretic concepts [11,27]. Similarly, during evolution, biological systems accumulate information from their environment, process it and use it quasi-optimally to increase their own fitness [28,29]. In this context, transfer entropy-based methods have the potential to identify the directional interactions in co-evolution processes, which could be for instance the genomic evolution of a virus compared to that of its antigenes [30]. With the recent advances in high-throughput techniques and Table 4 −1 ). The analysis was performed with a sampling τ = 6 min and 100 time series of 10 6 points. Information-theoretic analysis of the directional influence between cellular processes experimental evolution, we might soon be able to predict reliably the evolution of biological systems [31], and without doubt tools of information theory will play a key role in these advances.

Methods
In this section, we provide a detailed analysis of the information-theoretic quantities for the various models considered in this paper. The section is organized as follows: • Basic information-theoretic measures • Transfer entropy and information flow in the feedback cooling model • Transfer entropy rates and information flows in the model of Ref. [4] for a metabolic network • Transfer entropy rates and information flows in the modified model for the metabolic network

Basic information-theoretic measures
Below we briefly recall some definitions and properties of the information-theoretic measures. A fundamental quantity is the Shannon entropy which quantifies the uncertainty associated with the measurement x of a random variable X: where P(x) is the probability that event x is realized, given an ensemble of possible outcomes. With this convention, the entropy is measured in nats. Similarly, for two random variables X and Y, one defines the joint Shannon entropy where P(x, y) and P(x|y) are joint and conditional probability distribution functions, respectively. The mutual information I(X : Y) is then a symmetric measure defined as and characterizes the undirected information exchanged between the two processes. The conditional mutual information is defined similarly. In contrast, the transfer entropy T X!Y is a information-theoretic measure that is both asymmetric and dynamic as it captures the amount of information that a source process X provides about the next state of a target process Y. More precisely, as defined by Eq (1) in the introduction, where k and l define the lengths of the process histories, i.e., Y ðkÞ In this work, we have focused on a history length of 1 (i.e. k = l = 1) and denoted the corresponding TE by " which is an upper bound to T X!Y (k, l) for l = 1 when the joint process {X, Y} obeys a Markovian dynamics [11].
On the other hand, the information flow from X to Y is defined as the time-shifted mutual information and informs on the reduction of uncertainty in Y i when knowing about X i+1 as compared to what we had with X i only. In practice, I flow X!Y can be obtained by shifting in time one time series with respect to the other one. Contrary to the transfer entropy which is always a positive quantity, the information flow I flow X!Y may be negative or positive, depending on whether X sends information to Y (or X gains control of Y), or Y sends information to X (or X looses control over Y). In a bipartite system one has I flow X!Y ¼ À I flow Y!X in the stationary regime. This is no longer true when the system is non-bipartite.

Transfer entropy and information flow in the feedback cooling model
We first recall the theoretical expressions of the transfer entropy rates and the information flows for the feedback-cooling model described by Eq (6). These quantities were computed in Ref. [16]. The transfer entropy rates in the stationary state are given by Note that 2T/(γσ 2 ) is the signal-to-noise ratio that quantifies the relative size of the measurement accuracy to the thermal diffusion of the velocity. Accordingly, the TE rate T V!Y diverges when the control is deterministic. The information flow I flow V!Y is given by where jSj is the determinant of the covariance matrix. The analytical expressions of the elements of the matrix, hv 2 i, hy 2 i and hvyi, are given by Eqs (A2) in Ref. [16]. In contrast with T V!Y , the information flow I flow V!Y remains finite as the noise intensity vanishes.
The upper bounds to the transfer entropies (see Eq (2)) were computed in Ref. [24] in the general case of coupled linear Langevin equations. For the feedback cooling model, one obtains As shown in Fig 1, the estimate of the transfer entropy obtained by the inference method is in good agreement with the theoretical value (we stress that the figure shows the rates multiplied by the sampling time τ = 10 −3 ). In Fig 6, we also obtain satisfactory agreement between inferred value of the information flow I flow V!Y and theoretical value, when representing these quantities against the noise intensity σ 2 . These results of this figure confirm the inequalities Transfer entropy rates and information flows in the model of Ref. [4] for a metabolic network Stationary distributions and correlation functions. We first compute the stationary probability distributions (pdfs) associated with Eq (12) were the coefficients a j and b j are given We recall that μ E = μ 0 (1 + T μE − T EE ) sets the timescale of E-fluctuations [4]. Since Eq (12) describe a set of coupled Markovian Ornstein-Uhlenbeck processes, the stationary pdf p xuvy (x, u, v, y) is Gaussian and given by p xuvy ðx; u; v; yÞ ¼ 1 where S is the covariance matrix which obeys the Lyapunov equation [32] AS þ SA T ¼ 2D; where As an illustration, the steady-state pdf pðmÞ ¼ 1 m 0 p y y ¼ mÀ m 0 m 0 is plotted in Fig 7 for   with R μμ (0) = σ 44 . The correlation functions R μμ (τ), R EE (τ), and R Eμ (τ), obtained by taking the inverse Fourier transform of Eqs (6) in the Supplementary Information of [4] are plotted in Fig 4. In passing, we correct a few misprints in these equations: i) The correct expression of R μμ (τ) is obtained by replacing A E (τ) by R EE (τ) in the first term of Eq (12) in the Supplementary Information of [4]. ii) Eq (10) corresponds to R Eμ (τ) and not to R μE (τ) = R Eμ (−τ). Eq (8) then gives the correct expression of R Eμ (τ) (and not of R μE (τ)) provided the function A X (τ) defined in Eq (10) is altered. For τ ! 0, one should have Transfer entropy rates. We now address the computation of the conditional probabilities p y x 0 y 0 ðy; t þ tjx 0 ; y 0 ; tÞ and p y y 0 ðy; t þ tjy 0 ; tÞ at first order in τ. This will allow us to obtain the expressions of the upper bounds to the transfer entropy rates defined by I½y tþt : x t ; y t À I½y tþt : y t t I½x tþt : x t ; y t À I½x tþt : where I is the mutual information, for instance I½y tþt : x t ; y t ¼ R dy dx 0 dy 0 p y x 0 y 0 ðy; t þ t; x 0 ; y 0 ; tÞ ln ½p y x 0 y 0 ðy; t þ t; x 0 ; y 0 ; tÞ=½p y ðyÞp xy ðx 0 ; y 0 Þ in the steady state (where p xy (x 0 , y 0 ) and p y (y) become time independent pdfs). Therefore, dy dx 0 dy 0 p y x 0 y 0 ðy; t þ t; x 0 ; y 0 ; tÞ Â ln p y x 0 y 0 ðy; t þ tjx 0 ; y 0 ; tÞ p y y 0 ðy; t þ tjy 0 ; tÞ Note that the actual transfer entropy rates are defined as I½y tþt : x t ; fy t 0 g t 0 t À I½y tþt : fy t 0 g t 0 t t I½x tþt : fx t 0 g t 0 t ; y t À I½x tþt : fx t 0 g t 0 t t : Due to the linearity of Eq (12) and the Gaussian character of the pfds, one simply has " g y ðx; yÞ ¼ ax þ by and g y ðyÞ ¼ cy, where a, b, c are complicated functions of the model parameters which we do not display here. Eq (41) (resp. Eq (42)) merely shows that p y x 0 y 0 ðy; t þ tjx 0 ; y 0 ; tÞ (resp. p y y 0 ðy; t þ tjy 0 ; tÞ) at the lowest order in τ is identical to the transition probability associated with an Ornstein-Uhlenbeck process with drift coefficient " g y ðx; yÞ (resp. g y ðyÞ) and diffusion coefficient b m Z 2 m þ b G Z 2 G . To proceed further, it is then convenient to use to the Fourier integral representation of the δ function and re-express p y x 0 y 0 ðy; t þ tjx 0 ; y 0 ; tÞ and p y y 0 ðy; t þ tjy 0 ; tÞ for small times as ½yÀ y 0 À t" g y ðx 0 ;y 0 Þ 2 and p y y 0 ðy; t þ tjy 0 ; tÞ ¼ up to corrections of the order τ 2 [32]. This leads to ln p y x 0 y 0 ðy; t þ tjx 0 ; y 0 ; tÞ p y y 0 ðy; t þ tjy 0 ; tÞ ½2ðy À y 0 Þ À t½" g y ðx 0 ; y 0 Þ þ g y ðy 0 Þ Â½" g y ðx 0 ; y 0 Þ À g y ðy 0 Þ; ð46Þ and from Eq (39) and the definition of the transfer entropy rate [Eq (34)], 1 t Z dy dx 0 dy 0 p y x 0 y 0 ðy; t þ t; x 0 ; y 0 ; tÞ½2ðy À y 0 Þ À t½" g y ðx 0 ; y 0 Þ þ g y ðy 0 Þ½" g y ðx 0 ; y 0 Þ À g y ðy 0 Þ ¼ lim t!0 1 t Z dy dx 0 dy 0 p xy ðx 0 ; y 0 Þ½dðy À y 0 Þ À t@ y ½" g y ðx 0 ; y 0 Þ À ðb m Z 2 m þ b G Z 2 G Þ@ y dðy À y 0 Þ Â½2ðy À y 0 Þ À t½" g y ðx 0 ; y 0 Þ þ g y ðy 0 Þ½" g y ðx 0 ; y 0 Þ À g y ðy 0 Þ We then use Z dy ðy À y 0 Þ½dðy À y 0 Þ À t@ y ½" g y ðx 0 ; y 0 Þ À ðb m Z 2 m þ b G Z 2 G Þ@ y dðy À y 0 Þ ¼ t" g y ðx 0 ; y 0 Þ; ð48Þ and R dx 0 p xy ðx 0 ; y 0 Þ" g y ðx 0 ; y 0 Þ ¼ p y ðy 0 Þg y ðy 0 Þ ¼ R dx 0 p xy ðx 0 ; y 0 Þg y ðy 0 Þ; ð49Þ to finally arrive at Eq (13), namely Z dx dy p xy ðx; yÞ½" g 2 y ðx; yÞ À g 2 y ðyÞ: ð50Þ A similar expression can be found in Ref. [11] (see Eq (A.31) in that reference). Note also that the result given in Ref. [24] is obtained as a special case. Inserting into Eq (13) the values of the parameters given in Table S1 of Ref. [4], we obtain the values given in Table 2. Note that T E!m ¼ 0 for the high IPTG concentration because T μE = 0, and therefore μ(t) no longer depends on E(t) as can be seen from Eq (10).
There is no need to detail the calculation of T m!E (i.e. T Y!X ) because it goes along the same line, with y replaced by x. The crucial difference is that there is no white noise acting on _ x. Therefore, the denominator in Eq (13), which is the variance of the noise ξ y , is replaced by 0. This implies that T m!E is infinite. I½y t : x t À I½y tþt : x t t : Let us first consider the second flow I flow Y!X which requires the knowledge of p y x 0 ðy; t þ t; x 0 ; tÞ whose expression is obtained by integrating Eq (39) over x 0 . This yields p y x 0 ðy; t þ t; x 0 ; tÞ ¼ p xy ðx 0 ; yÞ À t@ y ½" g y ðx 0 ; yÞ À ðb m Z 2 m þ b G Z 2 G Þ@ y p xy ðx 0 ; yÞ þ Oðt 2 Þ: Hence I½y tþt : x t ¼ Z dx 0 dy p y x 0 ðy; t þ t; x 0 ; tÞ Â ln p y x 0 ðy; t þ t; x 0 ; tÞ p y ðyÞp x ðx 0 Þ ¼ I½y t : x t À t Z dx dy @ y ½" g y ðx; yÞ À ðb m Z 2 m þ b G Z 2 G Þ@ y p xy ðx; yÞ ln p xy ðx; yÞ p y ðyÞp x ðxÞ : for the information flows I flow E!m and I flow m!E . It is worth noting that the symmetry relation _ I flow E!m ¼ À _ I flow m!E no longer holds, except for the high IPTG concentration (as T μE = 0). This contrasts with the preceding case where N E was modeled by an Ornstein-Uhlenbeck noise. We also observe that the information flows are not always smaller than the transfer entropy rates, contrary to what occurs in bipartite systems. Therefore, the concept of a "sensory capacity" as introduced in Ref. [11] is here ineffective.