Advertisement

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Dynamic compensation, parameter identifiability, and equivariances

Abstract

A recent paper by Karin et al. introduced a mathematical notion called dynamical compensation (DC) of biological circuits. DC was shown to play an important role in glucose homeostasis as well as other key physiological regulatory mechanisms. Karin et al. went on to provide a sufficient condition to test whether a given system has the DC property. Here, we show how DC can be formulated in terms of a well-known concept in systems biology, statistics, and control theory—that of parameter structural non-identifiability. Viewing DC as a parameter identification problem enables one to take advantage of powerful theoretical and computational tools to test a system for DC. We obtain as a special case the sufficient criterion discussed by Karin et al. We also draw connections to system equivalence and to the fold-change detection property.

Author summary

A recently introduced mathematical notion called dynamical compensation of biological circuits was shown to play an important role in glucose homeostasis and other key physiological regulatory mechanisms. This paper explains how dynamical compensation can be formulated in terms of a well-known concept in systems biology, statistics, and control theory—that of parameter structural non-identifiability. Viewing dynamical compensation as a parameter identification problem enables one to take advantage of powerful theoretical and computational tools to test a system for dynamical compensation. As a special case, one obtains the sufficient criterion for dynamical compensation. The paper also draws connections to system equivalence and to the fold-change detection property. The non-identifiability characterization brings up an interesting contrast in the way in which one thinks of these properties in the two fields. From the point of view of robustness of behavior, one wishes that parameters do not influence much the response of a system. On the other hand, from the systems and parameter identification point of view, the more that a parameter affects behavior, the easier it is to estimate it, and poor sensitivity is taken as an indication of a poorly parametrized model.

Citation: Sontag ED (2017) Dynamic compensation, parameter identifiability, and equivariances. PLoS Comput Biol 13(4): e1005447. https://doi.org/10.1371/journal.pcbi.1005447

Editor: Natalia L. Komarova, University of California Irvine, UNITED STATES

Received: December 31, 2016; Accepted: March 6, 2017; Published: April 6, 2017

Copyright: © 2017 Eduardo D. Sontag. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper.

Funding: This work was supported by: National institutes of Health 1R01GM100473, to EDS https://www.nih.gov/; Air Force Office of Scientific Research FA9550-14-1-0060, to EDS http://www.wpafb.af.mil/afrl/afosr; Office of Naval Rsearch N00014-13-1-0074, to EDS https://www.onr.navy.mil/. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The author has declared that no competing interests exist.

Introduction

The recent paper [1] argued that physiological control systems should ensure a precise dynamical response despite variations in certain parameters, lest pathological conditions arise. The authors highlighted the biological significance of this robustness property through the analysis of several biological systems, including models of plasma glucose response in the face of changes in insulin sensitivity, parathyroid hormone control of calcium homeostasis, and arterial oxygen regulation in response to hypoxia. They formally introduced the system property of dynamical compensation (DC) with respect to variations in a parameter p, meaning that the complete output dynamics is exactly the same, for any time dependent input, independently of the precise value of p. They went on to provide a sufficient condition for DC to hold, and applied this condition to verify that their physiological examples have the DC property.

In this paper, we frame the notion of DC in the context of two equivalent notions in systems biology: equivariances and parameter (non-)identifiability. We provide a necessary and sufficient condition for DC in the language of equivariances and partial differential equations, for which the sufficient condition given in [1] becomes a particular case, as does the condition given in [2] for fold-change detection (FCD) or input symmetry invariance, a Weber-like law in psychophysics [3]. We also provide alternative necessary and sufficient conditions for DC, using some of the rich tools in the existing literature on systems identification and nonlinear control theory, for which effective algorithms are available. The non-identifiability characterization brings up an interesting contrast in the way in which one thinks of these properties in the two fields. From the point of view of robustness of behavior, one wishes that parameters do not influence much the response of a system. On the other hand, from the systems and parameter identification (“reverse engineering”) point of view, the more that a parameter affects behavior, the easier it is to estimate it, and poor sensitivity is taken as an indication of a poorly parametrized model.

We focus our paper on a self-contained discussion, with examples, of the fact that DC is equivalent to other known concepts. There is, in addition, an extensive literature on parameter identifiability, which may be consulted for more details, mathematical theory, and effective algorithms based on computer algebra methods [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16].

This paper is organized as follows. After formally setting up the questions to be studied, we present several characterizations: equivariances, Lie derivatives, and derivatives of outputs at the initial time. We then re-visit the examples from [1] in view of these characterizations, working out each example. A Discussion section summarizes the results, compares methodologies, and mentions weaker versions of DC. The Methods section discusses the proofs of the various results.

Systems and equivalence

We formally state definitions in the language of dynamical systems with inputs and outputs, the standard paradigm in control systems theory [17]: (1) or, more explicitly, The functions f, h describe respectively the dynamics and the read-out map; u = u(t) is an input (stimulus, excitation) function, assumed to be piecewise continuous in time, x(t) is an n-dimensional vector of state variables, y(t) is the output (response, reporter) variable, and p is the parameter (or vector of parameters) that we wish to focus our attention on. In typical applications, y(t) = xi(t) is a coordinate of x. Values of states, inputs, outputs, and parameters are constrained to lie in particular subsets , , , respectively, of Euclidean spaces . Typically in biological applications, one picks as the set of positive vectors in , x = (x1, x2, …, xn) such that xi > 0 for all i, and similarly for the remaining spaces. The state is an initial state, which could be different for different parameters; thus we view ξp as a function . (We prefer the notation ξp instead of ξ(p), so that we do not confuse p with the time variable.) For each input , we write the solution of Eq (1) with an initial condition x(0) = ξ (for example, x(0) = ξp) as and the corresponding system response, which is obtained by evaluating the read-out map along trajectories, as We assume that for each input and initial state (and each fixed parameter p), there is a unique solution of the initial-value problem , x(0) = ξ, so that the mapping φ is well-defined; see [17] for more discussion, regularity of f, properties of ODE’s, global existence of solutions, etc. We refer to Eq (1) as a parametrized (because of the explicit parameter) initialized (because of the specification of a given initial state for each parameter) family of systems.

Depending on the application, one might wish to impose additional restrictions on ξp. For example, in [1] an additional requirement is that and is an equilibrium when u(0) = 0, which translates into f(ξp, 0, p) = 0, and a similar requirement is made in [2] (this latter reference imposes the requirement that for each constant input u(t) ≡ μ there should exist an equilibrium, in fact). One may also impose stability requirements on this equilibrium; nothing changes in the results to be stated.

The “dynamic compensation” property in [1]—we prefer to use the terminology -invariance—is the property that outputs should be always the same, independent of the particular parameter. Formally: (2) The question addressed in [1] was that of providing conditions on the functions f, h, and ξ so that Property Eq (2) holds.

Results

We start by re-stating -invariance in a more convenient form. Let us fix from now on an arbitrary , which we write as “1” (for example, if is the set of all positive vectors (p1, …, ps), 1 could naturally be taken as the vector (1, 1, …, 1)). Let us write f(x, u) ≔ f(x, u, 1), h(x, u) ≔ h(x, u, 1), ξ ≔ ξ1, φ(t, ξ, u) ≔ φ(t, ξ, u, 1), and ψ(t, ξ, u) ≔ ψ(t, ξ, u, 1).

In control theory, one says that, for any given parameter p, the system (1) is said to be input/output (I/O) equivalent to the system (3) provided that (4)

Therefore, Eq (2) is equivalent to the property that, for every parameter p, Eq (1) should be I/O equivalent to Eq (3). Thus, the -invariance property is equivalent to the property that the parameters p are (structurally) unidentifiable from input/output data, a notion which was originally introduced by Bellman and Astrom in the 1970s [4]. Two approaches to testing I/O equivalence (or its lack) are described next.

Equivariances

Definition 1 Given a parametrized initialized family of system (1), a set of differentiable mappings is an equivariance family provided that: (5a) (5b) (5c) for all , , and , where in general ρ* denotes the Jacobian matrix of a transformation ρ.

An important observation is as follows.

Proposition 1 If an equivariance family exists, then the parametrized family is -invariant.

Observe that Eq (5a) is a first order quasilinear partial differential equation on the components of the vector function ρp (for each constant value u ∈ U and each parameter ), subject to the algebraic constraints given by Eqs (5b) and (5c). Such equations are usually solved using the method of characteristics [18]. In principle, testing for existence of a solution through a “certificate” such as an equivariance is far simpler than testing all possible time-varying inputs u(t) in the definition of equivalence, in a fashion analogous to the use of Lyapunov functions for testing stability or of value functions in the Hamilton-Jacobi-Bellman formulation of optimal control theory [17]. The paper [2] discusses the relation to classical equivariances in actions of Lie groups and symmetry analysis of nonlinear dynamical systems. Proposition 1 is proved in the Methods section.

It is a deeper result that the existence of an equivariance is also necessary as well as sufficient. This converse result, as some of the other converses to be mentioned, requires that the vector fields f(⋅, u), for each constant u, as well as the output function h, be real-analytic, that is, that they can be expanded into locally convergent power series. The expressions that commonly appear in systems biology models, such as mass action kinetics (polynomial functions) or Hill functions (rational functions) are real analytic, as are trigonometric, logarithmic, and exponential functions, so analyticity is not a strong restriction on the theory. An additional technical assumption, verified by all reasonable models including all examples in this paper, is that the systems are irreducible, meaning accessible from the initial state and observable. These notions are reviewed in the Methods section. Intuitively, accessibility means that no conservation laws restrict motions to proper submanifolds. For analytic systems, accessibility is equivalent to the property that the set of points reachable from ξ has a nonempty interior. Intuitively, observability means that no pairs of distinct states can give rise to an identical temporal response to all possible inputs.

Proposition 2 If the parametrized family is -invariant and the systems are irreducible for each p, then an equivariance family exists.

Necessity tells us that it is always worth searching for an equivariance, when attempting to prove that a system has -invariance. Often one can guess such functions by looking at the structure of the equations. Proposition 2 is proved in the Methods section.

Identifiability and Lie derivatives

For simplicity, we now restrict attention to systems in which inputs appear linearly (or more precisely, affinely); that is, systems defined by differential equations of the following general form: (6) where we assume that the input-value set is a convex subset of . Linearity in inputs is not a serious restriction, nor is not having an explicit input in the read-out map, and it is easy to generalize to more general classes of systems, but the notations become far less elegant. We write the s coordinates of h as h = (h1, …, hs).

Consider the following set of functions (“elementary observables” of the system): We are using the notation for the directional or Lie derivative of the function H with respect to the vector field X, and an expression as LYLXH means an iteration LY (LXH). We include in particular the case in which k = 0, in which case the expression in the defining formula is simply hj. For example, if the system is (initial states do not matter at this point; the parameters of interest could be all or a subset of α, β, γ, δ), then and we compute, for example,

A necessary condition for invariance is as follows.

Proposition 3 If a system is -invariant, then all elements in , when evaluated at the initial state, are independent of the values of parameters p.

The Methods section discusses a proof of this Proposition. This condition is most useful when proving non-invariance. One simply finds one element of which depends on parameters. This test can also be used as a test for identifiability, that is to say the possibility of recovering all parameters perfectly from outputs: if one can reconstruct the values of p from the elementary observables (evaluated at the initial state), that is to say if the map is one-to-one, then the parameters are identifiable. We discuss two examples in the section “Examples from the paper [1]”.

There is a converse of this property, under the additional assumption of analyticity.

Proposition 4 If the system is analytic and if all elements in , when evaluated at the initial state, are independent of the values of parameters p, then the system is -invariant.

An interpretation of this result in terms of identifiability is as follows: if one can recover all parameters as functions (perhaps nonlinear) of the elementary observables, then there is complete parameter identifiability (two different parameters give two different outputs, at least for some input). In other words, if the elementary observables give the same values for some two specific parameters , then the output for any possible inputs, whether step inputs or arbitrary inputs, is the same for these two parameters. See the Methods section for a discussion of this converse statement.

Identifiability and time derivatives at t = 0

Suppose that an input u(t) is infinitely differentiable on t. Then, and assuming that f and h are infinitely differentiable on their arguments, the corresponding output y(t), for any initial state, will also be infinitely differentiable (see e.g. the differential equations appendix in [17]). We assume now that the system is linear in controls, that is, that the equations have the form in Eq (6). It follows by iterating inputs and using the chain rule that the ith derivative y(i) of the output (i > 0) is a polynomial on the first i − 1 derivatives of the inputs. For example, and (taking for simplicity m = 1) Now let us consider the derivatives mi ≔ y(i)(0+). The mi’s are independent of parameters if and only if the coefficients of these polynomials on the inputs and their derivatives, which are themselves linear combinations of Lie derivatives, are independent of parameters.

Proposition 5 If a system is -invariant, then all the derivatives mi are independent of the values of parameters p.

A converse holds as well.

Proposition 6 If a system is analytic and if all the derivatives mi are independent of the values of parameters p, then the system is -invariant.

Typically, all higher-order derivatives of outputs can be expressed as a function of a finite number of derivatives, which helps make computations effective using computer algebra packages. The theory is developed in [19, 20, 21], and is based on work by Fliess and others [22, 23, 24, 25, 26]. We discuss an example in the Methods section.

Examples from the paper [1]

In its Supplement, the paper [1] presents the following class of systems that have -invariance, called there “dynamic compensation” (for arbitrary n, but the main text specializes to n = 1): (7a) (7b) (7c) (7d) (7e) The state variables are positive, that is, , inputs and outputs are scalar and positive, , , and the parameters for which we desire invariance are two positive numbers, . The additional parameters ηi and μi are fixed, and g, k, ℓ are three scalar (differentiable) functions, with the following positive homogeneity property for g: (8) The initial state is implicitly specified in [1] by the requirement that for any parameter vector p = (p1, p2) there is a unique steady state when u = 0; we call this state ξp. We take the reference parameter set to be 1 = (1, 1).

In the paper [1], when n = 1 this system is motivated as a mechanism for a hormonal circuit in which the output y is a regulated variable, x1 is a hormone that regulates y = x3, and the variable xn+1 = x2 represents the functional mass of the tissue that secretes the hormone x1. There is feedback on the regulated variable y, with a gain p1, and a growth rate in xn+1 that is controlled by y, for instance through increase of proliferation. The parameter p2 quantifies the magnitude of the effect of the mass on the hormone production rate. In one of the examples in [1], the input u(t) is the meal intake of glucose, xn+1 represents β cells, x1 is insulin, and y is plasma glucose concentration; the coefficients ηi correspond to transport rates, and the μi combine degradation rate and transport rates. In the more general case n > 1, the coordinates x1, …, xn represent the various physiological compartments that the hormone may circulate through.

We show -invariance by exhibiting an equivariance. In fact, the proof of dynamical compensation in [1] is based (without using that terminology) on showing that the following mapping is an equivariance family: Since h(x, u, p) = xn+2 is not changed by ρp, we have that h(ρp(x), u, p) = h(x, u), and since as equivariances always map steady states into steady states, and steady states are assumed to be unique, there is no need to test the condition ρp(ξ) = ξp. So we only need to check that: for all p, x, u. The map ρp is linear, and its Jacobian matrix is thus we need to verify: the last n + 1 of which are trivial, and the first one requires , which holds because of the homogeneity property Eq (8).

We next discuss the “linear integral feedback” and “linear proportional integral feedback” examples in the paper [1], which are given as examples of systems that are not -invariant (the paper does this by simulation). The first system is: (9a) (9b) (9c) with the unique steady state ξp = ((a − p2b)/p1, b) for u(0) = 0. Let us show that there is no possible equivariance ρp(x1, x2) = (αp(x1, x2), βp(x1, x2)). The requirement that h(ρp(x), u, p) = h(x, u) means that βp(x1, x2) = x2, so ρp(x1, x2) = (αp(x1, x2), x2). The PDE f(ρp(x), u, p)) = (ρp)*(x)f(x, u) translates into: Comparing coefficients of u in the first component, we have that , and this in turn implies that . Thus the only possible choice is a linear function αp(x1, x2) = c + x1. On the other hand, comparing second components, we have contradicting this formula. Thus, no possible equivariance exists, even if we only ask one of the parameters to vary. Combined with the necessity result, this implies non-invariance.

Another way to analyze this example is as follows. Let us compute the derivatives mi ≔ y(i)(0+) when applying an input u(t) that is differentiable for t > 0. We have that m0 = b, m1 = u0, m2 = −p2u0 + u1, , where ui ≔ u(i)(0+). Therefore, from the output y (and in particular from its derivatives at time zero) we can reconstruct p2 and p1, for example using a step function u ≡ 1: u0 = 1 and ui = 0 for i ≥ 1 implies that p2 = −m2 and . When there are no possible positive parameters producing this output, and when there is a unique pair p1 > 0, p2 > 0 giving the observed derivatives, but in any case, there is never more than one such pair. This means that the parameter vector p is identifiable, which implies that the system is not -invariant.

Alternatively, using the formalism of Lie derivatives, with (10) we can compute and we again see that we can recover the parameters, giving identifiability and hence non-invariance.

The second example of a system that fails invariance in the paper [1] is the “linear proportional integral feedback” given by: with the unique steady state ξp = (b, (a − p2b)/p1, b) for u(0) = 0. The non-existence of equivariances is proved similarly to the previous example, and is not shown. For simplicity, we only perform the Lie derivative test. One can find so once more we can recover the parameters, giving identifiability and hence non-invariance.

Let us now illustrate the method of derivatives with the example in Eq 7, which was already shown to be invariant by the method of equivariances. The equations are, in this case: where p = (p1, p2) are the parameters of interest, a, b are other positive parameters, and the equations evolve in the positive orthant xi > 0. Using, for clarity, superscripts “(i)” to indicate derivative of order i, and since y = x3, we have that: and thus, taking derivatives of this expression and substituting the equation for : so, taking again derivatives and substituting the equations for and : Observe that these are all functions of t (argument not shown). We can solve for x1 and x2 in terms of y(1) and y(2), obtaining: and and, finally, substitute these expressions into the formula for y(3), to obtain: which involves only y, y(1), and y(2) but neither x1 and x2 nor the parameters p1 and p2. Taking derivatives on both sides, and iterating, this implies, in turn, that higher-order derivatives of y can also be expressed in terms of y, y(1), and y(2) and neither of the other state coordinates nor p1 and p2. This holds for all t, so in particular at t = 0. We remark that the expression for y(3) can also be obtained, when the system equations happen to be given by polynomials, or even by rational functions, using tools from differential algebra. This is discussed in the Methods section.

In the expressions obtained, y(1) and y(2) depend on the parameters p = (p1, p2). However, these expressions did not yet use the initial state constraint. Next we prove that y(0), y(1)(0), and y(2)(0) are independent of parameters, so the same will be true for all derivatives at t = 0, of any order, and therefore Proposition 6 will give invariance. Now, specializing at the equilibrium initial state we have that, for any differentiable input, y(0) = b, y(1)(0) = u(0), and y(2)(0) = −(a/b)u(0) + u(1)(0). Thus, indeed neither of these derivatives depends on the parameters p, which confirms invariance.

It is interesting to observe that, in this example, there is no invariance to the other parameters, a and b; in fact, these parameters are identifiable. To see this, let us write, for convenience, α ≔ a/b. Then, allows identification of α; for example, using the input u(t) ≡ 1, Y2,1 ≔ y(2)(0) = −α, so can recover α = −Y2,1. From we cannot yet identify a and b individually. Taking one more derivative: and therefore, if the input is u ≡ 1, we obtain and if the input is u ≡ 2, we obtain and combining these two results, we conclude that 2Y4,1 − Y4,2 = 6α/b, allowing recovery of the parameter b (observe that α = a/b ≠ 0, since the parameters a and b are both positive).

Relations to FCD

The papers [2, 3] studied a notion of scale invariance, or more generally invariance to input-field symmetries. We restrict attention here to linear scalings although the same ideas apply to more general symmetries as well. The property being studied is the invariance of the response of the following parametrized family of systems: where the initial states ξp are the (assumed unique) steady states associated to p ⊙ u(0). This property is also called “fold change detection” (FCD), because the only changes that are detectable are possibly “fold” changes in u, not simply scalings such as changes of units. It is clear that this is merely a special case of the current general setup.

Discussion

The dynamical compensation (DC) property was shown in [1] to play an important role in physiological control systems. Independence of parameters means that complete dynamical responses, which may affect phenotype, are robust to potentially large parameter variations. The DC property was verified for various biological circuits including the dependence of glucose under parameters such as insulin sensitivity, using a mathematical test. We showed how the DC property can be re-interpreted in terms of more classical systems notions, and how effective tests for checking DC are then possible, by using the extensive literature on identifiability. We also drew connections to issues of system equivalence, and showed that the “fold change detection” or “scale invariance” property is another manifestation of the same phenomenon.

We have discussed a number of different techniques for testing identifiability (or lack thereof). Each of these techniques has its advantages and disadvantages. Equivariances amount to groups of symmetry and help uncover interesting connections among states and parameters which may have a mechanistic interpretation. When a system is DC, usually an explicit equivariance is easy to guess and it helps “certify” the property, with far less computation than “input/output” methods. The latter involve far less guessing (and no need to solve a partial differential equation), but on the other hand can have huge computational complexity and are difficult to apply to large systems even when there is an obvious structure (as in the example in Eq (7)). Among input/output methods, Lie derivatives or output derivatives at time zero have different advantages for different problems, and it is hard to compare their relative effectiveness. For Lie derivatives, the dimension of the state space is a major parameter in determining computational complexity, while in the case of output derivatives, it is the order of i/o equations that matters, and these two numbers are related in non-trivial ways [27]. In the Lie method, the number of directional derivatives grows exponentially with the input dimension, even for a fixed total order of output derivative. On the other hand, the output derivative method involves symbolic computations with large rational functions, which can easily exceed the available computational storage.

A rather striking aspect of the relation between DC and identifiability is that, while unidentifiability in the context of systems identification is viewed as a “bad” property, meaning that a model is overparametrized or that it is impossible to design experiments to gather enough information about a parameter, in contrast in the context of “dynamical compensation” it becomes a desirable property, as it means that system behavior is robust to changes in these parameters.

It is the case sometimes that even if system behavior theoretically depends on a given parameter, this dependence may be weak. A textbook example is that of an enzymatic reaction operating at near-saturation, where exact concentrations of substrates, if thought of as parameters, may have little influence. Another classical example is furnished by proteases acting in a close to zeroth-order regime. This notion of “almost independence” is also studied in the identification literature, under names such as “practical identifiability” (or more precisely, lack thereof) and is closely related to (though not exactly the same as) the concept of “sloppy” models in systems biology [28, 29, 30, 14, 15, 31]. When applied in the “dynamical compensation” context, these notions should be useful in understanding weaker notions of almost-robustness of homeostatic control systems in physiology.

Methods

Proofs of main characterizations

Proof of Proposition 1.

Fix a parameter p and an input u(⋅); we must show that Eq (4) is satisfied. Let x(t) = φ(t, ξ, u), t ≥ 0, be the solution of the initial-value problem , x(0) = ξ, so that ψ(t, ξ, u) = h(x(t), u(t)). Viewing the mapping x ↦ ρp(x) as a change of variables, we define z(t) ≔ ρp(x(t)). Differentiating with respect to t, where we used Eq (5a) applied with x = x(t) and u = u(t). Moreover, z(0) = ρp(x(0)) = ρp(ξ) = ξp, because of Eq (5c). Thus, since the solution of the initial-value problem , x(0) = ξp is unique, it follows that z(t) = φ(t, ξp, u, p). Now, by definition, where the last equality follows from Eq (5b). On the other hand, since ψ(t, ξ, u) = h(x(t), u(t)), we conclude that Eq (4) holds, as desired.

Proof of Proposition 3.

Consider a piecewise constant control which is equal to u1 on [0, t1), equal to u2 on [t1, t1 + t2), …, and equal to uk on [t1 + … + tk−1, t1 + … + tk), starting from x(0) = ξp. By invariance, the resulting output at time t = t1 + … + tk is independent of the parameters. Let us denote the jth coordinate of this output value as It follows that the derivatives with respect to the ti’s of this output are also independent of parameters, for every such piecewise constant control. By induction, one has where we are using the notations . This expression is a multilinear function of the ’s, and a further derivation with respect to these control value coordinates gives the elementary observables, which therefore must also be independent of parameters. See [17], Remark 6.4.2 for more details and references.

Proof of Proposition 5.

If the output is independent of parameters, so are all its derivatives, and in particular the derivatives evaluated at time t = 0 are independent of parameters.

We discuss next the notions of accessibility and observability required for the validity of the converse to Proposition 1.

Accessibility

We consider systems: with the technical condition that f is a real-analytic function of x. (Outputs do not matter for this section.)

For any fixed parameter vector , this system is said to be accessible from the initial state ξp if the set of all states reached from ξp, has a non-empty interior. In other words, cannot be “too thin” (as would happen if there are conservation laws among the species xi, in which case one should first reduce the model to independent variables). See for instance the textbooks [32, 17] for details.

Accessibility can be tested as follows. The Lie bracket of two smooth vector functions (more generally, one may extend to vector fields on a smooth manifold, but we restrict here to Euclidean spaces) is the function defined by the following formula: Let , that is, the set of vector fields obtained when we plug-in an arbitrary constant input (still for our fixed parameter vector p). We now define the set that consists of all possible iterated brackets [fℓ, …, [f3, [f2, f1] …]], over all choices of fi’s in and all ℓ ≥ 1. Formally, is the union of the sets that are recursively defined as follows: The accessibility (or controllability) rank condition is said to hold at ξp if there are n (the dimension of the state space ) linearly independent vectors among all these vectors when evaluated at the initial state. An equivalent way to state this condition is in terms of the accessibility Lie algebra of the system, which is the Lie algebra of vector fields generated by , which is the linear span of , the smallest Lie algebra of vector fields which contains . The condition is that , that is to say, the evaluations of these vector fields at the initial state span the entire tangent space. It can be shown that, for affine systems as in Eq (6),

As an illustration, take the example considered earlier. This is a system affine in inputs, with and we may compute: Since the determinant of the matrix with columns g1 and [g1,[g0, g1]] is equal to 2α2βγ ≠ 0 for every x, the accessibility rank condition holds, in particular, at the initial state.

As a second illustration, consider the linear integral feedback system in Eq (9). With the vector fields in Eq (10), we have that g1 = (0, 1)T and [g1, g0] = (1, −p2)T are linearly independent and hence span .

Observability

Consider again systems as in Eq (1): where f and h are real-analytic (initial states do not matter for this section). For any fixed parameter vector , this system is said to be observable provided that, for any two distinct states ξ and ξ′, there is some input function u (which might depend on the particular pair of states ξ and ξ′) such that the temporal responses ψ(t, ξ, u) and ψ(t, ξ′, u) differ at at least one time t. The equivalent contrapositive of this statement is “ψ(t, ξ, u) = ψ(t, ξ′, u) for all u, t implies ξ = ξ′.” For an analytic input-affine system (6) with output h = (h1,…, hp), one can restate the observability property as follows. We consider the set of elementary observables . Two states ξ and ξ′ are said to be separated by observables if there exists some such that k(ξ) ≠ k(ξ′). Observability is equivalent to the property that any distinct two states can be separated by observables, or, in more intuitive terms, that all coordinates of the state can be obtained as (possibly nonlinear) functions of observables. This is analogous to the similar result for parameter identifiability, and its proof is also based on approximating arbitrary inputs by piecewise constant inputs, combining Proposition 2.8.2 and Proposition 6.1.11 in [17] (see also Section 4 in [33]).

As an illustration, take once more We need to show that the coordinates of the states can be recovered from the elements in . Since x2 = h(x), so only need to give a formula for x2. Using , or, if we want to avoid division by zero when y = 0, .

Converses under irreducibility and/or analyticity

A system is said to be irreducible (alternative terminologies are “minimal” or “canonical”) if it is accessible from the initial state ξp, and observable.

Proof of Proposition 2.

The converse of Proposition 1 is entirely analogous to the proof of Theorem 1 in [2] for the special case of FCD, and relies upon the theory of minimal realizations of nonlinear systems. We outline it next. Let us fix any parameter . We must find a differentiable mapping so that ρp(ξ) = ξp and f(ρp(x), u, p) = (ρp)*(x)f(x, u), h(ρp(x), u, p) = h(x, u) for all and , This means that ρ should be an isomorphism [34] between the two systems (1) and (3). Now, Theorem 5 in [34] shows that for two analytic and irreducible systems, I/O equivalence implies the existence of an isomorphism, which is exactly what we require. See [2] for more discussion and details.

Proof of Proposition 4.

The proof of this converse fact consists two parts: (1) showing that outputs coincide for piecewise constant inputs, which is true because hj(t1, t2,…, tk, u1, u2,…, uk) can be expressed as a power series in terms of the observables, followed by (2) an approximation of arbitrary inputs by step ones. This step follows, applied to parameter identification instead of state observability, by combining Proposition 2.8.2 with Proposition 6.1.11 in [17]; see also Section 4 in [33], and [21]).

Proof of Proposition 6.

The proof is analogous to that of Proposition 4. We first consider inputs which are polynomial functions of time. Equality of output derivatives when using polynomial inputs is equivalent to equality using arbitrary differentiable inputs, since as remarked output derivatives are polynomial functions of the inputs and a finite number of their derivatives. Polynomials are in particular analytic inputs, and the system being analytic, the corresponding output functions y(t) are analytic in t (see e.g. the differential equations appendix in [17]). If two analytic outputs have the same derivatives of all orders at t = 0, then they are identical (Taylor expansions and principle of analytic continuation) Therefore, for any two parameters, the responses to all polynomial inputs coincide, and therefore one obtains equality for arbitrary inputs, again by an approximation argument as in Proposition 6.1.11 in [17], see also Section 4 in [33] and [21]).

Differential algebra computation of example

We show here the code needed in order to compute a differential equation for the output derivatives for the example described by Eq (7). Using the package Maple (Maplesoft, Waterloo Maple Inc.), one may employ the command

RosenfeldGroebner(sys,R)

to compute a representation of the radical of the differential ideal generated by a set of equations, in our case (from the equations defining the system)

S ≔ [diff(x1(t), t)-p_2*x2(t)*x3(t)+x1(t), \

diff(x2(t), t)-x2(t)*(x3(t)-b), \

diff(x3(t), t)-a+p_1*x1(t)*x3(t)-u(t), \

y(t)-x3(t)]

and R is a differential polynomial ring, in our case

R ≔ DifferentialRing(blocks = [x1, x2, x3, y, u], \

derivations = [t], arbitrary = [p_1, p_2, a, b])

in which we list the state variables, output variable, and input (in that order), followed by the time variable and the parameters. The command

G ≔ RosenfeldGroebner(S, R)

then computes the ideal, and a final elimination step

M ≔ Equations(G[1], solved)

provides a vector M whose coordinates are, respectively, the expressions for x1, x2, x3 = y worked out in the paper, as well as the expression for y(3).

Author Contributions

  1. Conceptualization: EDS.
  2. Data curation: EDS.
  3. Formal analysis: EDS.
  4. Funding acquisition: EDS.
  5. Investigation: EDS.
  6. Methodology: EDS.
  7. Project administration: EDS.
  8. Resources: EDS.
  9. Software: EDS.
  10. Supervision: EDS.
  11. Validation: EDS.
  12. Visualization: EDS.
  13. Writing – original draft: EDS.
  14. Writing – review & editing: EDS.

References

  1. 1. Karin O, Swisa A, Glaser B, Dor Y, Alon U. Dynamical compensation in physiological circuits. Mol Syst Biol. 2016;12(11):886. pmid:27875241
  2. 2. Shoval O, Alon U, Sontag ED. Symmetry invariance for adapting biological systems. SIAM Journal on Applied Dynamical Systems. 2011;10:857–886.
  3. 3. Shoval O, Goentoro L, Hart Y, Mayo A, Sontag ED, Alon U. Fold change detection and scalar symmetry of sensory input fields. Proc Natl Acad Sci USA. 2010;107:15995–16000. pmid:20729472
  4. 4. Bellman R, Åström KJ. On structural identifiability. Mathematical Biosciences. 1970;7:329–339.
  5. 5. Sontag ED. On the observability of polynomial systems. I. Finite-time problems. SIAM J Control Optim. 1979;17(1):139–151.
  6. 6. Chappell MJ, Gunn RN. A procedure for generating locally identifiable reparameterisations of unidentifiable non-linear systems by the similarity transformation approach. Math Biosci. 1998;148(1):21–41. pmid:9597823
  7. 7. Meshkat N, Eisenberg M, Distefano JJ. An algorithm for finding globally identifiable parameter combinations of nonlinear ODE models using Grbner Bases. Math Biosci. 2009;222(2):61–72. pmid:19735669
  8. 8. Nemcova J. Structural identifiability of polynomial and rational systems. Math Biosci. 2010;223(2):83–96. pmid:19913563
  9. 9. Saccomani MP, Audoly S, Bellu G, D’Angio L. Examples of testing global identifiability of biological and biomedical models with the DAISY software. Comput Biol Med. 2010;40(4):402–407. pmid:20185123
  10. 10. Lillacci G, Khammash M. Parameter estimation and model selection in computational biology. PLoS Comput Biol. 2010;6(3):e1000696. pmid:20221262
  11. 11. Chis O, Banga JR, Balsa-Canto E. GenSSI: a software toolbox for structural identifiability analysis of biological models. Bioinformatics. 2011;27(18):2610–2611. pmid:21784792
  12. 12. Anguelova M, Karlsson J, Jirstrand M. Minimal output sets for identifiability. Math Biosci. 2012;239(1):139–153. pmid:22609467
  13. 13. Kreutz C, Raue A, Timmer J. Likelihood based observability analysis and confidence intervals for predictions of dynamic models. BMC Syst Biol. 2012;6:120. pmid:22947028
  14. 14. Raue A, Karlsson J, Saccomani MP, Jirstrand M, Timmer J. Comparison of approaches for parameter identifiability analysis of biological systems. Bioinformatics. 2014;30(10):1440–1448. pmid:24463185
  15. 15. Janzen DL, Bergenholm L, Jirstrand M, Parkinson J, Yates J, Evans ND, et al. Parameter Identifiability of Fundamental Pharmacodynamic Models. Front Physiol. 2016;7:590. pmid:27994553
  16. 16. Villaverde AF, Barreiro A, Papachristodoulou A. Structural Identifiability of Dynamic Systems Biology Models. PLoS Comput Biol. 2016;12(10):e1005153. pmid:27792726
  17. 17. Sontag ED. Mathematical Control Theory. Deterministic Finite-Dimensional Systems. vol. 6 of Texts in Applied Mathematics. 2nd ed. New York: Springer-Verlag; 1998.
    • 18. Evans L. Partial Differential Equations. Providence: American Mathematical Society; 1998.
      • 19. Wang Y, Sontag ED. Generating series and nonlinear systems: analytic aspects, local realizability, and i/o representations. Forum Math. 1992;4(3):299–322.
      • 20. Wang Y, Sontag ED. Algebraic differential equations and rational control systems. SIAM J Control Optim. 1992;30(5):1126–1149.
      • 21. Sontag ED. Spaces of observables in nonlinear control. In: Proceedings of the International Congress of Mathematicians, Vol. 1, 2 (Zürich, 1994). Basel: Birkhäuser; 1995. p. 1532–1545.
        • 22. Fliess M. Fonctionnelles causales non linéaires et indéterminées non commutatives. Bull Soc Math France. 1981;109:3–40.
        • 23. Fliess M. Réalisation locale des systèmes non linéaires, algèbres de lie filtrées transitives et séries génératrices non commutatives. Invent Math. 1983;71:521–537.
        • 24. Crouch PE, Lamnabhi-Lagarrigue F. State space realizations of nonlinear systems defined by input-output differential equations. In: Bensoussan A, Lions JL, editors. Analysis and Optimization of Systems. Berlin, Heidelberg: Springer; 1988. p. 138–149. https://doi.org/10.1007/BFb0042209
          • 25. Conte G, Moog GH, Perdon A. Un théorème sur la représentation entrée-sortie d’un système non linéaire. C R Acad Sci Paris. 1988;307:363–366.
          • 26. Diop S. Elimination in control theory. Mathematics of Control, Signals and Systems. 1991;4(1):17–32.
          • 27. Wang Y, Sontag ED. Orders of input/output differential equations and state-space dimensions. SIAM J Control Optim. 1995;33(4):1102–1126.
          • 28. Gutenkunst RN, Waterfall JJ, Casey FP, Brown KS, Myers CR, Sethna JP. Universally sloppy parameter sensitivities in systems biology models. PLoS Comput Biol. 2007;3(10):1871–1878. pmid:17922568
          • 29. Chaves M, Sengupta A, Sontag ED. Geometry and topology of parameter space: investigating measures of robustness in regulatory networks. J of Mathematical Biology. 2009;59:315–358.
          • 30. Dayarian A, Chaves M, Sontag ED, Sengupta A. Shape, Size and Robustness: Feasible Regions in the Parameter Space of Biochemical Networks. PLoS Computational Biology. 2009;5:e10000256.
          • 31. Chis OT, Villaverde AF, Banga JR, Balsa-Canto E. On the relationship between sloppiness and identifiability. Math Biosci. 2016;282:147–161. pmid:27789352
          • 32. Isidori A. Nonlinear Control Systems II. Springer-Verlag, London; 1999.
            • 33. Sontag ED. Bilinear realizability is equivalent to existence of a singular affine differential I/O equation. Systems Control Lett. 1988;11(3):181–187.
            • 34. Sussmann HJ. Existence and Uniqueness of minimal realizations of nonlinear systems. Math Systems Theory. 1977;10:263–284.