Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states

Polina Lakrisenko; Dilan Pathirana; Daniel Weindl; Jan Hasenauer

doi:10.1371/journal.pone.0312148

Abstract

Estimating parameters of dynamic models from experimental data is a challenging, and often computationally-demanding task. It requires a large number of model simulations and objective function gradient computations, if gradient-based optimization is used. In many cases, steady-state computation is a part of model simulation, either due to steady-state data or an assumption that the system is at steady state at the initial time point. Various methods are available for steady-state and gradient computation. Yet, the most efficient pair of methods (one for steady states, one for gradients) for a particular model is often not clear. In order to facilitate the selection of methods, we explore six method pairs for computing the steady state and sensitivities at steady state using six real-world problems. The method pairs involve numerical integration or Newton’s method to compute the steady-state, and—for both forward and adjoint sensitivity analysis—numerical integration or a tailored method to compute the sensitivities at steady-state. Our evaluation shows that all method pairs provide accurate steady-state and gradient values, and that the two method pairs that combine numerical integration for the steady-state with a tailored method for the sensitivities at steady-state were the most robust, and amongst the most computationally-efficient. We also observed that while Newton’s method for steady-state computation yields a substantial speedup compared to numerical integration, it may lead to a large number of simulation failures. Overall, our study provides a concise overview across current methods for computing sensitivities at steady state. While our study shows that there is no universally-best method pair, it also provides guidance to modelers in choosing the right methods for a problem at hand.

Citation: Lakrisenko P, Pathirana D, Weindl D, Hasenauer J (2024) Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states. PLoS ONE 19(10): e0312148. https://doi.org/10.1371/journal.pone.0312148

Editor: Joshua Kiddy K. Asamoah, Kwame Nkrumah University of Science and Technology, GHANA

Received: June 11, 2024; Accepted: October 2, 2024; Published: October 23, 2024

Copyright: © 2024 Lakrisenko et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data for this study are publicly available from the Zenodo repository (https://doi.org/10.5281/zenodo.11147151).

Funding: This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, https://www.dfg.de/) under Germany’s Excellence Strategy (EXC 2047—390685813, EXC 2151—390873048; J.H.) and under the project IDs 432325352 – SFB 1454 and 443187771 – AMICI (D.P.), by the German Federal Ministry of Education and Research (BMBF, https://www.bmbf.de) within the e:Med funding scheme (junior research alliance PeriNAA, grant no. 01ZX1916A; D.W., P.L.), and by the University of Bonn (https://www.uni-bonn.de, via the Schlegel Professorship of J.H.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

At every step of dynamic modeling of biochemical reaction networks, from model formulation and calibration to model quality assessment, one has to choose from a variety of methods and approaches [1, 2]. While some decisions can be made relatively quickly based on the scope of the problem and available data, other decisions require an assessment of alternatives on a trial-and-error basis. Therefore, studies that benchmark different algorithms on diverse problems help modelers make informed decisions and recognize potential problems. Many benchmark studies have been published that compare methods available for different modeling steps, such as numerical integration methods for ordinary differential equation (ODE) models [3], computation of derivatives of differential equation solutions [4], solving differential equations and parameter estimation in Julia [5], optimization using different objective functions and approaches to link data to model simulations [6], and for parameter estimation in large kinetic models [7]. Collections of benchmark problems for model simulation or parameter estimation are valuable resources for such method evaluations [8, 9].

Model calibration, i.e. estimation of model parameters from experimental data, is a particularly challenging step that requires a large number of model simulations and, thus, can be computationally demanding [2]. Therefore, for working with larger models in particular, choosing the most efficient model simulation method is important for keeping computations tractable.

In many applications, steady-state computation is required during a model simulation. For example, in a benchmark collection of mathematical models with experimental data curated from the systems biology literature, 22% (7/32) require steady-state computation [9]. One can distinguish two cases: pre-equilibration, where the system is assumed to be in a steady state at the initial time point, then is perturbed and enters a dynamic state; and post-equilibration, which is required if steady-state data is available, where the system is assumed to reach a steady state having started from a dynamic state [10]. These two cases can occur together in the same data set.

Steady-state constraints constitute important prior knowledge, or assumptions, and allow one to reduce the search space during optimization. However, these constraints also pose a challenge, as they often come in a form of nonlinear equalities that can cause efficiency and convergence problems and require special treatment [11–13]. To eliminate steady-state constraints, various approaches have been developed to derive analytical expressions for steady states [13–16]. Furthermore, manifold optimization techniques have been adapted for steady-state constraints [11]. Still, the derivation of analytical expressions for steady states is only applicable to some application problems and manifold optimization techniques can be difficult to tune. Accordingly, most state-of-the-art implementations still rely on the numerical computation of steady states for given parameters. Often, numerical integration is run until a steady state is reached. An alternative is to directly solve for the steady state by using Newton’s method (and variations thereof) to find the zeroes of the ODE right-hand-side. However, the choice is non-trivial because Newton’s method can be 100 times faster, but numerical simulation is much more robust [12].

As the steady state is usually parameter dependent, multiple methods have also been introduced to compute the sensitivity of the steady state and the objective function gradient. The most widely used approaches for computing local sensitivities include finite differences, automatic differentiation, forward sensitivity analysis (FSA), and adjoint sensitivity analysis (ASA). These methods have been compared for dynamic states [4, 17]. Also, for the computation of steady-state sensitivities, tailored FSA and ASA methods have been implemented that avoid numerical integration and require solving a system of linear equations instead [10, 13]. Yet, a comprehensive comparison of steady-state sensitivity analysis methods and pairs of steady-steady computation and sensitivity analysis schemes has not been performed.

In this study, we consider steady-state and sensitivity-at-steady-state methods that are broadly applicable, i.e. methods that: do not rely on the model structure, are applicable to both pre- and post-equilibration, and work both with steady states alone and in combination with dynamic states. We study six different pairs of methods and apply them to six real-world problems, and investigate how each method pair affects computation time and failure rate. Overall, we find that Newton’s method should be used with caution, as it is less robust than numerical integration. We conclude that the two method pairs that combine numerical integration for the steady state with a tailored method for the sensitivities at steady-state stand out favorably in terms of overall efficiency and robustness.

Methods

We consider dynamic models of biochemical processes that can be described by systems of ODEs and are defined as initial value problems (1) which determine the evolution in time, denoted by , of a vector of state variables , representing, e.g., abundances of biochemical species. The vector denotes unknown parameters. The vector denotes known inputs that can specify experimental conditions by directly specifying model component values that represent, for example, various external stimuli such as growth factors, receptor ligands, drug administration regimes or changes in cell culture medium composition. The vector field of the ODE model is , which is assumed to be Lipschitz-continuous with respect to x, and the initial condition at t = t₀ is . Depending on a specific application, domains of x, θ, and u might be further restricted to ensure plausibility. In systems biology applications, both state variables and unknown parameters are often constrained to be non-negative, as x, for example, might describe species concentrations and θ might denote reaction rate constants. This restriction, however, is not necessary for the methods described in this study.

The connection between data and the model is provided by the observation map and the vector of observables, The experimental data for these observables are denoted by , with measurement time points t_j and measurements .

Maximum likelihood estimation with forward or adjoint sensitivity analysis

Frequently, the only way to obtain appropriate values for parameters θ is to estimate them from experimental data . This can be done, for example, using the maximum likelihood approach, where the likelihood () is the probability of the measurements given the model [2]. In the case of independent, normally distributed measurements, the negative log-likelihood is (2) where are measurements with additive noise , σ_ij(θ) are standard deviations, and y_i(t_j, θ) are model simulations.

The maximum likelihood estimate of the parameters is the solution to the optimization problem This problem can be efficiently solved using gradient-based global and multi-start optimization methods [1, 7]. These methods require the computation of the objective function gradient, which can be achieved, e.g., by using the finite differences approach, FSA, or ASA. Both FSA and ASA are more reliable and efficient than finite differences, which can be numerically unstable [1, 17]. In the following, we describe both in detail and how they can be modified at steady state.

The gradient of the objective function (2) with respect to the parameters is given by where the sensitivity of the output y_i with respect to parameter θ_k can be computed by (3) The term is the vector of state sensitivities with respect to parameter θ_k, also denoted by . The state sensitivities can be tricky to compute, as x is only given as the solution of the ODE (1).

One can calculate state sensitivities by using FSA. By differentiating the equations (1) with respect to θ, one can get an ODE system governing state sensitivities : (4) where k = 1, …, n_θ and represents the Jacobian of the system (1)

Solving this ODE system yields state sensitivities, from which output sensitivities can be easily calculated by (3). Numerical integration of the state sensitivities ODEs (4) is often coupled with integration of the system (1), which improves computational efficiency [18, 19].

An alternative to FSA is ASA, which allows for the calculation of the objective function gradient while avoiding calculation of output sensitivities [17]. It is achieved by introducing the adjoint state , such that ∀j = n_t…, 1, p on interval (t_j−1, t_j] satisfies the backward differential equation (5) with boundary values The adjoint state is used to reformulate the gradient (6) in which ∂x₀/∂θ_k denotes the sensitivity of the initial state with respect to parameter θ_k. As was shown in [17], this approach is computationally more efficient than finite differences or FSA for models with high numbers of state variables or parameters.

Steady-state computation

In this study, we focus on cases in which evaluating the objective function involves the computation of a parameter-dependent steady state x*(θ, u): (7) This is rather common, as before and after perturbations the biological systems are often in an asymptotically-stable steady state or approach one. Exceptions are oscillating and chaotic systems.

There are two typical cases [10]:

pre-equilibration, where at t = t₀, x is the steady state associated with input u^e; and
post-equilibration, where as t → ∞, x approaches the steady state x*(θ, u) associated with input u.

In both cases, according to Picard-Lindelöf theorem, the system (1) has a unique steady state for each initial condition (x₀(θ, u)) [20].

To compute the steady state of the ODE system (1), one must solve for x the equation which is generally nonlinear and a closed-form solution is not available. The most straightforward numerical approach is to integrate the system of ODEs until time derivatives become sufficiently small. For example, the integration can be performed until condition (8) is fulfilled, where “rtol” and “atol” denote relative and absolute tolerances, respectively.

Another well-known approach is Newton’s method, where the approximation x^l of the solution is computed iteratively by formula until convergence criterion, e.g. (8), is fulfilled. The method is very sensitive to the initial guess x⁰ and may not converge if started too far away from the solution. One can extend the radius of convergence by modifying the step length: (9) where γ ≤ 1 [12]. The factor γ is increased if the error ( for condition (8)) is reduced and decreased otherwise. When successful, Newton’s method can be orders of magnitude faster than numerical integration [12].

Objective function gradient computation at steady state

If either pre- or post-equilibration is required, it affects objective function gradient computation:

Pre-equilibration. In this case, the initial condition is defined as the steady state corresponding to a pre-equilibration input u^e: x₀(θ, u) = x*(θ, u^e), where x* is the steady state of Here, x* affects x(t_j, θ, u) and, consequently, the simulated observable values y(t_j, θ, u). Moreover, it influences sensitivity computation both for FSA and ASA, namely, the initial condition in (4) and the last term in (6).
Post-equilibration. In this case one needs to account for steady-state measurements in which y_i*(θ, u) = h_i(x*(θ, u), θ, u) denotes the values of the observable y_i at the steady state (7), and denotes the noise variance. In practice, post-equilibration measurements arise when measuring a real system that appears to have reached a steady state, e.g. long after the last input change occurred. It is possible that both steady-state and time-course measurements are available. Therefore, in this study we consider the most general case.
The steady-state measurements need to be included in the objective function, and its gradient,

Tailored methods for handling steady states

The steady-state condition facilitates alternative approaches for sensitivity-at-steady-state computation. In this section we describe how it can be used to simplify gradient computation both in FSA and ASA.

As for all k = 1, …, n_θ, the forward sensitivities ODE system (4) simplifies to (10) Therefore, if the Jacobian of the system (1) has full rank, the system of n_xn_θ FSA ODEs (4) simplifies to a system of the same number of linear algebraic equations [11, 13]. This approach is applicable to both the pre- and post-equilibration cases.

If the adjoint sensitivities approach is used, computations can be simplified as well. In the pre-equilibration case, one can extend ASA to the pre-equilibration time interval [−t′, t₀], where −t′ is such that the steady state has already been achieved. Then objective function gradient is given by (11) On this time interval the system (5) is a linear ODE system with constant matrix. At t = −t′ this system is at steady state p = 0, therefore, the last term in (11) vanishes. The third term in (11) reduces to a matrix-vector product where p_integral can be computed as the solution of the system of linear algebraic equations (12)

In the post-equilibration case the gradient is given by (13) where t′′ is a time point at which time derivatives are negligible and x(t) is a good approximation of the steady-state x*.

The sixth term in (13) reduces to a matrix-vector product where p_integral can be computed as the solution to (14) The tailored ASA approach for both pre-equilibration and post-equilibration is described in more detail in [10].

One should note that, if the (transposed) Jacobian is not full rank, the tailored approaches are not applicable. In this case, none of the systems (10), (12) and (14) has a unique solution and standard integration must be carried out. A possible reason for a singular Jacobian is the presence of conserved quantities in the model [21]. Various algorithms are available for identification of conserved quantities that facilitate applicability of the tailored methods [22–24].

Implementation

The six test problems (Table 1) were downloaded from the PEtab benchmark collection [25]. Model simulations and sensitivity computations were performed using AMICI [26]. For numerical integration, an algorithm based on the backward-differentiation formula (BDF) was used. Optimization was performed using the Fides trust-region optimizer [27] via pyPESTO [28]. Simulator options and further details are provided in Section “Implementation” of the S1 Text.

Download:

Table 1. Overview of the models and optimization problems considered in this study.

Here, n_x is the number of state variables, n_θ is the number of unknown parameters, is the number of data points, n_u is the number of inputs u and “x*type” is the equilibration type.

https://doi.org/10.1371/journal.pone.0312148.t001

Results

The combination of computing steady states by numerical integration or by Newton’s method with FSA, ASA, or their tailored-to-steady-state versions gives rise to eight different method pairs (Fig 1). In this study, we consider the six pairs that are feasible and meaningful (in the following, we use a shorthand notation to refer to the method pairs, where each method pair is represented by a tuple of steady-state and sensitivities-at-steady-state method, ∫ signifies numerical integration, whereas ϕ signifies solving an algebraic equation):

numerical integration for both steady-state and sensitivities computation ( and );
numerical integration for steady-state computation combined with the sensitivities computation approach tailored to steady-state case ( and ); and
Newton’s method for steady-state computation combined with the sensitivities computation approach tailored to steady-state case ( and ).

Download:

Fig 1. Method pairs considered in this study.

Columns correspond to two methods for steady-state computation: numerical integration and Newton’s method. Rows correspond to different approaches for sensitivities computation at steady state: both for forward sensitivity analysis and adjoint sensitivity analysis one can either use numerical integration or an approach tailored to the steady-state case that only requires solving a linear system of equations. Two method pairs are not feasible or not meaningful (see main text). Each method pair is represented by a tuple of steady-state and sensitivities-at-steady-state method, ∫ signifies numerical integration, whereas ϕ signifies solving an algebraic equation.

https://doi.org/10.1371/journal.pone.0312148.g001

Note that with FSA, it is not possible to use Newton’s method for steady-state computation together with numerical integration approach for sensitivities, as numerical integration of the state sensitivities ODEs (4) is coupled with integration of the system (1) [18]. While the ASA variant of this method pair is feasible, it is not practical, as the tailored method for sensitivities is always applicable when Newton’s method can be used for steady-state computation. Therefore, we do not consider these two method pairs in this study.

As it is not immediately clear which of the six method pairs are best in practice, we assess their performance on a selection of real-world problems of varying complexity that required steady-state computation. These models describe epigenetic [29, 30] and signal transduction processes [31–34]. Four of the problems possess pre-equilibration constraints and two possess post-equilibration constraints. The number of states ranges from 6 to 1396 and the number of parameters from 9 to 4088. For details we refer to the information on Table 1. The original Blasi, Brännmark and Isensee models contain conserved quantities that lead to a rank-deficient Jacobian, which is not compatible with the Newton’s method and the tailored sensitivities-at-steady-state methods. The conserved quantities were removed prior to the analysis details are given in Section “Conserved quantities” of the S1 Text.

We apply the six method pairs on these six real-world problems to assess robustness, accuracy, and speed, which are crucial for gradient-based parameter estimation.

Numerical integration is more robust than Newton’s method

Model analysis, parameter estimation and related tasks require robust model evaluations. To assess whether this is achieved by the different pairs of steady-state and sensitivity-at-steady-state computation methods, we evaluated their failure rates. Therefore, for each model, we sampled 1000 parameter vectors log-uniformly within the parameter bounds specified in the PEtab files, then performed simulation and gradient computation with each vector. Only for the Fröhlich model the case was not considered since it has already been shown to be computationally prohibitive [32]. We consider a model simulation for a given parameter vector as failed if for any input u the simulation (a) fails due to any numerical errors, or (b) returns negative state variables in the solution. The latter accounts for the fact that the state variables of all six real-world problems represent physically non-negative quantities.

We observed a broad range of failure rates. For the Blasi and Zheng models, we did not encounter any failures for the 1000 parameter vectors. For the Brännmark, Isensee and Weber models, the failure rate for the six method pairs ranged from 31.5% with and to 48.8% with (Fig 2a), from 7.7% with to 54.3% with (Fig 2c), and from 22.4% with to 48.3% with (Fig 2d), respectively. The experimental conditions for the Weber model are described using a discontinuous function, which caused numerical problems in about 17% of the simulations for each of the six method pairs (S1 Fig in S1 Text). With the Brännmark model, we observed that for about 30% of sampled parameters simulations failed independently of the applied method (S1 Fig in S1 Text). This suggests that the model does not exhibit a steady state for those parameter sets. For the Fröhlich model, and failed for 100% of the parameter vectors, while the three remaining pairs had a failure rate less than 2.5% percent (Fig 2b).

Download:

Fig 2. Failure rates for different method pairs based on model simulations with 1000 randomly sampled parameter vectors.

We did not encounter any failures for the Blasi and Zheng models.

https://doi.org/10.1371/journal.pone.0312148.g002

The reason for the high failure rate with Newton’s method varied between models. The most common reason for all models is that the factor γ in (9) reached a lower bound while the condition (8) was not fulfilled, i.e. the steady state could not be computed because the initial guess was too far away from it, and even the modification described in (9) could not resolve the issue. For the Fröhlich model, Newton’s method failed with every parameter vector due to numerical errors in at least one of the n_u steady-state computations. Indeed, only 6.6% (9,454/143,000) of the attempts to compute a steady state using and did not have numerical errors. The hypothesis that the failure rate for Newton’s method depends on the number of state variables could not be confirmed based on the given set of problems—both the Fröhlich model with a large number of state variables and the Brännmark model with very few state variables exhibited high failure rates. For the Brännmark, Isensee and Weber models, some steady-state concentrations computed with Newton’s method had negative values, some of which were several orders of magnitude away from zero, which is not biologically feasible. This issue did not occur when numerical integration was used. For example, for the Fröhlich model, concentrations computed with Newton’s method had negative values for most inputs u simulations that did not have any numerical issues (9,437/9,454). In other words, only 0.01% of inputs u (17/143,000) could be simulated successfully with method pairs and .

In summary, this assessment revealed that numerical simulation allows for a higher fraction of successful simulations than Newton’s method.

All method pairs provide accurate steady-state and gradient computation

Given that a computation is successful, the most important factor is accuracy. To assess the accuracy, we compared the steady-state values obtained by applying the method pairs with the sampled parameter vectors.

For all successful simulations, the steady-state values computed using either numerical integration or Newton’s method were similar (S2, S3 Figs in S1 Text). Having confirmed that the different methods yielded approximately equal steady states, we assessed the agreement of computed objective function gradient values. For all models, the objective function gradient values computed using different approaches were similar (Fig 3 and S4 Fig in S1 Text).

Download:

Fig 3. Comparison of objective function gradients obtained from different method pairs.

The heatmaps show Pearson correlation coefficients between objective function gradient values computed with the six different method pairs. Data was log-transformed prior to the computation of the coefficients. Scatter plots visualize the difference between objective function gradient values for selected method pairs. Points on the diagonal indicate a good agreement, darker points indicate higher density. The maximum and median deviations were computed as defined Section “Accuracy of gradient computation” of the S1 Text.

https://doi.org/10.1371/journal.pone.0312148.g003

In summary, this assessment revealed that for successful simulations the results for different method pairs are highly comparable. We generally saw improved accuracy (increased similarity between method pairs) with stricter simulation tolerances. The minor differences between method pairs seen in Fig 3 may be resolved by tolerance tuning.

Using tailored sensitivity-at-steady-state methods significantly reduces simulation time

After ensuring the accuracy of the steady-state and gradient values, we assessed the computation time for different method pairs. For all models, the total simulation time is comprised of equilibration time (pre- or post-), including sensitivities computation, and some overhead (Fig 4a). Additionally, the Brännmark, Isensee, Weber and Zheng models had dynamic simulation time, because these problems also include dynamic-state measurements. Therefore, we considered again the successful simulations for the previously sampled 1000 parameter vectors, for which we recorded the total equilibration time, and the overall total simulation time (Fig 4b).

Download:

Fig 4. Comparison of total simulation times and equilibration times for different combinations of problems and steady-state sensitivity methods.

(a) Composition of total simulation time. The main components are, depending on the model, pre- or post-equilibration including sensitivities computation and dynamic simulation (only for Brännmark, Isensee, Weber and Zheng models). (b) Violin plots comparing simulation efficiency of the method pairs. The left side on the violin plots shows total simulation time, the right side—total equilibration time.

https://doi.org/10.1371/journal.pone.0312148.g004

The method pairs considered in this work address equilibration. Therefore, the effect of the method pairs on total equilibration time is substantial (Fig 4b, right half of the violins), but it doesn’t always translate to the total simulation time (Fig 4b, left half of the violins). The reason is that the difference in the considered method pairs relates only to equilibration time, but not the dynamic simulation, which is a part of model simulations for the Brännmark, Isensee, Weber and Zheng models. For example, for the Weber model the pre-equilibration speedup of over is 28.7, while the total simulation time speedup for the same method pairs is 4.2.

In most cases, using Newton’s method instead of numerical integration to compute steady states, as well as using tailored sensitivity-at-steady-state methods, reduced simulation time. Speedups of total simulation time ranged from 3.5 to 7.3 for over (1.2 to 3.6 for over ), from 1.05 to 1.2 for over (1 to 2.2 for over ). Our analysis shows that, while was the slowest approach for all problems, the fastest approach was model-dependent. For all models (except for the Fröhlich model) the fastest method pair was either or .

Using sensitivity-at-steady-state methods significantly reduces optimization time

Simulation efficiency is especially important when large numbers of simulations have to be performed. This is often the case during parameter estimation where thousands to millions of objective function evaluations, and thus, model simulations are required. Accordingly, total optimization time is comprised of the cumulative total simulation time over all optimization steps, the time taken by the optimizer, and any additional overhead (Fig 5a).

Download:

Fig 5. Optimization efficiency, total time vs. equilibration time.

(a) Composition of total optimization time, where n is the number of optimization steps. Multiple simulations are required during an optimization, each, depending on the model, might include equilibration and dynamic simulation. (b) Violin plots comparing optimization efficiency of the method pairs.

https://doi.org/10.1371/journal.pone.0312148.g005

Therefore, to assess how the choice of steady-state and sensitivity-at-steady-state methods affects optimization efficiency and robustness, we ran 1000 local optimizations for the Blasi, Brännmark, Isensee, Weber and Zheng models for each method pair. With the Fröhlich model, only 50 local optimizations were performed with a maximum of 30 optimizer steps, due to the need for massive computational resources. Furthermore, for the Fröhlich model we disregarded and due to high failure rates and due to inefficiency.

The assessment of the optimization results revealed that the number of optimization failures correlated with the number of simulation failures (S5 Fig in S1 Text) and all method pairs had comparable efficacy (S6 Fig in S1 Text). Therefore, we focused on comparing the method pairs efficiency. The speedups of compared to were 2.1, 5.6, 8.2, 8.6, 2.6 for Blasi, Brännmark, Isensee, Weber and Zheng models, respectively, and the speedups of compared to were 2.7, 1.7, 2.0, 1.7, 1.1 for Blasi, Brännmark, Isensee, Weber and Zheng models, respectively. The speedups of compared to were 2.0, 2.9, 3.3, 4.4, 3.5 for Blasi, Brännmark, Isensee, Weber and Zheng models, respectively, and compared to was 1.3, 1.2, 1.4, 2.0, 1.0, 1.4 times as fast for the Blasi, Brännmark, Isensee, Fröhlich, Weber and Zheng models, respectively.

Overall, as with total simulation time, we observed that both using Newton’s method instead of numerical integration, and using tailored sensitivities-at-steady-state approaches, substantially reduces total equilibration time, i.e. cumulative equilibration time over all optimization steps, (Fig 5b, right half of the violin plots). The effect on total optimization time is not as pronounced, because the difference in methods is only in relation to simulation time, particularly equilibration time, but not the time taken in the optimizer itself, or other overhead. For most model, the fastest method pair was either and . One should keep in mind, however, that for some models these method pairs may not be applicable or result in many multi-start failures at the starting point. Fortunately, the more robust method pairs and are more efficient compared to and .

Discussion and conclusion

In this study, we explored six combinations of methods for computing steady states and sensitivities at steady state, and evaluated their accuracy, efficiency and robustness based on six real-world problems. All those methods have been introduced before, but have not yet been evaluated on a joint set of problems [1, 10, 12, 17]. The considered methods are widely applicable and do not require any specific model properties. However, it should be taken into account that the standard FSA approach is the least scalable and becomes prohibitively computationally expensive for large-scale models, while the more scalable ASA approach avoids computation of state sensitivities that may be needed for further analysis. Our results indicate that tailored sensitivities computation methods are more robust and more efficient than the standard FSA or ASA approaches that rely on numerical integration. In contrast, we found that using numerical integration for steady-state computation was more robust than using Newton’s method. However, when Newton’s method worked reliably, it was more than twice as fast.

An important issue we encountered was that computing steady states using Newton’s method led to non-physical solutions, in our case negative concentrations. This is a general problem with Newton’s method and alternative approaches able to respect non-negativity constraints, or better general constraints on the steady-state, while maintaining much of the efficiency would be valuable. However, we are not aware of any such implementation in the context of ODE model simulators.

Different subsets of the methods evaluated here have also been investigated in earlier studies [10–12]. Using Newton’s method for steady-state computation was demonstrated and evaluated in [12]. While we also observed a speedup when using Newton’s method, we found it only to be roughly twice as fast, instead of 100 times as fast as in the earlier study for the same model. This might be explained best by the comparatively small distances of the initial states from the steady state in [12]. The high failure rate we observed for the Fröhlich model is in line with the earlier observations. The tailored ASA method for computation of steady-state sensitivities () was introduced and compared to the standard ASA approach () in [10]. Simulation and optimization speedup values we observed in this study are in line with the previous results for the same models.

While we would like to derive general guidelines for choosing the best method for a given model, we have to note that our results might not generalize fully. Our study showed that the impact of the explored approaches on failure rates and speedup are quite problem-dependent. We based our analysis on six of the seven models available in the PEtab benchmark collection [9]. Further extension of the model collection, in particular by models that require steady-state computation, would allow for a more comprehensive analysis. Furthermore, much of the robustness and efficiency will depend on the exact implementation of the different algorithms and the chosen hyperparameters. It has been demonstrated elsewhere in the context of optimization, how different implementations of the supposedly same algorithm can perform quite differently [27]. Our results are based on AMICI [26]. We were not able to find another tool that would have allowed us to compare all the different methods explored here.

In summary, we illustrated different approaches for computing sensitivities for ODE simulations involving steady-states and demonstrated their advantages and disadvantages. While we focused here on systems biology applications, we think that these results mostly transfer to other fields, and can give modelers at least a first orientation on which methods to choose. Our comparison suggests one should be cautious when using Newton’s method for steady-state computation, as it might lead to a high number of failures or unphysical solutions. On the other hand, using tailored sensitivity-at-steady-state methods ( or ) is advantageous as these methods are robust and lead to a significant computational speedup.

Supporting information

S1 Text. Supplementary notes on accuracy assessment, implementation details, conserved quantities, as well as supplementary figures.

https://doi.org/10.1371/journal.pone.0312148.s001

(PDF)

References

1. Raue A, Schilling M, Bachmann J, Matteson A, Schelker M, Schelke M, et al. Lessons learned from quantitative dynamical modeling in systems biology. PloS one. 2013;8:e74335. pmid:24098642
- View Article
- PubMed/NCBI
- Google Scholar
2. Villaverde AF, Pathirana D, Fröhlich F, Hasenauer J, Banga JR. A protocol for dynamic model calibration. Briefings in Bioinformatics. 2021.
- View Article
- Google Scholar
3. Städter P, Schälte Y, Schmiester L, Hasenauer J, Stapor PL. Benchmarking of numerical integration methods for ODE models of biological systems. Scientific Reports. 2021;11(1):2696. pmid:33514831
- View Article
- PubMed/NCBI
- Google Scholar
4. Rackauckas C, Ma Y, Dixit V, Guo X, Innes M, Revels J, et al. A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions. CoRR. 2018;abs/1812.01892.
- View Article
- Google Scholar
5. Rackauckas C, Nie Q. DifferentialEquations.jl—A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia. The Journal of Open Research Software. 2017;5(1).
- View Article
- Google Scholar
6. Degasperi A, Fey D, Kholodenko BN. Performance of objective functions and optimisation procedures for parameter estimation in system biology models. npj Systems Biology and Applications. 2017;3(1):20. pmid:28804640
- View Article
- PubMed/NCBI
- Google Scholar
7. Villaverde AF, Fröhlich F, Weindl D, Hasenauer J, Banga JR. Benchmarking optimization methods for parameter estimation in large kinetic models. Bioinformatics. 2018;35(5):830–838.
- View Article
- Google Scholar
8. Villaverde AF, Henriques D, Smallbone K, Bongard S, Schmid J, Cicin-Sain D, et al. BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Systems Biology. 2015;9(1):8. pmid:25880925
- View Article
- PubMed/NCBI
- Google Scholar
9. A collection of mathematical models with experimental data in the PEtab format; 2022. Available from: https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/.
10. Lakrisenko P, Stapor P, Grein S, Paszkowski L, Pathirana D, Fröhlich F, et al. Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks. PLOS Computational Biology. 2023;19(1):1–19. pmid:36595539
- View Article
- PubMed/NCBI
- Google Scholar
11. Fiedler A, Raeth S, Theis FJ, Hausser A, Hasenauer J. Tailored parameter optimization methods for ordinary differential equation models with steady-state constraints. BMC Systems Biology. 2016;10(1). pmid:27549154
- View Article
- PubMed/NCBI
- Google Scholar
12. Terje Lines G, Paszkowski L, Schmiester L, Weindl D, Stapor P, Hasenauer J. Efficient computation of steady states in large-scale ODE models of biochemical reaction networks. IFAC-PapersOnLine. 2019;52(26):32–37.
- View Article
- Google Scholar
13. Rosenblatt M, Timmer J, Kaschek D. Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models. Frontiers in Cell and Developmental Biology. 2016;4. pmid:27243005
- View Article
- PubMed/NCBI
- Google Scholar
14. Chou KC. Applications of graph theory to enzyme kinetics and protein folding kinetics. Biophysical Chemistry. 1990;35(1):1–24. pmid:2183882
- View Article
- PubMed/NCBI
- Google Scholar
15. Feliu E, Wiuf C. Variable Elimination in Chemical Reaction Networks with Mass-Action Kinetics. SIAM Journal on Applied Mathematics. 2012;72(4):959–981. pmid:22311196
- View Article
- PubMed/NCBI
- Google Scholar
16. Loriaux PM, Tesler G, Hoffmann A. Characterizing the Relationship between Steady State and Response Using Analytical Expressions for the Steady States of Mass Action Models. PLoS Computational Biology. 2013;9(2):e1002901. pmid:23509437
- View Article
- PubMed/NCBI
- Google Scholar
17. Fröhlich F, Kaltenbacher B, Theis FJ, Hasenauer J. Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks. PLOS Computational Biology. 2017;13(1):1–18. pmid:28114351
- View Article
- PubMed/NCBI
- Google Scholar
18. Leis JR, Kramer MA. The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans Math Softw. 1988;14(1):45–60.
- View Article
- Google Scholar
19. Hindmarsh AC, Brown PN, Grant KE, Lee SL, Serban R, Shumaker DE, et al. SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. ACM Trans Math Softw. 2005;31(3):363–396.
- View Article
- Google Scholar
20. Coddington EA, Levinson N. Theory of Ordinary Differential Equations; 1955. Available from: https://api.semanticscholar.org/CorpusID:122952679.
- View Article
- Google Scholar
21. Oellerich T, Emelianenko M, Liotta L, Araujo RP. Biological networks with singular Jacobians: their origins and adaptation criteria. bioRxiv. 2021.
- View Article
- Google Scholar
22. Contejean E, Devie H. An Efficient Incremental Algorithm for Solving Systems of Linear Diophantine Equations. Information and Computation. 1994;113(1):143–172.
- View Article
- Google Scholar
23. De Martino A, De Martino D, Mulet R, Pagnani A. Identifying All Moiety Conservation Laws in Genome-Scale Metabolic Networks. PLOS ONE. 2014;9(7):1–11. pmid:24988199
- View Article
- PubMed/NCBI
- Google Scholar
24. Pasechnik D. On computing Hilbert bases via the Elliot-MacMahon algorithm. Theoretical Computer Science. 2001;263.
- View Article
- Google Scholar
25. Schmiester L, Schälte Y, Bergmann FT, Camba T, Dudkin E, Egert J, et al. PEtab—Interoperable specification of parameter estimation problems in systems biology. PLOS Computational Biology. 2021;17(1):1–10. pmid:33497393
- View Article
- PubMed/NCBI
- Google Scholar
26. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski L, Lines GT, et al. AMICI: high-performance sensitivity analysis for large ordinary differential equation models. Bioinformatics. 2021;37(20):3676–3677. pmid:33821950
- View Article
- PubMed/NCBI
- Google Scholar
27. Fröhlich F, Sorger PK. Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models. PLOS Computational Biology. 2022;18(7):1–28. pmid:35830470
- View Article
- PubMed/NCBI
- Google Scholar
28. Schälte Y, Fröhlich F, Jost PJ, Vanhoefer J, Pathirana D, Stapor P, et al. pyPESTO: a modular and scalable tool for parameter estimation for dynamic models. Bioinformatics. 2023;39(11):btad711. pmid:37995297
- View Article
- PubMed/NCBI
- Google Scholar
29. Blasi T, Feller C, Feigelman J, Hasenauer J, Imhof A, Theis FJ, et al. Combinatorial Histone Acetylation Patterns Are Generated by Motif-Specific Reactions. Cell Syst. 2016;2(1):49–58. pmid:27136689
- View Article
- PubMed/NCBI
- Google Scholar
30. Zheng Y, Sweet SMM, Popovic R, Martinez-Garcia E, Tipton JD, Thomas PM, et al. Total kinetic analysis reveals how combinatorial methylation patterns are established on lysines 27 and 36 of histone H3. Proceedings of the National Academy of Sciences. 2012;109(34):13549–13554. pmid:22869745
- View Article
- PubMed/NCBI
- Google Scholar
31. Brännmark C, Palmér R, Glad ST, Cedersund G, Strålfors P. Mass and Information Feedbacks through Receptor Endocytosis Govern Insulin Signaling as Revealed Using a Parameter-free Modeling Framework. Journal of Biological Chemistry. 2010;285(26):20171–20179. pmid:20421297
- View Article
- PubMed/NCBI
- Google Scholar
32. Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient Parameter Estimation Enables the Prediction of Drug Response Using a Mechanistic Pan-Cancer Pathway Model. Cell Systems. 2018;7(6):567–579.e6. pmid:30503647
- View Article
- PubMed/NCBI
- Google Scholar
33. Isensee J, Kaufholz M, Knape MJ, Hasenauer J, Hammerich H, Gonczarowska-Jorge H, et al. PKA-RII subunit phosphorylation precedes activation by cAMP and regulates activity termination. Journal of Cell Biology. 2018;217(6):2167–2184. pmid:29615473
- View Article
- PubMed/NCBI
- Google Scholar
34. Weber P, Hornjik M, Olayioye MA, Hausser A, Radde NE. A computational model of PKD and CERT interactions at the trans-Golgi network of mammalian cells. BMC Systems Biology. 2015;9(1):9. pmid:25889812
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Raue A, Schilling M, Bachmann J, Matteson A, Schelker M, Schelke M, et al. Lessons learned from quantitative dynamical modeling in systems biology. PloS one. 2013;8:e74335. pmid:24098642
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Villaverde AF, Pathirana D, Fröhlich F, Hasenauer J, Banga JR. A protocol for dynamic model calibration. Briefings in Bioinformatics. 2021.
View Article
Google Scholar

[6] View Article

[7] Google Scholar

[ref3] 3. Städter P, Schälte Y, Schmiester L, Hasenauer J, Stapor PL. Benchmarking of numerical integration methods for ODE models of biological systems. Scientific Reports. 2021;11(1):2696. pmid:33514831
View Article
PubMed/NCBI
Google Scholar

[9] View Article

[10] PubMed/NCBI

[11] Google Scholar

[ref4] 4. Rackauckas C, Ma Y, Dixit V, Guo X, Innes M, Revels J, et al. A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions. CoRR. 2018;abs/1812.01892.
View Article
Google Scholar

[13] View Article

[14] Google Scholar

[ref5] 5. Rackauckas C, Nie Q. DifferentialEquations.jl—A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia. The Journal of Open Research Software. 2017;5(1).
View Article
Google Scholar

[16] View Article

[17] Google Scholar

[ref6] 6. Degasperi A, Fey D, Kholodenko BN. Performance of objective functions and optimisation procedures for parameter estimation in system biology models. npj Systems Biology and Applications. 2017;3(1):20. pmid:28804640
View Article
PubMed/NCBI
Google Scholar

[19] View Article

[20] PubMed/NCBI

[21] Google Scholar

[ref7] 7. Villaverde AF, Fröhlich F, Weindl D, Hasenauer J, Banga JR. Benchmarking optimization methods for parameter estimation in large kinetic models. Bioinformatics. 2018;35(5):830–838.
View Article
Google Scholar

[23] View Article

[24] Google Scholar

[ref8] 8. Villaverde AF, Henriques D, Smallbone K, Bongard S, Schmid J, Cicin-Sain D, et al. BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Systems Biology. 2015;9(1):8. pmid:25880925
View Article
PubMed/NCBI
Google Scholar

[26] View Article

[27] PubMed/NCBI

[28] Google Scholar

[ref9] 9. A collection of mathematical models with experimental data in the PEtab format; 2022. Available from: https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/.

[ref10] 10. Lakrisenko P, Stapor P, Grein S, Paszkowski L, Pathirana D, Fröhlich F, et al. Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks. PLOS Computational Biology. 2023;19(1):1–19. pmid:36595539
View Article
PubMed/NCBI
Google Scholar

[31] View Article

[32] PubMed/NCBI

[33] Google Scholar

[ref11] 11. Fiedler A, Raeth S, Theis FJ, Hausser A, Hasenauer J. Tailored parameter optimization methods for ordinary differential equation models with steady-state constraints. BMC Systems Biology. 2016;10(1). pmid:27549154
View Article
PubMed/NCBI
Google Scholar

[35] View Article

[36] PubMed/NCBI

[37] Google Scholar

[ref12] 12. Terje Lines G, Paszkowski L, Schmiester L, Weindl D, Stapor P, Hasenauer J. Efficient computation of steady states in large-scale ODE models of biochemical reaction networks. IFAC-PapersOnLine. 2019;52(26):32–37.
View Article
Google Scholar

[39] View Article

[40] Google Scholar

[ref13] 13. Rosenblatt M, Timmer J, Kaschek D. Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models. Frontiers in Cell and Developmental Biology. 2016;4. pmid:27243005
View Article
PubMed/NCBI
Google Scholar

[42] View Article

[43] PubMed/NCBI

[44] Google Scholar

[ref14] 14. Chou KC. Applications of graph theory to enzyme kinetics and protein folding kinetics. Biophysical Chemistry. 1990;35(1):1–24. pmid:2183882
View Article
PubMed/NCBI
Google Scholar

[46] View Article

[47] PubMed/NCBI

[48] Google Scholar

[ref15] 15. Feliu E, Wiuf C. Variable Elimination in Chemical Reaction Networks with Mass-Action Kinetics. SIAM Journal on Applied Mathematics. 2012;72(4):959–981. pmid:22311196
View Article
PubMed/NCBI
Google Scholar

[50] View Article

[51] PubMed/NCBI

[52] Google Scholar

[ref16] 16. Loriaux PM, Tesler G, Hoffmann A. Characterizing the Relationship between Steady State and Response Using Analytical Expressions for the Steady States of Mass Action Models. PLoS Computational Biology. 2013;9(2):e1002901. pmid:23509437
View Article
PubMed/NCBI
Google Scholar

[54] View Article

[55] PubMed/NCBI

[56] Google Scholar

[ref17] 17. Fröhlich F, Kaltenbacher B, Theis FJ, Hasenauer J. Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks. PLOS Computational Biology. 2017;13(1):1–18. pmid:28114351
View Article
PubMed/NCBI
Google Scholar

[58] View Article

[59] PubMed/NCBI

[60] Google Scholar

[ref18] 18. Leis JR, Kramer MA. The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans Math Softw. 1988;14(1):45–60.
View Article
Google Scholar

[62] View Article

[63] Google Scholar

[ref19] 19. Hindmarsh AC, Brown PN, Grant KE, Lee SL, Serban R, Shumaker DE, et al. SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. ACM Trans Math Softw. 2005;31(3):363–396.
View Article
Google Scholar

[65] View Article

[66] Google Scholar

[ref20] 20. Coddington EA, Levinson N. Theory of Ordinary Differential Equations; 1955. Available from: https://api.semanticscholar.org/CorpusID:122952679.
View Article
Google Scholar

[68] View Article

[69] Google Scholar

[ref21] 21. Oellerich T, Emelianenko M, Liotta L, Araujo RP. Biological networks with singular Jacobians: their origins and adaptation criteria. bioRxiv. 2021.
View Article
Google Scholar

[71] View Article

[72] Google Scholar

[ref22] 22. Contejean E, Devie H. An Efficient Incremental Algorithm for Solving Systems of Linear Diophantine Equations. Information and Computation. 1994;113(1):143–172.
View Article
Google Scholar

[74] View Article

[75] Google Scholar

[ref23] 23. De Martino A, De Martino D, Mulet R, Pagnani A. Identifying All Moiety Conservation Laws in Genome-Scale Metabolic Networks. PLOS ONE. 2014;9(7):1–11. pmid:24988199
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref24] 24. Pasechnik D. On computing Hilbert bases via the Elliot-MacMahon algorithm. Theoretical Computer Science. 2001;263.
View Article
Google Scholar

[81] View Article

[82] Google Scholar

[ref25] 25. Schmiester L, Schälte Y, Bergmann FT, Camba T, Dudkin E, Egert J, et al. PEtab—Interoperable specification of parameter estimation problems in systems biology. PLOS Computational Biology. 2021;17(1):1–10. pmid:33497393
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref26] 26. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski L, Lines GT, et al. AMICI: high-performance sensitivity analysis for large ordinary differential equation models. Bioinformatics. 2021;37(20):3676–3677. pmid:33821950
View Article
PubMed/NCBI
Google Scholar

[88] View Article

[89] PubMed/NCBI

[90] Google Scholar

[ref27] 27. Fröhlich F, Sorger PK. Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models. PLOS Computational Biology. 2022;18(7):1–28. pmid:35830470
View Article
PubMed/NCBI
Google Scholar

[92] View Article

[93] PubMed/NCBI

[94] Google Scholar

[ref28] 28. Schälte Y, Fröhlich F, Jost PJ, Vanhoefer J, Pathirana D, Stapor P, et al. pyPESTO: a modular and scalable tool for parameter estimation for dynamic models. Bioinformatics. 2023;39(11):btad711. pmid:37995297
View Article
PubMed/NCBI
Google Scholar

[96] View Article

[97] PubMed/NCBI

[98] Google Scholar

[ref29] 29. Blasi T, Feller C, Feigelman J, Hasenauer J, Imhof A, Theis FJ, et al. Combinatorial Histone Acetylation Patterns Are Generated by Motif-Specific Reactions. Cell Syst. 2016;2(1):49–58. pmid:27136689
View Article
PubMed/NCBI
Google Scholar

[100] View Article

[101] PubMed/NCBI

[102] Google Scholar

[ref30] 30. Zheng Y, Sweet SMM, Popovic R, Martinez-Garcia E, Tipton JD, Thomas PM, et al. Total kinetic analysis reveals how combinatorial methylation patterns are established on lysines 27 and 36 of histone H3. Proceedings of the National Academy of Sciences. 2012;109(34):13549–13554. pmid:22869745
View Article
PubMed/NCBI
Google Scholar

[104] View Article

[105] PubMed/NCBI

[106] Google Scholar

[ref31] 31. Brännmark C, Palmér R, Glad ST, Cedersund G, Strålfors P. Mass and Information Feedbacks through Receptor Endocytosis Govern Insulin Signaling as Revealed Using a Parameter-free Modeling Framework. Journal of Biological Chemistry. 2010;285(26):20171–20179. pmid:20421297
View Article
PubMed/NCBI
Google Scholar

[108] View Article

[109] PubMed/NCBI

[110] Google Scholar

[ref32] 32. Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient Parameter Estimation Enables the Prediction of Drug Response Using a Mechanistic Pan-Cancer Pathway Model. Cell Systems. 2018;7(6):567–579.e6. pmid:30503647
View Article
PubMed/NCBI
Google Scholar

[112] View Article

[113] PubMed/NCBI

[114] Google Scholar

[ref33] 33. Isensee J, Kaufholz M, Knape MJ, Hasenauer J, Hammerich H, Gonczarowska-Jorge H, et al. PKA-RII subunit phosphorylation precedes activation by cAMP and regulates activity termination. Journal of Cell Biology. 2018;217(6):2167–2184. pmid:29615473
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref34] 34. Weber P, Hornjik M, Olayioye MA, Hausser A, Radde NE. A computational model of PKD and CERT interactions at the trans-Golgi network of mammalian cells. BMC Systems Biology. 2015;9(1):9. pmid:25889812
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

Figures

Abstract

Introduction

Methods

Maximum likelihood estimation with forward or adjoint sensitivity analysis

Steady-state computation

Objective function gradient computation at steady state

Tailored methods for handling steady states

Implementation

Results

Numerical integration is more robust than Newton’s method

All method pairs provide accurate steady-state and gradient computation

Using tailored sensitivity-at-steady-state methods significantly reduces simulation time

Using sensitivity-at-steady-state methods significantly reduces optimization time

Discussion and conclusion

Supporting information

S1 Text. Supplementary notes on accuracy assessment, implementation details, conserved quantities, as well as supplementary figures.

References