Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks

Polina Lakrisenko; Paul Stapor; Stephan Grein; Łukasz Paszkowski; Dilan Pathirana; Fabian Fröhlich; Glenn Terje Lines; Daniel Weindl; Jan Hasenauer

doi:10.1371/journal.pcbi.1010783

Abstract

Dynamical models in the form of systems of ordinary differential equations have become a standard tool in systems biology. Many parameters of such models are usually unknown and have to be inferred from experimental data. Gradient-based optimization has proven to be effective for parameter estimation. However, computing gradients becomes increasingly costly for larger models, which are required for capturing the complex interactions of multiple biochemical pathways. Adjoint sensitivity analysis has been pivotal for working with such large models, but methods tailored for steady-state data are currently not available. We propose a new adjoint method for computing gradients, which is applicable if the experimental data include steady-state measurements. The method is based on a reformulation of the backward integration problem to a system of linear algebraic equations. The evaluation of the proposed method using real-world problems shows a speedup of total simulation time by a factor of up to 4.4. Our results demonstrate that the proposed approach can achieve a substantial improvement in computation time, in particular for large-scale models, where computational efficiency is critical.

Author summary

Large-scale dynamical models are nowadays widely used for the analysis of complex processes and the integration of large-scale data sets. However, computational cost is often a bottleneck. Here, we propose a new gradient computation method that facilitates the parameterization of large-scale models based on steady-state measurements. The method can be combined with existing gradient computation methods for time-course measurements. Accordingly, it is an essential contribution to the environment of computationally efficient approaches for the study of large-scale screening and omics data, but not tailored to biological applications, and, therefore, also useful beyond the field of computational biology.

Citation: Lakrisenko P, Stapor P, Grein S, Paszkowski Ł, Pathirana D, Fröhlich F, et al. (2023) Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks. PLoS Comput Biol 19(1): e1010783. https://doi.org/10.1371/journal.pcbi.1010783

Editor: Attila Csikász-Nagy, Pázmány Péter Catholic University: Pazmany Peter Katolikus Egyetem, HUNGARY

Received: August 16, 2022; Accepted: December 1, 2022; Published: January 3, 2023

Copyright: © 2023 Lakrisenko et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All the presented approaches have been integrated into AMICI version 0.11.32, an open-source tool for efficient simulation and sensitivity analysis (https://github.com/AMICI-dev/AMICI). All code and data to reproduce the analysis has been deposited at https://doi.org/10.5281/zenodo.6963596.

Funding: This work was supported by the European Union’s Horizon 2020 research and innovation program (CanPathPro, https://cordis.europa.eu/project/id/686282; Grant No. 686282; P.S., G.T.L., L.P.), the German Federal Ministry of Education and Research (BMBF, https://www.bmbf.de) within the e:Med funding scheme (junior research alliance PeriNAA, grant no. 01ZX1916A; D.W., P.L.), the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, https://www.dfg.de/, Project ID 432325352, 443187771; S.G, D.P; EXC 2047 390873048, EXC 2151 390685813; J.H.), the Human Frontier Science Program (https://www.hfsp.org/, Grant no. LT000259/2019-L1; F.F.), and the National Cancer Institute (https://www.cancer.gov, Grant no. U54-CA225088; F.F.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

This is a PLOS Computational Biology Methods paper.

Introduction

Ordinary differential equation (ODE) models are widely used to describe the dynamics of biochemical processes such as signalling [1–3], metabolism [4, 5] or gene regulation [6, 7]. These models can capture the mechanistic details of interactions between biochemical species, aggregate current knowledge and integrate heterogeneous data types. However, such models often possess a large number of parameters. While some parameter values can possibly be extracted from databases such as BRENDA [8] or SABIO-RK [9], others usually need to be inferred from problem-specific experimental data. The complexity of this inference problem depends on the model size, the number of unknown parameters and data availability.

Data availability and, in general, the structure and information content of data sets differs substantially between projects. On the experimental side, one often distinguishes between time-course and dose-response data. On the computational side, different experimental setups correspond to different types of simulation setups (Fig 1a): (i) the initial conditions of the simulation can either be specified explicitly as numerical values or parameters, or implicitly with a steady-state constraint; and (ii) either a dynamic phase has to be simulated or only a steady-state has to be determined. The experimental setup and the corresponding formulation of the simulation can be encoded using formats, such as the Simulation Experiment Description Markup Language (SED-ML) [10] or the Parameter Estimation tables (PEtab) [11].

Download:

Fig 1. Steady-state simulation scenarios, and computation methods for ASA.

(a) Different scenarios requiring computation of the model steady state. Top: The pre-equilibration case, where the system is at steady state at the beginning of the experiment. This means that by t = t₀ the system has reached its steady state (x*) under some condition (u^e) (blue line). At t = t₀ the system was perturbed and measurements were taken at time points t_j > t₀ (orange crosses). The pre-equilibration steady state (x*) is the initial state under the experimental conditions (u), i.e. x₀(u) = x*. Bottom: The post-equilibration case, where at some point after the beginning of the experiment, the system reaches its steady state (x*) and measurements for this time point (t = t′′) are available (orange cross at t = t′′). Measurements of the transient state may also be available (orange crosses in t < t′′). (b) Alternative approaches for computing sensitivities. Top: The standard ASA approach that requires numerical integration until convergence to the steady state and subsequent backward integration of the adjoint state (p) ODE on the same time interval. Bottom: The proposed adjoint method that circumvents backward numerical integration and requires solving a system of linear algebraic equations instead.

https://doi.org/10.1371/journal.pcbi.1010783.g001

In many studies, there is some data available regarding the system’s steady-state [12–15]. There are two distinct cases: (1) the system is assumed to start in a steady state, then it is perturbed and enters a dynamic state; or (2) the system is assumed to start in a dynamic state and after some time it reaches a steady state (Fig 1a). These two cases will be referred to as pre- and post-equilibration, respectively. The system can start in a steady state and reach a steady state again after a perturbation, therefore, the two cases are not mutually exclusive.

Pre- and post-equilibration as part of the simulation complicate parameter estimation and other subsequent tasks as the numerical computation of steady states of nonlinear systems is challenging. Therefore, a variety of computational methods have been proposed to derive analytical expressions for steady states. These methods rely, for example, on graph theory [16, 17], py-substitution [18] and some can even take positivity constraints into account [19]. Yet, for a large number of application problems these analytical approaches are not applicable and one needs to resort to numerical approaches. Fortunately, steady states can be computed without time-course simulations [20, 21]. Nowadays, variants of Newton’s method are widely used for equilibration. Tailored implementations of Newton’s method achieve a speedup of up to 100-fold compared to steady-state calculation via numerical simulation [21].

Efficient numerical simulation and steady-state calculation are cornerstones for parameter estimation [22]. However, several studies found that the availability of the objective function gradient is also highly beneficial for the parameterization for dynamical modelling (see, e.g. [23, 24]). Indeed, gradient-based multi-start local and global optimization methods seem to often outperform gradient-free optimization methods [23].

Objective function gradients can be computed using automatic differentiation (see, e.g., [25]), finite differences, and forward or adjoint sensitivity analysis [26]. While approximations obtained using finite differences are often numerically unreliable [24], forward and adjoint methods allow for an accurate assessment [26]. All sensitivity analysis methods are, in principle, available for pre- and post-equilibration. With forward sensitivity analysis, one can simulate state sensitivity ODEs along with the dynamical system until approximate convergence to a steady state. This comes at a high computational cost and scales poorly with the number of states and parameters, as one has to simulate a system of ODEs with the dimension (n_x + 1)n_θ. The existing adjoint method for pre- and post-equilibration simulates the model until approximate convergence to a steady state, and then solves the corresponding backward equation for the same time interval [12, 27]. This is computationally costly as well but possesses, in contrast to the forward method, improved scalability; here, one has to sequentially simulate two systems of the size n_x. However, the forward method can be easily adapted to exploit steady-state constraints. One can find state sensitivities at steady state without numerical integration, by solving a system of linear equations per parameter (see, e.g., [20]). The downside is that this approach is not applicable for systems with singular Jacobians. A corresponding approach for adjoint methods is currently missing.

In this paper, we introduce a novel adjoint method for the computation of objective function gradients for problems with steady-state constraints. The proposed method exploits the steady-state constraint to circumvent numerical backward integration for pre- and post-equilibration. Instead, a system of equations is solved to compute parts of the objective function gradient. We provide an implementation of the method in the open-source toolbox AMICI and investigate its accuracy and efficiency on three real-world problems. Our study demonstrates that the new method achieves a substantial speedup for gradient computation, which is particularly valuable for large models.

Methods

In this section, we introduce the considered classes of mathematical models and parameter estimation problems. Subsequently, we describe the established method for gradient calculation via adjoint sensitivity analysis (ASA) and propose a new formulation for the steady-state case. We discuss how the method can be applied to models of biochemical reaction networks. The method, however, is not restricted to the field of computational biology. It can be used for any model of the form described in this section with pre- or post-equilibration, or both.

Mathematical modeling and simulation

We consider ODE models of the form (1) in which is the vector of state variables, is the parameter vector, is the vector of constant inputs and is the time. The vector field of the ODE model is , which is Lipschitz-continuous with respect to x, and the initial condition at t₀ is . The ODE model determines the time evolution of the state variables, given the parameters. In case of biochemical reaction networks, this can be e.g., concentrations of biochemical species and reaction rate constants, respectively. The inputs (u) describe experimental conditions, e.g., regime of administration of a drug or composition of a cell culture medium. In many models, the state variables approach a steady-state x*(θ, u) for large values of t: (2)

In this paper we assume that for each initial condition (x₀(θ, u)) there exists an exponentially stable steady state. Both pre- and post-equilibration cases assume existence of a steady state that the dynamical system can reach fast enough to appropriately describe the modeled biochemical process. This assumption is often implicitly made when working with steady-state data.

Additionally, we denote by u^e the inputs corresponding to a pre-equilibration condition, if pre-equilibration is required. In this case, the initial condition is specified as a steady state corresponding to pre-equilibration condition (x(t₀) = x*(θ, u^e)), i.e. as a steady state of

The solution of the ODE system (1) is usually not available in closed form. Therefore, numerical simulation algorithms are widely used to determine the time-course of state variables. Here, we employ an algorithm based on the backward-differentiation formula (BDF) to study the—often stiff—ODE models of biochemical processes. For the computation of steady states of ODE models (2) analytical methods, numerical simulation, and numerical equation solvers (e.g. Newton’s method) can be employed [16–19, 21]. Here, we employ numerical integration until time derivatives become sufficiently small. For example, one can run numerical integration until the condition (3) is fulfilled, where “rtol” and “atol” denote relative and absolute tolerances, respectively.

Experimental data

Experiments provide information about observable components of the biochemical processes, which are in general subsets or functions of the state variables (x). The dependence of the observables on the state variables and parameters is modelled as (4) with output map . As measurements are corrupted by noise, we model them as noise-corrupted realizations of the observables. Here, we consider two types of measurements:

Time-course measurements with independent, normally-distributed noise, (5) in which i = 1, …, n_y and j = 1, …, n_t index observables and time points, and denotes the noise variance. We assume without loss of generality that t_j < t_j+1 and denote the collection of these time-course measurements by .
Independent, normally distributed measurements of the steady state, (6) in which y_i*(θ) = h(x*(θ, u), θ, u) denotes the values of the observable y_i at the steady state x*(θ, u), and denotes the noise variance. For the case of multi-stable systems, the state for the starting point of the post-equilibration needs to be considered. We denote the collection of these measurements in steady state by .

Measurements from multiple experiments may be available, which can be described by a set of inputs . Further in this section, without loss of generality, we assume that measurements from one experiment, described by u are available.

Remark: For the subsequent sections we assume, without loss of generality, that the noise is normally distributed and the variance is known. The results also apply to unknown noise variances and other noise distributions, given that measurements are independent and the mean of the noise is equal to zero.

Parameter estimation

As the parameters of biochemical processes are often difficult to determine experimentally, they are typically unknown. Hence, the values of the parameters (θ) have to be estimated from experimental data. This is usually achieved using frequentist or Bayesian inference.

Frequentist inference focuses on maximum likelihood estimates (MLEs) and the assessment of their uncertainties. MLEs maximize the likelihood function (), which is a measure for the distance between measurement and simulation, and this is equivalent to minimizing the negative log-likelihood function (). For independent, normally-distributed measurements, the negative log-likelihood function is given by (7) in which the first part accounts for the time-course measurements () and the second part for the steady state measurements (). The MLE is computed by solving the optimization problem

The parameters are often constrained, e.g. to be non-negative, and might be log-transforms of the original model parameters in order to improve convergence of the optimizer [24, 28].

Bayesian inference focuses on the assessment of the posterior distribution, which encodes the information provided by the data as well as prior knowledge. In practice the first step is usually to compute the maximum a posteriori estimator. This is achieved by minimizing the (often unnormalized) negative log-posterior, yielding an optimization problem similar to the one encountered in frequentist inference [29].

Parameter estimation for ODE models often involves objective functions (e.g. negative log-likelihood or negative log-posterior functions) that are non-convex and possess local minima. To solve these parameter estimation problems, global optimization methods are used. A globalization strategy that has proven to be efficient is multi-start local optimization with gradient-based local optimization methods [23, 24]. The computation time of these methods is usually dominated by the computation time required for the evaluation of the objective function gradient. For the objective function (7), the gradient is given by (8) in which ∂y_i/∂θ_k denotes the sensitivity of the observable y_i with respect to the parameter θ_k at a specific time point t_j or in steady state.

Adjoint sensitivity analysis (ASA)

For large-scale ODE models, the objective function gradient for time-course data is usually calculated using ASA. This approach has a long history and relates to Green’s function method of sensitivity analysis (see e.g. [30] and references therein) and is used across scientific disciplines. [26] adopted it to parameter estimation problems in biological applications for the case of time-course measurements without pre- or post-equilibration. For a description of ASA, readers are referred to [26]. This approach circumvents the evaluation of the observable sensitivities ∂y_i/∂θ_k by introducing the adjoint state , such that ∀j = n_t…, 1 on interval (t_j−1, t_j] it satisfies the backward differential equation (9) with boundary values and Jacobian matrix J, (10)

Given the adjoint state, the objective function gradient can be computed as (11) in which ∂x₀/∂θ_k denotes the sensitivity of the initial state with respect to parameter θ_k. If the initial condition is given as an explicit function, this derivative can be computed easily; however, its calculation is non-trivial in the case of pre-equilibration.

To use ASA for the pre- and post-equilibration cases, the steady-state calculation is currently performed via a long simulation (see, e.g. [12, 27]):

For the case of a time-course followed by a post-equilibration, the simulation is run until a time for which time derivatives are negligible. We used (3) as convergence criterion. The values of the observables at time t′′, y_i(t′′, θ, u), are considered a good approximation of the observables at the steady state, y_i*(θ, u), and used for the evaluation of the objective function.
For the case of pre-equilibration followed by a time-course, in a first step a simulation is run for the pre-equilibration condition (u^e) until the state derivatives are sufficiently small. The initial time point for this simulation is denoted by −t′. In the second step, the condition is switched to u and a simulation is performed for the transient phase using the final state of the pre-equilibration simulation as the initial state. This can be interpreted as running the simulation for t ∈ [−t′, t₀] under the pre-equilibration condition (u^e) and then switching for to the time course condition (u).

These approximations of the exact setup are good if the convergence criteria are tight. In practice, one often uses values on the order of 10⁻⁸ and 10⁻¹⁶ as the relative and absolute tolerance, respectively [31]. One should note, that ASA only provides the objective function gradient, but not the sensitivity of the steady state with respect to the parameters.

Adjoint sensitivity analysis at steady state (ssASA)

As steady-state constraints are encountered in a large fraction of application problems (see, e.g., the benchmark collection in [28]), we propose here an adjoint sensitivity analysis method that is tailored for problems with pre- and post-equilibration.

Post-equilibration case.

For experiments with a post-equilibration, steady-state measurements () are available in addition to time-course measurements (). To compute the objective function gradient (11), one needs to compute the term (12)

Note, that the upper limit of the integral is different from the one in (11) as the steady-state measurements time point t = t′′ were added. On the time interval the integral should be computed using the standard ASA, while on the interval the proposed approach can be applied.

Let us introduce a time point t′ such that and for t ≥ t′ the system (1) is at steady state. Then for t ≥ t′ the Jacobian (10) is a constant matrix. Due to the exponential stability of the steady state, the precise choice of t′ is not important, as long as t′ has been chosen large enough. Then, on the time interval t ≥ t′, the adjoint system of ODEs (9) simplifies to (13)

This system has the solution

The steady state of the system (1) is exponentially stable if and only if the eigenvalues of the Jacobian at the steady state (J(x*(θ, u), θ, u)) have negative real parts. This follows that the steady state p = 0 of the system (13) is asymptotically stable in reverse time. Hence, as the time interval [t′, t′′] can be chosen large enough, it holds that which is illustrated in Fig 2a. As the steady state p = 0 is stable, the system (13) will not diverge from it on the interval .

Download:

Fig 2. Computation of the adjoint state integral term.

The top row illustrates the solution of the system (1) for (a) the post-equilibration case and (b) the pre-equilibration case for n_t = 3. The bottom row illustrates the integral term in (11).

https://doi.org/10.1371/journal.pcbi.1010783.g002

As a consequence, the integral (14) reduces to a matrix-vector product: since

To obtain the integral one can solve the linear system of equations (15) see Section “Adjoint state integral computation at steady state” of the S1 Text for the derivation. Additionally, the calculations from this section performed for a simple example of a conversion reaction can be found in Section “Conversion reaction example” of the S1 Text.

Pre-equilibration case.

For experiments with a pre-equilibration, in order to do standard ASA, one requires initial state sensitivities () (see Eq (11)). They can be computed by performing forward sensitivity analysis during pre-equilibration.

Alternatively, instead of computing , the ASA method can be extended to the pre-equilibration time interval [−t′, t₀]. On the intervals [t_N, t_N−1), …, [t₁, t₀) standard ASA can be performed as described in Section “Adjoint sensitivity analysis (ASA)”. As described for the post-equilibration case, at t = −t′, the system (13) is at steady state p = 0, hence the scalar product of the initial state sensitivities of pre-equilibration, with the adjoint state at t = −t′, vanishes. Therefore, one only needs to compute the quadratures (16)

For this, the adjoint state from the time-course adjoint simulation must be passed on to the initial adjoint state of backward pre-equilibration (p(t₀, θ, u)). Accordingly, for the case of pre-equilibration, the ODEs solved for pre-equilibration and time-dependent observations are coupled, and cannot be computed independently.

Similarly to the post-equilibration case, there are two possibilities for computing this integral. The first possibility is to compute it by simulating the system (1) until it reaches its steady state and then compute the integral (16) backward on the same time interval. The second possibility is to use the proposed ssASA method. One can assume, without loss of generality, that the system (1) is at the steady state that corresponds to the pre-equilibration condition for all t < t₀. On this interval, the same considerations as in the previous section apply and the adjoint system simplifies to (13), but in this case x*(θ, u) = x*(θ, u^e). To find the integral one can solve the linear system of equations (17)

This case is illustrated in Fig 2b.

Applicability of the proposed method.

The systems (15) and (17) have a unique solution only if the (transposed) Jacobian is non-singular. Otherwise, the proposed approach is not applicable. A common cause for singular Jacobians is, amongst others (cf. [32] for a review), presence of conserved quantities in the system. Conserved quantities or conserved moieties are functions of the sates of the dynamical system that remain constant over time. Identifying each conserved quantity allows to reduce model dimension by excluding one state, expressing it in terms of other states contained in the conserved quantity. A simple example can be found in Section “Conversion reaction example” of the S1 Text. There are various deterministic [33, 34] algorithms available to identify conserved quantities, which, however, are not suitable for large/genome-scale reaction networks due to combinatorial complexity. A fast, scalable, heuristic-based approach presented in [35] is better suited for large-scale reaction networks.

Removing conserved quantities from the system allows to apply the proposed method. If it is not possible or if the Jacobian remains singular, the adjoint state ODEs have to be integrated numerically.

Implementation

The new algebraic approach for computing objective function gradient as well as a heuristic-based conserved moieties identification approach [35] have been made available via the open-source AMICI package (https://github.com/AMICI-dev/AMICI/) [36], which interfaces the SUNDIALS solver CVODES [37]. Implementation details can be found in Section “AMICI Implementation” of the S1 Text.

During the numerical study, described in the next section, the AMICI package was used for model simulations [38]. Parameter estimation was performed using the open-source Python Parameter EStimation TOolbox (pyPESTO) [39], which internally used the Interior Trust Region Reflective algorithm for boundary constrained optimization problems implemented in the Fides Python package for optimization [40, 41].

Results

In this section we investigate whether using the proposed ssASA method is beneficial, compared to the standard ASA approach, to computing the objective function gradient. In both cases we used ASA, but in the first case gradient computation was done via numerical integration of the ODE system (9), whereas in the second case the new approach was applied, where the system of Eqs (15) and (17) is solved. In both cases numerical integration was used to find the steady state of the system (1), i.e. to find the solution x*(θ, u) of . The two cases will be referred to as “standard ASA for sensitivities” and “ssASA for sensitivities”. We assess accuracy by comparing the gradient values computed with the two methods, as well as the difference in computation time.

The comparison is done on three previously published ODE-constrained optimization problems of varying complexity that required either pre- or post-equilibration (Table 1). The problems were taken from the PEtab Benchmark Collection, a GitHub repository with parameter estimation problems from published studies implemented in the PEtab format [11, 42].

Download:

Table 1. Overview of the models and optimization problems considered in this study.

https://doi.org/10.1371/journal.pcbi.1010783.t001

The Blasi et al., 2016 model describes acetylation of the histone H4 [13]. It is linear and has 16 state variables and 9 optimized parameters, which is the dimension of objective function gradient (8). Only steady-state data is available for this model. The Blasi et al., 2016 model contains conserved quantities, which were automatically removed during model import by AMICI to ensure non-singularity of the Jacobian. The model by Zheng et al., 2012 is also linear and describes histone H3 methylation pattern formation. For this model dynamic data is available and the initial condition is determined by pre-equilibration. The model by Fröhlich et al., 2018 is a large-scale nonlinear model describing the drug response of cancer cell lines [12]. The measurement data used for optimization was collected when the system was assumed to be at steady state.

Accuracy with ssASA is preserved

To investigate the accuracy of objective function gradients computed with the proposed ssASA method, we performed 500 model simulations with different parameter vectors sampled uniformly within the parameter bounds specified in the parameters PEtab file [11]. For each model, the same initial state, specified in the PEtab files, was used for each simulation and no simulation failed.

For all three models the computed objective function gradients were very similar for both compared approaches (Fig 3). The maximum and median deviations (as defined in the S1 Text, Section “Accuracy of gradient computation”) are respectively equal to 1.9 ⋅ 10⁻⁴ and numerically 0 for the Blasi et al., 2016 model, 2.8 ⋅ 10⁻¹¹ and 1.6 ⋅ 10⁻¹⁷ for the Zheng et al., 2012 model, 3.1 ⋅ 10⁻² and 3.04 ⋅ 10⁻²⁰ for the Fröhlich et al., 2018 model. Therefore, we conclude that the new approach gives accurate results.

Download:

Fig 3. Accuracy of objective function gradients of the proposed method.

In each of the three subplots one point corresponds to an objective function gradient value computed during one simulation using either standard ASA (x-axis) or ssASA (y-axis) for sensitivities computation. The number of points in each subplot is equal to the number of optimized parameters multiplied by the number of simulations. Points close to the diagonal indicate a good agreement.

https://doi.org/10.1371/journal.pcbi.1010783.g003

Simulation with ssASA is faster

We then compared how simulation time differs for the two approaches. For all three models, all simulations were significantly faster with the new approach (Fig 4). The precise speedup depends on the model. The simulations were on average 3.3, 1.2, 3.3 times faster with the new approach for the Blasi et al., 2016, Zheng et al., 2012 and Fröhlich et al., 2018 models, respectively (Fig 4d).

Download:

Fig 4. Simulation efficiency of the proposed method.

Each point corresponds to a total simulation time with ASA (x-axis) and ssASA (y-axis) for sensitivities computation. Points on the diagonal correspond to simulations that took equal time with both approaches. (a) Blasi et al., 2016, (b) Zheng et al., 2012 and (c) Fröhlich et al., 2018 models. (d) Computation speedup of simulations using ssASA for sensitivities computation compared to using standard ASA. Each bar height corresponds to a mean of computation speedups of all simulations and each error bar corresponds to the sample standard deviation.

https://doi.org/10.1371/journal.pcbi.1010783.g004

Additionally, we investigated how simulation time and computation speedup scale with respect to the number of optimized parameters. We simulated the Fröhlich et al., 2018 model with 11 different estimation problem sizes (1, 2, 5, 12, 28, 64, 147, 337, 775, 1780, or 4088 estimated parameters), each 200 times, for a total of 2200 simulations. For each simulation, the subset of parameters set to be estimated or fixed was randomly chosen, and all parameter values were sampled uniformly within the parameter bounds in the PEtab parameters table. During simulation, sensitivities of the objective function were only computed for the estimated parameters. As expected, simulation time did not change with the number of optimized parameters for both sensitivities computation methods, (Fig 5a), as the dimension of both the adjoint state ODE (9) and the system (15) do not change with the number of optimized parameters and is equal to n_x. Consequently, computation speedup does not change with the number of optimized parameters, and the proposed method is faster, independent of the number of free parameters (Fig 5b).

Download:

Fig 5. Scaling of simulation time and computation speedup of simulations with respect to the number of optimized parameters.

Fröhlich et al., 2018 model, 200 simulations with 1, 2, 5, 12, 28, 64, 147, 337, 775, 1780, 4088 optimized parameters. (a) Each point is the mean simulation time for corresponding number of free parameters, and the error bars are the standard deviations. (b) Each point represents computation speedup of simulations performed with ssASA for sensitivities compared to using standard ASA for sensitivities, the error bars show the standard deviation.

https://doi.org/10.1371/journal.pcbi.1010783.g005

Optimization with ssASA is faster

Furthermore, we investigated how the speedup we observed for individual model simulations with different parameter vectors translates to parameter estimation tasks. A large number of model simulations might be required during optimization for parameter estimation, which also means that if pre- or post-equilibration is needed it will be executed during each simulation.

We compared the difference in computation time for solving parameter estimation problems using multi-start local optimization between the two approaches for sensitivities computation, (Fig 6a–6c). Same as for the simulation comparison, initial parameter vectors were sampled uniformly within the parameter bounds specified in the parameters PEtab file. The same initial parameter values were used for optimizations with ASA and ssASA. For the Blasi et al., 2016 and Zheng et al., 2012 models we ran 500 multi-starts. As the Fröhlich et al., 2018 model is computationally demanding, in this study, we considered only a subset of data used in [12]. We used only the control conditions, that is 143 out of the total 5281 conditions. For this model, we ran 50 multi-start optimizations with the number of optimization steps limited to 50. This would not yield good model fits, but should provide a representative performance estimate.

Download:

Fig 6. Optimization efficiency of the proposed method.

Each scatter point shows the total computation time required for one multi-start optimization using standard ASA (x-axis) or ssASA (y-axis) for sensitivities computation (a) Blasi et al., 2016 model, (b) Zheng et al., 2012 model, (c) Fröhlich et al., 2018 model. Points on the diagonal correspond to multi-starts that took equal time with both approaches. (d) Computation speedup of optimizations using ssASA for sensitivities computation compared to using standard ASA for sensitivities. Each bar height corresponds to a mean of multi-start local optimization computation speedups and each error bar corresponds to the sample standard deviation.

https://doi.org/10.1371/journal.pcbi.1010783.g006

For the Blasi et al., 2016 and Zheng et al., 2012 models all multi-starts finished successfully. For the Fröhlich et al., 2018 model for both methods one out of 50 multi-starts failed at the first step and the other 49 finished as the number of optimization steps exceeded 50. Optimizations were on average 1.5, 1.1, 1.9 times faster with the new approach for Blasi et al., 2016, Zheng et al., 2012 and Fröhlich et al., 2018 models respectively (Fig 6d).

The speedups for optimization were lower than those observed for model simulation, because the proposed method only affects simulation time, but not the time taken in the optimizer itself, or other overhead. The Blasi et al., 2016 model is rather small and linear and simulations made up only approximately 50% of total optimization time. Therefore, the 3-fold speedup in simulations resulted in 1.3-fold speedup of optimizations. For the Zheng et al., 2012 model speedup is about the same as for simulations. For the Fröhlich et al., 2018 model simulations made up around 25% of the total optimization time. Here, model simulation is very costly, but also significant time was spent in the optimizer due to the large number of optimized parameters. Generally, the speedup for optimization will be highly implementation-dependent.

Discussion and conclusion

It has been shown, that if finding a steady state is required during model simulation, whether it is pre-equilibration, post-equilibration, or both, it is possible to partially avoid numerical integration and to solve a system of algebraic equations instead [21]. In this study, we showed that this idea can also be used to simplify objective function gradient computation. We introduced a new method for efficient adjoint sensitivities computation at steady state. We showed that an intermediate step of objective function gradient computation with ASA, i.e., backward numerical integration of the adjoint state ODEs, can be avoided by solving a linear system of equations for the time interval including steady-state measurement time point. We also showed that in case pre-equilibration is required, computation of initial state sensitivities () with forward sensitivity analyses can be avoided by extending ASA to the pre-equilibration interval. The proposed method is applicable on this time interval as well, and a linear system of equations can be solved instead of backward numerical integration.

We described how the new approach can be applied in the case of modelling biochemical processes, tested it on three published models and demonstrated that it is accurate and allows for efficient sensitivity computation. The exact speedup over the standard approach depends on the model and dataset. More specifically, it depends on the computational complexity of computing the integral (14) (or (16) for the pre-equilibration case) on the time interval between the last finite time point () and the time point where the steady-state condition is satisfied. It is the difference between solving an n_x-dimensional linear non-autonomous ODE system, and solving a linear system of n_x equations. The speedup is expected to be higher for nonlinear systems. It also depends on the fraction of computational cost spent on computing the steady-state. For example, if the time spent in the optimizer is large compared to the time for model simulation, or if mostly time-course data is available, the speedup will be lower. As expected, the speedup does not significantly depend on the number of optimized parameters, which we demonstrated with the example of one of the three models. We observed a speedup of up to 3.3-fold for simulations and up to two-fold for parameter estimation, for the models and datasets considered in this study.

The new approach is only applicable if for each initial value there exists a unique, exponentially stable steady state. This implies that the (transposed) Jacobian of the right-hand side is non-singular. One of the main reasons for a singular Jacobian is the presence of conserved quantities in the dynamical system [32]. In order to facilitate applicability of the proposed method we have implemented a fast heuristic-based approach for conserved quantities identification suitable for large-scale reaction networks as presented in [35]. Both the conserved moieties identification approach and the proposed ssASA approach have been made available via the open-source AMICI package. If the uniqueness of the steady state cannot be ensured, e.g. because the system might be multi-stable, the method has to be applied carefully. While there also is a unique mapping from initial condition to steady state for multi-stable systems with Lipschitz-continuous right-hand sides, this mapping might not be obeyed by Newton’s method. We do not observe this problem in practice, but recommend to check this after optimization and to provide Newton’s method with a refined initial value (e.g. obtained via a short simulation run, which is already supported by AMICI) if necessary. We have tested this case on the Blasi et al., 2016 model. Conserved quantities have been removed before applying the proposed ssASA approach.

Solving parameter estimation problems consist of various steps that depend on the model and available data. Different methods, for finding steady states, numerical integration, or sensitivity computation, might be applicable or more efficient for a particular model. Further analysis and benchmarking of different approaches and their combinations would be beneficial. For example, a broader study of the steady-state case that considers different methods for steady-state computation in combination with different approaches for sensitivity analysis for both forward and adjoint sensitivity methods, to find especially efficient combinations, would be of interest.

Supporting information

S1 Text. Supplementary notes on the ssASA method derivation, accuracy assessment and AMICI implementation, as well as the conversion reaction example.

https://doi.org/10.1371/journal.pcbi.1010783.s001

(PDF)

References

1. Kemmer S, Berdiel-Acer M, Reinz E, Sonntag J, Tarade N, Bernhardt S, et al. Disentangling ERBB Signaling in Breast Cancer Subtypes—A Model-Based Analysis. Cancers. 2022;14(10). pmid:35625984
- View Article
- PubMed/NCBI
- Google Scholar
2. Schmucker R, Farina G, Faeder J, Fröhlich F, Saglam AS, Sandholm T. Combination treatment optimization using a pan-cancer pathway model. PLOS Computational Biology. 2022;17(12):1–22.
- View Article
- Google Scholar
3. Swameye I, Müller TG, Timmer J, Sandra O, Klingmüller U. Identification of nucleocytoplasmic cycling as a remote sensor in cellular signaling by databased modeling. Proceedings of the National Academy of Sciences. 2003;100(3):1028–1033. pmid:12552139
- View Article
- PubMed/NCBI
- Google Scholar
4. Ben Guebila M, Thiele I. Dynamic flux balance analysis of whole-body metabolism for type 1 diabetes. Nature Computational Science. 2021;1(5):348–361.
- View Article
- Google Scholar
5. Hurbain J, Thommen Q, Anquez F, Pfeuty B. Quantitative modeling of pentose phosphate pathway response to oxidative stress reveals a cooperative regulatory strategy. iScience. 2022;25(8):104681. pmid:35856027
- View Article
- PubMed/NCBI
- Google Scholar
6. van Sluijs B, Maas RJM, van der Linden AJ, de Greef TFA, Huck WTS. A microfluidic optimal experimental design platform for forward design of cell-free genetic networks. Nature Communications. 2022;13(1):3626. pmid:35750678
- View Article
- PubMed/NCBI
- Google Scholar
7. Schuh L, Loos C, Pokrovsky D, Imhof A, Rupp RAW, Marr C. H4K20 Methylation Is Differently Regulated by Dilution and Demethylation in Proliferating and Cell-Cycle-Arrested Xenopus Embryos. Cell Systems. 2020;11(6):653–662.e8. pmid:33296683
- View Article
- PubMed/NCBI
- Google Scholar
8. Chang A, Jeske L, Ulbrich S, Hofmann J, Koblitz J, Schomburg I, et al. BRENDA, the ELIXIR core data resource in 2021: new developments and updates. Nucleic Acids Research. 2020;49(D1):D498–D508.
- View Article
- Google Scholar
9. Wittig U, Kania R, Golebiewski M, Rey M, Shi L, Jong L, et al. SABIO-RK—database for biochemical reaction kinetics. Nucleic Acids Research. 2011;40(D1):D790–D796. pmid:22102587
- View Article
- PubMed/NCBI
- Google Scholar
10. Waltemath D, Adams R, Bergmann FT, Hucka M, Kolpakov F, Miller AK, et al. Reproducible computational biology experiments with SED-ML—The Simulation Experiment Description Markup Language. BMC Systems Biology. 2011;5(1):198. pmid:22172142
- View Article
- PubMed/NCBI
- Google Scholar
11. Schmiester L, Schälte Y, Bergmann FT, Camba T, Dudkin E, Egert J, et al. PEtab—Interoperable specification of parameter estimation problems in systems biology. PLOS Computational Biology. 2021;17(1):1–10. pmid:33497393
- View Article
- PubMed/NCBI
- Google Scholar
12. Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient Parameter Estimation Enables the Prediction of Drug Response Using a Mechanistic Pan-Cancer Pathway Model. Cell Systems. 2018;7(6):567–579.e6. pmid:30503647
- View Article
- PubMed/NCBI
- Google Scholar
13. Blasi T, Feller C, Feigelman J, Hasenauer J, Imhof A, Theis FJ, et al. Combinatorial Histone Acetylation Patterns Are Generated by Motif-Specific Reactions. Cell Syst. 2016;2(1):49–58. pmid:27136689
- View Article
- PubMed/NCBI
- Google Scholar
14. Zheng Y, Sweet SMM, Popovic R, Martinez-Garcia E, Tipton JD, Thomas PM, et al. Total kinetic analysis reveals how combinatorial methylation patterns are established on lysines 27 and 36 of histone H3. Proceedings of the National Academy of Sciences. 2012;109(34):13549–13554. pmid:22869745
- View Article
- PubMed/NCBI
- Google Scholar
15. Gopalakrishnan S, Dash S, Maranas C. K-FIT: An accelerated kinetic parameterization algorithm using steady-state fluxomic data. Metabolic Engineering. 2020;61:197–205. pmid:32173504
- View Article
- PubMed/NCBI
- Google Scholar
16. Chou KC. Applications of graph theory to enzyme kinetics and protein folding kinetics. Biophysical Chemistry. 1990;35(1):1–24. pmid:2183882
- View Article
- PubMed/NCBI
- Google Scholar
17. Feliu E, Wiuf C. Variable Elimination in Chemical Reaction Networks with Mass-Action Kinetics. SIAM Journal on Applied Mathematics. 2012;72(4):959–981.
- View Article
- Google Scholar
18. Loriaux PM, Tesler G, Hoffmann A. Characterizing the Relationship between Steady State and Response Using Analytical Expressions for the Steady States of Mass Action Models. PLoS Computational Biology. 2013;9(2):e1002901. pmid:23509437
- View Article
- PubMed/NCBI
- Google Scholar
19. Rosenblatt M, Timmer J, Kaschek D. Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models. Frontiers in Cell and Developmental Biology. 2016;4. pmid:27243005
- View Article
- PubMed/NCBI
- Google Scholar
20. Fiedler A, Raeth S, Theis FJ, Hausser A, Hasenauer J. Tailored parameter optimization methods for ordinary differential equation models with steady-state constraints. BMC Systems Biology. 2016;10(1). pmid:27549154
- View Article
- PubMed/NCBI
- Google Scholar
21. Terje Lines G, Paszkowski Ł, Schmiester L, Weindl D, Stapor P, Hasenauer J. Efficient computation of steady states in large-scale ODE models of biochemical reaction networks. IFAC-PapersOnLine. 2019;52(26):32–37.
- View Article
- Google Scholar
22. Fröhlich F, Loos C, Hasenauer J. 1. In: Sanguinetti G, Huynh-Thu VA, editors. Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes. New York, NY: Springer New York; 2019. p. 385–422. Available from: https://doi.org/10.1007/978-1-4939-8882-2_16.
23. Villaverde AF, Fröhlich F, Weindl D, Hasenauer J, Banga JR. Benchmarking optimization methods for parameter estimation in large kinetic models. Bioinformatics. 2018;35(5):830–838.
- View Article
- Google Scholar
24. Raue A, Schilling M, Bachmann J, Matteson A, Schelke M, Kaschek D, et al. Lessons Learned from Quantitative Dynamical Modeling in Systems Biology. PLoS ONE. 2013;8(9):e74335. pmid:24098642
- View Article
- PubMed/NCBI
- Google Scholar
25. Naumann U. The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation. No. 24 in Software, Environments, and Tools. Philadelphia, PA: SIAM; 2012. Available from: http://bookstore.siam.org/se24.
26. Fröhlich F, Kaltenbacher B, Theis FJ, Hasenauer J. Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks. PLOS Computational Biology. 2017;13(1):1–18. pmid:28114351
- View Article
- PubMed/NCBI
- Google Scholar
27. Schmiester L, Schälte Y, Fröhlich F, Hasenauer J, Weindl D. Efficient parameterization of large-scale dynamic models based on relative measurements. Bioinformatics. 2019;36(2):594–602.
- View Article
- Google Scholar
28. Hass H, Loos C, Raimúndez-Álvarez E, Timmer J, Hasenauer J, Kreutz C. Benchmark problems for dynamic modeling of intracellular processes. Bioinformatics. 2019;35(17):3073–3082. pmid:30624608
- View Article
- PubMed/NCBI
- Google Scholar
29. Villaverde AF, Pathirana D, Fröhlich F, Hasenauer J, Banga JR. A protocol for dynamic model calibration. Briefings in Bioinformatics. 2021.
- View Article
- Google Scholar
30. Lemke M, Cai L, Reiss J, Pitsch H, Sesterhenn J. Adjoint-based sensitivity analysis of quantities of interest of complex combustion models. Combustion Theory and Modelling. 2019;23(1):180–196.
- View Article
- Google Scholar
31. Städter P, Schälte Y, Schmiester L, Hasenauer J, Stapor PL. Benchmarking of numerical integration methods for ODE models of biological systems. Scientific Reports. 2021;11(1):2696. pmid:33514831
- View Article
- PubMed/NCBI
- Google Scholar
32. Oellerich T, Emelianenko M, Liotta L, Araujo RP. Biological networks with singular Jacobians: their origins and adaptation criteria. bioRxiv. 2021.
- View Article
- Google Scholar
33. Contejean E, Devie H. An Efficient Incremental Algorithm for Solving Systems of Linear Diophantine Equations. Inf Comput. 1994;113(1):143–172.
- View Article
- Google Scholar
34. Pasechnik D. On computing Hilbert bases via the Elliot-MacMahon algorithm. Theoretical Computer Science. 2001;263.
- View Article
- Google Scholar
35. De Martino A, De Martino D, Mulet R, Pagnani A. Identifying All Moiety Conservation Laws in Genome-Scale Metabolic Networks. PLOS ONE. 2014;9(7):1–11. pmid:24988199
- View Article
- PubMed/NCBI
- Google Scholar
36. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski Ł, Lines GT, et al. AMICI: high-performance sensitivity analysis for large ordinary differential equation models. Bioinformatics. 2021;37(20):3676–3677. pmid:33821950
- View Article
- PubMed/NCBI
- Google Scholar
37. Hindmarsh AC, Brown PN, Grant KE, Lee SL, Serban R, Shumaker DE, et al. SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. ACM Trans Math Softw. 2005;31(3):363–396.
- View Article
- Google Scholar
38. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski L, Lines GT, et al.. AMICI: High-Performance Sensitivity Analysis for Large Ordinary Differential Equation Models; 2022. Available from: https://doi.org/10.5281/zenodo.6850965.
39. Schälte Y, Fröhlich F, Stapor P, Vanhoefer J, Weindl D, Jost PJ, et al.. pyPESTO—Parameter EStimation TOolbox for python; 2022. Available from: https://zenodo.org/record/6606687.
40. Fröhlich F, Sorger PK. Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models. PLOS Computational Biology. 2022;18(7):1–28. pmid:35830470
- View Article
- PubMed/NCBI
- Google Scholar
41. Froehlich F, Weindl D. Fides; 2021. Available from: https://doi.org/10.5281/zenodo.6038127.
42. A collection of mathematical models with experimental data in the PEtab format; 2022. Available from: https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/.

[ref1] 1. Kemmer S, Berdiel-Acer M, Reinz E, Sonntag J, Tarade N, Bernhardt S, et al. Disentangling ERBB Signaling in Breast Cancer Subtypes—A Model-Based Analysis. Cancers. 2022;14(10). pmid:35625984
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Schmucker R, Farina G, Faeder J, Fröhlich F, Saglam AS, Sandholm T. Combination treatment optimization using a pan-cancer pathway model. PLOS Computational Biology. 2022;17(12):1–22.
View Article
Google Scholar

[6] View Article

[7] Google Scholar

[ref3] 3. Swameye I, Müller TG, Timmer J, Sandra O, Klingmüller U. Identification of nucleocytoplasmic cycling as a remote sensor in cellular signaling by databased modeling. Proceedings of the National Academy of Sciences. 2003;100(3):1028–1033. pmid:12552139
View Article
PubMed/NCBI
Google Scholar

[9] View Article

[10] PubMed/NCBI

[11] Google Scholar

[ref4] 4. Ben Guebila M, Thiele I. Dynamic flux balance analysis of whole-body metabolism for type 1 diabetes. Nature Computational Science. 2021;1(5):348–361.
View Article
Google Scholar

[13] View Article

[14] Google Scholar

[ref5] 5. Hurbain J, Thommen Q, Anquez F, Pfeuty B. Quantitative modeling of pentose phosphate pathway response to oxidative stress reveals a cooperative regulatory strategy. iScience. 2022;25(8):104681. pmid:35856027
View Article
PubMed/NCBI
Google Scholar

[16] View Article

[17] PubMed/NCBI

[18] Google Scholar

[ref6] 6. van Sluijs B, Maas RJM, van der Linden AJ, de Greef TFA, Huck WTS. A microfluidic optimal experimental design platform for forward design of cell-free genetic networks. Nature Communications. 2022;13(1):3626. pmid:35750678
View Article
PubMed/NCBI
Google Scholar

[20] View Article

[21] PubMed/NCBI

[22] Google Scholar

[ref7] 7. Schuh L, Loos C, Pokrovsky D, Imhof A, Rupp RAW, Marr C. H4K20 Methylation Is Differently Regulated by Dilution and Demethylation in Proliferating and Cell-Cycle-Arrested Xenopus Embryos. Cell Systems. 2020;11(6):653–662.e8. pmid:33296683
View Article
PubMed/NCBI
Google Scholar

[24] View Article

[25] PubMed/NCBI

[26] Google Scholar

[ref8] 8. Chang A, Jeske L, Ulbrich S, Hofmann J, Koblitz J, Schomburg I, et al. BRENDA, the ELIXIR core data resource in 2021: new developments and updates. Nucleic Acids Research. 2020;49(D1):D498–D508.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref9] 9. Wittig U, Kania R, Golebiewski M, Rey M, Shi L, Jong L, et al. SABIO-RK—database for biochemical reaction kinetics. Nucleic Acids Research. 2011;40(D1):D790–D796. pmid:22102587
View Article
PubMed/NCBI
Google Scholar

[31] View Article

[32] PubMed/NCBI

[33] Google Scholar

[ref10] 10. Waltemath D, Adams R, Bergmann FT, Hucka M, Kolpakov F, Miller AK, et al. Reproducible computational biology experiments with SED-ML—The Simulation Experiment Description Markup Language. BMC Systems Biology. 2011;5(1):198. pmid:22172142
View Article
PubMed/NCBI
Google Scholar

[35] View Article

[36] PubMed/NCBI

[37] Google Scholar

[ref11] 11. Schmiester L, Schälte Y, Bergmann FT, Camba T, Dudkin E, Egert J, et al. PEtab—Interoperable specification of parameter estimation problems in systems biology. PLOS Computational Biology. 2021;17(1):1–10. pmid:33497393
View Article
PubMed/NCBI
Google Scholar

[39] View Article

[40] PubMed/NCBI

[41] Google Scholar

[ref12] 12. Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient Parameter Estimation Enables the Prediction of Drug Response Using a Mechanistic Pan-Cancer Pathway Model. Cell Systems. 2018;7(6):567–579.e6. pmid:30503647
View Article
PubMed/NCBI
Google Scholar

[43] View Article

[44] PubMed/NCBI

[45] Google Scholar

[ref13] 13. Blasi T, Feller C, Feigelman J, Hasenauer J, Imhof A, Theis FJ, et al. Combinatorial Histone Acetylation Patterns Are Generated by Motif-Specific Reactions. Cell Syst. 2016;2(1):49–58. pmid:27136689
View Article
PubMed/NCBI
Google Scholar

[47] View Article

[48] PubMed/NCBI

[49] Google Scholar

[ref14] 14. Zheng Y, Sweet SMM, Popovic R, Martinez-Garcia E, Tipton JD, Thomas PM, et al. Total kinetic analysis reveals how combinatorial methylation patterns are established on lysines 27 and 36 of histone H3. Proceedings of the National Academy of Sciences. 2012;109(34):13549–13554. pmid:22869745
View Article
PubMed/NCBI
Google Scholar

[51] View Article

[52] PubMed/NCBI

[53] Google Scholar

[ref15] 15. Gopalakrishnan S, Dash S, Maranas C. K-FIT: An accelerated kinetic parameterization algorithm using steady-state fluxomic data. Metabolic Engineering. 2020;61:197–205. pmid:32173504
View Article
PubMed/NCBI
Google Scholar

[55] View Article

[56] PubMed/NCBI

[57] Google Scholar

[ref16] 16. Chou KC. Applications of graph theory to enzyme kinetics and protein folding kinetics. Biophysical Chemistry. 1990;35(1):1–24. pmid:2183882
View Article
PubMed/NCBI
Google Scholar

[59] View Article

[60] PubMed/NCBI

[61] Google Scholar

[ref17] 17. Feliu E, Wiuf C. Variable Elimination in Chemical Reaction Networks with Mass-Action Kinetics. SIAM Journal on Applied Mathematics. 2012;72(4):959–981.
View Article
Google Scholar

[63] View Article

[64] Google Scholar

[ref18] 18. Loriaux PM, Tesler G, Hoffmann A. Characterizing the Relationship between Steady State and Response Using Analytical Expressions for the Steady States of Mass Action Models. PLoS Computational Biology. 2013;9(2):e1002901. pmid:23509437
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref19] 19. Rosenblatt M, Timmer J, Kaschek D. Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models. Frontiers in Cell and Developmental Biology. 2016;4. pmid:27243005
View Article
PubMed/NCBI
Google Scholar

[70] View Article

[71] PubMed/NCBI

[72] Google Scholar

[ref20] 20. Fiedler A, Raeth S, Theis FJ, Hausser A, Hasenauer J. Tailored parameter optimization methods for ordinary differential equation models with steady-state constraints. BMC Systems Biology. 2016;10(1). pmid:27549154
View Article
PubMed/NCBI
Google Scholar

[74] View Article

[75] PubMed/NCBI

[76] Google Scholar

[ref21] 21. Terje Lines G, Paszkowski Ł, Schmiester L, Weindl D, Stapor P, Hasenauer J. Efficient computation of steady states in large-scale ODE models of biochemical reaction networks. IFAC-PapersOnLine. 2019;52(26):32–37.
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref22] 22. Fröhlich F, Loos C, Hasenauer J. 1. In: Sanguinetti G, Huynh-Thu VA, editors. Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes. New York, NY: Springer New York; 2019. p. 385–422. Available from: https://doi.org/10.1007/978-1-4939-8882-2_16.

[ref23] 23. Villaverde AF, Fröhlich F, Weindl D, Hasenauer J, Banga JR. Benchmarking optimization methods for parameter estimation in large kinetic models. Bioinformatics. 2018;35(5):830–838.
View Article
Google Scholar

[82] View Article

[83] Google Scholar

[ref24] 24. Raue A, Schilling M, Bachmann J, Matteson A, Schelke M, Kaschek D, et al. Lessons Learned from Quantitative Dynamical Modeling in Systems Biology. PLoS ONE. 2013;8(9):e74335. pmid:24098642
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref25] 25. Naumann U. The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation. No. 24 in Software, Environments, and Tools. Philadelphia, PA: SIAM; 2012. Available from: http://bookstore.siam.org/se24.

[ref26] 26. Fröhlich F, Kaltenbacher B, Theis FJ, Hasenauer J. Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks. PLOS Computational Biology. 2017;13(1):1–18. pmid:28114351
View Article
PubMed/NCBI
Google Scholar

[90] View Article

[91] PubMed/NCBI

[92] Google Scholar

[ref27] 27. Schmiester L, Schälte Y, Fröhlich F, Hasenauer J, Weindl D. Efficient parameterization of large-scale dynamic models based on relative measurements. Bioinformatics. 2019;36(2):594–602.
View Article
Google Scholar

[94] View Article

[95] Google Scholar

[ref28] 28. Hass H, Loos C, Raimúndez-Álvarez E, Timmer J, Hasenauer J, Kreutz C. Benchmark problems for dynamic modeling of intracellular processes. Bioinformatics. 2019;35(17):3073–3082. pmid:30624608
View Article
PubMed/NCBI
Google Scholar

[97] View Article

[98] PubMed/NCBI

[99] Google Scholar

[ref29] 29. Villaverde AF, Pathirana D, Fröhlich F, Hasenauer J, Banga JR. A protocol for dynamic model calibration. Briefings in Bioinformatics. 2021.
View Article
Google Scholar

[101] View Article

[102] Google Scholar

[ref30] 30. Lemke M, Cai L, Reiss J, Pitsch H, Sesterhenn J. Adjoint-based sensitivity analysis of quantities of interest of complex combustion models. Combustion Theory and Modelling. 2019;23(1):180–196.
View Article
Google Scholar

[104] View Article

[105] Google Scholar

[ref31] 31. Städter P, Schälte Y, Schmiester L, Hasenauer J, Stapor PL. Benchmarking of numerical integration methods for ODE models of biological systems. Scientific Reports. 2021;11(1):2696. pmid:33514831
View Article
PubMed/NCBI
Google Scholar

[107] View Article

[108] PubMed/NCBI

[109] Google Scholar

[ref32] 32. Oellerich T, Emelianenko M, Liotta L, Araujo RP. Biological networks with singular Jacobians: their origins and adaptation criteria. bioRxiv. 2021.
View Article
Google Scholar

[111] View Article

[112] Google Scholar

[ref33] 33. Contejean E, Devie H. An Efficient Incremental Algorithm for Solving Systems of Linear Diophantine Equations. Inf Comput. 1994;113(1):143–172.
View Article
Google Scholar

[114] View Article

[115] Google Scholar

[ref34] 34. Pasechnik D. On computing Hilbert bases via the Elliot-MacMahon algorithm. Theoretical Computer Science. 2001;263.
View Article
Google Scholar

[117] View Article

[118] Google Scholar

[ref35] 35. De Martino A, De Martino D, Mulet R, Pagnani A. Identifying All Moiety Conservation Laws in Genome-Scale Metabolic Networks. PLOS ONE. 2014;9(7):1–11. pmid:24988199
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

[ref36] 36. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski Ł, Lines GT, et al. AMICI: high-performance sensitivity analysis for large ordinary differential equation models. Bioinformatics. 2021;37(20):3676–3677. pmid:33821950
View Article
PubMed/NCBI
Google Scholar

[124] View Article

[125] PubMed/NCBI

[126] Google Scholar

[ref37] 37. Hindmarsh AC, Brown PN, Grant KE, Lee SL, Serban R, Shumaker DE, et al. SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. ACM Trans Math Softw. 2005;31(3):363–396.
View Article
Google Scholar

[128] View Article

[129] Google Scholar

[ref38] 38. Fröhlich F, Weindl D, Schälte Y, Pathirana D, Paszkowski L, Lines GT, et al.. AMICI: High-Performance Sensitivity Analysis for Large Ordinary Differential Equation Models; 2022. Available from: https://doi.org/10.5281/zenodo.6850965.

[ref39] 39. Schälte Y, Fröhlich F, Stapor P, Vanhoefer J, Weindl D, Jost PJ, et al.. pyPESTO—Parameter EStimation TOolbox for python; 2022. Available from: https://zenodo.org/record/6606687.

[ref40] 40. Fröhlich F, Sorger PK. Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models. PLOS Computational Biology. 2022;18(7):1–28. pmid:35830470
View Article
PubMed/NCBI
Google Scholar

[133] View Article

[134] PubMed/NCBI

[135] Google Scholar

[ref41] 41. Froehlich F, Weindl D. Fides; 2021. Available from: https://doi.org/10.5281/zenodo.6038127.

[ref42] 42. A collection of mathematical models with experimental data in the PEtab format; 2022. Available from: https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/.

Figures

Abstract

Author summary

Introduction

Methods

Mathematical modeling and simulation

Experimental data

Parameter estimation

Adjoint sensitivity analysis (ASA)

Adjoint sensitivity analysis at steady state (ssASA)

Post-equilibration case.

Pre-equilibration case.

Applicability of the proposed method.

Implementation

Results

Accuracy with ssASA is preserved

Simulation with ssASA is faster

Optimization with ssASA is faster

Discussion and conclusion

Supporting information

S1 Text. Supplementary notes on the ssASA method derivation, accuracy assessment and AMICI implementation, as well as the conversion reaction example.

References