Data

Dynamical calmodulin data.

We use the calcium uncaging data from Faas et al. [5], in which different concentration mixes of the fluorescent Ca²⁺ indicator OGB-5N, the light sensitive Ca²⁺ chelator DM-nitrophen (DMn), calmodulin and titrated free Ca²⁺ were used to make seven different groups of solutions A–G (see S2 Appendix for specific concentrations). For different batches of each group of solutions, a sequence of laser pulses of increasing strength was used to induce Ca²⁺ uncaging from DMn while OGB-5N fluorescence was observed at 35°C. The stronger the pulse, the more calcium is released. Due to technical limitations, it was not possible to predict the amount of released calcium for each laser pulse precisely. We elaborate on how we model the fraction of uncaged Ca²⁺ below. Fig 1 shows the fluorescence time courses for three of the seven groups—A, B and G—and different uncaging laser strengths. We use a subset of the data and split it into training, validation and test data sets (see Supplemental Text S1 Appendix for more information).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Relative fluorescence (ΔF/F₀) time-series data from [5] for three different initial condition groups (A, B and G).

Different lines within a plot are due to different laser uncaging strength (the higher the laser strength, the larger the ΔF/F₀ value).

https://doi.org/10.1371/journal.pone.0318646.g001

Calmodulin equilibrium data.

Steady-state calcium-calmodulin binding came from an experiment in Shifman et al. [11], which measured the number of Ca²⁺ ions bound per calmodulin molecule at different free Ca²⁺ concentrations (their Fig 1B). Their experimental chamber contained a fluorescent indicator Fluo4FF (5μM), calmodulin (5μM) and a varying amount of free Ca²⁺. The amount of free Ca²⁺ was titrated until a required concentration (between approximately 10⁻⁷M and 5.5 × 10⁻⁵M) was reached. We used a digital tool (https://automeris.io/WebPlotDigitizer/) to extract this data from their plots, giving the 107 points shown in Fig 2. This data was obtained at 25°C but calmodulin does not show significant temperature dependent changes in equilibrium behaviour [27], so we do not adjust for temperature dependent changes in calmodulin kinetics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Equilibrium measurements of the number of Ca²⁺ ions per calmodulin molecule from [11].

Experiments were done using 5μM Fluo4FF and 5μM calmodulin.

https://doi.org/10.1371/journal.pone.0318646.g002

Kinetic schemes and published reaction rate constants

We investigated six different calcium-calmodulin binding schemes from the literature that span the complexity of the most commonly used calmodulin models (Fig 3). The reference we give for a scheme may be its original source, or a source that is frequently cited for the scheme. There are more complex published calmodulin schemes that we did not use [14], because they would be prohibitively computationally expensive to fit.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Six calmodulin kinetic schemes to which we fit parameters, and compare to performance with published parameter values.

Scheme 1—due to strong co-operativity, each calmodulin lobe binds two Ca²⁺ ions at a time, with the lobes modelled sequentially (first C lobe then N lobe). Scheme 1 is parameterised by reaction rate constants . Scheme 2—due to co-operativity the first reaction has two Ca²⁺ ions binding as a single event and then the next two Ca²⁺ ions binding sequentially. It is parameterised by reaction rate constants . Scheme 3—fully expanded sequential calmodulin scheme where each binding event is represented individually. Depending on the reaction rate constants, the binding events could be mixed between the lobes, e.g. first binding event could be in the C lobe, the second in the N lobe, or partial combinations of different lobes. The visualised scenario is where the first two events are in the C lobe. This scheme is parameterised by reaction rate constants . Scheme 4—calmodulin binds two Ca²⁺ ions at a time and, contrary to Schemes 1–3, the lobes are independent. It is parameterised by eight reaction rate constants which, along with free Ca²⁺, are used to calculate the effective reaction rate constant (see S3 Appendix for more details). Scheme 5—this scheme has independent N and C lobes, with a single Ca²⁺ ion binding at a time. Binding sites within a single lobe are identical. It is parameterised by reaction rate constants . Scheme 6—this scheme has independent N and C lobes, with a single Ca²⁺ ion binding at a time. In contrast to Scheme 5, the binding sites within a single lobe are distinct (indicated by different shades of green/purple). The scheme is parameterised by 16 reaction rate constants . In all schemes green circles indicate Ca²⁺-bound C lobe sites, purple circles indicate Ca²⁺-bound N lobe sites and arrows indicate bidirectional reactions (Ca²⁺ ions not shown).

https://doi.org/10.1371/journal.pone.0318646.g003

The simplest scheme (Scheme 1, Fig 3), from Kim et al. [9], is made up of three calmodulin states—CaM0, CaM2Ca, CaM4Ca, respectively calmodulin bound to no, two and four Ca²⁺ ions. In Scheme 1, Ca²⁺ binding is assumed to be highly co-operative and binding of two Ca²⁺ ions is treated as a single reaction. In principle Scheme 1 does not assume which lobe binds first; the first two Ca²⁺ ions could bind to the C lobe or the N lobe. However, the parameterisation of Scheme 1 by Kim et al. [9] implies that they treat the first Ca²⁺ binding event as being to the C lobe. The published reaction rate constants in Kim et al. [9] are based on stopped-flow fluorescence measurements [7]. Scheme 1 is parameterised by four reaction rate constants , which can be used to derive two dissociation constants: and .

The next scheme (Scheme 2, Fig 3), from Bhalla and Iyengar [10], is made up of four calmodulin states—CaM0, CaM2Ca, CaM3Ca, CaM4Ca, respectively calmodulin bound to no, two, three and four Ca²⁺ ions. In Scheme 2, binding of the first two Ca²⁺ ions is assumed to be highly co-operative and treated as a single reaction, whereas the next two Ca²⁺ ions bind individually. The parameters in Bhalla and Iyengar [10] (as given in https://doqcs.ncbs.res.in/, also see [28]) do not match neatly to either lobe and the description of how the rate constants were derived was unavailable at the time of writing. Scheme 2 is parameterised by six reaction rate constants , which can be used to derive three dissociation constants: , and .

The final linear scheme that ignores calmodulin lobe-based structure (Scheme 3, Fig 3) is from Shifman et al. [11]. It comprises five calmodulin states—CaM0, CaM1Ca, CaM2Ca, CaM3Ca, CaM4Ca—respectively calmodulin bound to no, one, two, three and four Ca²⁺ ions. The dissociation constants based on experiments in Shifman et al. [11] are 7.9μM, 1.7μM, 35μM, 8.9μM respectively for Ca²⁺ binding events one to four. Reactions in this scheme do not neatly map onto individual Ca²⁺ binding sites within calmodulin lobes; instead they are abstract binding events where, depending on the parameters, they may be probabilistic combinations between different binding sites. Scheme 3 is parameterised by eight reaction rate constants , which can be used to derive four dissociation constants , , and . This scheme is the most complex linear CaM scheme possible (without adding conformational calmodulin changes), modelling each Ca²⁺ binding site individually.

Our Scheme 4 (Fig 3) is from Pepke et al. [12] and comprises four states—CaM0, CaM2C, CaM2N, CaM4Ca—respectively calmodulin bound to no Ca²⁺ ions, two at the C lobe, two at the N lobe and four across both lobes. It is the simplest scheme that captures the lobe-based structure of calmodulin. It has eight reaction rate constants and is based on Scheme 5 (described below), but was simplified used a quasi-steady state approximation for calmodulin species that have a single bound Ca²⁺ ion. This approximation results in elimination of partially bound species from simulations by setting their derivatives to 0 and expressing the partially bound species in terms of the unbound and the fully bound species and permitting the appropriate substitutions in the equations for the unbound and the fully bound species (see S4 Appendix).

We draw our Scheme 5 (Fig 3) from model 1 in Pepke et al. [12], which is identical to the scheme used in [5]. It is made up of nine states—CaM0, CaM1C, CaM1N, CaM2C, CaM2N, CaM1C1N, CaM2C1N, CaM1C2N, CaM4—with the number of Ca²⁺ ions bound to calmodulin indicated by numbers preceding C and N. Even though in total there are nine states, since in this study calmodulin does not bind to any downstream species, we do not need to track individual calmodulin molecules. Therefore, we simulate the lobes as independent species which decreases the number of states we need to track from nine to six—CaM0N, CaM1N, CaM2N, CaM0C, CaM1C, CaM2C—without changing the scheme itself. Scheme 5 is parameterised by eight reaction rate constants , which can be used to derive four dissociation constants , , and . Pepke et al. [12] used two data sources on calmodulin equilibrium behavior: (1) data from wild-type and tryptic calmodulin fragments (one lobe expressed, other eliminated) [6]; (2) data from competition assays (calmodulin, either wild type or mutants with one active and one inactive lobe, and fluorescent indicator Fluo4FF) [11]. Pepke et al. [12] (in their supplemental information) give reaction rate constants as ranges—we take specific numerical values from this model’s entry (model identifier: MODEL1001150000) in the BioModels Database [29, 30]. Faas et al. [5] tuned the model to their own UV-flash photolysis data.

Finally, our Scheme 6 (Fig 3) is from Byrne et al. [13]. It is made up of sixteen states, but similar to Scheme 5, we simulate the lobes as independent species which reduces the number of states to eight—CaM0N, CaMN₁, CaMN₂, CaM2N, CaM0C, CaMC₁, CaMC₂, CaM2C, where CaM0X denotes an unbound calmodulin lobe, CaMX₁ denotes Ca²⁺ bound to the first site of a lobe, CaMX₂ denotes Ca²⁺ bound to the second site of a lobe and CaM2X denotes a fully bound lobe. This scheme is parameterised by 16 reaction rate constants , which can be used to derive eight dissociation constants , , and . Reaction rate constants in Byrne et al. [13] are based on stopped-flow fluorescence and competitive binding assay data [31].

For each of the six schemes we use the reaction rate constants from the associated publication (we use both Pepke et al. [12] and Faas et al. [5] for Scheme 5). All of the reaction rate constants we used are given in S4 Appendix.

Ca²⁺ uncaging model

Faas et al. [5] used a linear model for laser induced Ca²⁺ uncaging (1) where U is the uncaged DMn fraction, PCD is the specified Pockels cell delay, where a larger value results in a higher energy laser pulse and more Ca²⁺ uncaging, and x is used to account for the uncertainty of the actual PCD value, i.e. the difference between the specified and physically realised values.

Even though Faas et al. [5] used the linear model successfully in their study, its performance is not quantified and it has some limitations. Most importantly, U is not bounded to [0, 1] and can take negative values or values above 1, which would result in physically unrealistic initial conditions. Moreover, it is not clear that a simple linear relationship is optimal to accurately model the relationship between the PCD value and the uncaging fraction. Finally, the variable x is additive, and it is not clear that this formulation is optimal—it could be multiplicative or some more complex functional relationship.

Due to lack of the necessary data, we could not develop our own model of how the fraction of calcium uncaged depends on the PCD. Instead, of the linear model (Eq 1) we use an even simpler model that performed better in practice than either the linear model from Faas et al. [5] or a neural network. Our uncaging model does not take the specified PCD value and is a simple sigmoid function, bounding its output to [0, 1]. (2) With this approach, we do not claim that uncaging is completely independent of PCD; rather we use a single equation to capture both uncertainty in estimating the PCD, as well as other sources of variance.

Fitting our reaction rate constants

We combine and adapt the definitions and notation of NLME provided in [24, 32] and we present it using Scheme 1 as an example. NLME modeling framework comprises a two level hierarchical structure (shown visually in Fig 4) with fixed effects Θ at the upper level, which can be broadly grouped into

• model parameters θ, e.g. for scheme 1—reaction rate constants
• random effect prior distribution parameters Ω, e.g. μ and ω used to parameterize the prior of η_n for the Ca²⁺ uncaging fraction (see paragraph below)
• observation model noise parameters σ

and do not vary between recordings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Visual representation of an NLME model, rectangle nodes in the top box denote parameters (fixed effects), circles denote random quantities which are either latent (unfilled) or observed (filled), diamonds are deterministic given the inputs, and nodes without a border are constant.

Each symbol in the node can be either a number or a vector depending on the context.

https://doi.org/10.1371/journal.pone.0318646.g004

The lower level is random effects η_n which account for the inter-individual variability of the observations y_nj, generally, by individualizing model parameters θ. We assume that both Ca²⁺ and calmodulin molecules are identical between experiments, hence reaction rate constants do not have the random effect-enabled individualization for each experimental run. The sole usage of random effects in this paper is in fitting the fraction of uncaged Ca²⁺ by passing to Eq 2.

Furthermore, there is a set of covariates Z_n associated with each recording, i.e. the total concentration of calmodulin, Ca²⁺ and OGB5N, which are known. These three sets of values are collated via the parameter model g into the dynamical parameter vector p_n of the nth recording (3) The dynamical parameters p_n are then fed into the structural model (e.g. an ordinary differential equation (ODE) system) (4) where u are the dynamical variables being solved for (DMn, OGB5N, Ca²⁺ and their combinations and the various calmodulin species determined by the scheme being used). For Scheme 1 the system of equations (for brevity omitting DMn, OGB5N, Ca²⁺ and means of its input amplitude, frequency and duration into the model) would be (5) Note that enter into Eq 4 via p_n, which can also be used to initialize the ODE system.

The final step is to link the numerical solution of the ODE system to the experimentally observed quantities. The jth observable quantity y_nj for the nth entity is calculated using the simulated variables u_n(t) and the times t_m at which the observations were made via the observational model h (6) In this study there are two observable quantities: the relative fluorescence ΔF/F₀ over time being fit to recordings from Faas et al. [5] and Ca²⁺ per calmodulin at equilibrium being fit to Shifman et al. [11]. ΔF/F₀ is derived from OGB5N as follows (7) where F_max/F_min = 39.364 [5] and F_max, F_min are maximal and minimal recorded fluorescence values. Ca²⁺ per calmodulin is simply the sum of calmodulin species multiplied by the number of bound Ca²⁺ ions for each species divided by total calmodulin. After obtaining the observable quantities a Gaussian observation model is used to account for observational noise.

There are many ways to fit NLME models, both frequentist and Bayesian [33]. In this study we use the maximum aposterior (MAP) conditional log-likelihood objective which can be stated as (8) where Θ* is the mode of the fixed effects, η* is the mode of the random effects for each subject and p(Θ) is the fixed effect prior distribution. Conditional likelihood is much more numerically efficient due to Θ and η_n being optimized jointly whereas, for example, marginal likelihood generally requires a two level optimization scheme and Markov Chain Monte Carlo requires many more likelihood evaluations due to sampling. However, conditional likelihood requires appropriate handling (either fixing or priors, see next section) of Ω to avoid overly broad random effect distributions which barely penalize extreme η_n values and effectively result in different individual models due to the learning being offloaded mostly to the random effects.

We use the Pumas.jl [24] Julia package to fit to solve Eq 8. Pumas.jl contains efficient and powerful algorithms for NLME modelling, which was essential when fitting the ηs used to model the uncaging fraction. Specifically, we used the BFGS optimization algorithm from Optim.jl with the gradient calculations handled by Pumas.jl.

All fitting was done on the JuliaHub (https://juliahub.com/) cloud computing platform using nodes with 8 vCPUs and 64GB of memory. Individual fits took between one to ten minutes, depending on which scheme was used and whether some of the parameters were fixed. For each scheme we conducted 20 fitting runs with different initial conditions. All the code that was used to define the models, run the simulations and perform the analysis is accessible at https://github.com/dom-linkevicius/FaasCalmodulin.jl.git, the data is accessible at [34].

Prior distributions

We incorporated the existing knowledge about calmodulin reaction rate constants for different kinetic schemes via per-scheme prior distributions that depend on the amount data available for each scheme. All of our priors are in log₁₀ space as optimizing rate constants in log-space was more performant.

For Schemes 1 and 2, since they are simplified and contain fewer reaction rate parameters than an actual calmodulin molecule would, mapping from experimental data to reaction rate constants is difficult. Therefore, we opted to use wide uniform priors that reflects the small amount of available prior information: for the forward reaction rate constants (corresponds to 10² M^-2ms^-1 to 10⁹ M^-2ms^-1) and for the dissociation constants (corresponds to a range of 1 nM² to 10 μM²).

For Scheme 3, which models Ca²⁺ binding events individually, there is a significant amount of prior information. Specifically, we use set priors on the dissociation constants based on Shifman et al. [11]. We used priors of the form , where r is a dissociation constant from Shifman et al. [11] Table 2 in log₁₀. Unfortunately, setting a prior that could similarly constrain the forward reaction rate constants was not possible, therefore we again opted for wide uniform priors that we used for Schemes 1 and 2: .

For Schemes 4 and 5, since they share the same set of reaction rate constants, we used the same set of prior distributions. For each forward reaction rate and dissociation constant we used , where r are reaction rate constants from Faas et al. [5] in log₁₀. We chose Faas et al. [5], rather than Pepke et al. [12], because their rate constants are based on dynamical data and upon initial simulation runs were performing better. Similarly for Scheme 6, we used the same approach, but centered the Gaussian priors on parameters from Byrne et al. [13].

Finally, we restrict the values of μ and ω which parameterize the prior distribution of random effects to avoid over-fitting due to the usage of the MAP conditional likelihood as the optimization objective. Specifically, we use and limit its domain to [1, ∞], as well as limiting the domain of μ to be in [−5, 5]. We found that these were the minimal set of restrictions that prevented over-fitting of Ω.

Numerical ODE solving

We use the Julia programming language for numerical ODE solving both during and outside of parameter fitting. Specifically, we use the DifferentialEquations.jl package [35]. We use the Rodas5P numerical solver which can handle significant stiffness in the ODE system and which performed the best of the methods tried. We used it with the default settings, except for reducing the absolute error tolerance to abs_tol = 1e-16 since some simulations that contained low concentrations of species suffered from significant errors in the numerical solution.

Model comparison

There are many ways to compare model performance, but for the purposes of this study we use two metrics: root-mean-square error (RMSE) and the Akaike information criterion (AIC) [26]. The RMSE for a single experimental observation vector and model prediction is defined as (9) We calculated the RMSE values for each recording in each of the test data sets for each of the 20 optimization runs for a given scheme, pooling them. We also calculated the RMSE values for the same scheme but with published reaction rate constants, again pooling them. This gave us two samples of RMSE values, r₁ that was obtained using the reaction rate constants we fitted and r₂ that was obtained using the published reaction rate constants. We then compared r₁ and r₂ using a two sample T-test (assuming unequal variance) and calculated Cohen’s d to establish the effect size of using our rates and the ones published in the literature. Cohen’s d is defined as (10) where and are the means of each sample and s is the pooled variance. We used the RMSE to focus directly on a models predictive performance.

In contrast, we used the AIC for selecting the model that performs the best when its complexity is taken into account. The AIC for model with parameters θ and given data d is defined as (11) where k is the length of the parameter vector θ (in this paper—reaction rate constants for a particular scheme, noise parameter σ and random effect prior parameters μ and ω). Even though in model optimization we use the conditional likelihood, in AIC calculations we used the marginal likelihood obtained via the Laplace approximation [36]. The AIC is a measure that evaluates model performance, but also penalizes model complexity via the 2k term. There are many other model comparison metrics [37], but the AIC is sufficient for the present study due to the inclusion of predictive model performance and penalizing model complexity along with it being computationally simple to calculate. For each of the 20 different optimization runs we calculated the AIC value for the test data set of a run using the given scheme with our reaction rate constants, as well as if the published reaction rate constants were used. This gave us one sample of AIC values per combination of scheme + reaction rate constants.

General model fitting results

We fit each of the six kinetic schemes shown in Fig 3 to the fluorescence traces from Faas et al. [5] and the steady state calcium-calmodulin binding data from Shifman et al. [11]. We used the root mean square error (RMSE) to evaluate the goodness of fit between the models and the data. The fitting procedure was repeated 20 times with different random seeds, which set the random sampling of the training, validation and test data (see S1 Appendix) along with the initial parameters for optimization. Therefore, due to variability in training data and initial parameters, RMSE values (especially for our parameter fits) for each seed can be significantly different.

We now compare the performance of each kinetic scheme with the reaction rate constants we fit and the published ones. Table 1 shows the training (split between dynamical data from Faas et al. [5] and equilibrium data from Shifman et al. [11]), validation and test data set performance summary statistics for the six investigated kinetic schemes, reporting the average RMSE values for the 20 different seeds used. The distribution of the test RMSE values is shown in Fig 5. On average Schemes 5–6 performed the best, whereas other schemes were not able to capture either the equilibrium data (Schemes 3–4) or both the dynamical and the equilibrium data (Schemes 1–2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Violin plots of RMSE values for the test data set for each seed for all schemes for our own and the published parameter sets.

https://doi.org/10.1371/journal.pone.0318646.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of training, validation and test performance (RMSE ± SD) for different kinetic schemes with either parameters fit from scratch, fixed to values from publications or our modifications.

https://doi.org/10.1371/journal.pone.0318646.t001

Dynamical behaviour.

To illustrate the differences between model fits, Fig 6 shows the measured fluorescence traces that were used as the validation data set for seed 1 and corresponding model predictions for each kinetic scheme with our fitted rate constants and the published rate constants. Each row shows the same 24 experimental traces (black), split between the seven groups of solutions which were used experimentally in Faas et al. [5]. Within each group, each trace corresponds to a different laser uncaging strength used—the stronger the laser, the more calcium gets released, the larger the ΔF/F₀ values that are measured.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Sample dynamics for validation data for seed 1 for all trained models.

Each column is a single data group A–G from Faas et al. [5], whereas each row is a different combination of scheme + reaction rate constants (specific combination given in the rightmost column). In all subplots y axis is ΔF/F₀ and x axis is time. Black lines are empirical data and red lines are model outputs. Scale bars are given at the bottom row, each tick on the y-axis corresponds to the same value in each column.

https://doi.org/10.1371/journal.pone.0318646.g006

The biggest differences in measured and predicted dynamics are for Scheme 1, for which the RMSE differences are also the largest. In Fig 6 the main difference between our parameters and rate constants from Kim et al. [9] is that our rate constants give rise to traces that follow the experimental dynamics to some extent, whereas the published rate constants simply equilibrate to a value and barely display any dynamics (e.g. groups D–G). However, even though our rate constants result in dynamical behaviour, they do not show good equilibrium performance and in fact do not reach equilibrium when the data has long reached it.

The comparison of dynamics for Scheme 2 is similar to that of Scheme 1. Comparing our reaction rate constants with those in Bhalla and Iyengar [10], we see that the published reaction rate constants make the system equilibrate and not follow the data closely, whereas our reaction rate constants achieve a more accurate fit. However, with either our rate constants or the published rate constants, the traces and the average RMSE values indicate that Scheme 2 fits the data quite poorly.

Models with the level of complexity of Scheme 3 and higher are able to capture the dynamical data much better than the simpler Schemes 1 and 2. As shown in Fig 6, both Scheme 3 models perform adequately. However, our reaction rate constants trained from scratch still perform better, especially in capturing the initial rise and fall in ΔF/F₀ (for example see columns D–F). Note that for this scheme the comparison is not entirely equivalent to other cases as we had to fit the forward reaction rate constants while we kept the dissociation rate constants fixed to those in Shifman et al. [11].

For Scheme 4, our reaction rate constants significantly outperform those of Pepke et al. [12] as shown in Fig 6. This is reflected in a smaller mean RMSE value of our rate constants and is evident in most experimental groups, where our rate constants result in reasonably accurate predictions whereas the Pepke et al. rate constants significantly under-predict ΔF/F₀.

Looking at the dynamics for Scheme 5, based on the RMSE values in Table 1, our rate constants perform significantly better than the rate constants from either Faas et al. [5] or Pepke et al. [12], but the gap is much smaller for the former than the latter. The differences in dynamics between our fits and rate constants in Faas et al. are subtle, but generally our rate constants perform better for small amounts of uncaged Ca²⁺. In contrast, comparing dynamics with our rate constants to dynamics with rate constants in Pepke et al. [12], their reaction rate constants result in significant mismatches to the data, to the point that the optimization procedure has to inject amounts of Ca²⁺ that lead to incorrect equilibrium levels (see Fig 7, which shows that their dissociation constants can fit equilibrium data well).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Calmodulin equilibrium behaviour when the free amount of Ca²⁺ is varied for all kinetic schemes for the 20 seeds that we tested (solid line is the median and shaded area is the 95% confidence interval).

Model behaviour with our reaction rate constants are plotted in blue, whereas published ones are in the colors indicated. Note that at times the published reaction rate constants (Scheme 1 and Scheme 2) result in behaviours that are much more right-shifted, therefore show up as zero in the relevant range. We also include data points for wild type calmodulin for an equivalent experimental setup in Shifman et al. [11].

https://doi.org/10.1371/journal.pone.0318646.g007

Finally, for Scheme 6, the main differences between the dynamics resulting from our reaction rate constants and those in Byrne et al. [13] are generally seen for small amounts of uncaged calcium (columns D–G bottom traces). Our reaction rate constants (for this seed) managed to capture calmodulin behaviour with low amounts of Ca²⁺ more accurately. Even though for some traces the published rate constants can outperform ours (e.g. top traces in either groups A or C), our reaction rate constants on average show a smaller RMSE value.

Model comparison via AIC

We now compare both the published reaction rate constants to our reaction rate constants and between the kinetic schemes via AIC evaluated on the test set of a random seed. AIC is a useful model comparison tool because it takes into account both model predictive performance as well as model complexity (number of parameters). Fig 8 shows the box plots for the AIC values for all the combinations of kinetic scheme and parameter set for all 20 seeds. As shown in Table 2, our reaction rate constants have lower median AIC values (lose less information) compared to the published ones. Moreover, the more complex the scheme, the lower the AIC value, with Scheme 6 performing the best. The median AIC values seem to asymptote and reach a lower value by Scheme 6 (the big change is after Schemes 2–3), so only qualitatively different model improvements are likely to decrease the AIC value more.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. AIC box plots for all the different combinations of kinetic schemes and either previously published or our own reaction rate constants.

Each box plot is based on training the model on 20 different random seeds, the AIC value is calculated on the test data set of a given seed.

https://doi.org/10.1371/journal.pone.0318646.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Median AIC values for all combinations of kinetic schemes and reaction rate constants (our fits or published).

https://doi.org/10.1371/journal.pone.0318646.t002

Calculating the relative likelihoods from median AIC values, where our reaction rate constants are the reference, all the published parameter sets have negligibly low relative likelihoods (largest being on the order of e⁻¹⁰⁰). Therefore, our reaction rate constants are significantly more likely compared to the published ones.

Since our reaction rate constants have lower AIC values (lose less information) compared to the published ones, we use our reaction rate constants to compare between different kinetic schemes. Given the results in Fig 8 and Table 2 and using the median AIC for Scheme 6 with our rate constants as reference, the other schemes with our parameter sets have a negligibly small relative likelihoods (again on the order of e⁻¹⁰⁰). Therefore, of the combinations of schemes + reaction rate constants that we found, Scheme 6 with our reaction rate constants is relatively the most likely.

Calmodulin Ca²⁺ integration properties

Having established that our reaction rate constants are significantly more likely than the published ones, we now ask whether this difference is meaningful practically. To answer this question we probe the Ca²⁺ signal integration properties of calmodulin. CA1 pyramidal cell Schaffer collateral synapses undergo long-term potentiation dependent on CaMKII (and therefore on calmodulin) in response to three 1s trains of 50Hz stimulation [39]. Given these results, it is likely that calmodulin integrates the Ca²⁺ signal within a single train. Therefore, we set up a series of simulations where a model was stimulated by a 1s train of Ca²⁺ injections but the frequency was varied from 2Hz to 100Hz. Based on the results in Sabatini et al. [40], Ca²⁺ influx due to single synaptic stimulation event for a neuron at resting voltage is around 0.7μM (this mimics the experimental setup in Bayazitov et al. [39] best). To mimic the competition between calmodulin and other buffers and pumps we implemented a minimal Ca²⁺ extrusion model using values in Sabatini et al. [40] Table 1 for a CA1 pyramidal cell spine—Ca²⁺ decaying to a baseline of 100nM with a time constant τ = 12ms. Finally, we use a biologically realistic calmodulin concentration of 20μM [41]. After the simulation, we evaluate the calmodulin signal integration by calculating the area under the curve of both partially bound calmodulin and fully bound calmodulin, where bigger values indicate a larger level of Ca²⁺ signal integration.

As shown in Fig 9 columns one and two, Schemes 1 and 2 are not capable of integrating Ca²⁺ signals in the tested frequency range (except for a few outlier runs with Scheme 2). For Scheme 1 it is likely due to the fact that the models are not sensitive enough to Ca²⁺ (see Fig 7 top row). The same interpretation, however, does not hold for Scheme 2, whose equilibrium behaviour with our parameter fits was usually too sensitive to Ca²⁺ compared to experimental data. This behaviour for Scheme 2 may be explained by slow reaction rate constants compared to Ca²⁺ decay.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Ca²⁺ signal integration properties (measured as area under the curve) of partially (first row) and fully bound (second row) calmodulin species in response to a 1sec train of 2–100Hz stimulation that delivers 0.7μM Ca²⁺ per spike (same general patterns hold if 12μM Ca²⁺ per spike is delivered, results not shown).

A different calmodulin scheme is used in each column and shows our parameter fits (deep blue lines) and the published parameter values (all other colours). The solid lines are median model behaviour and shaded areas are the 95% confidence intervals.

https://doi.org/10.1371/journal.pone.0318646.g009

The results are somewhat different for Schemes 3 and 4 (Fig 9 columns three and four), where both the published reaction rate constants and our own fits show significant Ca²⁺ integration in the partially bound calmodulin species, but barely any in the fully bound calmodulin. In both cases our reaction rate constants result in significantly higher Ca²⁺ signal integration. However, results for Scheme 3 with our reaction rate constants which show significant Ca²⁺ integration at 2Hz should be taken with caution due to the same fits being overly sensitive to Ca²⁺ at equilibrium (Fig 7 third row from the top).

Finally, for Schemes 5 and 6 we see integration of Ca²⁺ signals that results in both fully and partially bound calmodulin species (Fig 9 columns five and six). For both schemes our reaction rate constants predict that Ca²⁺ signal integration would result in more partially bound calmodulin compared to predcitions from the published rate constants. As for fully bound calmodulin, our reaction rate constants predict a lower level of fully saturated calmodulin than the predcitions from the published rate constants.

The difference between the partially and fully bound calmodulin signals is more pronounced with our reaction rate constants than with the published ones. For Scheme 5 the difference is around an order of magnitude for our reaction rate constants and under 2-fold for the published reaction rate constants, whereas for Scheme 6 the difference is around 4-fold for our reaction rate constants and around 2-fold for the published reaction rate constants. Given that our models reaction rate constants perform better, we predict that partially bound calmodulin species play a more significant part in Ca²⁺ signalling integration and propagation than predicted by previously published models.

Parameter correlations

We next examine the relationships between our parameter fits within a given scheme. Analysing relationships between parameters may point future experimental research questions. For example, if some reaction rate constants are correlated, they may be under-determined. Therefore, future model development would benefit from additional, more directed data to better constrain the correlated parameters. We use partial correlation as a measure of relationship between parameters [19]. Briefly, partial correlation quantifies the degree of association between two variables when the variance from a set of controlling variables is taken into account. For example, in Scheme 1 the partial correlation between k₁ and k₃ would indicate the relationship between these two rate constants when and is accounted for.

Since structurally there is nothing to distinguish between the C and the N lobes for Schemes 4–6, we calculate the dissociation constant for the two binding reactions in Scheme 4 and for the first binding reaction for both lobes in Scheme 5 and Scheme 6 and compare their values—the one that has a lower K_D value we call the C lobe and the one that has a higher value we call the N lobe. This is to avoid what we call the C lobe being functionally the N lobe and vice versa, which would result in artificially higher parameter spread or obscure parameter correlations. We show the parameter pair plots for all six schemes in the S5 Appendix, along with tables of individual reaction rate fits. We show partial correlation coefficients for all schemes in Fig 10.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Partial correlation coefficients for our reaction rate constant fits for all schemes.

White stars in the rectangle indicate p < 0.05 with null hypothesis that the partial correlation is zero.

https://doi.org/10.1371/journal.pone.0318646.g010

First of all, even though the data set we use is richer, as it includes both the dynamical and the equilibrium data, there is still significant variance in our model parameter fits. For some parameters the pairs can span 5 to 10 orders of magnitude (see S5 Appendix). Moreover, there are significant correlations between multiple parameters in most schemes.

There is only one significant negative correlation for Scheme 1, between and . This correlation is most likely due to a limited number of degrees of freedom offered by this scheme. Assuming that a calmodulin molecule has an overall dissociation constant that is a function of the dissociation constants of individual reactions, the dissociation constants of individual reactions have to co-vary in order to maintain the same overall behaviour.

Reaction rate constants for Schemes 2 have 8/15 significant correlations which indicates a high level of sloppiness in the system. Similarly to Scheme 1, given the high level of simplification used in this scheme, in order to maintain the same overall model behaviour, parameters have to co-vary. This is especially true for the dissociation constants, which are all are negatively correlated. We also see that the on-rate for the first reaction is not correlated to any other parameter and is therefore fairly well-determined.

Scheme 3 has proportionally fewer correlations than Scheme 2 (7 out of 28 parameters correlated). Most correlations are with the dissociation constant for the final Ca²⁺ binding event. This can be explained by thinking about a general dissociation constant for calmodulin, which would be a function of , , and —since is likely better constrained due to the first binding event needing to be of particular speed to fit the rising/falling phase of the individual time series, the other dissociation constants may co-vary more freely and balance each other out. The on-rate for the first binding event is only correlated to one variable, which indicates that it is quite well determined.

Scheme 4 shows the largest fraction of correlations of all the schemes (15/28). This is most likely due to the quasi-steady state approximation which results in steady state reaction rate constants that provide a lot of room for sloppiness via products and quotients of the full set of reaction rate constants .

The point that model structure is of utmost importance in determining the levels of sloppiness in the system is further reinforced by Scheme 5, where 10 out of 28 reaction rate constants were correlated. More importantly, a significant number of correlations are within-lobe, for example k₁ and k₃—the first and second on-rate constants. There are also some cross-lobe correlations, for example and which are the second Ca²⁺ binding events for C and N lobes respectively.

Curiously, even though Scheme 6 is the most complex in terms of number of parameters and number of states, it shows only six significant correlations between reaction rate constants. Moreover, all correlations are within a lobe, rather than between lobes. More specifically, most of them are for parameters in the C lobe, rather than the N lobe.

Necessary structural components of a calmodulin model

As shown in Table 1 and in Fig 6, there is a large gap in training performance between Schemes 1–4 and Schemes 5–6. Even though training RMSE in both dynamical and equilibrium data significantly decreases going from Scheme 2 to Scheme 3, only from Scheme 5 onwards can both dynamics and equilibrium behaviour be captured well. There are two main differences between Schemes 1,2,4 and and 5–6: independence of lobes and structural assumption of co-operativity. Both Scheme 3 and Schemes 5–6 allow co-operativity (via reaction rate constants) but do not assume it structurally. Schemes 3 does not allow for independence of lobes, while Schemes 5–6 assume it structurally. In this section we provide an empirical argument that links model features to gaps in performance, focusing on event-based (as opposed to binding site-based) and structurally co-operative (especially for the C lobe) schemes to model calmodulin.

Assuming that the real calmodulin dynamics operate in a k-dimensional space, any model capable of modeling the dynamics would have to have at least that many dimensions (along with an appropriate structure). Calmodulin models framed in terms of events (fully abstracted from binding sites) can operate at most in a four dimensional linear subspace (since rank of such a network is four, see page 30 in [42]) of the five dimensional state space (see Scheme 3 in Fig 3). Therefore, an immediate conclusion of this may be that k > 4, real calmodulin dynamics operate in a higher dimensional space than an event-based model allows for. However, Scheme 5, which is able to model both calmodulin dynamics and equilibrium behaviour (see Table 1), has rank 4 as well. The main difference between Schemes 3 and 5 are the independence of the lobes: Scheme 5 contains two independent subnetworks (each of which is rank 2). Therefore, based on our results, in order to accurately model both calmodulin dynamics and equilibrium behaviour, two independent subnetworks (independence of lobes) is a necessary model feature.

We next analyze whether a structural assumption of co-operativity, modelling the binding of two Ca²⁺ ions as a single event, within calmodulin lobes is reasonable. This is not the only way of modelling co-operativity, but it results in models with a smaller state space vector and therefore can be preferable computationally. Fractional calmodulin occupancy of the N and the C lobes using a well performing model (Scheme 6 with parameters from Byrne et al. [13]) is shown in Fig 11 columns one and two. Starting with the dynamics of the partially occupied N lobe, the model predicts around 20% of calmodulin molecules would have the first site occupied, with a negligible fraction having the second site occupied. Moreover, the dynamics of partially occupied sites in the N lobe do not show fast changes over the simulated time period, so the quasi-steady state approximation would hold reasonably well. The dynamics of the C lobe paint an opposite picture. It is immediately obvious that, due to its slower speed, the quasi-steady state approximation (d[CaMC₁]/dt = 0) does not hold for the C lobe as there are calmodulin dynamics occurring over the whole simulated time of 35ms. Therefore, even though it is a theoretically appealing tool to reduce the number of calmodulin states, the quasi-steady state approximation is too inaccurate for the C lobe and results in significant errors in either calmodulin dynamics or equilibrium behaviour.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Scheme 6 behaviour with parameters from Byrne et al. [13] on dynamical data from Faas et al. [5].

Each line is one of the initial conditions (solution + uncaging strenght) Faas et al. used. Calmodulin has been normalized to total calmodulin used in an experiment, the first row shows N lobe dynamics, the second row shows C lobe dynamics. Each column shows a different calmodulin state—completely unbound (first column), Ca²⁺ bound to the first site on a lobe (second column), Ca²⁺ bound to the second site on a lobe (third column), Ca²⁺ bound to both sites of a lobe (fourth column). Above the plots we provide reaction rate constants from Byrne et al. [13] for the reactions a specific calmodulin state participates in. Note that time on the x axis is in log10 space to better show the initial dynamics.

https://doi.org/10.1371/journal.pone.0318646.g011