Fitting and comparison of calcium-calmodulin kinetic schemes to a common data set using non-linear mixed effects modelling

Domas Linkevicius; Angus Chadwick; Guido C. Faas; Melanie I. Stefan; David C. Sterratt

doi:10.1371/journal.pone.0318646

Peer Review History

Original SubmissionNovember 1, 2024
29 Nov 2024 Decision Letter - Pan Li, Editor Transfer Alert This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present. PONE-D-24-49899Fitting and comparison of calcium-calmodulin kinetic schemes to a common data set using non-linear mixed effects modellingPLOS ONE Dear Dr. Linkevicius, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jan 13 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Pan Li, PhD Academic Editor PLOS ONE Journal Requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: I Don't Know ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The main contribution of the manuscript is fitting a set of calcium-calmodulin kinetic models to data from two different sources viz., dynamical/transient data from Faas et.al. [5] and data of equilibrium states from Shifman et. al. [11]. These models are borrowed from the literature and these correspond to varying levels of complexity in their Ca2+ binding schemes. In my opinion, the manuscript lacks novelty, since, there is no new modelling scheme that the authors propose, nor do they propose novel schemes for data fitting. All the models as well as data fitting schemes used in the manuscript are known and the only contribution of the authors is the use of data from two different sources in order to fit parameters that match both data sets using well known methods. The other contribution is an analysis of the six different model schemes in the light of the new parameters that they have estimated. Since there is no novelty in terms of a modelling or estimation/data fitting approach or an experimentally discovered unknown biological phenomenon, I believe that the manuscript is not eligible to be published in PLOS One. In summary, there is neither a mathematical/computational novelty nor a biological/experimental novelty in the manuscript. Another general remark that I have is the lack of explanation of the parameter estimation procedure. Although the authors have mentioned exactly the known packages/programming languages that have been used, the authors could put more efforts in explaining in detail the underlying mechanism for estimating the parameters. I have a very specific remark regarding the Methods section in page 9. It is not clear what F denotes in equation (7). Some other remarks are as follows: 1. In page 6, 7, one of the references does not appear, instead it is replaced with ??, see e.g. the end of line 3 in page 7 and the caption of Fig. 3 in page 6. 2. The second paragraph of page 7 needs more explanation. Why the nine states described in the beginning of the paragraph can be reduced to 5 states? What is the biological implication of this reduction? 3. At several places, the term "rates" actually means "rate constants". For example, the term "rates" has been used to mean "rate constants" in Table 1. By the way, no rows have been highlighted in this Table as claimed at the beginning of its caption. 4. There is a typo in line 3 of page 10. Correct the spelling of "available". 5. You have mentioned in page 12 as well as in the Discussion section that the parameters that you have estimated fit the data of Faas et.al. [5] and Shifman et. al. [11] better than the parameters in existing literature in many cases as also seen from the RMSE values in Table 1. I think this is obvious, because the published rate constants may have been derived using other sets of data. 6. In page 16, 17, there is a talk of frequency, periodic signal trains etc. How is this incorporated in your model and how did you simulate these conditions? The model that is currently described in equation (5) does not depend on any of these characteristics (frequency/signal trains etc.). Reviewer #2: Terminology Line 239: The authors use the terminology of Rackaukas et al. (2022) [1] and call the function g the “structural model”. I find this to be misleading, if not plain wrong. Please compare e.g. the terminology of Duchesne et al. (2021) who use definitions that I find correct: The structural model describes the dynamics of the system (could be an ODE model as described by u’ in Equation 4), the function g would then be the parameter model, and then there is an error model (here a simple observation noise standard deviation). Also, I think that using the symbol eta as a parameter in equation 2 is a suboptimal choice, because in my mind, eta is the deviation of a random effect parameter from the typical value, i.e. should have a mean of zero (also compare [2]). Correspondingly, only the random effect covariance Omega enters the random effects in Figure 4, while the eta in equation 2 is also influenced by mu, which would be part of the fixed effects parameter vector theta. The use of eta for the uncaging model led me to believe that a random effect was only assumed for the uncaging of calcium, and not for all the individual rate constants. Please clarify early in the manuscript where random effects were employed. It seems to me, that calmodulin molecules should be sufficiently homogeneous that no random effects are needed to account for “interindividual differences”, but I am not sure from the model description if random effects were incorporated for the k values. Optimization method The explanation to equation (8) suggests that the conditional likelihood was optimized. If I read this correctly, the covariance matrix Omega was fixed in the optimizations? If this is the case, then the formulation “requires appropriate constraints on Omega” (line 266) is misleading. Please clarify. Model comparison Only the AIC is explained in the Methods section, but in Table 1, the AIC values are not shown but a comparison of RMSE values by using Cohens d. Please clarify in the methods section how the model comparison was set up (20 model fits), how Cohens d was calculated and if it is appropriate (is the number of model parameters equal in the comparisons?). This leads me to the AIC – please document how the number of parameters is counted. This is not self-understood in the mixed modelling community, as far as I know different versions exist. If you use the conditional likelihood in the AIC omputation I think this means that you actually have to count all individual random effects eta_n to have a valid comparison. Please specify the method of likelihood computation used for the AIC values in Table 2. You mention it is easy to calculate (line 315), but e.g. the saemix R package provides three different methods for the likelihood computation, because the full likelihood does not have closed form and has to be approximated, e.g. by Gaussian quadrature, importance sampling or a linearization. “Model comparison via AIC” Reading this section (line 426ff) makes me wonder if the increasing complexity of the models is penalized properly, as the observed variables are quite simple and do not allow to distinguish between all the different states of calmodulin. But maybe I am too suspicious and your model specification and fitting method is just very efficient. Parameter correlations In NLME models, if a parameter correlation is suspected, this can be specified in the random effects covariance matrix Omega. Now I still do not know if random effects were allowed for the k values. By now this should be very clear to the reader (but I cannot exclude that it is just me not being able to comprehend, please excuse me if this is the case). Various further notes: The acronym NLME should be introduced at its first occurrence. p. 6/31 One reference is not correctly specified in the legend to Figure 3 (“??”). Presumably the same reference concerning Scheme 4 is also not resolved on page 7/31. p. 7 Please specify the reference by author and year, in addition to the number on line 200 p. 7 Parameter eta in equation 1: is that a random variable? What distribution is assumed, if yes? The fact that Equation 2 fits better than Equation 1 means that the strength of the laser pulse is completely random and cannot be controlled by specifying a PCD to a relevant degree. I think this should be clarified. Line 233: “The basis of NLME”? Please clarify. Maybe “The basic NLME”? Line 605: Closing parenthesis missing Line 655: Spike train instead of spike trains? References: Please fix ref 11. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0318646.r001
Revision 1
7 Jan 2025 Author Response We are grateful to the Reviewers for their insightful comments and observations regarding our manuscript. We will first address what we see as the major points of discussion, followed by more minor points. Reviewer #1 wrote: “In my opinion, the manuscript lacks novelty, since, there is no new modelling scheme that the authors propose, nor do they propose novel schemes for data fitting.” It is true that the current manuscript does not contain a novel calmodulin scheme or a new fitting technique. However, creation of novel models or fitting techniques is not the only possible source of novel scientific insight. The calmodulin model comparison work contained in this manuscript has not been done before and, therefore, leads to new insights: the needlessness of new calmodulin modeling schemes given the current available data, the level of detail necessary to model calcium-calmodulin binding. Furthermore, the work highlights the potential pitfalls of using calmodulin schemes that are too simple to model calcium-calmodulin binding, which calls for a reassessment of existing work. Therefore, despite the manuscript not containing new modeling schemes or parameter estimation methods, it is still original and insightful. Reviewer #1 also wrote: “You have mentioned in page 12 as well as in the Discussion section that the parameters that you have estimated fit the data of Faas et.al. [5] and Shifman et. al. [11] better than the parameters in existing literature in many cases as also seen from the RMSE values in Table 1. I think this is obvious, because the published rate constants may have been derived using other sets of data.” The fact that our estimates result in better fits than the previously published rate constants is obvious only at a first glance. Upon deeper reflection it brings up a number of important points, especially taking into account the magnitude of the difference in some comparisons. Even though different published rate constants have been derived using different sets of data, there still ought to be broad agreement in model predictions if the data sets are broadly consistent. If the model predictions are not consistent (especially qualitatively, rather than quantitatively), then either the different data sets are contradictory or some of the models are not powerful enough to fit certain data sets (or both). Our comparisons revealed a number of cases where the differences are qualitative, rather than quantitative, namely Schemes 1–3 and parameters from Pepke et al. show significant qualitative differences from our fits. This, however, is not the case for the Faas et al. or the Byrne et al. parameters, where the differences are much smaller and keep to the same qualitative trends as our fits. Therefore, yes, it is unsurprising that our fits result in lower errors on the data we use to fit them than other models which were tuned to different data sets, but the size of errors is, in some cases, surprising. Reviewer #1 also pointed out that the parameter estimation procedure could be explained better. We have added additional explanations, expanding and deepening the level of explanation. We have also addressed the following list of more minor remarks (following the numbering in the review): 1. In page 6, 7, one of the references did not appear; this has been corrected. 2. The explanation of why the nine states described in the beginning of the second paragraph of page 7 can be reduced to 5 states was found to be lacking. We clarified this point and used more precise language, making it clear that this is not a conceptual reduction of the model, but simply of the states being recorded, the underlying kinetic scheme is the same. 3. The term “rates” was often used instead of “rate constants”; this has been corrected. 4. We have fixed the spelling error pointed out by the Reviewer. 5. This point about the argument about our parameters fitting better than previous ones because previous parameters have been tuned to other data has been addressed in the paragraph above. 6. The Reviewer pointed out that it is not clear how the model in equation (5) represents frequency/signals. We have added additional description to clarify that calcium injections are a part of the differential equation system and can be used to model calcium signals of different frequency or amplitude. Moving on to the comments provided by Reviewer #2, the Reviewer wrote: “The fact that Equation 2 fits better than Equation 1 means that the strength of the laser pulse is completely random and cannot be controlled by specifying a PCD to a relevant degree. I think this should be clarified.” We appreciate why the reviewer would draw this conclusion. With our approach, we do not claim that uncaging is completely independent of PCD; rather we use a single equation to capture both uncertainty in estimating the PCD, as well as other sources of variance. The amount of variance captured by the specified PCD value and finer points of calcium uncaging modeling are left for future research. We have included a statement to this effect in the revised version. We addressed the feedback by Reviewer #2 concerning the usage of terminology and adjusted the relevant parts to follow Duchesne et al. (2021). We clarified and reduced the abuse of notation for the NLME parameters. We changed the symbol η to a more neutral symbol x to show that it is simply a description of a function. We clarified that the only place we used random effects is the calcium uncaging fraction. We clarified how the random effect prior distribution parameters were constrained. We expanded the section on model comparison, specifying that Laplace approximated marginal likelihood was used in AIC computations, described how and why we used Cohen’s d. We also addressed the further more minor notes provided by Reviewer #2, correcting the missing references, parentheses and other grammatical errors. We again thank the Reviewers for their time, insightful feedback and contributing to the scientific community. We hope that our clarifications and revisions address their feedback satisfactorily. Many thanks for your consideration, Domas Linkevicius Attachments Attachment Submitted filename: Respone_to_reviewers.pdf https://doi.org/10.1371/journal.pone.0318646.r002
21 Jan 2025 Decision Letter - Pan Li, Editor Fitting and comparison of calcium-calmodulin kinetic schemes to a common data set using non-linear mixed effects modelling PONE-D-24-49899R1 Dear Dr. Linkevicius, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Pan Li, PhD Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0318646.r003
Formally Accepted
25 Jan 2025 Acceptance Letter - Pan Li, Editor PONE-D-24-49899R1 PLOS ONE Dear Dr. Linkevicius, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Pan Li Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0318646.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .