Abstract
In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading) have commented on our paper in which we proposed a simple scaling method to test structural identifiability. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries. We agree with the limitations raised by these authors but, also, we emphasize that the method is still valuable for its applicability to a wide variety of models, its simplicity, and even as a tool to introduce the problem of identifiability to investigators with little training in mathematics.
Citation: Castro M, de Boer RJ (2021) The limitations, dangers, and benefits of simple methods for testing identifiability. PLoS Comput Biol 17(10): e1009425. https://doi.org/10.1371/journal.pcbi.1009425
Editor: Hugues Berry, Inria, FRANCE
Received: July 22, 2021; Accepted: September 6, 2021; Published: October 14, 2021
Copyright: © 2021 Castro, de Boer. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was funded by Agencia estatal de investigación (AEI) and Fondo Europeo de Desarrollo Regional (Grant Number: PID2019-106339GB-I00) awarded to MC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading [1]) have commented on our paper in which we proposed a simple scaling method to test structural identifiability [2]. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries (we indeed indicated but not investigated the importance of generalizing the method to other symmetries). Thus, we agree that our simple method provides a necessary but not sufficient condition for identifiability, and we appreciate their careful analysis and constructive criticism.
We nevertheless think that the simple method remains useful because it is so simple. Even for investigators with little training in mathematics, the method provides a necessary condition for structural identifiability that can be derived in a few minutes with pen and paper. Similarly, we have found its pedagogic strength by teaching the method to our own graduate students and colleagues. More advanced methods (such as STRIKE-GOLDD [3,4], COMBOS [5], or SIAN [6]) are typically intimidating for researchers with a background in Biology or Bioinformatics. This simple method can help those practitioners to familiarize themselves with the identifiability problem and better understand their models.
Finally, it is worth noting that if scaling invariance is the only symmetry (as it was in all the cases we analyzed), our SIM remains valuable (albeit uncontrolled), and surprisingly effective for a wide variety of problems (as the extensive list collected in the Supplementary Material our paper [2]). We guess that the SIM especially fails when applied to linear models (as more potential rotations of the variables leave the system invariant), and in non-linear scenarios where some parameters are identical. For instance, the FitzHugh-Nagumo model raised by Villaverde and Massonis,
could have been written as
where λ1 = λ2 = λ3 = 1/λ4 = c. One of the reasons why our method fails, in this case, might be these additional symmetries introduced in this more elaborate notation of the model.
Hence, it is worth understanding generic conditions under which the SIM method is expected to be fragile, possibly using STRIKE-GOLDD to test large families of nonlinear models.
As a final remark, we appreciate that Villaverde and Massonis have shared their source code, so researchers might have a gold standard to test identifiability.
References
- 1. Villaverde AF, Feixas GM. On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading. bioRxiv. 2020.
- 2. Castro M, de RJ. Testing structural identifiability by a simple scaling method. PLoS Comput Biol. 2020;16 (11):e1008248. pmid:33141821
- 3. Villaverde AF, Barreiro A, Papachristodoulou A. Structural Identifiability Analysis via Extended Observability and Decomposition. IFAC-PapersOnLine. 2016;49 (26):171–7.
- 4. Villaverde AF, Barreiro A, Papachristodoulou A. Structural Identifiability of Dynamic Systems Biology Models. PLoS Comput Biol. 2016;12 (10):1–22. pmid:27792726
- 5. Meshkat N, Kuo CEz DSJ. On finding and using identifiable parameter combinations in nonlinear dynamic systems biology models and COMBOS: a novel web implementation. PLoS One. 2014;9 (10):e110261. pmid:25350289
- 6. Hong H, Ovchinnikov A, Pogudin G, Yap C. SIAN: software for structural identifiability analysis of ODE models. Bioinformatics. 2019;35 (16):2873–4. pmid:30601937