Fig 1.
Ultradian Endocrine model: Flow diagram.
The circles represent the three main state variables (Ip, Ii, G), the solid arrows represent the input and output flows and rate of exchange, and the dashed arrows represent functional relationships.
Table 1.
Ultradian Endocrine model: List of parameters for the model.
The search ranges are listed only for the five parameters used for the parameter discovery in our simulations.
Fig 2.
AI-Aristotle framework for gray-box identification: 1. The observed data and the partial knowledge of physics are used to train the selected neural network-based module. 2. The selection of the neural networks-based module needs to be done between (a) X-TFC, recommended for high-resolution data and missing terms discovery, and (b) PINN, recommended for sparse data and parameter estimation. The neural network outputs are the time-dependent representations of the missing terms of the dynamical systems, which are fed into the symbolic regression algorithm. 3. The selected Symbolic Regression module identifies the mathematical expressions of the missing terms. It is recommended to use both symbolic regressors for cross-validation. 4. The full knowledge of physics is now available, allowing forward modeling performance.
Fig 3.
Pharmacokinetics model: Schematic of the X-TFC algorithm.
Input weights and biases are randomly selected. The last step solves iteratively a least squares system, thus no back-propagation is involved in the training, allowing fast computational times.
Fig 4.
Pharmacokinetics model: Schematic of the PINNs algorithm for predicting the unknown term h(t; θ2) and the values of parameters simultaneously.
Here, u(t; θ1) is a vector that contains all three output states. Unlike the X-TFC network, PINN requires back-propagation, which is the expensive computational component.
Fig 5.
Ultradian Endocrine model: Schematic of the PINNs algorithm for solving a gray-box identification problem.
Table 2.
Pharmacokinetics model: Performance of X-TFC and PINNs for parameter discovery for time range [0, 50] hours.
Refer to Table 1 for X-TFC hyperparameters.
Table 3.
Pharmacokinetics model: PINN parameters setup for the discovery of unknown terms over a time range of [0, 50] hours.
The initial and second numbers in the ‘Number of Iterations’ Row represent the iterations during the primary and secondary training stages using Adam optimization. The third number corresponds to the training stage utilizing L-BFGS. The first and second numbers in the ‘Architecture of Neural Networks’ indicate the width and depth, respectively.
Table 4.
Pharmacokinetics model: Unknown term discovery for time range [0, 50] hours.
Comparison between X-TFC and PINNs performance via MAE, RMSE, RE, and computational time for different numbers of data points. The initial number in the ‘# of Iter.’ column for PINNs represents the iterations during the primary training stages using Adam optimization while the second number corresponds to the training stage utilizing L-BFGS.
Table 5.
Pharmacokinetics model: X-TFC hyperparameters setup for parameter discovery and unknown term discovery, for time range [0, 50] hours.
Fig 6.
Pharmacokinetics model: comparison between exact solution vs. X-TFC and PINNs solutions, for (A) the variables B, G, and U, with 20 data points per variable, and for (B) for the unknown term h(t).
Fig 7.
Pharmacokinetics model: (A) Comparison between exact solution B, G, and U and solution of PINNs and X-TFC with noisy data (noise std = 0.05). (B) Comparison between exact solution vs. X-TFC and PINNs solutions for unknown term h(t) with noisy data (noise std = 0.05).
Table 6.
Pharmacokinetics model with noisy data: Unknown term discovery for time range [0, 50] hours.
Comparison between X-TFC and PINNs performance via MAE, RMSE, RE, and computational time for different values of noise.
Table 7.
Ultradian Endocrine model: Parameter discovery via X-TFC and PINNs algorithms.
The performance of the two methods is given by the absolute difference between nominal values and inferred values. On the right, we also present computational times in seconds.
Fig 8.
Glucose-insulin interaction model: Comparison between exact solution vs. X-TFC and PINNs solutions for (A) the variables Ip, Ii, and G (top to bottom), and (B) unknown terms f(t) and g(t) (top to bottom).
Table 8.
Ultradian Endocrine model: Unknown terms discovery for time range [0, 1800] minutes.
X-TFC and PINNs performance in terms of MAE, RMSE, RE, number of iterations, and computational time for different numbers of data points.
Table 9.
Ultradian Endocrine model: X-TFC hyperparameters setup for parameter discovery and unknown terms discovery, for time range [0, 1800] minutes.
Table 10.
Ultradian Endocrine model: PINNs parameters setup for unknown terms discovery, for the time range [0, 1800] minutes.
The first and second numbers in the ‘Architecture of Neural Networks’ indicate the width and depth, respectively. The initial and second numbers in the ‘Number of Iterations’ Row represent the iterations during the primary and secondary training stages.
Table 11.
Pharmacokinetics model: Results of symbolic regression for gray-box identification using the PySR package (top) proposed by Cranmer [33] and the method implemented in gplearn [34] package (bottom).
Fig 9.
Pharmacokinetics model: Validation metrics for the Pharmacokinetics model using for X-TFC and PINN based gray-box models.
(A) represents variation in loss and score of symbolic models obtained from PySR with respect to the complexity of expressions. Once convergence is achieved, the score remains constant as the complexity of the recovered expression increases, and thus, the criteria for selection of candidate symbolic with expression shown in Table 11. (B) represents variation in loss of symbolic models, obtained from gplearn, with respect to the length of expression. We choose the length of expression 9 and 19 for PINNs and X-TFC, respectively. These lengths of expressions correspond to the minimum loss for the regressed symbolic models with closed form expression shown in Table 11.
Fig 10.
Pharmacokinetics model: gplearn based evolved tree of binary operations in symbolic model recovered for gray-box model hsym obtained from X-TFC and PINNs.
It is to be noted that the number of nodes in the tree corresponds to the length of expressions, which is 9 for the PINNs method.
Table 12.
Mathematical expressions distilled with symbolic regression for both PINN and X-TFC methods for different noise levels, compared with the true expression.
Table 13.
Results of symbolic regression for gray-box discovering of Ultradian Endocrine model using the PySR package (top) developed by Cranmer [33] and the method implemented in gplearn [34] package (bottom).
Fig 11.
Ultradian Endocrine model: Validation metrics for PySR method.
(A) fsym and (B) gsym are expressed by score and loss metrics against the complexity of the expressions recovered using PySR. It is to be noted that, in both the plots, once convergence is achieved, the score remains unchanged as complexity increases.
Fig 12.
Ultradian Endocrine model: Validation metrics for gplearn method.
(A) fsym and (B) gsym are expressed by MSE loss against length of the expressions recovered using gplearn and presented in Table 13. For fsym, we choose length of expression 18 and 25 for PINNs and X-TFC, respectively. However, for gsym, we choose length of expression 13 and 25 for PINNs and X-TFC, respectively.
Fig 13.
Ultradian Endocrine model: Tree of binary operations recovered for gsym.