Fig 1.
Profile likelihood for identifiability analysis.
(A) Illustrative profile likelihood as blue solid line for an identifiable parameter. For analyses of the identifiability status, the intersection with statistical threshold value (red line) at is crucial. Furthermore, profile likelihood-based confidence intervals can be constructed from the projections of the intersections on the parameter axis, defining the lower θlb and upper bound θub of the confidence interval, if these intersections exist. The MLE
is indicated by the black asterisk. (B) Profile likelihood of a practically non-identifiable parameter that is open to the left resulting in an unbounded (C) Likelihood profile of a structural non-identifiability indicated by a flat line with unbounded confidence interval in both directions.
Fig 2.
Empirical likelihood ratio classification.
A: Probability-probability plot (pp-plot) with four possible graph scenarios. B: -samples show a noticeable deviation from the diagonal in the pp-plot. The pp-plot in panel B shows graphs from 1000 samples each with Nreal = 500 random numbers drawn from a
-distribution. C: Classifications plot with classification regions and classification scheme for mixed cases: the white graph is classified as conservative, although its upper part lies in the perfect consensus area. Likewise, the light gray graph is identified with the anti-conservative case, while the black graph represents an alternating case which populates all three areas.
Table 1.
Table of all analyzed benchmark models.
Fig 3.
Overview of benchmark model properties and identifiability status.
(A) Estimated parameters of all 19 models and distribution of parameter types. (B) Identifiability of model parameters based on the profile likelihood using the original data and a threshold of Δα=0.05 = 3.84. Either identifiable parameters with finite profile likelihood-based confidence intervals (darker colors) or practically non-identifiable parameters (lighter colors) were identified. Structural non-identifiability was not observed.
Fig 4.
Classification results of whole pp-plots graphs compared to the -distribution.
Results are sorted by the percentage of non-problematic, i.e. conservative and perfect consensus cases. The anti-conservative and alternating fraction of cases is indicated by a negative sign. (A) Overall results for all model parameters and (B-E) results separately for each model parameter type.
Fig 5.
All pp-plots graphs of the estimated model parameters, separated by their identifiability status.
(A) identifiable parameters and (B) practically non-identifiable based on their profile likelihoods. All graphs are classified in perfect (red), conservative (yellow), anti-conservative (blue) or alternating (purple) cases.
Fig 6.
Average distance of pp-plot graphs from the diagonal.
Distances are normalized by the maximal possible distance value, i.e. a graph lying on the upper x-axis at pemp = 1 for all pemp in the pp-plot. Models are ordered according to the ranking of the classification results from Fig 4A. The histogram on the right shows the distribution of average distances from the diagonal summarized for all parameters.
Fig 7.
Classification results of pp-plots graphs at the 95% confidence level.
Results are sorted by the percentage of non-problematic, i.e. conservative and perfect consensus cases. The model ranking remains similar to the whole pp-plot graph analysis (cf. Fig 4).
Fig 8.
Appropriateness of the -distribution for empirical likelihood ratios at specific confidence levels.
(A) Heatmap of conservativeness ratio CR, i.e. fraction of non-problematic cases in the respective parameter group or model. CR-values close to 1 indicate a good agreement with asymptotic theory, while lower values occur in situations where the asymptotic approximation might not be appropriate. (B) Parameter groups or models with CR larger than 95% at the confidence level are indicated by black tiles in the upper panel. (C) A less strict criterion checks if the CR value is larger than 1 − α% of the respective confidence level and parameter groups or model.
Fig 9.
Data-space representation of four different scenarios using the nodal curve model manifold example.
(A,D,G,J) Different shapes of the model manifold (black solid line) in the data space with a choice of data realisations (N = 25 out of 200, gray crosses) drawn from a n-ball around the true parameter (blue cross) and fit onto the model manifold (gray solid line and red circles). Distances between the true parameter and the fits (N = 200) correspond to the empirical likelihood ratio. (B,E,H,K) Corresponding histograms to each scenario illustrate the distribution of the distances, i.e. likelihood ratios compared to a -distribution (red line). (C,F,I,L) The pp-plot panel allows for a detailed comparison of the empirical likelihood ratios to a
-distribution (N = 200).
Fig 10.
Impact of experimental designs with increased amounts of data.
Whole pp-plot graph classification results and average normalized distances from the pp-plot graphs to the diagonal for three illustrative benchmark models. The outcome for the originally published design , and two artificial designs
and
with increased temporal sampling is shown. The respective number of estimated parameters, total number of data points and their ratio is given below.
Fig 11.
Bartlett correction results of pp-plot classifications at the 95% confidence level.
Heatmap colors according to pp-plot classification for each Bartlett correction factor as indicated on the y-axis. Black tiles show optimal Bartlett correction factor closest to 1 of the individual parameter in each column. Since columns are ordered by this correction factor, the x-axis corresponds to the approximative conservativeness ratio aCR. White line and black bold numbers indicate the uncorrected outcome for C = 1, whereas green line and numbers show overall optimal Bartlett correction for a CR ≈ 95%. Deviations from 0.95 on x-axis originate from binning issues. (A) Results for all 768 parameters, (B) same results grouped by parameter type and (C) identifiability status. (D) Outcome for individual models reveal that a comprehensive overall optimal correction factor is difficult to determine. (E) Overall optimal Bartlett correction factors C* from exhaustive search with resulting thresholds for the likelihood ratio statistic.
Fig 12.
Conservativeness ratio CR at asymptotic and adapted thresholds.
(A) Histogram of conservativeness ratio CR for the typically utilized 95%-threshold from asymptotic theory for all analyzed parameters, separately for parameter groups and individual models as a reference. (B) Table summarizes the CR for all parameters for alternative confidence levels using asymptotic theory and for adapted thresholds or adapted confidence levels using Chebyshev’s or Cantelli’s inequality. (C, F) All blue histograms including panel (A) share the same threshold value T = 3.84, but differ in the interpretation of the confidence level. (D, G) The green histograms correspond to a confidence level of 90% with slightly different thresholds using Chebyshev’s or Cantelli’s inequality. (E, H) Likewise, purple histograms correspond to a confidence level of 95%. Vertical black solid lines in the histograms indicate the CR > 1 − α and CR > 0.95 limit, which coincides for the asymptotic case.