Computer simulations of mathematical models open up the possibility of assessing hypotheses generated by experiments on pathogen immune evasion in human whole-blood infection assays. We apply an interdisciplinary systems biology approach in which virtual infection models implemented for the dissection of specific immune mechanisms are combined with experimental studies to validate or falsify the respective hypotheses. Focusing on the assessment of mechanisms that enable pathogens to evade the immune response in the early time course of a whole-blood infection, the least-square error (LSE) as a measure for the quantitative agreement between the theoretical and experimental kinetics is combined with the Akaike information criterion (AIC) as a measure for the model quality depending on its complexity. In particular, we compare mathematical models with three different types of pathogen immune evasion as well as all their combinations: (i) spontaneous immune evasion, (ii) evasion mediated by immune cells, and (iii) pre-existence of an immune-evasive pathogen subpopulation. For example, by testing theoretical predictions in subsequent imaging experiments, we demonstrate that the simple hypothesis of having a subpopulation of pre-existing immune-evasive pathogens can be ruled out. Furthermore, in this study we extend our previous whole-blood infection assays for the two fungal pathogens Candida albicans and C. glabrata by the bacterial pathogen Staphylococcus aureus and calibrated the model predictions to the time-resolved experimental data for each pathogen. Our quantitative assessment generally reveals that models with a lower number of parameters are not only scored with better AIC values, but also exhibit lower values for the LSE. Furthermore, we describe in detail model-specific and pathogen-specific patterns in the kinetics of cell populations that may be measured in future experiments to distinguish and pinpoint the underlying immune mechanisms.
Citation: Lehnert T, Prauße MTE, Hünniger K, Praetorius J-P, Kurzai O, Figge MT (2021) Comparative assessment of immune evasion mechanisms in human whole-blood infection assays by a systems biology approach. PLoS ONE 16(4): e0249372. https://doi.org/10.1371/journal.pone.0249372
Editor: Aijaz Ahmad, University of the Witwatersrand, SOUTH AFRICA
Received: December 4, 2020; Accepted: March 17, 2021; Published: April 1, 2021
Copyright: © 2021 Lehnert et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets supporting the conclusions of this article are available in the ImmuneEvasion repository, https://asbdata.hki-jena.de/LehnertEtAl_PLOS/.
Funding: This work was financially supported by the International Leibniz Research School for Microbial and Biomolecular Interactions (ILRS) and by the Deutsche Forschungsgemeinschaft (DFG) through the CRC/TR124 FungiNet under DFG project number 210879364 (project B4 to MTF and project C3 to OK).
Competing interests: The authors have declared that no competing interests exist.
As integral part of the systems biology cycle, mathematical models based on experimental measurements allow unraveling the complex network of immune reactions during infections. Representing the immune response by a mechanistic network, mathematical models have the strength to not only quantify immune reactions that are not directly accessible in experiments but also to predict the expected outcome of novel immune mechanisms. Moreover, the possibility to easily change model parameters or even the structure of a model enables making concrete predictions under well-defined conditions. Thereby, mathematical models help to guide future experimental measurements and by that can drastically reduce the financial and time expenditure.
In previous studies, we applied the iterative cycle between biological experiments and virtual infection modeling to unravel the immune response to fungal pathogens in human whole-blood infection assays [1–3]. The combination of experimental investigations with mechanistic modeling facilitated, on the one hand, the quantification of functional characteristics of the immune defense by specific immune cells and promoted, on the other hand, the identification of a fungal population that can be neither phagocytosed nor killed but instead evades the immune response. Virtual infection modeling was realized in terms of a state-based model (SBM), where the states are occupied by the populations of essential immune cells, i.e. polymorphonuclear neutrophils (PMNs) and monocytes, as well as pathogens that can be either alive or killed and that are located either in extracellular space or within the immune cells. The biological processes that take place during infection are implemented by state transitions, which are characterized by transition rates. In these SBMs, we defined rates for transitions representing phagocytosis of pathogens by PMNs and monocytes, intracellular and extracellular killing of pathogens as well as the acquisition of the immune-evasive state. We found that the SBM does not only mimic the infection dynamics in human whole-blood assays but also enables quantitative predictions of treatment strategies against fungal pathogens for neutropenic patients .
In our initial SBM the mechanism of becoming immune-evasive was implemented by a constant rate, because at that time there were no details known about this mechanism. Thus, we assumed that immune evasion takes place spontaneously . Therefore, we will refer to this model as spon-IE model in the following. Even though, we could not yet identify the exact mechanism causing immune evasion, we were able to reject potential theories based on our previous experimental observations. For example, we observed that adding fresh blood from the same donor to an infected blood sample did not lead to elimination of the immune-evasive pathogens . Consequently, immune evasion is not a result of exhausted immune cells. Additionally, we observed that immune-evasive C. albicans cells are still measureable when performing the whole-blood infection assays with thimerosal-killed C. albicans or with the non-filamentous efg1Δ, cph1Δ mutant of C. albicans. Thus, we could exclude that C. albicans proliferation or any process that is actively induced by the C. albicans cells is responsible for their immune evasion. Based on these experimental observations, we theoretically investigated the possibility that immune evasion is mediated by the host. In the study by Prauße et al. , we exploited the strength of mathematical modeling and implemented a host-mediated mechanism in the SBM in order to make predictions by comparing various infection scenarios. More precisely, we hypothesized that the degranulation of PMNs in response to initial phagocytosis events does not only cause the release of antimicrobial peptides but also leads to secretion of effector molecules that induce immune evasion. Since this potential evasion mechanism is mediated by molecules that originate from PMNs, the respective SBM is denoted by PMNmed-IE model in the following. In addition to whole-blood infection with C. albicans, we comparatively analyzed the immune response during infection with another fungal pathogen, Candida glabrata. We previously studied these two fungal pathogens with regard to differences in the PMN response against them [5, 6]. However, while we could identify remarkable differences in the model predictions for various infection scenarios under neutropenia, we could neither accept nor reject the PMNmed-IE model nor the spon-IE model on the basis of the existing experimental data . Therefore, generating and analyzing additional experimental data will be one of the essential steps towards the identification of the immune evasion mechanism.
In the present study, we expand our search for the mechanism of immune evasion by performing a broader theoretical study involving further potential immune-evasion mechanisms and combine the corresponding models with the analysis of additional infection scenarios. According to the iterative cycle of systems biology, the resulting model predictions were partly tested experimentally within the scope of this study and, thereby, one potential immune-evasion model was rejected. First, we considered the possibility that immune evasion is not developing during the host response, but takes place before infection of the whole-blood sample by pathogens. This was realized in the so-called pre-IE model by eliminating the rate for immune evasion and instead implementing constant cell populations of alive and killed immune-evasive pathogens that are present before infection. Based on the quantitative predictions made by this model, we performed experiments that clearly negate the presence of killed immune-evasive pathogens before infection, such that the pre-IE model could be rejected. Instead, we continued our study with the alivePre-IE model, where we assume that solely a population of alive and immune-evasive pathogens is present before infection. Consequently, we were left with three base models that each contains one way to acquire immune evasion (see Fig 1): the spon-IE model, the PMNmed-IE model and the alivePre-IE model. In addition to these base models, we took into consideration that more than one immune-evasion mechanism could act in parallel and implemented all combinations of the three base models. Taken together, we obtained eight model combinations (see Table 1) and calibrated these models to the time-resolved experimental data on the viability of pathogens and the association of pathogens to PMNs and monocytes that were measured during whole-blood infection assays.
PMN (purple) containing granules and pathogens that are either alive (green), killed (red), or immune-evasive (gray). Immune-evasive pathogens are either alive (green outer line) or killed (red outer line). (A) In the spontaneous immune evasion model (spon-IE model) pathogens acquire immune evasion with constant rate ρ. (B) PMN-mediated acquisition of immune evasion is mediated by factors released from neutrophils (PMNmed-IE model) that were secreted by PMNs in response to the first phagocytosis of pathogens with rate ρ(t). (C) In the pre-existing immune-evasion model (pre-IE model), immune-evasive properties have been acquired before infection. The alive as well as the killed subpopulation of immune-evasive cells is estimated by this model. (D) Schematics of the alivePre-existing immune-evasion model (alivePre-IE model). The difference to the pre-IE model is that in the alivePre-IE model only the population of alive immune-evasive cells is estimated and there is no possibility for killed immune-evasive cells to develop.
For each model, the implemented immune-evasion mechanisms (indicated by a cross as table entry), the respective a priori unknown parameters that contribute to immune evasion and the total number of parameters are given. The spon-IE model represents a spontaneous immune-evasion mechanism by a constant rate ρ. The hypothesis on immune evasion mediated by PMN is implemented in the PMNmed-IE model and represented by a time-dependent rate ρ(t) that is determined by the two constants and γIE (see Eq 10). The populations of immune-evasive pathogens that are alive (PAIE) and killed (PKIE), and that are pre-existent at infection time t = 0 are estimated in the pre-IE model. The alivePre-IE model contains only the population of evasive pathogens that are alive (PAIE) before infection. The SBMs containing more than one evasion mechanisms (combined models) are composed of at least two base models.
In addition to infection scenarios with C. albicans or C. glabrata infection, we extended the present study by a third scenario with the bacterial pathogen Staphylococcus aureus. These three microbes are opportunistic pathogens that colonize the human body and can become pathogenic in patients with a compromised immune system. Therefore, these pathogens constitute a serious risk, not only of superficial infections but also of severe systemic infections like bloodstream infections. The Candida species C. albicans and C. glabrata together colonize more than 50% of the human population [7–9] and are the most frequent species causing fungemia . The gram-positive bacterium S. aureus is the leading cause of bacteremia [11–13]. A nationwide surveillance study on bloodstream infections in US hospitals revealed that S. aureus, C. albicans and C. glabrata together account for up to 30% of all cases and moreover, the associated mortality rates are extremely high . Thus, there is an urgent need to investigate how these pathogens manage to evade the immune response and can frequently cause bloodstream infections potentially leading to organ failure and death.
After calibrating the models of immune evasion to the experimental data, we evaluated the goodness of these models using different quantitative measures: as a measure for the quantitative agreement between the theoretical and experimental kinetics, we compute the least-square error (LSE) that is minimized during the parameter estimation procedure [2–4, 12]. The LSE evaluation is combined with the Akaike information criterion (AIC) as a measure for the model quality depending on its complexity in terms of model parameterization . Our results reveal that, by performing a comparative analysis of the different evasion models for C. albicans, C. glabrata and S. aureus infection, we can identify relative differences between the model predictions, which in turn enables us to suggest experimental investigations by which predicted immune-evasion mechanisms can be rejected or confirmed.
In this study, we tested hypotheses on the mechanism of immune evasion of pathogens in human whole blood by implementing various potential candidate mechanisms and their combinations into virtual infection models. These comprise four base models with a single mechanism to acquire immune evasion (see Fig 1 and Table 1) and four models with at least two evasion mechanisms, also referred to as combined models (see Table 1). In order to evaluate the different models, we calibrated each model to the experimentally determined kinetics of whole-blood infection assays with one of the three different pathogens C. albicans, C. glabrata and S. aureus.
S. aureus induces specific immune response pattern in whole-blood infection assays
In our previous studies, we performed whole-blood infection assays with either C. albicans or C. glabrata and observed that infection outcomes strongly depend on the pathogen [1, 2]. In this study, we additionally performed whole-blood infection assays with the bacterial pathogen S. aureus. During the time course of four hours, we measured the viability of the pathogens by survival assays as well as the association of the pathogens to PMNs and monocytes using flow cytometry. The experimentally measured kinetics of pathogen association and viability are depicted in Fig 2 for S. aureus (orange lines) together with the kinetics from infection with C. albicans (pink lines) or C. glabrata (blue lines). Similar to C. albicans and C. glabrata, most S. aureus cells were killed in the whole-blood infection assay during four hours. A fraction of only 5.0% ± 2.4% alive S. aureus cells was measured four hours post infection (see Fig 2A). The time course of alive S. aureus cells in the whole-blood infection was qualitatively similar to that of alive C. glabrata cells, but in comparison to alive C. albicans cells a much faster dynamics was observed. The flow cytometry analysis revealed association of S. aureus cells to PMNs and monocytes, but no interaction with lymphocytes. Similar to infections with the fungal pathogens, association to immune cells increased with time and the majority of S. aureus cells was associated with PMNs (76.7% ± 6.5%), whereas a smaller fraction was associated with monocytes (20.1% ± 6.4%) at four hours post infection. However, the dynamics of S. aureus interaction with both immune cell types were faster compared to the two fungal pathogens, resulting in a smaller population of extracellular S. aureus cells after four hours of infection, and showed a markedly higher association to monocytes. (see Fig 2B).
Experimental data from whole-blood infection assays infected with either C. albicans (pink line), C. glabrata (blue line) or S. aureus (orange line). Survival assays were performed to measure fractions of alive pathogens (A) and flow cytometry was used to determine the distribution of pathogens in human blood, shown as extracellular pathogens (B), pathogens associated with PMN (C) and with monocytes (D).
Taken together, we found that, similar to the fungal pathogens, also S. aureus could not be cleared completely from human blood, which is a strong evidence for the existence of immune-evasive S. aureus cells at four hours post infection. Furthermore, we found that the immune response to S. aureus showed pathogen-specific properties, such as the greater impact of monocytes and the larger elimination of pathogens from whole blood in comparison to C. albicans and C. glabrata.
Rejection of pre-IE model by image-based systems biology approach
Since we observed for all three types of pathogens that microbial cells can remain extracellular (see Fig 2B) and can remain alive, we investigated the hypothesis of a pre-existing immune-evasive phenotype in a proportion of the initial population used for infection (see Fig 1C). In the pre-IE model, we set the population of immune-evasive cells to a constant value that does not change over time (see Methods). This pre-existing subpopulation of immune-evasive cells can consist of alive and/or killed cells. In order to test whether this model captures the immune response as measured in the whole-blood infection assays with any of the three pathogens, we applied the parameter estimation algorithm Simulated Annealing based on the Metropolis Monte Carlo scheme (see S2 Fig of S1 File). This algorithm minimizes the least-square error (LSE); i.e. it determines the model parameter values for the best fit between model simulations and experimental data by minimizing the LSE. The resulting parameter values are given in S3 Table of S1 File. We observed that the corresponding model simulations matched the experimental data for all three infection scenarios. As shown in S10, S18 and S26 Figs of S1 File, respectively, for infection with C. albicans, C. glabrata and S. aureus, the model simulations were close to the mean values of the experimental data and within their corresponding standard deviations.
In the following we focus on model simulations for extracellular pathogens and alive pathogens. For infection with C. albicans cells, the pre-IE model predicted that four hours post infection a fraction of 15.85% ± 0.0% pathogens was remaining in extracellular space (see Fig 3B) and that most of these cells were immune-evasive (15.81% ± 0.0%, see S10 Fig of S1 File). Furthermore, the model predicted that this fraction of evasive cells is composed of 7.1% ± 0.0% killed evasive cells and 8.7% ± 0.0% alive evasive cells. Thus, the model predicted that at the time point of infection (t = 0 min) a fraction of 7.1 ± 0.0% pathogens were already dead, i.e. not all pathogens were alive initially. This can be seen in Fig 3A, where the fraction of alive C. albicans cells was 92.9% ± 0.0% at t = 0 min. Similarly, the simulated dynamics of alive pathogens for infection with C. glabrata and S. aureus, respectively, started at t = 0 min with 10.3 ± 0.1% and 1.6 ± 0.01% killed evasive pathogens (see Fig 3C) and hence with 89.7% ± 0.1% and 98.4% ± 0.1% alive pathogens (see Fig 3A). Taken together, the pre-IE model predicted fractions of alive cells and killed cells that are immune-evasive before infection for all three infection scenarios.
The thickness of the solid lines indicates the mean ± standard deviation (SD) of 50 simulations with transition rate values that were randomly sampled within their respective SD. Dashed lines correspond to the experimental data to which the model was fitted. The simulation results of the pre-IE model are depicted for an infection with C. albicans (pink line), C. glabrata (blue line) or S. aureus (green line). The time course of alive pathogens is shown at the top (A), the time course of extracellular pathogens in the middle (B) and the predicted time course total of immune-evasive (PIE), alive immune-evasive (PAIE) and killed immune-evasive cells (PKIE) is shown at the bottom (C).
So far, it is not possible to experimentally determine the fraction of immune-evasive cells before infection. However, we can test the model prediction by quantifying the amount of dead pathogens before infection using the live/dead staining with propidium iodide (PI) and subsequent microscopy imaging (see Methods). Example microscopy images for the three types of pathogens are shown in S27 Fig of S1 File. In order to count the number of alive and dead cells in these images, we applied the automated segmentation algorithm implemented in the framework AMIT (Algorithm for Migration and Interaction Tracking) , as described in Methods. In total, we analyzed six samples with C. albicans cells, six samples with C. glabrata cells and nine samples with S. aureus cells. Independent of the type of pathogen, we found that almost all cells in a viable pathogen inoculum were alive and that the number of dead pathogens was vanishingly small (see S28 Fig of S1 File): on average 0.0% ± 0.0% C. albicans cells, 0.2 ± 0.4% C. glabrata cells and 0.13% ± 0.07% S. aureus cells were dead before infection.
Taken together, for all three types of pathogens the experimentally determined fractions of dead pathogens before infection are vanishingly small and much smaller than the fractions of killed evasive pathogens that were predicted by the pre-IE model. These results strongly suggest that no dead pathogens are present at the beginning of the infection assays. On the other hand, since it cannot be excluded that alive evasive pathogens do exist before infection, we replace the pre-IE model by the alivePre-IE model. The latter model allows for the existence of an initial subpopulation of evasive pathogens that are all alive (see Methods). However, since dead cells do exist after four hours of infection (see Fig 2), the alivePre-IE model can only be considered in combination with other mechanisms of immune evasion. This conclusion motivated us to perform a comparative analysis of virtual infection models.
Comparative analysis of virtual infection models for whole-blood assays
In order to find the most realistic immune-evasion model from the set of eight immune evasion models (see Table 1), we calibrated these models to the experimental data of all three types of pathogens using the global parameter estimation algorithm Simulated Annealing based on the Metropolis Monte Carlo scheme (see Methods section and S2 Fig of S1 File). Afterwards, we comparatively analyzed different model characteristics such as the estimated model parameter values and the goodness of the models, using the Akaike information criterion (AIC) in addition to the LSE. In the following paragraphs, we compare the predictions of the different immune-evasion models for each type of pathogen separately. Here, we first focus on the comparison of the alivePre-IE model with the other two base models that contain solely one way to acquire immune evasion, i.e. the spon-IE model and the PMNmed-IE model (see Fig 1A, 1B and 1D, Table 1 and Methods). Afterwards, we comparatively analyze the predictions made by the combined models, where three of them contain two evasion mechanisms (the spon-PMNmed-IE model, the spon-alivePre-IE model and the PMNmed-alivePre-IE model) and one model with all three potential evasion mechanisms, the spon-PMNmed-alivePre-IE model.
Whole-blood infection with C. albicans
By fitting the immune-evasion models to data from whole-blood infection assays with the fungal pathogen C. albicans , we could estimate values of the model parameters and thereby quantify the immune reaction rates. The resulting parameter values are given in S1 Table of S1 File and depicted in Fig 4 for each immune-evasion model. We observed that the alivePre-IE model predicted larger phagocytosis rates of neutrophils (ΦN) and monocytes (ΦM) in comparison to the spon-IE model (, ) and the PMNmed-IE model (, ). The combined units that were directly influenced by these rates are PN and PM, respectively, representing pathogens in neutrophils and in monocytes. Despite these differences in the phagocytosis rates between the base models, the dynamics of PN and PM were similar during the first 60 minutes post infection and differed only slightly at four hours post infection (see Fig 5B and 5C). However, the phagocytosis rates were larger in the alivePre-IE model, because at t = 0 min the fraction of 12.7% ± 0.4% pathogens were alive evasive pathogens and 87.3% ± 0.4% of non-evasive pathogens in extracellular space were accessible for phagocytosis, which is a value that is smaller in comparison to the two other base models. In the spon-IE and the PMNmed-IE model, no evasive cells were present before infection. Thus, 100% of the infected pathogens were in extracellular space and could be phagocytosed. In comparison to the spon-IE model and PMNmed-IE model, the alivePre-IE model predicted a smaller fraction of extracellular pathogens (PE) at t = 240 min and nearly all of them were alive and evasive (see S3C Fig of S1 File for spon-IE model, S4C Fig of S1 File for PMNmed-IE model and S5C Fig of S1 File for alivePre-IE model). Furthermore, we observed differences between models in rates of killing. Compared to the spon-IE model and the PMNmed-IE model, the alivePre-IE model predicted a larger value for the rate of intracellular killing by neutrophils (κN) (see Fig 4B), whereas the rate for extracellular killing by antimicrobial peptides (κEK(t)) was smaller (see Fig 4C). This inverted ratio of intracellular and extracellular killing rates between the alivePre-IE model and the two other base models seems to compensate the effect on the dynamics of the combined unit for killed pathogens (PK). Consequently, the course of PK during 60 minutes post infection was similar for each of these three base models (see Fig 5A). At four hours post infection, the alivePre-IE model predicted a smaller fraction of killed pathogens (PK = 87.3 ± 0.4%) and a larger fraction of alive pathogens (PA = 12.7 ± 0.4%) in comparison to the spon-IE model with PK = 88.6 ± 0.6% and PA = 11.4 ± 0.6%, and the PMNmed-IE model with PK = 88.9 ± 0.9% and PA = 11.1 ± 0.9%.
Mean values (data points) and standard deviations (error bars and line thickness) of transition rates were quantified by the global parameter estimation algorithm Simulated Annealing based. Dots depict the transition rate values for the spon-IE model (light green), PMNmed-IE model (pink), alivePre-IE model (dark blue), spon-alivePre-IE model (light blue), PMNmed-alivePre-IE model (orange), spon-PMNmed-IE model (red) and spon-PMNmed-alivPre-model (dark green). The transition rates for phagocytosis are defined by ϕN for PMN and by ϕM for monocytes. Intracellular killing rates are defined by κN for PMN and κM\ for monocytes. Transition rates γ and determine the time-dependent extracellular killing rate κEK(t). Depending on the chosen model, the transition rates ρ, and γIE determine the immune-evasion rate over time (ρ(t)). Transition rate values for infection scenario with C. albicans for phagocytosis (A), intracellular killing (B), time course of the extracellular killing rate (C) and the time course of the immune-evasion rate (D). Transition rate values for infection scenario with C. glabrata infection for phagocytosis (E), intracellular killing (F), time course of the extracellular killing rate (G) and time course of the immune-evasion rate (H). Transition rate values for infection scenario with S. aureus infection for phagocytosis (I), intracellular killing (J), time course of the extracellular killing rate (K) and time course of the immune-evasion rate (L). Transition rates and start populations quantified for all base and combined models for each pathogen infection are also shown in S1-S3 Tables of S1 File.
Experimental data from whole-blood infection assays (dashed lines) with corresponding standard deviations (SDs) are compared to the simulated data (solid lines) for infection scenario with C. albicans. The simulated kinetics by the spon-IE model (light green line), PMNmed-IE model (pink line), alivePre-IE model (dark blue line), spon-alivePre-IE model (light blue line), PMNmed-alivePre-IE model (orange line), spon-PMNmed-IE model (red line) and spon-PMNmed-alivePre-IE model (dark green line) are shown in their corresponding color. The thickness of solid lines indicates the mean ± SD of 50 simulations with transition rate values that were randomly sampled within their respective SD. Simulated kinetics of killed cells (A), pathogens associated with monocytes (B), pathogens associated with PMN (C) and immune-evasive cells (D).
Taken together, the alivePre-IE model predicted parameter values, especially for ΦN, ΦM, κN and κEK(t), that differed in comparison to the spon-IE and the PMNmed model. Furthermore, we observed differences between these models in the simulation of the combined units PE, PK and PA (see Fig 5A–5C and S3-S5 Figs of S1 File). As described above, these differences in the parameter values and the combined units were a direct consequence of the initial presence of alive evasive pathogens at time t = 0 min. Furthermore, by comparing the degree of agreement of model simulations with the experimental data using the LSE (see Methods), we found that the alivePre-IE model simulations had the largest LSE (see Fig 6A) and, moreover, this was mainly caused by deviations in the combined units PE and PA, which were shown to be mainly affected by the presence of evasive pathogens that are alive. In other words, the model could not fill in the gap between extracellular pathogens (15.0±5.8%) and alive pathogens (6.5±4.2%) at t = 240 min post infection. In addition to the LSE, we compared the goodness of the models using the AICC (corrected Akaike information criterion, see Methods), which involves not only the deviation to the experimental data but also the number of model parameters (see Methods). We observed that the spon-IE model not only had the smallest LSE, among the three base models (see Figs 6 and 7A), but also showed the smallest AICC (see Fig 7A). The goodness of the two other base models was evaluated by their ΔAICC, which is defined as the absolute difference to the model with the smallest AICC, i.e. the spon-IE model. The differences ΔAICC = 4.78 for the PMNmed-IE model and ΔAICC = 5.86 for the alivePre-IE model show that these models are in the category of models with considerably less support in comparison to the spon-IE model (see Methods).
Each bar refers to the mean LSE value ± SD of 50 simulations with transition rate values that were randomly sampled within their respective SD by the spon-IE model (light green line), PMNmed-IE model (pink line), alivePre-IE model (dark blue line), spon-alivePre-IE model (light blue line), PMNmed-alivePre-IE model (orange line), spon-PMNmed-IE model (red line) and spon-PMNmed-alivePre-IE model (dark green line). The LSE is compared for a simulated infection scenario with C. albicans (A), C. glabrata (B) and S. aureus (C).
The LSE distribution of 50 simulations for each immune evasion model with transition rate values that were randomly sampled within their respective standard deviation are depicted in the left column. The colors refer to the spon-IE model (light green), PMNmed-IE model (pink bars), alivePre-IE model (dark blue), spon-alivePre-IE model (light blue), PMNmed-alivePre-IE model (orange), spon-PMNmed-IE model (red) and spon-PMNmed-alivePre-IE model (dark green). The mean of the LSE was used to calculate the corresponding AICC value for each model (same color as in the left column). The most supported model achieves the lowest AICC value and for every other model the ΔAICC value is calculated and depicted. Areas colored by different shades of grey represent the different categories of ΔAICC that are defined in the Methods section. The white regions indicate the areas between the ΔAICC categories. The horizontal red line depicts the median LSE of all LSE values. The subfigures show the LSE distribution and the AICC values for each model, respectively, for C. albicans infection (A, B), C. glabrata infection (C, D) and S. aureus infection (E, F).
By involving the results of the four combined immune-evasion models into the comparative analysis, we observed that the PMNmed-alivePre-IE model predicted killing rates that have also an inverse relationship to the other combined models (see Fig 4B and 4D), but the resulting dynamics of PK was not affected (see Fig 5A).
However, similar to the alivePre-IE model, the PMNmed-alivePre-IE model predicted a smaller fraction of immune-evasive cells (PIE) in comparison to the other models (see Fig 5D). This resulted in much larger LSEs of these two models (see Fig 6A). For example LSEs of the alivePre-IE model (P′ < 0.001) and PMNmed-alivePre-IE model (P′ < 0.001) are significantly different in comparison to the LSE of the PMNmed-IE model and this is also the case when comparing to the other models. Since the complexity of the PMNmed-alivePre-IE is also large (nine parameters), the difference ΔAICC to the model with the smallest AICC, which is the spon-IE model, indicates that this model is in the category of models that have essentially no support (see Methods for the respective categorization). The spon-alivePre-IE model simulations caused a similar LSE to the spon-IE model simulations and the corresponding ΔAICC is the smallest among the compared models. The other combined models have an LSE similar to the spon-IE model, but are either close to or within the category of models that have essentially no support according to their ΔAICC values.
Taken together, we observed that the immune-evasion models can be calibrated to the experimental data of whole-blood infection with C. albicans. However, differences in the parameter values caused differences in the goodness of the fit with respect to the LSE and the AICC. We found that the spon-IE model simulations caused the smallest LSE and the best AICC. Since no killed evasive cells are allowed in the alivePre-IE model, the simulated kinetics causes the highest LSE. The PMNmed-alivePre-IE model and the spon-PMNmed-alivePre-IE model are the evasion models with the largest complexity, i.e. with the largest number of model parameters, causing the highest AICC values. The respective difference to the best AICC value, the ΔAICC, indicates that these models belong to the category of models with essentially no support.
Whole-blood infection with C. glabrata
In our previous study, we found remarkable differences in the model predictions for C. glabrata infection regarding the transition rate values–especially for the rate of extracellular killing–from the spon-IE model and the PMNmed-IE model. In this study, we additionally compared the predictions from the alivePre-IE model and found its transition rate values to be either similar to or between the respective values quantified by the two other base models (see Fig 4F–4J). For example, the phagocytosis rate of neutrophils (Φ0) was predicted to be highest in the spon-IE model, followed by the alivePre-IE model and lowest in the PMNmed-IE model (). Consequently, this led to a quicker increase in the dynamics of PN for the spon-IE model in comparison to the alivePre-IE model and the PMNmed-IE model, which is also visible by comparing the fraction of PN at ten minutes post infection (, , ) as shown in Fig 8C. However, four hours post infection with C. glabrata, the fraction of PN was largest in the model with the smallest phagocytosis rate, i.e. the PMNmed-IE model. This was caused by additionally predicting a lower immune evasion rate in comparison to the spon-IE and the alivePre-IE model (see Fig 4H), which consequently gave the neutrophils and monocytes more time to phagocytose C. glabrata cells.
Experimental data from whole-blood infection assays (dashed lines) with corresponding standard deviations (SDs) are compared to the simulated data (solid lines) for C. glabrata infection scenario. The simulated kinetics by the spon-IE model (light green line), PMNmed-IE model (pink line), alivePre-IE model (dark blue line), spon-alivePre-IE model (light blue line), PMNmed-alivePre-IE model (orange line), spon-PMNmed-IE model (red line) and spon-PMNmed-alivePre-IE model (dark green line) are shown in their corresponding color. The thickness of solid lines indicates the mean ± SD of 50 simulations with transition rate values that were randomly sampled within their respective SD. Simulated kinetics of killed cells (A), pathogens associated with monocytes (B), pathogens associated with PMN (C) and immune-evasive cells (D) for a C. glabrata infection.
Despite the fact that the model dynamics of the combined units appear similar for the base models, the quantitative differences in the transition rate values are associated with somewhat different LSEs (see Fig 6B). The LSE of the alivePre-IE model is smaller than the LSE of the PMNmed-IE model and larger than the LSE of the spon-IE model. By additionally involving the model complexity into the model assessment using the AICC (see Methods), we found that the spon-IE model had the lowest AICC, which indicates that the corresponding LSE is small enough to compensate for the higher number of parameters in the spon-IE model (see Table 1) in comparison to the alivePre-IE model. Moreover, the absolute difference between AICC values of the alivePre-IE model and the spon-IE model () indicates that the alivePre-IE model has less support than the spon-IE model (see Fig 7D). The PMNmed-IE model not only showed a larger LSE value than the spon-IE model, but also a higher number of parameters. Thereby, the AICC is larger and, moreover, the corresponding lies between the two categories referred to as considerably less support and essentially no support.
Interestingly, the four combined models of immune evasion predicted values for ΦN, ΦM and κN that were markedly different from the values predicted by the two base models, alivePre-IE and PMNmed-IE model (see Fig 4E–4G). By comparing the LSE, we found that simulations of all four combined models have LSEs that differ from the spon-IE model, which is the base model with the lowest LSE. The statistical analysis revealed that this difference is significant with P′ < 0.01 for the spon-alivePre-IE model and with P′ < 0.001 for the PMNmed-alivePre-IE model, the spon-PMNmed-IE model and the spon-PMNmed-alivePre-IE model. However, the combined models contain more model parameters and, therefore, their AICC values are larger in comparison to that of the spon-IE model. The spon-alivePre-IE model has a lower LSE and thus the larger model complexity was compensated in comparison to the spon-IE model. The corresponding ΔAICC = 2.7 clearly shows the proximity to the category of models with substantial support. All other combined models failed to compensate for their model complexity with a better fit and are either close to or within the category of models with essentially no support (see Fig 7D).
Taken together, the alivePre-IE model made predictions that are quantitatively between those of the two other base models regarding transition rate values, LSE and AICC. The largest differences were found for the rates of intracellular killing, extracellular killing and immune evasion. The spon-IE model was the best according to the AICC and the spon-alivePre-IE model was comparable with this model according to the categorization of the ΔAICC (see Methods). All other combined models contain more parameters and this larger complexity could not be compensated by a better fit to experimental data, leading to a larger difference ΔAICC.
Whole-blood infection with S. aureus
Extending to the infection scenario with S. aureus in this study, the immune reaction rates were quantified by calibrating the models to the observed association and killing kinetics of S. aureus cells. We found that the optimal reaction rate values of the spon-IE model and the alivePre-IE model showed a smaller deviation from each other than in comparison to the rates of the PMNmed-IE model. As can be seen in Fig 4I, the phagocytosis rates of monocytes (ΦM) and PMNs (ΦN) are smaller for the PMNmed-IE model in comparison to the other base models. Furthermore, the rate for immune evasion is smaller in the PMNmed-IE model (see Fig 4L). The evasion rate in this model is dependent on the first-phagocytosis events by PMNs, which mainly occurred during the first hour post infection, resulting in an immune-evasion rate close to zero at one hour post infection. At later times, extracellular pathogens (PAE and PKE) were phagocytosed predominantly by immune cells, i.e. mainly by neutrophils. This led to a larger fraction of PN for the PMNmed-IE model in comparison to the spon-IE model and the alivePre-IE model (see Fig 9). Additionally, the PMNmed-IE model predicted an optimal value for the intracellular killing rate for monocytes (κM) that was the smallest compared to the other base models, whereas the rate of extracellular killing (κEK(t)) was the largest. However, these differences do not have an effect on the resulting model dynamics with regard to the killed pathogens (PK), which is almost equal for all three base models (see Fig 9A). The LSE is larger for the PMNmed-IE model in comparison to the spon-IE model (P′ < 0.001) and the alivePre-IE model (P′ < 0.001). The three base models have similar LSE values that are not significantly different (P′ > 0.05) (see Fig 6C). However, in comparison to the other infection scenarios (see Fig 6A and 6B), the differences between the LSEs of the base models are smaller. Simulations of the alivePre-IE model led to the smallest AICC value being similar to the AICC of the spon-IE model with difference ΔAICC = 0.09, indicating that this model belongs to the category of models with substantial support (see Fig 7E and 7F). We observed a difference of ΔAICC = 4.19 for the PMNmed-IE model, indicating that this model belongs to the category of models with considerably less support than the alivePre-IE model with the smallest AICC value.
Experimental data from whole-blood infection assays (dashed lines) with corresponding standard deviations (SDs) are compared to the simulated data (solid lines) for the S. aureus infection scenario. The simulated kinetics by the spon-IE model (light green line), PMNmed-IE model (pink line), alivePre-IE model (dark blue line), spon-alivePre-IE model (light blue line), PMNmed-alivePre-IE model (orange line), spon-PMNmed-IE model (red line) and spon-PMNmed-alivePre-IE model (dark green line) are shown in their corresponding color. The thickness of solid lines indicates the mean ± SD of 50 simulations with transition rate values that were randomly sampled within their respective SD. Simulated kinetics of killed cells (A), pathogens associated with monocytes (B), pathogens associated with PMN (C) and immune-evasive cells (D) for a S. aureus infection.
The combined models of immune evasion could be calibrated to the whole-blood infection assays with S. aureus, resulting in LSEs that are not significantly different from those of the model with the smallest LSE, i.e. the spon-alivePre-IE model. Considering the AICC score, we observed that this model is better than the other combined models, because the difference ΔAICC classifies the model in the category of considerably less supported models, whereas the other combined models where classified in the category of models with essentially no support.
Taken together, for the infection scenario with S. aureus, we observed similar dynamics for all models with predicted reaction rates that are mostly similar. We found only minor differences in the predictions from the PMNmed-IE model. Furthermore, we observed that for S. aureus whole-blood infection the alivePre-IE model simulation led to the smallest AICC, being close to the AICC of the spon-IE model.
Patterns of immune reaction rates for the three infection scenarios are highly model-specific
After comparing the predictions by the various models for each infection scenario, we compared the predictions for all infection scenarios for each model separately. This is helpful in order to identify pathogen-specific patterns that differ between the immune evasion models. First, we compared the values of the rate of extracellular killing by antimicrobial peptides for the infection with C. albicans, C. glabrata and S. aureus at specific time points post infection. As shown in Fig 10A, the spon-IE model predicted the largest value of the extracellular killing rate for C. glabrata infection and the smallest value for C. albicans infection ( at t = 10 min) ten minutes post infection. In contrast, the PMNmed-IE model predicted a larger value of the extracellular killing rate for infection with S. aureus in comparison to C. albicans and C. glabrata infection, while the latter two rates are not significantly different from each other ( at t = 10 min0) (see Fig 10B). The third base model, the alivePre-IE model, also predicted that the infection by S. aureus cells leads to the largest extracellular killing rate, but the values for infection by C. glabrata and C. albicans are significantly different ( at t = 10 min) (see Fig 10C).
The predicted extracellular killing rate κEK(t) is shown for an infection with C. albicans (pink line), C. glabrata (blue line) or S. aureus (orange line). The subfigures show the time course of extracellular killing for the spon-IE model (A), the PMNmed-IE model (B), the alivePre-IE model (C), the spon-alivePre-IE model (D), the PMNmed-alivePre-IE model (E), the spon-PMNmed-IE model (F) and the spon-PMNmed-alivePre-IE model (G). The thickness of the lines indicates the mean ± standard deviations obtained by 50 simulations with normally distributed transition rates.
As shown in Fig 10D–10G, the various combined immune evasion models predicted strikingly similar relations of the extracellular killing rates between the infection scenarios at ten minutes post infection. Similar to the pattern predicted by the spon-IE model, infection with C. glabrata induced a larger extracellular killing rate than infection with S. aureus, which in turn is larger than the value for C. albicans infection, i.e. at t = 10 min. However, additional evaluation of the relation of extracellular killing rates between the pathogens at one hour post infection revealed that the spon-PMNmed-IE model predicted another pattern than the other four models. The extracellular killing rate is larger for infection with C. albicans in comparison to infection with C. glabrata (with P′ < 0.001) (Fig 10F). The other three combined immune evasion models (Fig 10D, 10E and 10G) and the spon-IE model (Fig 10A) predicted similar values for C. albicans and C. glabrata at one hour post infection. Therefore, we conclude that the rate for extracellular killing provides a marker that enables to differentiate between three immune evasion models, i.e. the PMNmed-IE model, the alivePre-IE model and the spon-PMNmed-IE model. Moreover, the other four models predicted a unique pattern, which in turn is distinct from the previous three models.
By comparing the relation of immune evasion rates between the models, we identified more model-specific patterns. Since the PMNmed-IE model, the alivePre-IE model and the spon-alive-IE model can be clearly differentiated based on the rate for extracellular killing, we here focus on the four models that have a unique pattern for extracellular killing, i.e. the spon-IE model (Fig 11A), the spon-alivePre-IE model (Fig 11C), the PMNmed-alivePre-IE model (Fig 11D) and the spon-PMNmed-alivePre-IE model (Fig 11F). The spon-IE model and the spon-alivePre-IE model have immune evasion rates that are constant during infection and the predicted patterns for the relation between the infection scenarios are similar. However, because of the constant nature of the immune evasion rate, these models can be differentiated from the PMNmed-alivePre-IE model and the spon-PMNmed-alivePre-IE model with their time-dependent immune evasion rates (see Fig 11D and 11F). Additionally, we observed for the spon-PMNmed-alivePre-IE model that the order of the immune evasion rates for the different infection scenarios was constant during infection, while the PMNmed-alivePre-IE model predicted that this order changed during infection. As can be seen in Fig 11D, ten minutes post infection the rate for C. albicans infection is the smallest in comparison to C. glabrata and S. aureus infection, while from 70 minutes post infection, the rate is the largest for C. albicans.
The predicted immune-evasion rate is shown for an infection with C. albicans (pink line), C. glabrata (blue line) or S. aureus (orange line). The subfigures show the results for the spon-IE model (A), the PMNmed-IE model (B), the alivePre-spon-IE model (C), the PMNmed-alivePre-IE model (D), the spon-PMNmed-IE model (E) and the spon-PMNmed-alivePre-IE model (F). Note that the alivePre-IE model does not contain a rate for the acquisition of immune evasion and therefore it is not included in this figure.
Additionally, we compared the relation of the other transition rates between the infection scenarios and we found that either all models predicted a similar pattern, such as for ΦM and ΦN with ΦS.a. > Φc.g. > Φc.a., or the differences are not sufficient to differentiate between the spon-IE model and the spon-alivePre-IE model.
Taken together, the analysis of the relation of immune reaction rates between the three infection scenarios separately for each immune evasion model enabled us to identify distinct patterns for most immune evasion models. We found unique patterns for the PMNmed-IE model, the alivePre-IE model and the spon-PMNmed-IE model by analyzing the model predictions for the rate of extracellular killing. While the spon-IE model and the spon-alivePre-IE model predict similar patterns for all immune reaction rates, evaluating the rate for immune evasion revealed unique patterns for the PMNmed-alivePre-IE model and the spon-PMNmed-alivePre-IE model. This means that five out of seven immune-evasion models showed unique patterns for the rates of extracellular killing and immune evasion.
We applied a systems biology approach in order to assess hypotheses on the mechanism of immune evasion of pathogens in human whole-blood infection assays. To this end, we implemented different hypotheses into virtual infection models and comparatively assessed the quantitative model predictions for three different infection scenarios. In addition to infection scenarios with the fungal pathogens C. albicans and C. glabrata, which were already under investigation in previous studies [1, 2, 4, 5, 12], we infected human whole-blood samples with the bacterial pathogen S. aureus. By applying the same experimental setup and measurement procedures as for the fungal pathogens, we were able to directly compare the measured kinetics for pathogen survival and immune cell association. We found bacteria-specific characteristics, such as a quicker and even higher association with immune cells, in particular, with monocytes. Consequently, the fraction of S. aureus that was not associated with immune cells but remained extracellularly was relatively small. Since only these extracellular pathogens can be immune-evasive, we conclude that the development of an immune-evasive phenotype plays a minor role in the response to S. aureus infection in human whole blood.
In addition to our previous models of the immune evasion of pathogens in whole blood, i.e. the spon-IE model with spontaneous and the PMNmed-IE model with PMN-mediated immune evasion, we here implemented the pre-IE model. This model represents the hypothesis that evasive pathogens do not evolve during immune response, but rather are present as a subpopulation in the whole pathogen population even before infection. It should be noted that this hypothesis has been frequently put forward to explain the existence of immune-evasive pathogens against the possibility that immune evasion is the result of some process going on during the time course of the human whole-blood assay. By calibrating the pre-IE model to the experimental measurements of the three whole-blood infection scenarios, we could predict not only the fraction of pathogens that were immune-evasive, but also the fractions of these evasive cells that need to be alive and killed before infection. In the spirit of an image-based systems biology approach [15, 16], we made use of microscopy imaging and automated image analysis to test the model predictions. In this way we could falsify our initial hypothesis by measuring the true fraction of killed pathogens before infection. Since it turned out that there were almost no killed pathogens measured before infection–rather that this fraction is smaller as the fractions of extracellular pathogens and killed immune-evasive pathogens–the pre-IE model with alive and killed evasive pathogens needed to be rejected. Based on these results, we were able to start a new lap within the cycle of systems biology and tested the hypothesis of having a subpopulation of exclusively alive evasive cells within the so-called alivePre-IE model.
Next, we compared the model predictions for the three immune evasion models, each containing only one evasion mechanism and referred to as the three base models. However, since it cannot be excluded that more than one mechanism causes immune evasion of pathogens, we also generated models for each possible combination of the three evasion mechanisms, which we refer to as combined models. These contain at least two of the evasion mechanisms of the base models, leading to an increased model complexity that is reflected by a larger number of model parameters.
The models were again calibrated to experimental data of the three infection scenarios and in order to comparatively assess the quality of each model, we used the agreement with the experimental data in terms of the least-square error (LSE). However, it is evident that this measure alone can overvalue models with larger numbers of parameters [17–19], because these models have a larger degree of freedom and thereby may be calibrated to the experimental data with higher accuracy. We cope with the differing number of parameters of our models including the corrected Akaike Information criterion (AICC) as a second measure. This measure involves a penalty for the number of model parameters, which is even more pronounced as for the Bayesian information criterion . Although such information criteria are powerful tools for model selection , it should be noted that, in contrast to other studies [22–25], we do not consider an information criterion that penalizes the number of model parameters as a sufficient argument against a particular model. Information criteria measure the quality of a model from a mathematical point of view, without considering that particular biological processes may inevitably be associated with a certain number of required parameters. Furthermore, there are specific sources of uncertainty that must be considered in the course of finding the best model for a biological system by model calibration to experimental data. Warne et al. , described four key sources of uncertainty, which are 1) unknown model parameter values, 2) uncertainty in the model choice, 3) uncertainty that arise by the stochastic nature of the model and 4) uncertainties that are caused by systemic or measurement errors during the biological experiments . We accounted for all these four sources of uncertainty by 1) applying a global estimation algorithm in order to provide optimal prerequisites for the search of unknown model parameters, 2) implementing various reasonable models in order to provide a set of potential models, 3) simulating the model with a large number of individuals to reduce the impact of stochasticity on the average model output and 4) repeating the experiments to account for biological uncertainties. Furthermore, the epistemic uncertainty of the model kinetics, which is caused by variation of the transition rate values [26, 27], was considered by performing 50 simulations with transition rate values that were randomly sampled within their respective standard deviation. In order to investigate how these uncertainties were caused by the different transition rates, a sensitivity analysis could be performed [26, 28, 29]. Since this analysis step does assess the transition rates but not the immune-evasion models in a comparative way, it would not help to reject or select immune-evasion mechanisms. Once the immune-evasion mechanism is experimentally validated by future studies, a sensitivity analysis should be performed. Furthermore, after reducing the number of immune evasion models, a long-term analysis of the model dynamics should be the focus of future studies. However, it is commonly known that this analysis requires a huge computational effort. Cornell et. al.  describe a method to analyse the dynamics of such individual-based models in a mathematical way. This more efficient method could be applied in future studies.
The calibration of the eight immune-evasion models to the three infection scenarios revealed model-specific predictions and differences in the quality of the fit to experimental data for each infection scenario. For infection with C. albicans, the spon-IE model showed the smallest LSE and AICC and may therefore be prioritized over the other models, whereas the alivePre-IE model is by far the model with the largest LSE and AICC. The spon-IE model and the spon-alivePre-IE model are the models with the smallest LSE and the best AICC for the infection scenario with C. glabrata. Moreover, for infection with either of the two fungal pathogens, we observed that the more complex models could not compensate for their penalized complexity by a better agreement with the experimental data in comparison to the simpler immune evasion models. Similarly, for the infection scenario with S. aureus, we observed that the spon-IE model and the alivePre-IE model had the smallest LSE and AICC, while in general all model predictions are more similar to each other in comparison to the other two infection scenarios.
In summary, even though we found differences in the model predictions for each infection scenario, it is not possible to either reject a model or to unambiguously identify one true model for all infection scenarios, because the majority of the models yielded simulation results within the experimentally measured standard deviations. However, we found that the differences between the models were larger for the infection scenarios with larger fractions of extracellular pathogens at four hours post infection, because the fraction of pathogens that can become evasive is larger in these scenarios. Infection scenarios with fungal pathogens, especially C. albicans, showed a larger fraction of extracellular cells and the differences between the model predictions and the corresponding LSEs and AICC are much larger in comparison to the infection scenario with S. aureus. This implies that the fungal pathogens seem much more suitable for further investigations of immune evasion mechanisms in human whole-blood assays. Nevertheless, by performing a whole-blood S. aureus infection in addition to the fungal infection scenarios, we found that these could still provide decisive information for the identification of the true evasion model or the rejection of unsuitable models. By analyzing specific model predictions for each of the three infection scenarios in a comparative fashion, we identified model-specific patterns. The relation between the infection scenarios was model-specific for the kinetics of the extracellular killing rate at ten and 60 minutes post infection and for the immune evasion rate. Thus, we identified unique patterns for five out of seven immune-evasion models, i.e. for the three base models, the spon-IE model, the PMNmed-IE model and the alivePre-IE model, as well as for two combined models, the spon-PMNmed-IE model and the spon-alivePre-IE model. For future studies, we suggest to exploit the pathogen-specific immune response in human whole-blood infection assays by extending the types of pathogens. For example, experimental investigations could also be done for Gram-negative bacteria, such as Escherichia coli, which was recently the focus of a study on infection assays with avian whole blood . It will be interesting to see whether including more types of pathogens will extend the pathogen-specific parameter space, allowing to differentiate between mathematical models whose distinct immune-evasion mechanisms are currently not experimentally distinguishable.
Based on these predictions, we can now guide validation experiments which could reject or accept any of the models or even give rise for suggesting other immune-evasion models. For example, one can comparatively analyze the growth of pathogens within activated serum, which was previously enriched with antimicrobial peptides in response to an infection. By using serum without any immune cells, a decrease in the number of pathogens could only be caused by extracellular antimicrobial activity. Note that we can compare these measurements with our predictions in qualitative way. In contrast, determining the concentration of known antimicrobial peptides that were released upon first phagocytosis by neutrophils–like lactoferrin, elastase 2 and myeloperoxidase–would not provide direct information about their activity in pathogen killing. A comparative analysis of the time-dependent killing effect of antimicrobial peptides would still help rejecting or accepting three of the immune evasion models, i.e. the PMNmed-IE model, the alivePre-IE model and the spon-PMNmed-IE model. Two further models, the spon-IE models and the spon-alivePre-IE model, can be tested by additionally analyzing the immune evasion rate. Unfortunately, the experimental measurement of this rate is currently challenging, because the characteristics of immune-evasive pathogens are largely unknown. Furthermore, if the concentration of evasive pathogens could be measured, it could be possible to investigate whether pathogens become immune-evasive in activated or non-activated serum. Subsequently, one could compare the relative development of evasive pathogens in each of these media and compare qualitatively with the predictions of the various models. Finally, we have shown that the model-specific patterns of the relation of reaction rates between the three infection scenarios can be used as key data to navigate future experimental measurements, which in turn could validate the immune evasion models and ultimately identify the mechanism of immune evasion.
Systems biology integrates computer-aided and experimental investigations to decipher mechanistic relationships in complex biological systems. With a focus on the network of immune reactions in terms of a state-based model, we investigated the undesired occurrence of immune evasion by pathogens in human bloodstream infections. We have already identified this phenomenon in previous studies, but one of the unanswered questions was whether these observations may be an artifact of the experimental procedure. In this study, we designed models and performed computer simulations to predict the consequences of assumptions, such as the initial existence of a fraction of immune-evasive pathogens. Subsequent imaging experiments revealed that this assumption cannot fully explain our observations in human whole-blood assays. The possibility to easily modify the structure of a mathematical model enabled making concrete predictions for additionally included mechanisms of immune evasion. Based on these predictions obtained from quantitatively estimating model parameters that best fit the experimental data, we derived model-specific patterns and propose new experiments that will contribute to the identification of immune-evasion mechanism in the future. From a general perspective, our present study is a prime example of applying the systems biology approach with the aim of directing research towards the identification of mechanisms in complex biological systems.
Materials and methods
Fungal strains and culture
C. albicans wild-type strain SC5314 , GFP-expressing C. albicans strain (CaGFP) , C. glabrata wild-type strain ATCC2001  and C. glabrata expressing GFP (CgGFP)  were routinely seeded in YPD medium (2% D-glucose, 1% peptone, and 0.5% yeast extract, in water) and grown overnight at 30°C (C. albicans strains) and 37°C (C. glabrata strains), respectively, in a shaking incubator. Fungal cells were reseeded 1:50 in fresh YPD medium, grown until they reach the mid-log phase (OD600 ~ 1) followed by harvesting in HBSS.
Killing of C. albicans wild-type strain SC5314 or C. glabrata wild-type strain ATCC2001 was performed after staining with Cell Tracker™Green (Invitrogen) for 1 hour using 0.1% thimerosal (Sigma-Aldrich) at 37°C for 1 hour and followed by extensively washing with HBSS.
Staphylococcus aureus strains and culture
S. aureus wild-type strain ATCC25923 and GFP-expressing S. aureus (SaGFP) [35, 36] were cultivated overnight in lysogeny broth (LB) medium (10 g/l tryptone, 5 g/l yeast extract, 10 g/l sodium chloride, pH 7) at 37°C in a shaking incubator. In case of SaGFP the lysogeny broth medium was supplemented with 10 μg/ml chloramphenicol. Bacterial cells were reseeded 1:100 in fresh lysogeny broth medium, grown until they reach the mid-log phase (OD600 = 0.6–0.7) followed by harvesting in HBSS containing 10% heat-inactivated human serum (hiHS, Sigma-Aldrich).
Killing of S. aureus wild-type strain ATCC25923 was performed after staining with Cell Tracker™Green for 30 minutes using 50% ethanol for four hours at 37°C and followed by extensively washing with HBSS+10%hiHS.
Human whole-blood infection assay
This study was conducted in accordance with the Declaration of Helsinki and all protocols were approved by the Ethics Committee of the University Hospital Würzburg (permit number: 37/17). Human peripheral blood was collected from healthy donors with written informed consent. The whole-blood infection assay was performed as described previously . Briefly, alive GFP-expressing C. albicans, C. glabrata or S. aureus cells were added to Hirudin-anti-coagulated blood (1x106 fungal or bacterial cells per milliliter of blood) and incubated at 37°C on a rolling mixer for indicated time points. Treatment of blood with either HBSS or HBSS containing 10% hiHS served as mock-infected control samples. After incubation, the samples were immediately placed on ice and subjected to flow cytometric analyses.
Analyses of immune cell populations in whole blood with regard to phagocytosis of fungal or bacterial cells were performed using differential FACS staining and subsequent measurement with a FACS Canto II (BD Bioscience). To distinguish different immune cells, whole blood was stained with mouse anti-human CD3 (clone SK7, T cells), CD19 (clone HIB19, B cells), CD56 (clone B159, NK-cells) and CD66b (clone G10F5, PMN) obtained from BD Biosciences. Monocytes were labeled with mouse anti-human CD14 antibody (clone 47-3D6, Abcam). The stained samples were treated with FACS Lysing solution (BD Biosciences) that lyses erythrocytes while preserving and fixing leukocytes, followed by washing and harvesting cells in CellWASH solution (BD Biosciences).
Detailed data analysis including gating of relevant immune cell populations was performed using FlowJo 7.6.4 software. The strategy used to evaluate the association of microorganisms to immune cells in human blood was shown for CaGFP in Hünniger et al. .
Propidium iodide staining and imaging
To evaluate the amount of dead CaGFP, CgGFP or SaGFP cells in a fresh induction culture, microorganisms were stained with 2.5 ng/ml of propidium iodide (Sigma-Aldrich). Propidium iodide (PI) is widely used as a vital dye that is not permeant to live cells, but labels the nucleus in dying cells, which lack an intact plasma membrane. Thimerosal-killed Cell Tracker™Green-labeled C. albicans or C. glabrata and ethanol-killed Cell Tracker™Green-labeled S. aureus cells, respectively, served as controls for a positive PI staining. Fluorescence images of C. albicans and C. glabrata samples were acquired with the Zeiss AxioObserver Z.1 microscope using a 40x objective (EC Plan-Neofluar 40x/0,75) and processed using the ZEN 2.3 software (Carl Zeiss). Images of S. aureus samples were acquired with the LSM 780 confocal microscope using a 63x objective (Plan-Apochromat 63x/1.4 Oil DIC) and processed using the ZEN 2.3 software.
Automated image analysis
The microscopy images consist of two different channels: the green fluorescence channel with GFP-labeled pathogens and the red fluorescence channel for pathogen live/dead PI staining (S27 Fig of S1 File). The microscopy images with either C. albicans cells or C. glabrata cells were analyzed and counted by the automated segmentation algorithm that is integrated within the framework AMIT (Algorithm for Migration and Interaction Tracking) [14, 37, 38]. In brief, the green fluorescence images were segmented and cell clusters were split into their constituent cells using the implemented cluster splitting algorithm that is based on the method of Farhan et al. . Here, a cluster was split after identifying the concave contour regions between the borders of neighboring convex cells. Since the parameters of this cluster splitting method are strongly dependent on pathogen specific properties, such as the average diameter of the respective cells, we adapted these parameters individually for each type of pathogen following the guidelines given by Brandes et al. . The respective parameter values are given in S4 Table of S1 File. In case of C. albicans cells and C. glabrata cells, we observed reproduction by budding where small outgrowths are formed. These smaller daughter cells adhere to the mother cell and are only evaluated as a new cell if they reach the size of an average cell. The cluster splitting algorithm detected cells with a large bandwidth of different area sizes. To prevent daughter cells from being counted as single cells, these were excluded by a histogram threshold method over the area sizes (see S4 Table of S1 File).
In the case of S. aureus cells with sizes of only 1 μm in diameter–corresponding to five pixels à 0.2 μm–an alternative segmentation approach was applied involving a pixel-based thresholding approach. Since the illumination was varying from image to image, handling of S.aureus images was done in the following way: Images showing local inhomogeneities with a small contrast between the cell border and the background intensity were pre-processed by a contrast enhancement, where the image intensity was stretched over the whole range of intensity values. In order to remove background noise and emphasize the cells itself, a median filter with kernel size 4 × 4 px was used. Finally, a global thresholding was performed on the image intensity histogram to reduce noise in the images. The number of S.aureus cells was then determined by dividing the amount of foreground pixels through the average cell size. In order to determine the total number of dead cells, the resulting segmented image was applied as a binary mask for the PI stained channel that was processed by the same three methods, i.e. thresholding, median filtering and contrast enhancement.
Whole-blood infection modeling
In this study, we generated mathematical models with different mechanisms for immune evasion of pathogens in whole blood. These state-based models (SBMs) were derived from our previous models of whole-blood infection [1, 12]. In order to compare the dynamics of the SBMs with the time-resolved experimental measurements, we defined the five combined units PE, PM, PN, PA and PK as variables that combine specific states, which represent specific cell populations. The combined unit PE is defined by (1) and incorporates all states that represent extracellular pathogens, i.e. states for extracellular pathogens that are alive (PAE) or killed (PKE), or immune-evasive pathogens that are alive (PAIE) or killed (PKIE). Pathogens phagocytosed by PMN or monocytes are respectively recorded within the combined units (2) and (3)
Here, the states Ni,j and Mi,j represent neutrophils and monocytes respectively, with the number of i alive pathogens and j killed pathogens. We limit the maximal number of phagocytosed pathogen cells to nine (i.e. j<10), a value far above what has been observed in experiments . The combined unit PA contains all states that represent alive pathogens and is defined by (4) whereas the combined unit of killed pathogens is given by (5)
Note that the total number of pathogens is given by P ≡ PE + PN + PM and P ≡ PA + PK. Biological processes that take place during the immune response to the pathogens are represented by state transitions characterized by transition rates. All SBMs contain at least four types of state transitions that are the phagocytosis by PMN or monocytes, the intracellular killing by PMN or monocytes and the extracellular killing of pathogens (see S5 Table of S1 File). The implementation of the immune-evasion mechanism of pathogens differs between the SBMs as described in the next subsection.
The SBMs were applied to simulate the immune response to 1 × 106 pathogens in 1 ml blood containing 5 × 106 PMN and 5 × 105 monocytes; thus, the initial conditions of the model simulation are PA(t = 0) = 1 × 106, N0,0(t = 0) 5 × 106, M0,0(t = 0) 5 × 105. The time-evolution of the SBMs is calculated by the random selection simulation algorithm using a time-step driven, synchronous update scheme  (see S1 Fig of S1 File for the flow chart of the algorithm). In this simulation algorithm, the simulation time is divided into discrete time-steps with distance Δt and at each simulation time-step a transition from state S to state S′ can be performed with probability PS→S′ = rS→S′ × Δt, where rS→S′ denotes the corresponding transition rate. In S5 Table of S1 File, a complete list of transition rates of the SBMs is given. Here it can be seen that the SBMs contain common transition rates for the following transitions: phagocytosis by PMN or monocytes, intracellular killing by PMN or monocytes as well as extracellular killing. The killing of pathogens in the extracellular space is caused by antimicrobial peptides released during the first-time phagocytosis of PMN. Therefore, the rate for extracellular killing κEK(t) is defined by (6) and describes the time-dependent effect of antimicrobial peptides by the constant rate , a half-life time characterized by the rate γ and the number of first-phagocytosis events (NPE) per PMN (N0,0).
For further information about the model structure, the simulation algorithm as well as the estimation of transition rate values using the global optimization algorithm Simulated Annealing we refer to our previous studies by Hünniger et al.  and Lehnert et al. .
In order to investigate the unresolved mechanism of immune evasion of pathogens, we generated various SBMs that differ in the implementation of the rate for the acquisition of an immune-evasive state. We implemented two models with solely one immune-evasion mechanism, two models without an evasion mechanism but instead with either populations of alive and killed pre-existing evasive cells or with only alive pre-existing cells. We refer to these models as base models in the following. Additionally, we implemented four so called combined models that are composed of at least two base models and thereby contain more than one option for the acquisition of immune evasion. The representation of the different immune-evasion mechanisms in the SBMs is described in the following section and an overview is given in Table 1. The source code of all models is available in the Immune evasion repository at https://asbdata.hki-jena.de/LehnertEtAl_PLOS/.
Modeling immune evasion of pathogens
The population of pathogens that can evade the immune response in whole blood was first described in our studies by Hünniger et al.  and Lehnert et al. . In these studies, experimental measurements revealed that at four hours post infection alive and extracellular pathogens are present. These observations gave rise to define the transitions (7) and (8) where extracellular pathogens that are alive (PAE) or killed (PKE) can become immune-evasive pathogens that are alive (PAIE) or killed (PKIE) and thereby cannot be phagocytosed by immune cells and/or killed. The corresponding transition rate was defined by (9) since there were no information about this mechanism known and we consequently assumed that the process occurs spontaneously (see Fig 1A). In the following, we refer to this model as spon-IE model.
In the subsequent study by Prauße et al. , we took advantage of the strength of mathematical modeling and implemented the PMNmed-IE model to test the hypothesis that the acquisition of immune evasion is mediated by PMN. More precisely, we speculated that during first-time phagocytosis by PMNs not only molecules causing extracellular killing will be released but also specific molecules that aims at protecting pathogens against the immune system (see Fig 1B). In the PMNmed-IE model, we replaced the constant transition rate of the spon-IE model and defined the time-dependent rate for the acquisition of immune evasion by (10)
Similar to the time-dependent rate for extracellular killing κEK(t) (see Eq 6), this rate represents the time-dependent effect of specific molecules that were secreted upon the first-time phagocytosis events by PMN (PPE). The effect of these molecules is specified by the constants and γIE and decreases exponentially.
In addition, we investigated the hypothesis that immune-evasive cells do not develop during immune response to pathogens but instead are present before infection as pre-existing immune-evasive cells (see Fig 1C). To this end, we implemented two models that do not contain the transition from the extracellular state to the immune-evasive state (see Eqs 7 and 8), but rather contain constant populations of immune-evasive cells that are present during the infection time. In the pre-IE model, we assume that both alive and killed evasive pathogens are present before infection and therefore set the number of alive and killed immune-evasive cells at time point of infection t = 0 to PAIE > 0 and PKIE > 0. The alivePre-IE model contains exclusively alive evasive pathogens (see Fig 1D) and therefore, we set PAIE > 0 and PKIE > 0 at time point of infection t = 0.
Thereby, the number of a priori unknown transition rates decreased in these two models in comparison to the spon-IE and the PMNmed-IE models. However, the numbers of alive and killed immune-evasive cells at time t = 0 min are additional a priori unknown values and have to be estimated in the course of the parameter estimation. The total number of parameters of each SBM that has to be estimated is given in Table 1. As a starting value of evasive cells, the algorithm randomly chooses a value in the range of the mean ± standard deviation of experimentally measured pathogens that are free in extracellular space and not associated with immune cells at four hours post infection (see Fig 2B).
Furthermore, we implemented four SBMs with more than one of the above described evasion mechanisms in order to investigate the possibility that immune evasion of pathogens could be developed in various ways. As given in Table 1, we implemented not only three SBMs with two evasion mechanisms, i.e. the spon-PMNmed-IE model, the spon-alivePre-IE model and the PMNmed-alivePre-IE model, but also one SBM with three immune-evasion mechanisms, the spon-PMNmed-alivePre-IE model.
Model parameter estimation by Simulated Annealing
The estimation of optimal values for the transition rates of the different immune evasion mechanism was rformedperformed by the global optimization method Simulated Annealing based on the Metropolis Monte Carlo scheme . Starting with a random position in the parameter space, the algorithm performs a random walk and decides for the next step based on a score of the deviations between the model simulation with model parameters and experimental data, i.e. the weighted sum of least-square errors (LSE) (11)
Here, ωc is defined as the specific weight of the LSE (12) which is defined as the sum of the squared difference between the experimental data () and the comparable simulated data (), which forms the combined unit c, for each time point k. The definition of each combined unit is provided in the methods section “Whole-Blood Infection Modeling”. The weights used for the infection scenarios with C. albicans and C. glabrata were taken over from our previous studies [1, 2]. For infection with S. aureus, we used the same weights as for C. albicans infection.
In course of the random walk through the parameter space, the next step will be performed by accepting a parameter set with a smaller in comparison to , the score of the current parameter set , i.e. . However, the Metropolis Monte Carlo scheme enables that also parameter sets with a larger LSE are accepted with a certain probability, the acceptance probability, which is dependent on the Boltzmann distribution and decreases with the increasing number of fitting steps f. The acceptance function plays the role of the “system’s temperature”, which decreases during the annealing process. The flowchart in S2 Fig of S1 File shows the algorithm. For a detailed description of this algorithm we refer to our previous studies of Hünniger et al.  and Lehnert et al. . The source code of this algorithm is available in the Immune evasion repository at https://asbdata.hki-jena.de/LehnertEtAl_PLOS/.
We repeated the whole fitting procedure to increase the statistical robustness of this procedure. Furthermore, we performed this procedure for different system sizes by starting with P = 100 cells, M0,0 = 50 cells and N0,0 = 500 cells and stepwise increasing the number of cells by a factor of ten, while keeping the ratio of pathogens and immune cells constant. The optimal parameter set of a system size was used as starting point for the fitting procedure of the next larger system size until the system size of 1 ml blood with P = 106 cells, M0,0 = 5 · 105 cells and M0,0 = 5 · 106 cells is reached.
Model comparison based on information criterion
In order to compare the different mathematical models for the immune evasion of pathogens, we applied the Akaike information criterion (AIC) . This score aims to rank the different models by including not only the agreement of the model kinetics with the experimental data, like the LSE (see Eq 12), but also by incorporating the complexity of the models regarding the number of parameters. For least-square estimation, the AIC is defined by (13) as described by Burnham et al. . Here, K denotes the number of parameters that is directly proportional to the penalty defined in the second term. The first term of Eq 13 represents the penalty by the number of independent experimental data points n and by the deviation of the model and the experimental data in terms of the sum of squared residuals (14)
This variable is calculated by the sum of the squared differences between experimental data (xexp) and simulated data () resulting from simulating the model with the optimal parameter set at each of the ith experimentally measured data points (i = 1, 2, 3, …, 27). The corrected version of the AIC, the (15) must be applied if n/K ≤ 40, i.e. the number of model parameters is large relative to the number of experimental data points n . For each of the m models, the AICC value was calculated and, subsequently, the relative measure (16) was used to rank the models. Here, denotes the relative AICC difference of the ith model with to the model with the smallest AICC (). Based on this difference the models were categorized by means of the following assessment guidelines 
Statistical analysis was performed by applying the following steps. First, we test whether the underlying data is normally distributed using the Shapiro-Wilk test. If the data was normally distributed, the unpaired t-test was used to test for significant differences. If the data was not normally distributed the Mann-Whitney U test (Wilcoxon rank-sum test) was applied to test unpaired samples for significant differences. If comparisons were made between several data sets, a multiple comparison correction (Bonferroni’s correction) was performed afterwards. The corrected P-value is given by P′. Significance is shown as *P′ < 0.05, **P′ < 0.001, ***P′ < 0.001.
- 1. Hünniger K, Lehnert T, Bieber K, Martin R, Figge MT, Kurzai O. A Virtual Infection Model Quantifies Innate Effector Mechanisms and Candida albicans Immune Escape in Human Blood. PLoS Comput Biol. 2014;10. pmid:24586131
- 2. Prauße MTE, Lehnert T, Timme S, Hünniger K, Leonhardt I, Kurzai O, et al. Predictive virtual infection modeling of fungal immune evasion in human whole blood. Front Immunol. 2018;9: 1–13. pmid:29403488
- 3. Hünniger K, Bieber K, Martin R, Lehnert T, Figge MT, Löffler J, et al. A Second Stimulus Required for Enhanced Antifungal Activity of Human Neutrophils in Blood Is Provided by Anaphylatoxin C5a. J Immunol. 2015;194: 1199–1210. pmid:25539819
- 4. Timme S, Lehnert T, Prauße MTE, Hünniger K, Leonhardt I, Kurzai O, et al. Quantitative simulations predict treatment strategies against fungal infections in virtual neutropenic patients. Front Immunol. 2018;9: 1–14. pmid:29403488
- 5. Duggan S, Essig F, Hünniger K, Mokhtari Z, Bauer L, Lehnert T, et al. Neutrophil activation by Candida glabrata but not Candida albicans promotes fungal uptake by monocytes. Cell Microbiol. 2015;17: 1259–1276. pmid:25850517
- 6. Essig F, Hünniger K, Dietrich S, Figge MT, Kurzai O. Human neutrophils dump Candida glabrata after intracellular killing. Fungal Genet Biol. 2015;84: 37–40. pmid:26385824
- 7. Calderone R, Clancy C. Candida and Candidiasis. American Society for Microbiology Press; 2011.
- 8. Brunke S, Hube B. Two unlike cousins: Candida albicans and C.glabrata infection strategies. Cell Microbiol. 2013;15: 701–708. pmid:23253282
- 9. Li L, Redding S, Dongari-Bagtzoglou A. Candida glabrata, an emerging oral opportunistic pathogen. J Dent Res. 2007;86: 204–215. pmid:17314251
- 10. Wisplinghoff H, Bischoff T, Tallent SM, Seifert H, Wenzel RP, Edmond MB. Nosocomial bloodstream infections in US hospitals: Analysis of 24,179 cases from a prospective nationwide surveillance study. Clin Infect Dis. 2004;39: 309–17. pmid:15306996
- 11. Thomer L, Schneewind O, Missiakas D. Pathogenesis of Staphylococcus aureus Bloodstream Infections. Annu Rev Pathol Mech Dis. 2016;11: 343–364. pmid:26925499
- 12. Lehnert T, Timme S, Pollmächer J, Hünniger K, Kurzai O, Figge MT. Bottom-up modeling approach for the quantitative estimation of parameters in pathogen-host interactions. Front Microbiol. 2015;6: 1–15. pmid:25653648
- 13. Akaike H. A New Look at the Statistical Model Identification. IEEE Trans Automat Contr. 1974;AC-19: 716–723.
- 14. Brandes S, Dietrich S, Hünniger K, Kurzai O, Figge MT. Migration and interaction tracking for quantitative analysis of phagocyte–pathogen confrontation assays. Med Image Anal. 2017;36: 172–183. pmid:27940225
- 15. Medyukhina A, Timme S, Mokhtari Z, Figge MT. Image-based systems biology of infection. Cytom Part A. 2015;87: 462–470. pmid:25641512
- 16. Mech F, Wilson D, Lehnert T, Hube B, Thilo Figge M. Epithelial invasion outcompetes hypha development during Candida albicans infection as revealed by an image-based systems biology approach. Cytom Part A. 2014;85: 126–139. pmid:24259441
- 17. Johnson JB, Omland KS. Model selection in ecology and evolution. Trends Ecol Evol. 2004;19: 101–108. pmid:16701236
- 18. Stoica P, Selen Y. Model-order selection. IEEE Signal Process Mag. 2004;21: 36–47.
- 19. Warne DJ, Baker RE, Simpson MJ. Using Experimental Data and Information Criteria to Guide Model Selection for Reaction–Diffusion Problems in Mathematical Biology. Bull Math Biol. 2019;81: 1760–1804. pmid:30815837
- 20. Schwarz G. Estimating the Dimension of a Model. Ann Stat. 1978;6: 461–464.
- 21. Portet S. A primer on model selection using the Akaike Information Criterion. Infect Dis Model. 2020;5: 111–128. pmid:31956740
- 22. Guan D, Ahmad F, Theobald P, Soe S, Luo X, Gao H. On the AIC ‑ based model reduction for the general Holzapfel–Ogden myocardial constitutive law. Biomech Model Mechanobiol. 2019. pmid:30945052
- 23. Yang H, Liu Y, Liang H. Focused information criterion on predictive models in personalized medicine. Biometrical J. 2015;57: 422–440. pmid:25653006
- 24. Posada D, Buckley TR. Model selection and model averaging in phylogenetics: Advantages of akaike information criterion and bayesian approaches over likelihood ratio tests. Syst Biol. 2004;53: 793–808. pmid:15545256
- 25. Sagar A, Dai W, Minot M, LeCover R, Varner JD. Reduced order modeling and analysis of the human complement system. PLoS One. 2017. pmid:29155837
- 26. Marino S, Hogue IB, Ray CJ, Kirschner DE. A methodology for performing global uncertainty and sensitivity analysis in systems biology. Journal of Theoretical Biology. 2008. pmid:18572196
- 27. Li C, Mahadevan S. Relative contributions of aleatory and epistemic uncertainty sources in time series prediction. Int J Fatigue. 2016;82: 474–486.
- 28. Zi Z. Sensitivity analysis approaches applied to systems biology models. IET Syst Biol. 2011. pmid:22129029
- 29. Wu J, Dhingra R, Gambhir M, Remais J V. Sensitivity analysis of infectious disease models: Methods, advances and their application. J R Soc Interface. 2013;10. pmid:23864497
- 30. Cornell SJ, Suprunenko YF, Finkelshtein D, Somervuo P, Ovaskainen O. A unified framework for analysis of individual-based models in ecology and beyond. Nat Commun. 2019;10: 1–14. pmid:30602773
- 31. Sreekantapuram S, Lehnert T, Prauße MTE, Berndt A, Berens C, Figge MT, et al. Dynamic Interplay of Host and Pathogens in an Avian Whole-Blood Model. Front Immunol. 2020;11. pmid:32296424
- 32. Gillum AM, Tsay EYH, Kirsch DR. Isolation of the Candida albicans gene for orotidine-5′-phosphate decarboxylase by complementation of S. cerevisiae ura3 and E. coli pyrF mutations. Mol Gen Genet MGG. 1984;198: 179–182. pmid:6394964
- 33. Dujon B, Sherman D, Fischer G, Durrens P, Casaregola S, Lafontaine I, et al. Genome evolution in yeasts. Nature. 2004;430: 35–44. pmid:15229592
- 34. Seider K, Brunke S, Schild L, Jablonowski N, Wilson D, Majer O, et al. The Facultative Intracellular Pathogen Candida glabrata Subverts Macrophage Cytokine Production and Phagolysosome Maturation. J Immunol. 2011;187: 3072–3086. pmid:21849684
- 35. Tuchscherr L, Heitmann V, Hussain M, Viemann D, Roth J, von Eiff C, et al. Staphylococcus aureus Small‐Colony Variants Are Adapted Phenotypes for Intracellular Persistence. J Infect Dis. 2010;202: 1031–1040. pmid:20715929
- 36. Tuchscherr L, Pöllath C, Siegmund A, Deinhardt-Emmer S, Hoerr V, Svensson C-M, et al. Clinical S. aureus Isolates Vary in Their Virulence to Promote Adaptation to the Host. Toxins (Basel). 2019;11: 135. pmid:30823631
- 37. Brandes S, Mokhtari Z, Essig F, Hünniger K, Kurzai O, Figge MT. Automated segmentation and tracking of non-rigid objects in time-lapse microscopy videos of polymorphonuclear neutrophils. Med Image Anal. 2015;20: 34–51. pmid:25465844
- 38. Al-Zaben N, Medyukhina A, Dietrich S, Marolda A, Hünniger K, Kurzai O, et al. Automated tracking of label-free cells with enhanced recognition of whole tracks. Sci Rep. 2019;9: 1–10. pmid:30626917
- 39. Farhan M, Yli-Harja O, Niemistö A. A novel method for splitting clumps of convex objects incorporating image intensity and using rectangular window-based concavity point-pair search. Pattern Recognit. 2013;46: 741–751.
- 40. Schönfisch B, de Roos A. Synchronous and asynchronous updating in cellular automata. 1999;51: 123–143. pmid:10530753
- 41. Kirkpatrick S, Gelatt CD, Vecchi MP. Optimization by Simulated Annealing. Science (80-). 1983;220: 671–680. pmid:17813860
- 42. Burnham KP, Anderson DR. Multimodel inference: Understanding AIC and BIC in model selection. Sociol Methods Res. 2004;33: 261–304.
- 43. Burnham KP, Anderson DR. Model Selction and Multimodel Inference: A Practical Information-Theoretic Approach. 2nd ed. Springer; 2002.