When to wake up? The optimal waking-up strategies for starvation-induced persistence

Prolonged lag time can be induced by starvation contributing to the antibiotic tolerance of bacteria. We analyze the optimal lag time to survive and grow the iterative and stochastic application of antibiotics. A simple model shows that the optimal lag time can exhibit a discontinuous transition when the severeness of the antibiotic application, such as the probability to be exposed the antibiotic, the death rate under the exposure, and the duration of the exposure, is increased. This suggests the possibility of reducing tolerant bacteria by controlled usage of antibiotics application. When the bacterial populations are able to have two phenotypes with different lag times, the fraction of the second phenotype that has different lag time shows a continuous transition. We then present a generic framework to investigate the optimal lag time distribution for total population fitness for a given distribution of the antibiotic application duration. The obtained optimal distributions have multiple peaks for a wide range of the antibiotic application duration distributions, including the case where the latter is monotonically decreasing. The analysis supports the advantage in evolving multiple, possibly discrete phenotypes in lag time for bacterial long-term fitness.

Introduction.When bacteria cells are transferred from a starving environment to a substraterich condition, it takes sometime before the cells start to grow exponentially.This lag phase [1] is often considered as the delay during which the cells modify their gene expression pattern and intracellular composition of macromolecules to adapt to the new environment [2,3].Therefore, the characteristics of the lag phase depend on the growth condition before the starvation, the environment during the starvation, and the new environment for the regrowth.In spite of this complexity, naıvely thinking, reducing the lag time as possible appears to be better for the bacterial species because it maximizes the chances of population increase.However, interestingly, it has been reported that the distribution of the lag time at a single-cell level has much heavier tail than a normal distribution [4][5][6].
Indeed, having a subpopulation with long lag time can be beneficial under certain circumstances, for instance, when the nutrients are supplied together with antibiotics.This is because antibiotics often target active cellular growth processes and hence dormant, non-growing cells are tolerant of the killing by antibiotics [7][8][9].In general, dormant cells tend to be less sensitive to environmental stress, providing a better chance of survival.Therefore, the lag phase can work as a shelter for the cells from lethal stress.
The phenotypic tolerance provided by a dormant subpopulation has been attracting attention as a course of bacterial persistence [9][10][11][12][13].Operationally persistence can be categorized into two types [11,13]; type I or triggered persistence, where the dormancy is triggered by external stress such as starvation, and type II or spontaneous persistence, where the cells switch to a dormant state even though the environment allows exponential growth of the population.The spontaneous persistence has been interpreted as a bet-hedging strategy [14][15][16][17], where the optimal switching rates between dormancy and growth is proportional to the switching rates of the environments with and without antibiotic.For the triggered persisters, there should also be an optimal lag time distribution under given antibiotics application.Analysis of the optimal lag time should be relevant in understanding bacterial persistence given a recent laboratory experiment with Escherichia coli showing that the starvation triggers the dominant fraction of the bacterial persistence [18], as well as their appearance in pathogenic bacteria Staphylococcus aureus and its correlation with antibiotic usage [19].
Previously, Friedman et al. [20] have conducted an experiment to see whether bacteria can evolve to increase lag-time by an iterated application of the antibiotics.In the experiment, they grew bacteria with fresh media supplemented with an antibiotic.The antibiotic was removed after a fixed time T had passed, and the culture was left for one day to let the survived bacteria grow and enter the stationary phase.Then, a part of the one-day culture was transferred to the next culture, supplemented fresh media with the antibiotic.By repeating the procedure, it was found that the mean lag time of the bacteria has evolved to the roughly same length to T , which is expected to be optimal for the long-term population growth.
The present work is motivated by this experiment.In their experiment, the antibiotics were applied at every re-inoculation.What will happen if the application of the antibiotic is probabilistic?What is the optimal average lag time?Is it better for the total growth to split into subpopulations with different lag times?
We here analyze the optimal waking-up strategy under the probabilistic antibiotic application by using a population dynamics model.Analytical and numerical calculations show that the evolution to increase the lag time occurs only if the effect of the antibiotics exceeds a certain threshold, at which a discontinuous transition of the optimal lag time from zero to finite happens.We then extend the model so that the population can be split into two subgroups with different average lag time, to show that there is a continuous transition from single-strategy to bet-hedging strategy when changing the antibiotics application probability and time.
Model.Motivated by ref. [20], we consider the following setup: Bacterial cells are transferred from stationary phase culture to a fresh media where all the cells are in a dormant state.At every re-inoculation, the fresh media is supplemented with the antibiotics with the probability p.The antibiotics are removed at time T , and after that, the culture will be left to grow long enough time until t T before entering the stationary phase.The population dynamics model is adopted from [20] and slightly modified.We assume that a cell can take two states, namely, the dormant (or lag) state and the growing state.A cell in the dormant state is assumed to be fully tolerant of the antibiotics but cannot grow.A cell in the growing state dies at a rate of γ if the antibiotic exist in the environment, and proliferate at a rate of µ if there is no antibiotic.We first analyze a simple case, where the cells in the dormant state transit to the growing state at a constant rate of 1/λ.Hence λ corresponds to the average lag time of the population, and the lag time distribution follows an exponential function.The transition from a growing state to the dormant state is not considered when there are nutrients in the culture.Hereafter, we set µ to unity by taking 1/µ as unit of time.Then, the temporal evolution of the population after an inoculation is ruled by where d and g represent the population in the dormant state and growing state, respectively.The population dynamics of the antibiotic-free case is obtained by setting T = 0. We set the initial population to unity without losing generality, i.e., d(0) = 1, g(0) = 0.By solving the equation, g(t; T ) under given T is written as Since the initial population is unity and d(t) is given as exp(−t/λ) ≈ 0, ln[g(t; T )] represents the order of magnitude of total population increase for time t.By noting that the population with zero-lag time grows as exp(t) under the antibiotic-free condition, f (T ) ≡ ln[g(t; T )/ exp(t)] = ln[g(t; T )] − t represents the logarithmic population growth normalized by the exponential growth without the antibiotics and the lag time per one round of the inoculation and growth.With many repetition of this process, the long-term average normalized growth per round F I (λ; γ, p, T ) is given by averaging f (T ) over the probability p of the antibiotics application [21][22][23] as +p −T + ln e −T /λ − e −γT γλ − 1 + e −T /λ 1 + λ .
Hereafter, we study the optimal lag time λ * which maximizes the fitness F I for a given environmental parameter set p, T , and γ.
Optimal lag time.In Fig. 1(a), the optimal lag time λ * is plotted as a function of p for various values of T , with γ = 1.The killing rate γ ≈ 1 is biologically reasonable range since it is often found to be same order of magnitude of the bacterial growth rate [7,24].When changing p, the optimal lag time λ * shows a discontinuous transition, with a critical p value dependent on T (and γ).Below each critical p value, the optimal lag time is zero, while above the threshold, it increases with p as λ * ∼ pT , reaching λ * ∼ T when antibiotic is always present (p = 1).The fitness F I is plotted as a function of 1/λ in Fig. 1(b), demonstrating the appearance of a local maximum at a positive finite λ leading to a discontinuous transition above a critical p.We found that transition takes place by changing the severeness of the application of the antibiotic, by changing one of the parameters among γ, p, and T with keeping the rest constant.In the γ → ∞ limit, it is easy to show that the optimal lag time is given by 1/λ * = (−1 + 1 + 4/pT )/2.Specifically for the killing rate γ, we give proof for the existence of the critical value γ c = γ c (p, T ) at which the transition occurs and an upper bound of γ c which is 1/p − 1 in Supplement., γ, p, T ).The local peak of the fitness is formed at p ≈ 0.2, and the fitness value at the peak exceeds its value at 1/λ → ∞.The local optimal of F I is formed at p ≈ 2.0, and it becomes the global optimal at p ≈ 2.5.γ is set to unity and T = 6 for (b).
Bet-hedging.So far, we have studied the optimal lag time where all the cells have the same transition rate (1/λ).However, it is known that even an isogenic bacterial population can split the population into several phenotypes.To see whether the best strategy changes in the multiphenotype case, we do the simplest extension of the model to the case where the bacteria is capable of having two subpopulations with different values of average lag time.
We split the total population into two parts a and b, and assume the transition rates from the dormant to growing state being 1/λ a and 1/λ b , respectively.Without loss of generality, we assume λ a ≤ λ b , and denote the fraction of the subpopulation b as x.The fitness function is then written down as where f a (T ) (f b (T )) is ln[g(t; T )/ exp(t)] with the transition rate 1/λ a (1/λ b ).In this case, the parameters that the bacteria can evolve to optimize are λ a , λ b , and x.
Figure 2 shows the optimal parameter values as a function of the antibiotics application time T for different probability p, obtained by the numerical optimization of λ a , λ b , and x.The optimal lag time of the single-strategy case is also shown for comparison.
Interestingly, the antibiotics application probability changes the qualitative behavior of the optimal fraction (x * ) to an increase of T .For a small p value (Fig. 2a), all the cells have zero lag time in the short T region (x * = 0), while as T gets longer, the population start to invest subpopulation to the non-zero lag time phenotype.This strategy corresponds to the bed-hedging strategy which is well-investigated in the population dynamics field [22,23] including type II persistence [14,15] as well as the finance [21]; as the risk of the antibiotic application becomes larger with longer T , it The optimal fraction of the strategy with non-zero lag time x * , the optimal lag times of two strategies (λ * a = 0 and λ * b ), and the optimal lag time for the one strategy case (λ * ) are plotted against the antibiotics application time T for different probability of the antibiotics application p.The left vertical axis is for x * while the right vertical axis for the others.We also plot the locally optimal solution of the fraction as x * L (see the main text for further description).p = 0.2 for (a) and 0.8 for (b).γ is set to unity.pays off to save small fraction of population to finite lag time to hedge the risk.On the other hand, if the value of p is relatively large (Fig. 2b), all the population has non-zero lag time (x * = 1) even if T is small, reflecting that the chance of having antibiotics is too high that it does not worth betting subpopulation into zero lag time going for more growth in no-antibiotic condition.However, as T gets larger, the optimal strategy changes to bet some fraction of the population to zero-lag time.This somewhat counterintuitive result is due to the trade-off between the benefit and cost of having a longer lag time.Cells can avoid getting killed by having the lag time being close to the antibiotics application time T .However, having a long lag time means waiving the opportunity to grow even when the fresh media is fortunately antibiotics-free.While the loss of the opportunity is negligible for small T , as T gets larger, the loss becomes sizable and it becomes better for the population to bet some fraction for the chance of the media to be antibiotics-free.
In the analysis, we also found some locally optimal strategies, which are shown in Fig. 2 as x * L .For both p = 0.2 and p = 0.8, the optimal strategy for short T is having a single phenotype (x * = 0 for (a) and x * = 1 for (b)).For p = 0.2 case, the single phenotype strategy is locally stable as long as the zero-lag time is the optimal lag time for that strategy (x * L = 0), while after the optimal lag time for single phenotype case λ * becomes non-zero, the single-strategy is no longer optimal, even locally.On the other hand, for p = 0.8 case, the single phenotype strategy is always optimal, at least, locally (x * L = 1).In this situation, the optimal lag time of single strategy is non-zero for all T , and thus, the cells can keep being adapted to prolonged antibiotics application time by simply increasing the lag time, and forcing a part of the cells to have slightly different lag time leads no advantage for the population growth.Therefore, the locally optimal branch exists for any T > 0.
In Fig. 3, the globally optimal fraction x * of the subpopulation with finite lag time is shown as a heat-map as functions of p and T .There are three phases, namely, (i) all the cells waking up immediately (x * = 0), (ii) all the cells have the finite lag time (x * = 1), and (iii) the bet-hedging phase (0 < x * < 1).The x * = 0 phase and the x * = 1 phase are placed in the region with small p and T and the region with large p and small T , respectively.For the shown case of γ = 1, behavior of x * to the increase of T changes at p ≈ 0.5.Increasing (decreasing) γ shifts the phase boundaries to smaller (larger) p and T .
Discussion.We have shown that the optimal waking-up strategy changes depending on the severeness of the antibiotic application.When only a single phenotype is allowed, the optimal average lag time exhibits a discontinuous transition from zero lag time to finite lag time as the severeness of the antibiotics is increased.If the cells can split the population to have two subpopulations with different average lag-time, bet-hedging behavior can occur depending on the severeness of antibiotics application, where one sub-population has zero lag time and the other sub-population has a positive finite lag time.
We like to comment on a few points regarding the connection between the present simple model and a more biologically realistic setup.First, the zero-lag time is biologically impossible, hence in reality, what is expected to happen when the zero-lag time is optimal is to evolve to have the shortest possible lag time.The finiteness of the shortest possible lag time affects the location of the critical points as one can infer from Fig. 1b, but as long as it is shorter than the finite optimal lag time after the transition, the qualitative nature of the transition stays the same.Second, in the main text, we presented the case with no killing by antibiotics when a cell is in the dormant state, but in reality, dormant cells are likely to die, too, just significantly slower than the growing cells.We have checked that introducing a finite but small death rate for dormant cells under antibiotic application does not alter the main conclusion (supplement).Third, we only presented a constant wake-up rate model, giving an exponential distribution of lag time for a given phenotype.Another simple case that analysis is straightforward is the case with δ-function distributed lag time, which is presented in the supplement.There are of course quantitative differences, but qualitatively we observe the parallel behaviors 1 , demonstrating that the transitions are a robust feature of the present system.Fourth, we analyzed only the single phenotype case and the two-phenotype case, but splitting into many more phenotypes or having continuous wide distribution is also possible.If the system split into a finite number of phenotypes, it is expected that there still will be a continuous transition from one phenotype being optimal to multiple phenotypes being optimal.When the continuous distribution of phenotype is possible, it is more likely to have a continuous change of the width of the distribution depending on the environmental parameters.Experimentally, a Gaussian distribution in short lag time with an exponential tail in long lag time [4] and bimodal distribution of lag time [5] have been reported, suggesting that relatively clear division into a few phenotypes can happen.It is worth mentioning that a power-law distribution of lag-time was inferred from the killing curve of bacteria population by antibiotic [6], hence the continuous distribution of the phenotype is also a possibility.Fifth, though repeating a fixed antibiotic application duration T is possible in a laboratory experiment, treating T as a stochastic variable can be more relevant in evolution in a natural environment.We have performed an analysis of the environment where T is exponentially distributed, for single-phenotype δ-function distributed lag time case.It has been found that the optimal lag time shows the continuous transition from zero to finite value when increasing the severeness of the antibiotic application (detail in supplement).
Assuming that the bacterial cells evolve to reach the optimal lag time, the present analysis implies the nontrivial evolution of tolerant phenotypes after repeated antibiotics application.If it is easier to evolve the population average lag time as single phenotype than having multiple phenotypes with different lag time, then the discontinuous transition predicted in the single-phenotype case imply the following: If the antibiotics are used very often (p higher than the critical probability for the transition), the treatment leads to the prolonged lag time.In contrast, if the antibiotics are used less often (low p), the lag-time may shorten as a result of selection, even though p is still non-zero.In other words, there is a critical frequency of antibiotic application below which one can avoid the evolution of more tolerant bacteria.Since the transition is expected to be sharp, it is expected to be relatively easy to detect the transition.If it is easy to evolve to have multiple phenotypes, the total population-averaged lag time would experience continuous transition because the transition of the fraction x * is continuous.Hence the transition may be harder to detect.Still, if the p and T are kept small enough to stay in x * = 0 phase, the evolution of tolerant phenotype can be avoided.Therefore, detection of the phase boundary can be clinically important.

Acknowledgement. This work is supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme under grant agreement No. [740704].
whole population avoid death by the antibiotic.This removes the transition in a single phenotype case with changing T .

Supplemental Information
The critical γ In this section, we describe detailed calculations.For the sake of readability, we first list the notations of some of the functions and parameters.
Here, we show that there is a value of γ at which the optimal λ value becomes non-zero.First, we see that there is a parameter region in which the optimal λ is zero.F I (λ, γ = 0, p, T ) is a strict monotonically decreasing function of λ, and is a continuous function of γ for 0 < p < 1 and T > 0. Thus, for a sufficiently small γ > 0, F I (λ, γ, p, T ) is also the monotonically decreasing function of λ meaning that the optimal λ is zero.
To see there is a transition of the optimal λ value from zero to non-zero, we derive a sufficient condition of λ = 0 being no longer optimal.Since γ represents the killing rate of the bacterial cells, ) hold regardless of the parameter values.Therefore, if λ = 0 is the optimal λ value, holds.Since the inequality is the necessary condition of λ = 0 being the optimal value, the contraposition of this argument is used as the sufficient condition of the transition.Note that F I (0, γ, p, T ) is a monotonically decreasing function of γ with −∞ as its value at γ → ∞ limit, whereas F ∞ I is the constant function.Therefore, there is a value of γ (= δ) which satisfies F I (0, δ, p, T ) = F ∞ I (λ * ∞ , p, T ).In γ > δ region, F I (0, γ, p, T ) is smaller than F ∞ I (λ * ∞ , p, T ), i.e., λ = 0 is no longer the optimal.From the previous argument, one can see that δ = 0 holds.
Note that F I (λ * , γ, p, T ) > F ∞ I (λ * ∞ , p, T ) always holds for finite γ values, and thus, the transition of the optimal λ takes place at value of γ (= γ c ) which is strictly lower than δ.
Next, we derive an upper bound of γ c .Note that λ * = 0 is equivalent to that the equation ∂F I /∂(1/λ) = 0 has no solution of v ≡ 1/λ in R + .By taking the partial derivative, we get where H(v, γ, p, T ) ≥ 0 holds (described later) regardless of parameter values being give as Here, H(0, γ, p, T ) = 0 and lim v→∞ H(v, γ, p, T ) = (1 + γ)p holds.Therefore, if γ is greater than 1/p − 1, ∂F I /∂v = 0 has a solution.This solution is only the solution if H is a monotonic function of v, but from the existence of δ, H always has a solution even if γ is smaller than 1/p − 1.Thus, γc = 1/p − 1 gives the upper bound of δ and γ c satisfying γc ≥ δ > γ c .As shown in Fig. 4, H approaches to a monotonic function as T becomes smaller, whereas for large T values, H shows the non-monotonic feature and ∂F/∂v = 0 has a solution even if γ is smaller than 1/p − 1. Lastly, we show that λ = 0 has no singularity while the transition is taking place to see the transition is discontinuous.While ∂F I /∂λ is not continuous at λ = 0, its right limit is given as p(1 + γ) − 1.Since 0 < γ c < γc holds and p(1 + γ) − 1 is negative in the region of γ < γc , λ = 0 is locally optimal at γ = γ c which means that λ * > 0 is not continuously branching out from λ * = 0.
H(v, γ, p, T ) ≥ 0 is shown as follows; first, the denominator can be rewritten as For the case of γ > v, (1 + γ)/(1 + v) > 1 and e (γ−v)T > 1 hold, and thus, the sign of the large parenthesis is positive and (γ − v) > 0 which means the denominator has the positive sign.For the opposite case, (1 + γ)/(1 + v)e (γ−v)T is less than one and (γ − v) < 0. Therefore, the denominator is again positive.The square bracket of the numerator is rewritten as where a = (γ − v)T .Here we introduce the function h(a) which is given as h(a) = 1 − (1 − a)e a .By noting that h(0) = 1 and dh/da = ae a hold, one can see that h(a) monotonically increases (decreases) with a in the region of a > 0 (a < 0), and h(0) is the minimum.Therefore, h(a) is always larger than one, and accordingly, positive.

Model extensions
In this section, we show the outcome of the models extended from what we studied in the main text to check the robustness of the present result.
Non-zero killing rate at the full dormant state In the main text, we assumed the zero killing rate at the full dormant state.To see whether this assumption is critical for the main results, we relax this assumption.
Here we introduce the non-zero killing rate also for the full dormant state.We assume that the killing rate at the full dormant state is smaller than that of the active state, and thus, we set it to αγ where 0 < α < 1.The rate equation is given as The solutions of the equations are given by d(t) = e −(1/λ+αγ)t (t < T ) e −αγT e −t/λ (t > T ) T − e −(1+1/λ+αγ)T e t + 1 1+λ e −(1+1/λ)T e t − e −t/λ e −αγT (t > T ) Thus, the fitness function is Fig. 5(a) shows the optimal lag time as the function of α and T .The optimal λ value still shows the transition by changing T .Since the nature of the transition is the same with α = 0 case, the transition is triggered by changing the severeness of the antibiotics application by changing either p, γ, and T .Also the model shows transition by changing α in the region T > ∼ 10.The transition takes place discontinuously as shown in Fig. 5(b).Note that the optimal lag time for α = 1 case is zero regardless of the other parameter values because there is no reason to stay at the dormant state.

The case of the delta function-type wake up
As the simplest case, we can virtually consider that the cells start to grow immediately after the time λ has passed without any variation, i.e., the lag time distribution follows the Dirac's delta function.Therefore, the temporal evolution of the bacterial population at time t (t > max{λ, T }) given as following (Table .
Table 1: The population of the cells at time t > max{λ, T} for each condition.
Then, the fitness function is given by By taking the first-order derivative of F δ I respect to λ, we obtain Therefore, the optimal lag time, λ * shows the discontinuous transition as It is worth noting that the transition point is determined only by γ and p.In contrast to the models described above and in the main text.The antibiotics application time T has no role in the transition.Also, the γ = 1/p − 1 is not upper bound of the transition point but exactly determines the transition point.
The calculations for the two-strategy case are also carried out analytically.For the two species case, the optimal parameter values are given as λ a = 0, λ b = T , and The optimal x is shown in Fig. 6 which has qualitatively the same features of the optimal fraction of the model described in the main text.The optimal lag time for stochastic antibiotic application duration T Lastly, we calculate the optimal lag time analytically for the delta-function case even if the period of the antibiotics application T is not fixed, but it follows a simple probability density function.Here, we assume that the antibiotics are applied for the period T with probability p(T ), where p(0) represents the probability of the new culture to be antibiotics-free.When we choose the exponential distribution with the parameter τ , τ −1 exp(−T /τ ), as p(T ).The fitness function is calculated as the weighted average of the fitness (9) as Since λ * < 0 is not allowed in this model, the optimal lag time is zero as long as p(1 + γ) < 1 holds, while it becomes non-zero when p(1 + γ) gets larger than unity and continuously increases as p or γ increases.Note that p(1 + γ) = 1 is the critical line for the fixed T case and the length of time of the antibiotics application never triggers the transition as well as the fixed T case.

Figure 1 :
Figure 1: (a).The optimal lag time value, λ * , is plotted as a function of p for several choices of T value.The optimal lag time shows the discontinuous transition.Below the transition point, λ * is zero.(b).The fitness function is plotted as a function of 1/λ for several values of p close to the critical value.Each dashed line represents lim 1/λ→∞ F I (λ, γ, p, T).The local peak of the fitness is formed at p ≈ 0.2, and the fitness value at the peak exceeds its value at 1/λ → ∞.The local optimal of F I is formed at p ≈ 2.0, and it becomes the global optimal at p ≈ 2.5.γ is set to unity and T = 6 for (b).

Figure 2 :
Figure2: The optimal fraction of the strategy with non-zero lag time x * , the optimal lag times of two strategies (λ * a = 0 and λ * b ), and the optimal lag time for the one strategy case (λ * ) are plotted against the antibiotics application time T for different probability of the antibiotics application p.The left vertical axis is for x * while the right vertical axis for the others.We also plot the locally optimal solution of the fraction as x * L (see the main text for further description).p = 0.2 for (a) and 0.8 for (b).γ is set to unity.

Figure 3 :
Figure 3: The phase diagram of the optimal fraction x * .The solid lines indicate the boundary between x * = 0 and x * > 0 (left region), x * = 1 (right region) and x * < 1. γ is set to unity.

Figure 6 :
Figure 6: The the optimal fraction x * is plotted as a function of p and T .The steep transition at p ≈ 0.5 in smaller T region is continuous.γ is set to unity.