Dynamics of Immune Escape during HIV/SIV Infection

Several studies have shown that cytotoxic T lymphocytes (CTLs) play an important role in controlling HIV/SIV infection. Notably, the observation of escape mutants suggests a selective pressure induced by the CTL response. However, it remains difficult to assess the definite role of the cellular immune response. We devise a computational model of HIV/SIV infection having a broad cellular immune response targeting different viral epitopes. The CTL clones are stimulated by viral antigen and interact with the virus population through cytotoxic killing of infected cells. Consequently, the virus population reacts through the acquisition of CTL escape mutations. Our model provides realistic virus dynamics and describes several experimental observations. We postulate that inter-clonal competition and immunodominance may be critical factors determining the sequential emergence of escapes. We show that even though the total killing induced by the CTL response can be high, escape rates against a single CTL clone are often slow and difficult to estimate from infrequent sequence measurements. Finally, our simulations show that a higher degree of immunodominance leads to more frequent escape with a reduced control of viral replication but a substantially impaired replicative capacity of the virus. This result suggests two strategies for vaccine design: Vaccines inducing a broad CTL response should decrease the viral load, whereas vaccines stimulating a narrow but dominant CTL response are likely to induce escape but may dramatically reduce the replicative capacity of the virus.


Introduction
HIV infection in humans and SIV infection in non-human primates is not cleared by the host's immune system. However, there is partial control of virus replication that is mainly attributed to the cytotoxic T lymphocyte (CTL) response [1,2]. Particularly the evolution of immune escape mutations posing severe fitness costs [3][4][5][6][7][8][9], suggests that there is a strong selective pressure induced by the CTL response [10,11].
It is a challenge to acquire longitudinal data on CTL escape since the virus diversity within a host has to be followed over a long phase of chronic infection. Additionally, little data is available during the early phase of infection, especially for HIV where the infection is difficult to diagnose during the first weeks. Asquith et al. [12] analyzed several data sets of immune escape in HIV, and concluded that killing by the CTL response is inefficient in infected humans. More recent studies described more rapid immune escape of SIV, and estimated more efficient killing in nonhuman primates [13][14][15]. Quantifying the process of CTL escape and estimating rates remains problematic since the underlying process of viral replication might be taken into account [16]. Furthermore, by analyzing escape at single epitopes it is difficult to assess the strength of the total response of different CTL clones recognizing different epitopes. Due to those difficulties it remains a challenge to devise a realistic model of HIV/SIV dynamics and the subsequent CTL escape.
The interaction of viral replication, mutation and selection by different CTL responses appears to be complex. Studies have observed CTL escape very early but also years after primary infection [17][18][19]. Also, it has been shown that the number of epitope-specific CTL responses varies over time [20][21][22]. Additionally, there is much controversy in the field of HIV evolution as to whether stochastic effects play a role in viral evolution within a host [23]. For example, it has been suggested that the waiting time for compensatory mutations prolongs the emergence of escape variants [24]. To better understand these processes, we present a computational model of HIV/SIV infection taking into account the dynamics of a broad cellular immune response targeting different viral epitopes. The simulations allow us to follow the dynamics in detail, and quantify several critical properties to compare them to data.

Model
We develop a computational model of HIV/SIV virus dynamics including a cellular immune response consisting of several CTL clones (Fig. 1). We have n different CTL clones recognizing n different epitopes derived from viral proteins. On the viral genome we allow two mutations to occur per epitope. One mutation confers escape from recognition by the specific CTL clone. Since escape mutations may be associated with a fitness cost in viral replication or infectivity, a second mutation can at least partially compensate for the fitness loss.
We translate those interactions into a set of ordinary differential equations (ODEs) and add stochastic events for viral mutation. Classically, the processes of infecting target cells or the killing of infected cells by CTLs have been described with simple massaction terms [25,26]. For instance, a previous study already described antigenic escape from CTL clones during HIV-1 infection with a simple mathematical model [27]. Recently, we developed terms describing a density-dependent infection that results in a better description of the dynamics of acute infection (i.e. less oscillatory) of the viral load and the immune response [28]. Moreover, the interaction of infected cells with effector CTLs, both for proliferation of effector cells and the killing of infected cells, were assumed to saturate according to Michaelis-Menten kinetics. Here, we integrate those interaction terms to devise a new virus dynamics model consisting of several CTL clones that is described by the following differential equations: dI i dt~b Non-infected CD4 + target cells T are produced at a rate of l cells per day, die at a rate d T and can become infected by virus particles of type V i with fitness f i at a maximal rate of b per day. Target cell availability for virus particles is density dependent, as the infection rate per virus particle is saturating over the total number of CD4 + cells (non-infected and infected). After infection of a target cell, reverse transcription occurs during which the virus can mutate with a probability of m per position (for further details see Methods). Having two positions to mutate per n epitopes, the number of different viral variants, m, is maximally 2 2n . Infected A number of n CTL clones can recognize n different epitopes and kill the cells infected with virus expressing those epitopes. The virus population can evade recognition from specific CTL clones by acquiring escape mutations (shown as e). Since escape mutations can be associated with a fitness cost in viral replication or infectivity, the virus additionally acquires compensatory mutations (shown as c) that can partially restore the viral fitness. doi:10.1371/journal.pcbi.1000103.g001

Author Summary
As a result of their high mutation rate, HIV and its counterpart SIV in non-human primates can evade recognition by the host immune response through the generation of viral variants, the so-called escape mutants. This avoidance of cytotoxic T lymphocyte (CTL) mediated killing seems to be one of the major reasons why virus replication is not controlled effectively. However, it remains difficult to investigate the critical properties of the dynamics of immune escape. To this end, we developed a new computational model of HIV/SIV infection consisting of several CTL clones that can recognize specific parts of viral proteins, i.e., epitopes. The simulations allow us to follow the dynamics of immune escape in detail and help to interpret longitudinal data of HIV/SIV infections. Interestingly, changing the relative sizes of the CTL clones leads to a different evolution of the virus. Instead of reducing the number of infected cells, an alternative strategy of vaccine design could be to reduce the replicative capacity of the virus that might have implications for disease progression.
cells I i die at a rate of d I per day, and are cleared by those CTL clones that can recognize an epitope on its surface. The matrix a ij defines the topology with which the CTL clone E j recognizes epitopes presented on the surface of the infected cell of type i, and contains either 1 (recognition) or 0 (no recognition). Following Michaelis-Menten kinetics, the CTL clones compete with each other for clearance of infected cells (Fig. 2). When P m l~1 I l a lj z P n l~1 E l a il wwh k , an infected cell is killed at a maximal rate of k per day. With increasing h k , we approach massaction kinetics for the killing, i.e. when P m l~1 I l a lj z P n l~1 E l a il vvh k cells are killed at a per capita rate of k h k P n j~1 E j a ij per day. Since the dynamics of virus particles V i is much faster than that of the cell populations [29], we assume a quasi-steady-state for the virus particles and set V i = pI i /d V [30]. 'Naive' CTLs E i are produced at a rate of s cells per day. If they recognize antigen produced by I j (i.e. a ji = 1) they proliferate at a maximal rate g per day and die at a rate d E . Upon infection, virus replicates rapidly and typically P m l~1 I l a lj ww P n l~1 E l a jl . Therefore, CTL effector cells are produced at a half-maximal rate when P m l~1 I l a li~hgi . To account for different avidities for the different CTL clones E i we draw h gi from a uniform distribution. Once the total number of CTL effector cells is high, P n l~1 E l a jl becomes important and we get inter-clonal competition between the CTL effector cells. An overview of the parameters is given in Table 1. For a more detailed description of the model see Methods.

Sequential Escape from Several CTL Clones
The most surprising phenomenon of CTL escape in HIV/SIV is the time scale at which it occurs. Selection of escape variants has been found to happen very early after acute infection, but also late after years [17][18][19]. Additionally, it has been shown that escape can occur sequentially [31]. Although it has been suggested that compensatory mutations delay the appearance of escape variants, it is still unclear why escape variants would occur so late when the CTL clones recognizing the epitope have been present since acute infection.
Our model describes the virus dynamics of an HIV/SIV infection and the subsequent immune escape in a very realistic manner (Fig. 3). Many escapes occur widely spaced out in time and we observe that their appearance is determined by the dynamics of the different CTL clones. Starting with the same CTL clone repertoire, Fig. 3A represents a simulation where killing of infected cells approaches mass-action dynamics (h k = 10 12 ). Fig. 3B shows a simulation where killing follows Michaelis-Menten kinetics and CTL clones compete for killing of infected cells (h k = 10 9 ). In both simulations, the total number of infected cells peaks a few weeks after infection and reaches a set-point level of around 10 7 to 10 8 cells (black line in top panels) [32]. The colored lines represent the amount of virus-infected cells containing an escape mutation at a specific epitope. Due to the large actual population size of infected cells and the high mutation rate, all single escape and compensatory mutants are produced rapidly during the acute phase of the infection and are maintained in a mutation-selection balance. Some escape variants are being selected early, and replace the wild-type variant at this epitope (e.g., blue line). However, many escapes occur late, despite their early presence in the virus population.
The dynamics of the CTL clones E i are depicted in the bottom panels. Upon infection, the clones become stimulated depending on the parameter h gi defining their avidity, which generates a single or a few dominant CTL clones and many sub-dominant clones. Escape preferentially occurs from dominant clones that more efficiently kill infected cells. However, due to a severe fitness cost for the escape mutation, there is no escape from the most dominant CTL clone in this particular simulation run. When an escape variant replaces the wild-type variant, the CTL clone looses antigenic stimulation and  declines. Because of inter-specific competition between CTL clones, previously sub-dominant clones can increase in size, increasing the selection pressure for the epitopes they recognize. It has been suggested that many escapes occur early, i.e. during the decline phase in viral load after the peak of infection and that they potentially prevent clearance of HIV/SIV. However, in our simulations immune escape does not occur before the set-point level is reached around two to three months after infection (Fig. 4A). This is because the CTL clones only become effective during the decline phase of virus after the peak of infection [28,33]. Due to a transient CD4 + target cell depletion [34][35][36], there is not enough viral replication for escape variants to increase in frequency during this phase. Therefore, escape variants become selected only after the set-point is approached. Nevertheless, many escapes occur during the first months after set-point levels have been attained. After about two years, the virus population stabilizes as the 'easy' escapes have been done, the replicative capacity is partially restored and only few escapes are expected to appear later during infection. However, it is important to note that Fig. 4A shows only the first appearance of escape variants. Some of those variants will start to fluctuate in frequency or revert back to wild-type and possibly re-emerge at a later time point. This has implications for the analysis of longitudinal data. If an escape is found to happen late it does not necessarily mean that it had not been selected earlier during infection Infected-cell death rates from different patients have been found to be close to a normal distribution with a mean of 0.45 d 21 [37]. We show that variation of the average infected-cell death rate is also expected to occur within a patient due to the transient decrease of the death rate during an immune escape from CTL mediated killing ( Fig. 4B and 4C). However, the death rate does not necessarily decrease over time, i.e. the half-life of infected cells is not expected to increase with disease progression.

Rates of Escape
Escape variants not only appear at different times during infection but also with different rates. The rate at which an escape variant replaces the wild-type, the so-called escape rate, is determined by the balance between the evaded rate of killing and the fitness cost of the escape mutation [12,38]. Furthermore, the heterogeneity of the wild-type and the escape variant population at all other epitopes can lead to a different selection induced by other CTL clones. In general, we can define the escape rate as the 'escape variant growth rate -wild-type growth rate'. A recent study quantified this process and concluded on the basis of slow escape rates and minimal fitness costs that killing of HIV-1-infected cells is inefficient in humans [12]. Unfortunately, the available data sets do not allow us to follow the process of escape in detail. In contrast, our model provides a unique tool to follow the dynamics and analyze rates of escape and link those to rates of killing. Fig. 5A shows the distribution of escape and the corresponding rates that occur in 1000 simulation runs over five years of infection. The histograms of killing and escape rates are shown in Fig. 5B and 5C, respectively. Although the total killing by all CTL clones is high (0.9 d 21 , see Table 1), most individual killing rates are below 0.4 d 21 , and most escape rates are below 0.2 d 21 . The lower rates of escape are due to the fitness cost of escape mutations that cannot totally be restored by compensatory mutations. Interestingly, the rates of escape are not distributed equally over the time of infection (Fig. 5A). We therefore calculate the mean of  killing and escape rates per year after infection (Fig. 5D). It can be seen that escape rates decrease late during infection and reach relatively low values. The corresponding killing rates decrease after the first year of infection and dramatically increase late during infection. This result shows a) that the selection strength of CTL clones can increase after there has been escape from other clones and b) that escape against efficient CTL clones can be associated with a dramatic cost in viral fitness that slows down the selection of the escape variant.
In Fig. 5A-D we assumed mass-action kinetics for the interaction of CTLs to kill infected cells, i.e., we set h k = 10 12 . However, when CTL effector cells form a complex with infected cells before delivering their lethal hit, the killing should follow Michaelis-Menten kinetics [28,39]. We analyzed the influence of a saturating killing term using several lower values of h k (Fig. 5E). Although the maximal total killing is constant (see Table 1), we can see that killing and escape rates decrease if the model approaches Michaelis-Menten kinetics. Moreover, escape occurs relatively frequent for high values of h k (mass-action) but rarely for low values of h k (saturation) (black dots).
To estimate rates of escape from in vivo data one has to obtain ratios of escape variants to the wild-type. However, a problem arises when escapes are too rapid to be followed by the relatively long sampling intervals [13]. This shortcoming can lead to an underestimation of the rate of escape. To analyze this effect we measure the frequency of an escape variant at four time points (100, 150, 200 and 250 days after infection) and estimate the escape rate as described in reference [12] (see also Methods). Fig. 5F shows the distribution of the estimated escape rates (gray bars) and the true escape rates (black line). Even though the sampling intervals were relatively short (50 days) it can be clearly seen that the estimated rates are generally lower, with a mean of about 50% of the true rates.
Our analysis shows several properties of the process of escape. Even though the total killing induced by all CTL clones is high, escape rates are expected to be slow. First, escape rates slow down due to the acquired fitness cost in viral replication, especially during later phases of infection. Secondly, if killing of infected cells follows Michaelis-Menten kinetics, the rates of escape are decreased. In addition, estimating rates of escape from ratios of escape variants and wild-type virus is likely to lead to an underestimation of the true rate. These findings highlight that slow escape rates do not necessarily infer low killing rates.

Immunodominance and Viral Evolution
CTL responses during HIV infection generally consist of several CTL clones recognizing many different epitopes. The size of those CTL clones can differ substantially resulting in dominant and subdominant responses [21,40]. In our model, the relative size of a CTL clone depends on how well it can recognize viral antigen, and is defined by h gi . By changing max h gi À Á we can change the sizes of the CTL clones relative to each other. For high max h gi À Á we get strong immunodominance with a single or few dominant clones and many sub-dominant clones, whereas low values of max h gi À Á result in CTL clones that are very similar in size (Fig. 6A). Note, that the total killing induced by the sum of all CTL clones remains constant (0.9 d 21 ) and is independent of max h gi À Á . To investigate the effect of immunodominance we run simulations for different values of max h gi À Á . Strong immunodo- minance, where single CTL clones can induce a strong selection pressure on the viral population, leads to more frequent escape (Fig. 6B). Concurrently, viral replication is controlled less efficiently and the number of infected cells increases with increasing immunodominance (Fig. 6C). However, viral escape can be associated with a fitness cost. Even though the number of infected cells increases, the replicative capacity of the viral population is reduced substantially (Fig. 6D).

Discussion
The interaction of the HIV quasispecies with the CTL response appears to be complex which makes the analysis of experimental data difficult. Mathematical and computational models have been helpful to investigate how different processes influence each other. For example, a previous model described how shifting immunodominance and antigenic oscillations can occur during HIV infection [27]. Now, with a model incorporating multiple CTL responses together with escape and compensatory mutations, we show that the dynamics of the CTL clones are sufficient to explain the sequential and late occurrence of escape variants. Upon loss of antigenic stimulation, CTL clones disappear slowly [41,42]. Concomitantly, other CTL clones can increase in size and induce more efficient killing that leads to further escape that is sequentially distributed over many years after infection. Interestingly, the outgrowth of escape variants does not occur earlier even though they are always present in the viral population. It has been suggested that a small effective population size of HIV-infected cells is responsible for the late production and selection of escape variants [43]. However, a recent study argues that stochastic effects play a minor role for the appearance of deleterious and beneficial mutations in HIV [23]. Also, our simulations show that viral evolution is fairly deterministic. The model allows us to simulate two identical 'patients' that are infected with the same virus and have the same CTL repertoire. In that case, the stochastic generation of the viral variants results in a slow increase in variation over years (results not shown). For example, the emergence of a certain escape variant is predictable during the first years after infection but becomes more variable later on. This is in line with a recent study where concordant evolution of HIV has been observed in mono-zygotic twins early during infection but more variation has been shown at later stages of the infection [21].
Another explanation for the late appearance of escape variants that has been put forward is the waiting time for compensatory mutations [24]. Our simulations show that single compensatory mutations are expected to be present in the viral quasispecies and therefore should not slow down the emergence of escape. However, if more compensatory mutations are needed to restore the fitness loss of an escape variant they can delay their occurrence (results not shown). Due to epistasis however, the fitness interactions of several mutations are highly complex and therefore simulation results depend mainly on a presumed fitness landscape. Therefore, we propose a way to test the two hypotheses for the late appearance of escape variants. First, if the delay was due to the waiting time for compensatory mutations, late escapes would be associated with more compensatory mutations than early escapes. If on the other hand, the dynamics of CTL clones mainly determines the late appearance, CTL clones where escape has been detected late should have increased in size relative to the other CTL clones in the period before. Both of these hypotheses can be tested by analyzing longitudinal data of HIV/SIV infections.
Our simulations proved to be useful to analyze the process of escape. We conclude that, although killing can be very efficient, escape rates are expected to be low. The association with a fitness cost and the way how CTL clones interact with infected cells critically influence the rates of escape. Hence, it appears to be important to study those interactions to derive realistic killing terms [39]. Furthermore, we have shown that estimating rates of escape is difficult from infrequent sequence measurements. We propose that sequence intervals should be shortened to follow the outgrowth of the variants. Additionally, there might be other reasons that slow down the selection of escape variants. For example, our model does not take into account multiple infected cells. As the fraction of multiple infected cells is large [44], escape variants are likely to appear in cells that are also infected with wildtype virus. As long as wild-type epitopes are presented on the cells surface, the escape variant does not gain any growth advantage compared to the wild-type. As a consequence, the selection of escape variants is expected to slow down.
Immunodominance affects the viral evolution within a host. A higher degree of immunodominance leads to more frequent escape with a reduced control of viral replication but a substantially impaired replicative capacity of the virus. This is interesting as vaccines generally aim to induce a broad CTL response where escape is unlikely to occur. However, a dramatic reduction of the replicative capacity of the virus due to escape could indeed slow down disease progression and/or reduce transmission [45]. Based on these results we identify two strategies for vaccine design: Vaccines inducing a broad CTL response should decrease the viral load whereas vaccines stimulating a narrow but dominant CTL response are likely to induce escape and consequently reduce the replicative capacity of the virus.
The balance between the CTL response and the viral population acquiring escape mutations appears to be a dynamical process over a long period of infection. We have shown that it is important to analyze the kinetics of this process and also the time scale (acute and chronic phase) at which it occurs. More longitudinal data of HIV infections will help to further investigate this process and future research is likely to go into this direction.

Stochastic Events
The set of ordinary differential equations (ODEs) is extended with stochastic events for viral mutation (similar as in [46]). Initially at day 0, infection occurs with the wild-type. Therefore the number of viral variants, m(0), is 1. In the following, the ODEs are integrated using the routine odeint from Numerical Recipes [47]. Every time step Dt = 1 day, we approximate the number of cells that have been infected with virus of type i during the last time step according to After infection, reverse transcription of the viral RNA occurs during which every position can mutate with the probability m. For the cells being newly infected with virus of type i, we calculate the integer number x ij of cells where the virus mutated into another type j. This is done by drawing from the binomial distribution x ij = B(DI i , m) for each of the two positions (escape and compensatory) over a total of n epitopes. Then, the infected cell populations are updated accordingly (i.e. I i = I i 2x ij and I j = I j +x ij ). Whenever a previously not existing viral variant j is generated, m(t) = m(t2Dt)+1, and the set of ODEs is expanded. We also take into account the actual population size of virus infected cells [23]. If the number of infected cells of a certain variant falls below 1, the variant is deleted. Note, that in order for the simulations to be computationally efficient, we only generate the viral variants stochastically but do not include possible fluctuations at small population sizes. However, this hybrid stochastic-deterministic approach is sufficient to keep the variants in a mutation-selection balance. Running the model using the Gillespie algorithm [48], we observed that whenever the different viral variants become selected, their growth approximates the deterministic description from the ODEs. A program of the model was written in C and simulations were run under Linux. The source code can be obtained freely on request from the authors.

Viral Fitness
Escape mutations in HIV and SIV are likely to confer a fitness cost in viral replication or infectivity [3][4][5][6][7][8][9]. A recent study by Parera et al. [49] showed that fitness effects caused by random single mutations are approximately uniformly distributed. Therefore, we draw the fitness of a virus containing a single mutation relative to the wild-type virus from a uniform distribution between 0.0 and 1.0. However, the combined effect of an escape mutation together with a compensatory mutation in a single epitope confers a higher fitness that is drawn uniformly between wild-type fitness (f wt = 1.0) and the higher fitness of the two single variants.

Estimating Rates of Escape
The outgrowth of an escape variant and the subsequent replacement of the wild-type can be considered as a competitive growth between two populations: where w is the wild-type and e is the escape variant. Cells infected with wild-type virus replicate at a net rate a and are killed by all CTL clones recognizing epitopes other than the escape epitope at rate b. The CTL clone recognizing the escape epitope kills wildtype cells at rate c. Cells infected with the escape variant replicate at a net rate a9 and are only killed by the CTL clones recognizing epitopes other than the escape epitope. For c.0, the escape variant gains a growth advantage as long as the fitness cost of the escape, d = a2a9, is not higher (i.e. d,c). As shown in Asquith et al. [12] this model can be fitted to longitudinal data of escape. This is done using the routine lmfit (http://sourceforge.net/projects/ lmfit) based on the Levenberg-Marquardt algorithm to solve nonlinear least-squares problems. However, the fitting is likely to underestimate the true rates of escape as it is illustrated in Fig. 7.

Killing Terms
We assume Michaelis-Menten kinetics for the killing of infected cells by CTL effector cells: kI i X n j~1 E j a ij h k z P m l~1 I l a lj z P n l~1 E l a il : ð8Þ This term takes into account the effector/target cell ratio of the cellular interaction: the total killing is proportional to infected cells at high effector cell densities and proportional to effector cells at high infected cell densities [28]. However, we also analyzed the outcome of other killing terms following different kinetics: Here, infected cells are killed by effector cells following simple mass-action kinetics. We mentioned that by increasing h k in Equation 8 we approach mass-action kinetics and the results are discussed.
Single saturation over CTL effector cells E is similar to the doublesaturating term used in Equation 8: once the set-point is reached, P n l~1 E l a il ww P m l~1 I l a lj , and Equation 8 approaches Equation 10.
Taking into account that CTL effector cells need some time to interact with infected cells before delivering their lethal hit, a saturation effect occurs as given in Equation 11 (see also reference [28]). We define k~0 :9 h k zIÃ ð Þ E Ã , to have infected cells being killed at a maximal rate of 0.9 d 21 during the chronic phase of infection. I* and E* are the numbers of infected cells and CTL effector cells, respectively, in the steady-state in absence of escape. During the acute phase, we observe three types of dynamics: 1) For h k ,5610 7 , the infection is always cleared since an infected cell can be killed at a rate of k E I per day that can be enormous. 2) For 5610 7 ,h k ,7610 7 , a very rapid escape during acute infection can prevent clearance of the infection. 3) For h k .7610 7 , we approach mass-action kinetics since h k ..I and the term becomes k h k IE.