Continuous antiretroviral therapy is currently the most effective way to treat HIV infection. Unstructured interruptions are quite common due to side effects and toxicity, among others, and cannot be prevented. Several attempts to structure these interruptions failed due to an increased morbidity compared to continuous treatment. The cause of this failure is poorly understood and often attributed to drug resistance. Here we show that structured treatment interruptions would fail regardless of the emergence of drug resistance. Our computational model of the HIV infection dynamics in lymphoid tissue inside lymph nodes, demonstrates that HIV reservoirs and evasion from immune surveillance themselves are sufficient to cause the failure of structured interruptions. We validate our model with data from a clinical trial and show that it is possible to optimize the schedule of interruptions to perform as well as the continuous treatment in the absence of drug resistance. Our methodology enables studying the problem of treatment optimization without having impact on human beings. We anticipate that it is feasible to steer new clinical trials using computational models.
Citation: Mancini E, Castiglione F, Bernaschi M, de Luca A, Sloot PMA (2012) HIV Reservoirs and Immune Surveillance Evasion Cause the Failure of Structured Treatment Interruptions: A Computational Study. PLoS ONE 7(4): e36108. https://doi.org/10.1371/journal.pone.0036108
Editor: Vladimir Brusic, Dana-Farber Cancer Institute, United States of America
Received: December 15, 2011; Accepted: March 30, 2012; Published: April 27, 2012
Copyright: © 2012 Mancini et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Partial support from the European Union under the EC contract FP6-2004-IST-4, No. 028069 (ImmunoGrid) http://igrid-ext.cryst.bbk.ac.uk/immunogrid/site/. No additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The increase in life expectancy of HIV positive individuals raised both costs and side effects of combination Anti-Retroviral Therapy (cART), stimulating research into Structured Treatment Interruptions (STI). Latest clinical trials on STI ,  indicated they are less effective than continuous treatment. We show that the STI tested so far in clinical trials would fail regardless of the emergence of drug resistance and that this failure is caused by HIV reservoirs and immune evasion.
For many HIV positive individuals treatment interruptions are unavoidable. Although clinical studies proved that increased risks are associated to cART interruptions, patient-initiated unstructured treatment interruptions are still quite common in the clinical practice . A recent systematic review  of cohort studies and clinical trials indicates a proportion of unstructured treatment interruptions ranging from 5.8% to 83.1% with a median of 23.1%. The mean duration of cART interruptions ranges from 11.5 days to 18 months with a median of 150 days. The main reported reasons for treatment interruptions are laboratory toxicity and clinical side effects. Less frequent arguments are patient compliance, treatment fatigue, intercurrent illness and other reasons.
The problem of unstructured interruptions is getting worse because of the increasing duration of treatment. Recent studies suggest an earlier use of cART as a way to fight effectively the HIV epidemics , . Current cART guidelines defer the treatment to the time when CD4+ counts drop below 350 (European guidelines - EACS) or 500 cells per microliter (US guidelines - DHHS and IAS-USA) whereas recent studies indicate that an early start of cART (CD4+ counts >500 cells per microliter) could significantly improve survival , –. Regardless of the success of anti-retroviral therapies, HIV's ability to mutate and evade both antiviral treatments and vaccines shifted the attention from curing affected individuals to fighting the epidemics. Some predictive models investigate the effects of different strategies on the HIV epidemics , . Recent clinical investigations  indicate that a more intensive and earlier use of cART is effective in reducing the spread of the virus. Indeed, another study highlighted the effectiveness of preventive use of cART in reducing the chance of being infected in case of sexual contact with a seropositive individual . Strategies aimed at reducing the spread of infection not only extend the duration of cART for an infected individual but also increase the number of individuals simultaneously under cART, raising the - already high - global cost of cART treatments.
STI aimed at discontinuing the therapy according to a schedule so as to minimize the side effects without losing substantial protection. In a large, randomized clinical trial ,  STI were associated with an increased risk of death and opportunistic diseases connected to treatment interruptions. Earlier studies ,  indicated positive results for STI, in some cases associated to supporting drugs . Subsequently, many clinical studies on different STI schedules resulted in generally neutral or negative outcomes –, although the reasons are still not fully understood. Results from the Staccato randomized trial indicated substantial drug savings and did not result in increased drug resistance in the STI arm, while treatment related adverse events were more frequent with continuous treatment .
Hereafter we resort to a computational model to gain a better understanding of the reasons of STI failure. A number of mathematical models describe the HIV infection dynamics and the related immune response. Some of them take into account the use of cART –. Most mathematical models use a continuous description of time, allowing the use of optimal control techniques to search for the best time to deliver the therapy –. Recently a mathematical model searched for the underlying reasons of STI failure . That model confirms that viral mutation and the emergence of drug resistant strains may be accelerated by STI, although only activated CD4+ cells and macrophages were considered as possible hosts for the virus. The study of HIV through cellular automata and agent-based models is also common – due to the discrete nature of the biological entities involved in the phenomena. In this work we use a well-established and validated agent-based model (ABM) of HIV infection –. See File S1 for a brief description of the model used in this work.
Our aim is to gain a better understanding of the reasons behind STI failure: are they inherent to the HIV dynamics or consequential of other mechanisms like the emergence of drug resistance? By excluding to model the resistance to the drugs we restrict the possible causes of STI failure to mechanisms such as virus reservoirs in macrophages and resting/memory CD4+ T lymphocytes. In addition, in this simplified formulation, we search for an optimal STI and compare it to the optimal one found in a previous work . A similar attempt to optimize clinical treatments using in silico modeling has been applied to cancer-preventing vaccinations . In that case the predictions of the computational model were validated through specific in vivo experiments on mice. That paper proved that an integrated in vivo-in silico approach is able to improve mathematical and biological models for cancer immunoprevention.
In the present work we investigate treatment interruptions with two in silico simulations: in the first simulation we test three STI used in clinical trials and compare their efficacy to that of the continuous (i.e., uninterrupted) treatment, while in the second simulation we search for an optimal STI scheduling and compare it with random treatment and continuous treatment. By ‘optimal schedule’ we mean the one that less impairs the ability to mount an immune response while keeping the amount of drugs used to a minimum. We use the Simulated Annealing (SA) technique to search for the optimal STI schedule. We simulate the disease progression for a group of 250 virtual (i.e., in silico) HIV positive patients. For each group of virtual patients we compare the effects of different treatment strategies on the HIV infection over a therapeutic period of 48 weeks, three years after seroconversion. We finally evaluate the efficacy of each STI schedule by challenging the immune system of the virtual HIV patients with a simulated opportunistic bacterial infection at the end of the treatment.
To quantitatively evaluate the effect of a STI schedule of cART we define a fitness function that measures the health of the virtual patient's immune system and the amount of drugs received over the 48 weeks period. The fitness function is described in the Materials and Methods section. We performed 50 SA optimizations by using different initial conditions. These optimizations resulted in 50 optimized treatment schedules with an average fitness score of 2.325 and a standard deviation of 0.012. The optimal treatment schedules have 29 weeks of therapy on average and a standard deviation of 3 weeks. The optimized schedules are shown in a histogram (Figure 1) in which each bin represents the frequency of a given week of therapy in the optimized schedules. The histogram shows a peak every 3 weeks spaced out by 2 weeks in which the drug administration has a lower but still significant frequency. Due to the requirement of providing a schedule as regular as possible, we define as “optimal" a schedule having 2 weeks of therapy followed by an interruption of 1 week for 32 weeks after the initial two months. The optimization shows a different pattern for the initial two months of treatment (Figure 2). The optimal schedule has 29 weeks of therapy and is shown in Figure 2. According to our simulations the last 4 weeks of the therapeutic period have limited importance. In other words, the effect of cART at later times is somehow less important, supporting the current opinion that it is more rewarding to support the immune system with anti-retroviral treatment at earlier times.
Each bin represents the frequency of a given week of therapy in the optimized schedules.
The therapeutic period of 48 weeks starts after 3 years of untreated infection (day 1096). The optimal therapy is shown in the figure. At the end of the 48 weeks of therapy we inject a fixed amount of bacteria in the virtual patient, challenging his immune system. The patient dies if the antigens exceed a threshold of 4×106 bacteria/ml. An efficient therapy is capable of restoring the patient's immune system enough to contain the bacterial infection.
The aim of the treatment was to suppress the HIV viral load and induce immune restoration, resulting in an immune system capable of dealing with opportunistic infections. To assess the efficacy of the optimal therapy indicated by the SA algorithm, we ran simulations of an opportunistic infection starting immediately after the therapeutic period. Bacteria were injected in the virtual patients as soon as the 48 weeks of therapy were over. The immune system reaction against those bacteria depended on its efficiency at injection time. We compared the result of the schedule optimized by the SA algorithm with three different simulated control groups (Figure 3). In the first group (“Continuous Therapy") the drugs were administered every day for a therapeutic period of 48 weeks as prescribed by the cART guidelines; in the second group (“Random Therapy") the total amount of drug was equal to that of the optimal therapy but administration within the therapeutic period was performed randomly; finally, in the last group, there was no therapy (“Void Therapy"). To measure the effectiveness of the optimized STI we monitored the survival curve in a population of 250 virtual HIV+ patients infected by a bacterium 4 years after their initial HIV infection. For the same 250 patients we compared the effects of continuous therapy, void therapy, optimal therapy and random therapy.
30 days after bacterial challenge survival rates are the following: continuous therapy 91.53%, optimal therapy 90%, random therapy 81.85% and void therapy 45.38%. A reduction of 40% of the drug intake corresponds to less than 5% reduction in the survival rate.
Our results show that the optimal therapy provided a survival rate comparable to the continuous therapy using 40% less drug. Survival rate for untreated patients was 45.4%. The optimal therapy performed significantly better than the void one (survival rate 90% versus 45.4%). The optimal therapy also performed better than a random therapy with the same amount of drugs (a survival rate of 90% vs. 81.9%). The latter result indicates that the schedule identified by the SA algorithm is optimized for both the amount of medicine and the administration schedule, supporting the importance of finding a specific scheduling to reduce therapeutic failures.
Comparison with clinical trials
Even though different structured treatment schedules have been tested in clinical trials, a comparison with multiple trials is often difficult due to the heterogeneity of their experimental conditions. The clinical trials that tested the STI simulated in our work have different durations and different criteria for the enrollment of patients. Such diversity makes it difficult to compare their results with a single optimized schedule. In terms of initial conditions and treatment duration our simulations can be related to the clinical trial by Dybul et al. . As in Dybul's clinical trial, our simulations have a treatment period of 48 weeks, enroll patients with similar CD4+ cells counts (CD4+ cells >300 cells/mm3) and test the STI schedule with 8 weeks of treatment and 4 weeks of interruption. The main difference is that in our simulations we use patients that have been never exposed to HAART and thus with a higher viral load whereas the clinical trial enrolled patients receiving a 3-drug HAART regimen with HIV RNA levels <500 copies/mL plasma for >6 months and <50 copies/mL at screening.
The parameters that control the pharmacodynamics of HIV in our model have been tuned by using clinical information from about twenty-two patients selected at the Clinical Department of the National Institute of Infectious Diseases “L. Spallanzani" in Rome . No special tuning has been performed for the model parameters of the three STI schedules tested in the present work. For this reason we can validate our model results by comparing the simulations for both continuous treatment and the structured treatment interruption with data from Dybul's clinical trial. Those data consists of the median and range of CD4+ cell counts at baseline, at week 40 and at week 48 of the treatment period observed in 52 patients randomized in the continuous and STI arm of the clinical trial. In (Figure 4) we compare the simulations results with the values observed in the clinical trial. Simulated CD4+ cell counts are within the ranges observed in the clinical trial. In Table S1 we show the variation of immunological parameters for all the schedules tested in our virtual cohort. In the table we report the median value and the range observed in the virtual cohort at week 40 and week 48 of the treatment period.
In the upper panel we compare the median CD4+ cell counts of 250 virtual patients with the data of real patients from the continuous treatment arm of Dybul's clinical trial. In the lower panel we compare the median CD4+ cell counts of 250 virtual patients with the data of real patients from the STI arm of Dybul's clinical trial.
Results of the STI strategies tested in clinical trials are shown in Figure 5. As observed in the clinical trials, all the STI strategies are associated with an increased number of deaths compared to the continuous treatment. The ratio of deaths associated to the “Week On/Week Off" strategy over that of continuous therapy is 1.95 at the end of the 30 days after opportunistic bacterial infection. The same ratio is 3.52 for the “4 Weeks On/4 Weeks Off" strategy and 2.05 for the “8 Weeks On/4 Weeks Off". We observe a ratio of deaths of 1.19 for the Optimal therapy and 2.14 for the Random therapy.
30 days after bacterial challenge, survival rates are the following: Week On/Week Off STI 83.60%, 8 Weeks On/4 Weeks Off 82.66% and 4 Weeks On/4 Weeks Off 70.28%. Survival rates for continuous therapy and void therapy are 91.53% and 45.38% respectively.
The effects of the different STI on virological and immunological parameters are shown in (Figure 6). The parameters that show the strongest correlation with the survival of the virtual patients in the long term are CD4+ cell count and provirus levels. The levels of provirus seem to be responsible for the failure of STI treatment as much as the CD4+ cells count. The “4 Weeks On/4 Weeks Off " schedule has the worst survival because of the high level of provirus even though it shows a CD4+ cells count comparable to that of the “Week On/Week Off " STI. The difference in the ratio of deaths associated to “ 8 Weeks On/4 Weeks Off " and the optimal STI (2.05 versus 1.19) should be due to their difference in the level of provirus, since both schedules show comparable CD4+ cell counts. These results lead us to point out the importance of HIV reservoirs as one of the main causes of STI failure. As expected, a treatment interruption longer than 10 days leads to a viral rebound and a decrease in CD4+ cell count. We observe that the viral rebound is proportional to the duration of the interruption. Treatment interruptions of one or two weeks have a smaller viral rebound whereas for longer interruptions the viral load reaches levels comparable to the ones before treatment (data not shown).
Median CD4+ cells count (upper panel) and provirus levels (lower panel) of the 250 virtual patients during the 48 weeks treatment period. In each panel we show the different STI tested in this study, the optimal therapy and the continuous therapy. The levels of provirus seem to be responsible for the failure of STI treatment as much as the CD4+ cells count. The “4 Weeks On/4 Weeks Off" schedule has the worst survival curve because of the high level of provirus even though it has a CD4+ cells count comparable to that of the “Week On/Week Off" STI.
For both optimal and continuous treatment the outcome of the opportunistic infection is correlated more to the level of provirus than to the level of CD4+ lymphocytes. For all other schedules it seems that both CD4+ and provirus are relevant in determining fatal outcomes of the bacterial infection. In Figure 7 we show the CD4+ cell counts versus provirus level of the 250 virtual patients for each of the treatment schedules tested in the present work. For each STI schedule we separate the 250 virtual patients in two groups (survivors and casualties) depending on the outcome of the opportunistic infection. We perform a Kolmogorov-Smirnov two-sample test to compare the distributions of the CD4+ cell counts at the end of the treatment period in both groups. The null hypothesis is that the CD4+ cell counts in the group of survivors and the one in the group of casualties are from the same distribution. For all the treatment schedules tested, except the optimal and continuous treatment, the null hypothesis is rejected with p-values between 1.1*10−5 and 1.6*10−2. Surprisingly for the optimal and continuous treatment the CD4+ cell count are not related to the outcome of the opportunistic infection (p-value 0.29 and 5.1*10−2). We perform the same test to compare the distributions of provirus in survivors and casualties. In this case the null hypothesis is rejected for all the schedules with p-values between 9.1*10−12 and 1.5*10−2.
In each panel we show the CD4+ cell counts versus provirus of the 250 virtual patients at the end of the treatment period. The black markers indicate the patients that survived the opportunistic bacterial infection, whereas the red markers indicate the patients that died within 30 days of the bacterial infection.
The three STI strategies (“Week On/Week Off", “4 Weeks On/4Weeks Off" and “8 Weeks On/4Weeks Off") have lower performances compared to continuous treatment regardless of the emergence of drug resistance since the ratio of death associated to a clinically tested STI strategy over that of continuous treatment is between 1.95 and 3.52. Since in the model we don't allow HIV to develop resistance to cART drugs, the failure of STI seems to be inherent to the HIV infection dynamics, rather than caused by the emergence of drug resistant strains. We conclude that the amount of HIV reservoirs both in macrophages and latently infected resting/memory CD4+ T lymphocytes are sufficient to cause a failure of the STI schedules tested so far in clinical trials regardless of drug resistance. This observation may have relevant implications for the design of future treatment strategies. Indeed, efforts directed towards the reduction of the size of HIV reservoirs in humans, might facilitate the adoption of subsequent strategies aimed at reducing the antiretroviral treatment exposure, such as STI or treatments with less drugs .
We also show that, without the emergence of drug resistance, it is possible to find an optimized STI whose efficacy is close to that of a continuous treatment with a reduction of 40% in drug administration. In addition by showing the failure of the “8 Weeks On/4 Weeks Off" STI we show that adherence to the optimal treatment schedule is more important than the amount of drug taken over the treatment period. The efficacy of random therapies with the same amount of drug of the optimal treatment is sensibly lower than that of optimal treatment. Results indicate a ratio of death of 2.14 for the random treatment instead of 1.19 observed for the optimal. We conclude that the SA optimization was successful in identifying an optimal schedule of drug administration for the set of conditions studied in this work.
The result obtained in a previous work  on a shorter therapeutic period (6 months) and optimized by using a genetic algorithm indicated an optimized therapy with a week-on week-off pattern after a period of about 2 months of continuous therapy. The initial period of continuous therapy is not necessary according to our study, possibly because the immune system is not compromised enough to require an initial strong recovery phase. In the previous study the percentage of survivors for full and optimal therapy was comparable (34.1% and 30.5% respectively) whereas random and void therapy had lower performances (21.9% and 20.2%). The differences in the optimized treatments indicate that the best therapeutic strategy changes with the progressive damage that HIV infection inflicts on the immune system. Since at 3 years post infection the immune system is usually not severely compromised, in the present study the percentage of survivors is higher than the one observed in the previous study in which the therapy started almost 8 years after HIV infection. An interesting difference is the effect of random therapy. In our study we notice that a random therapy has less negative effects if the patient receives early (CD4+ cell counts >500 cell/mm3) cART. If treatment starts later random therapy has virtually no effect, as shown in . This result indicates that unstructured interruptions have a less negative effect on the short term when the immune system is still healthy but further investigation is required to evaluate their effects in the long term.
The increased risk of opportunistic infections associated to provider-directed structured therapeutic interruptions led to abandoning this strategy in favor of the safety of a continuous treatment. Yet continuous treatment is not always possible as suggested by the proportion of patient-initiated unstructured interruptions. Since new studies suggesting the beneficial effects of early treatment may increase the duration of cART in a patient's life, a further growth of the already common unstructured interruptions can be foreseen. Moreover, so far, no clinical study has compared the risk of opportunistic infections associated with unstructured interruptions to the risk associated with STI. Finally, the sustainability of life-long antiretroviral therapy at global level, both from an economic and toxicity points of view, still represents a major challenge. For these reasons, exploring the mechanisms behind the failure of structured therapeutic interruptions and the possible optimal STI is still an open topic.
The ethical problems associated to further studies of STI in light of the previous failures make the use of modeling techniques appealing. The possibility of simulating STI in silico to predict the success or failure of a given STI strategy is a powerful tool that can support the design of clinical studies without having impact on human beings. The optimization of treatment interruptions is another clear example of the usefulness of modeling approaches to foster the understanding of complex problems.
For the HIV infection the lack of fully predictive animal models  makes it difficult to accurately validate the prediction of computational models. It is very difficult to address the ethical implications of testing computational predictions on humans, given the potential loss of human lives that could be caused by a wrong prediction. Yet, clinical trials are still needed and any tool that could be used to assist those trials should be considered.
Regardless of the difficulties in validating the predictions of our model, our results can be used to orientate qualitatively the design of clinical trials. In silico simulations could be used to predict if a STI schedule might fail because of the size of the HIV reservoir regardless of the emergence of drug resistance. In this way clinical trials could be directed toward schedules with a higher chance of success.
Materials and Methods
In silico approach
In this paper we adopt a two phases approach similar to the one used in Castiglione et al. 2007  to study the existence of an optimal treatment that minimizes the drug administration without compromising the immune system response to HIV. Firstly we search for the optimal STI schedule (“optimal therapy") by using the simulated annealing optimization algorithm. In this phase, looking at the conditions of the immune system at the end of the therapeutic period, we evaluate the potentially administered therapeutic schedule by means of computer simulations. The SA algorithm is applied to search for the optimal schedule of cART interruptions that maximizes both the immune system recovery and viral control and, at the same time, minimizes the amount of drugs used in a therapeutic period. Secondly, once we have found the optimal schedule, we test it by challenging the immune system with an opportunistic infection simulated by introducing a bacterium in a group of 250 virtual patients. Therefore, we compare the survival curves of the different control groups with the survival curve of the optimal therapy. In the first control group patients have a continuous course of therapy (i.e., uninterrupted) for the whole period (“continuous therapy"). Patients of the second control group receive the same amount of medicine as in the optimal therapy, but the drugs are randomly administered in time (“random therapy"). The last control group consists of untreated patients (“void therapy"), that is, patients that do not receive therapy at all. Performing this test in silico allows us to use the same virtual patients for each group, whereas in clinical trials the control groups consist of different individuals. Such choice enables us to directly relate the survival of rates of each group to the effect of the different therapies.
In addition to the search for an optimal STI we also study the effects of STI strategies tested in clinical trials by comparing their effect on the HIV dynamics of the virtual patients over a period of 48 weeks of treatment. As control groups for this experiment we use groups of 250 virtual patients under continuous treatment and void treatment. Since the model does not include drug resistance, the outcomes of the STI strategies can be traced back to the dynamics of the immune response to the HIV virus and in particular to the action of antibody producing B lymphocytes and to the CD4+ and CD8+ T cells.
Quantitative evaluation of STI
The simulation starts at time zero with an injection of HIV viral particles and the HIV infection progresses untreated for three years. At that point the disease is already in its chronic phase and we start a therapeutic period of 48 weeks. During the therapeutic period we administer the cART according to a given schedule represented by TherStr, a 48-bit-long string, where a 0 stands for a week without therapy (i.e., an interruption) and a 1 stands for a full week of therapy (Figure 8). We assume the efficacy of the therapy to drop to zero during an interruption so that the HIV life cycle may progress unhindered through all its stages during the interruptions. Most cART regimens administer a daily cocktail of at least three drugs, commonly two RTIs and one PI. Although we could optimize the therapy on a daily basis, we decided to constrain the minimum administration/interruption period to one week. Note that this is not a fundamental limitation in our approach, but rather a practical one since no patient would be able to follow a one-year-long therapeutic schedule changing on a daily basis. In addition this choice allows a direct comparison with a previous study based on a genetic algorithm. Another constraint to the simulated cART dictated by medical practice is that RTI and PI drugs are always administered at the same time. At the end of the 48 weeks we measure the efficacy of the therapy by assigning a score given by our fitness function. At the beginning of the therapeutic period, the average CD4+ count measured in the virtual patients is about 500 cells per microliter.
In the string, the jth bit set to 1 means that during the jth week the HAART is administered to the patient.
To quantitatively evaluate the effect of a STI schedule of cART we define a fitness function Fi for the ith virtual patient. The fitness function measures the health of the virtual patient's immune system as the outcome of a simulation and the amount of drugs received over the 48 weeks period. Since the whole immune/HIV dynamics depends on the therapy administered, the fitness score is a function of TherStr. Fi is the sum of three terms Hi, Di and Zi.
- Hi takes into account the fitness of HIV which is computed as the average of the sum of both virus contained in infected cells and free viral particles divided by the viral set point here defined as the sum of free virus and virus in infected cells at time the treatment starts ts;
- Di is a measure of the fitness of the immune system and is given by the average of the ratio between the CD4+ count of the healthy individual (i.e., CD4+ count at time = 0, just before the infection) and the CD4+ count during the treatment period;
- Zi measures the amount of drugs used as the ratio between the number of weeks of active therapy and the 48 weeks of treatment.
In order to partially account for inter-patient variability, the optimization algorithm tries to minimize F, the average fitness score of eight virtual patients.(1)where(2)(3)(4)In the formulas above ts indicates the time the therapy starts (three years post infection); te the time the therapy ends (i.e., 48 weeks after ts); Hi(t,TherStr) is the sum of free virions and proviral HIV in infected cells; Di(t, TherStr) is the T helper cell count in the simulated space and zi(t, TherStr) is a function such that zi(t) = 1 if the therapy is active at time t, zero otherwise. Minimizing the fitness function implies decreasing the viral load and the amount of drugs used, whereas increasing the CD4+ cell count over a period of 48 weeks. C-ImmSim uses a time-step corresponding to 8 hours of real life.
Simulated annealing optimization
To find the optimal schedule of the cART during the 48 weeks therapeutic period we use a Simulated Annealing algorithm . To optimize the STI we perform in silico experiments on a test group of 8 virtual patients. The size of the test group is purposefully chosen equal to the number of cores available in each node of the computer cluster used to perform the optimization. This choice allows us to simulate in parallel the HIV infection on 8 virtual patients, by performing the computation on a single node.
We adapted the classic simulated annealing algorithm to take as input a therapeutic schedule in the form of a 48 bits string where each bit represents a week of therapy. We define an algorithm to update the therapeutic schedule that modifies the previous schedule by adding, removing or shifting 1 week of therapy (1 bit). By using this algorithm we assume that two consecutive configurations in the Markov chain have comparable fitness scores (i.e., the energy of two consecutive configurations is comparable), which is a requirement for a correct annealing optimization.
The simulated annealing is characterized by a set of parameters described in Table S2. For the annealing we resort to a geometric cooling schedule . We tested also logarithmic and adaptive cooling schedules. The logarithmic cooling was soon discarded due to the long computational times required and the adaptive did not result in a better convergence.
A similar optimization problem has been studied in the past by following an approach based on a genetic algorithm (GA) . The GA study looked for the optimal therapy over a period of six months starting after 7.5 years post infection. One of the main differences between the present work and the previous one  is that we reduce the time between seroconversion and treatment from 7.5 to 3 years. This reduction allows considering the positive effect of early cART, but increases the difficulty of the optimization because the effect of cART on CD4 recovery is dampened by the healthier immune system at the time of treatment initiation. Another important difference is that we attempt to optimize the schedule over a longer period (close to 1 year) instead of just 24 weeks. Extending the therapeutic period to one year increases the complexity of the optimization problem. In the previous study the state space had 224 possible configurations. In the present work we face a state space of 248 configurations. Doubling the therapeutic period allows to test if the optimal STI can yield results similar to those observed in the shorter STI previously studied . Clinical studies on STI indicated a reduction of the benefits over longer time periods , , . By using an extended period of STI we have the chance to verify if we observe a similar behavior in the model.
In each iteration of the optimization algorithm we evaluated the fitness function as the average among N = 8 virtual patients. The evaluation of the fitness of a given STI schedule required the simulation of 4 years of HIV infection dynamics in each individual patient. Most of the computational time is due to the C-ImmSim simulations, whereas the annealing algorithm has a relatively small impact on the computation time. The longest annealing optimizations use MAXITER equal to 27000 and take about 15 days to complete (Figure 9) using a L5520 eight-core processor having a clock rate equal to 2.26 GHz.
Here we show 8 different annealing instances. The one corresponding to the lowest fitness is highlighted in red.
We explore stochastically the configuration space of 248 different therapeutic schedules. We stop the optimization when the standard deviation of the fitness score for the last homogeneous Markov chains (i.e., parameter MARKOVCHAIN in the Table S2 in Supporting Information) is lower than an arbitrary threshold set equal to 10−4. Each instance of the optimization algorithm results in an optimized schedule and a corresponding fitness score. We find several local optimal therapies with comparable fitness scores. As usual with most stochastic algorithms we do not have any warranty that a better solution could not be found by continuing the search.
Description of the computational model used to simulate HIV infection in virtual patients.
Variations of immunological and virological parameters: In the upper part of the table we report the data from Dybul's clinical trial . In the lower part of the table we show the results of our simulations. For each treatment schedule tested in our simulations we report the median value for the most important immunological and virological parameters. In parentheses we report the minimum and maximum values observed in the population of 250 virtual patients. For the viral load measurement ND indicates level of infectious virions below the detection level (<50 virions/ml).
Conceived and designed the experiments: EM FC. Performed the experiments: EM. Analyzed the data: EM. Contributed reagents/materials/analysis tools: EM FC MB AdL PMAS. Wrote the paper: EM. Revised the manuscript critically for important intellectual content: FC MB AdL PMAS.
- 1. The Strategies for Management of Antiretroviral Therapy (SMART) Study Group (2006) CD4+ Count–Guided Interruption of Antiretroviral Treatment. N Engl J Med 355: 2283–2296.
- 2. The Strategies for Management of Antiretroviral Therapy (SMART) Study Group (2008) Inferior Clinical Outcome of the CD4+ Cell Count–Guided Antiretroviral Treatment Interruption Strategy in the SMART Study: Role of CD4+ Cell Counts and HIV RNA Levels during Follow-up. J Infect Dis 197: 1145–1155.
- 3. Kranzer K, Lewis JJ, Ford N, Zeinecker J, Orrell C, et al. (2010) Treatment interruption in a primary care antiretroviral therapy program in South Africa: cohort analysis of trends and risk factors. J Acquir Immune Defic Syndr 55(3): e17–23.
- 4. Kranzer K, Ford N (2011) Unstructured treatment interruption of antiretroviral therapy in clinical practice: a systematic review. Trop Med Int Health 16(1): 134–39.
- 5. Kitahata MM, Gange SJ, Abraham AG, Merriman B, Saag MS, et al. (2009) Effect of Early versus Deferred Antiretroviral Therapy for HIV on Survival. N Engl J Med 360: 1815–1826.
- 6. Granich RM, Gilks CF, Dye C, De Cock KM, Williams BG (2009) Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model. Lancet 373: 48–57.
- 7. Pensieroso S, Cagigi A, Palma P, Nilsson A, Capponi C, et al. (2009) Timing of HAART defines the integrity of memory B cells and the longevity of humoral responses in HIV-1 vertically-infected children. Proceedings of the National Academy of Sciences 106: 7939–7944.
- 8. When To Start Consortium (2009) Timing of initiation of antiretroviral therapy in AIDS-free HIV-1-infected patients: a collaborative analysis of 18 HIV cohort studies. The Lancet 373: 1352–1363.
- 9. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, et al. (2011) HPTN 052 Study Team. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med 365(6): 493–505.
- 10. Mei S, Quax R, van de Vijver D, Zhu Y, Sloot PMA (2011) Increasing risk behaviour can outweigh the benefits of antiretroviral drug treatment on the HIV incidence among men-having-sex-with-men in Amsterdam. BMC Infectious Diseases 11: 118.
- 11. University of Washington (2011) PARTNERS PrEP STUDY, Pre exposure prophylaxis.
- 12. Dybul M, Chun T, Yoder C, Hidalgo B, Belson M, et al. (2001) Short-cycle structured intermittent treatment of chronic HIV infection with highly active antiretroviral therapy: Effects on virologic, immunologic, and toxicity parameters. Proceedings of the National Academy of Sciences 98: 15161–15166.
- 13. García F, Plana M, Ortiz GM, Bonhoeffer S, Soriano A, et al. (2001) The virological and immunological consequences of structured treatment interruptions in chronic HIV-1 infection. AIDS 15: F29–F40.
- 14. García F, Plana M, Arnedo M, Ortiz GM, Miró JM, et al. (2003) A cytostatic drug improves control of HIV-1 replication during structured treatment interruptions: a randomized study. AIDS 17: 43–51.
- 15. Ananworanich J, Nuesch R, Le Braz M, Chetchotisakd P, Vibhagool A, et al. (2003) Failures of 1 week on, 1 week off antiretroviral therapies in a randomized trial. AIDS 17: F33–F37.
- 16. Dybul M, Nies-Kraske E, Daucher M, Hallahan C, Csako G, et al. (2003) A randomized, controlled trial of long cycle structured intermittent versus continuous ARV therapy for chronic HIV infection. Abstracts from the 10th Conf Retrovir Oppor Infect.
- 17. Papasavvas E, Ortiz GM, Gross R, Sun J, Moore EC, et al. (2000) Enhancement of Human Immunodeficiency Virus Type 1—Specific CD4 and CD8 T Cell Responses in Chronically Infected Persons after Temporary Treatment Interruption. J Infect Dis 182: 766–775.
- 18. Ananworanich J, Gayet-Ageron A, Le Braz M, Prasithsirikul W, Chetchotisakd P, et al. (2006) CD4-guided scheduled treatment interruptions compared with continuous therapy for patients infected with HIV-1: results of the Staccato randomised trial. The Lancet 368: 459–465.
- 19. Perelson AS, Nelson PW (1999) Mathematical Analysis of HIV-I: Dynamics in Vivo. SIAM Rev 41: 3–44.
- 20. Nowak M, May RM (2000) Virus Dynamics: Mathematical Principles of Immunology and Virology. Oxford: Oxford University Press.
- 21. Simon V, Ho DD (2003) HIV-1 dynamics in vivo: implications for therapy. Nat Rev Micro 1: 181–190.
- 22. Hee-Dae K (2007) Optimal treatment strategies derived from a HIV model with drug-resistant mutants. Applied Mathematics and Computation 188: 1193–1204.
- 23. Wu Y, Zhang M, Wu J, Zhao X, Xia L (2011) Evolutionary game theoretic strategy for optimal drug delivery to influence selection pressure in treatment of HIV-1. J Math Biol 64(3): 495–512.
- 24. Aviran S, Shah PS, Schaffer DV, Arkin AP (2010) Computational Models of HIV-1 Resistance to Gene Therapy Elucidate Therapy Design Principles. PLoS Comput Biol 6(8): e1000883.
- 25. Ferreira J, Hernandez-Vargas E, Middleton RH (2011) Computer simulation of structured treatment interruption for HIV infection. Comput. Methods Programs Biomed 104: 50–61.
- 26. Pandey RB, Stauffer D (1990) Metastability with probabilistic cellular automata in an HIV infection. J Stat Phys 61: 235–240.
- 27. Zorzenon dos Santos RM, Coutinho S (2001) Dynamics of HIV Infection: A Cellular Automata Approach. Phys Rev Lett 87(16): 168102.
- 28. Ruskin HJ, Pandey RB, Liu Y (2002) Viral load and stochastic mutation in a Monte Carlo simulation of HIV. Physica A: Statistical Mechanics and its Applications 311: 213–220.
- 29. Shi V, Tridane A, Kuang Y (2008) A viral load-based cellular automata approach to modeling HIV dynamics and drug treatment. J Theor Biol 253: 24–35.
- 30. Burkead EG, Hawkins JM, Molinek DK (2009) A dynamical study of a cellular automata model of the spread of HIV in a lymph node. Bull Math Biol 71(1): 25–74.
- 31. Sloot PMA, Chen F, Boucher CAB (2002) Cellular Automata Model of Drug Therapy for HIV Infection, in S. Bandini; B. Chopard and M. Tomassini, editors, 5th International Conference on Cellular Automata for Research and Industry, ACRI 2002, Geneva, Switzerland, October 9–11, 2002. Proceedings, in series Lecture Notes in Computer Science 2493: 282–293.
- 32. Castiglione F, Pappalardo F, Bernaschi M, Motta S (2007) Optimization of HAART with genetic algorithms and agent-based models of HIV infection. Bioinformatics 23: 3350–3355.
- 33. Castiglione F, Bernaschi M (2006) Hiv-1 strategies of immune evasion. Int J Mod Phys C 16(12): 1869–1879.
- 34. Paci P, Martino F, Bernaschi M, D'Offizi G, Castiglione F (2011) Earlier is better: a timely HAART initiation may pave the way for best controllers. BMC Infectious Diseases 11: 56.
- 35. Castiglione F, Poccia F, D'Offizi G, Bernaschi M (2004) Mutation, Fitness, Viral Diversity, and Predictive Markers of Disease Progression in a Computational Model of HIV Type 1 Infection. AIDS Res Hum Retroviruses 20(12): 1314–1323.
- 36. Bernaschi M, Castiglione F (2001) Design and implementation of an immune system simulator. Comput Biol Med 31: 303–331.
- 37. Palladini A, Nicoletti G, Pappalardo F, Murgo A, Grosso V, et al. (2010) In silico modeling and in vivo efficacy of cancer-preventive vaccinations. Cancer Res 70(20): 7755–63.
- 38. Paci P, Carello R, Bernaschi M, D'Offizi G, Castiglione F (2009) Immune control of HIV-1 infection after therapy interruption: immediate versus deferred antiretroviral therapy. BMC Infectious Diseases 9: 172.
- 39. Lambert-Niclot S, Flandre P, Valantin MA, Peytavin G, Duvivier C, et al. (2011) Factors associated with virological failure in HIV-1-infected patients receiving darunavir/ritonavir monotherapy. J Infect Dis 204(8): 1211–6.
- 40. Girard MP, Plotkin SA (2012) HIV vaccine development at the turn of the 21st century. Current Opinion in HIV & AIDS 7(1): 4–9.
- 41. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by Simulated Annealing, Science. New Series 220(4598): 671–680.
- 42. van Laarhoven PJM, Aarts EHL (1987) Simulated Annealing: Theory and applications. Kluwer Academic Publishers.
- 43. Lafeuillade A, Halfon P, Chadapaud S, Hittinger G, Kiri H, et al. (2001) Potential adverse effects of structured therapeutic interruptions on the pool of HIV-infected cells. JAIDS 28(2): 197–199.