Simulation models

We simulate the tumor growth of each patient by incorporating different therapeutic strategies into the two-population Lotka-Volterra model [26], which is commonly used in adaptive therapy of tumors. The tumor cells are classified into drug-sensitive () and drug-resistant () populations. The growth dynamics of these two cell types, under treatment, can be described as follows:

(1)

Here, drug-sensitive and drug-resistant cells proliferate at rates and , respectively, and undergo natural death at rates and . The two subpopulations share a finite environmental carrying capacity , giving rise to resource competition. The parameters and represent competition coefficients quantifying competitive effects between the two cell types.

Since the primary aim of this study is to evaluate the impact of adaptive therapy strategies on prostate cancer treatment outcomes, we adopt model (1), a simplified competition-based framework, to describe tumor cell growth and drug effects. Biologically, androgen deprivation therapy (ADT) for prostate cancer acts primarily by depriving androgen-sensitive cells () of the hormonal signals required for cell-cycle progression, thereby inducing cell-cycle arrest (predominantly in the G0/G1 phase) rather than direct cytotoxicity [32–34]. Consequently, we model the therapeutic effect as a reduction in the intrinsic proliferation rate, represented by , where represents the cytostatic efficacy of ADT and denotes the treatment intensity. This formulation aligns with established mathematical models of prostate cancer dynamics in which therapy modulates growth potential rather than increasing cell death [11,35]. Within this framework, the resistant population is assumed to be completely insensitive to ADT, and treatment-driven phenotypic conversion from sensitive to resistant cells is not explicitly included. Similar assumptions have also been used in previous studies of tumor containment, where tumors are represented by pre-existing sensitive and fully resistant subpopulations and mutations after treatment initiation are neglected [27,28]. In particular, previous work has shown that ongoing mutations after treatment initiation do not substantially affect the main containment results [36].

In all adaptive-therapy protocols considered here, a common feature is that treatment is administered intermittently in an “on/off” manner. Accordingly, we restrict to being a binary indicator function taking values in {0,1}. This choice corresponds to the clinically standard on/off regimen: during treatment-on periods, the drug is administered continuously at the standard dose, whereas during treatment holidays no drug is given . As a result, within our simulation framework, specifying a therapeutic strategy is equivalent to specifying the function . The strategies considered in this study, together with their corresponding indicator functions, are summarized in Table 1. In the coming section, we define the different treatment strategies in detail.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Treatment strategies and their treatment functions.

https://doi.org/10.1371/journal.pcbi.1014457.t001

Therapeutic strategies

Continuous therapy (CT). Treatment is administered continuously [37–39], thus for all .

Intermittent therapy (IT). There are two predefined periods: treatment window () and treatment holiday (), and the treatment is alternately triggered on/off between the two periods [40–42]. Therefore, the control (treatment) time intervals can be defined as.

Two thresholds-guided adaptive therapy (AT50). Denote as the total number of tumor cells at time , and as the initial number of tumor cells. The standard AT50 treatment protocol relies on two thresholds [11,43]: treatment is triggered (or restarted) when the tumor size exceeds , and stopped when the tumor size falls below . Consequently, treatment remains suspended until the tumor size returns to . In addition to the standard AT50, we introduce an AT50 variant for comparative analysis. In this variant, the treatment initiation threshold is defined as a variable parameter (instead of fixed ), while the cessation threshold remains fixed at ; this variant is denoted as AT50.

Single threshold-guided adaptive therapy. In this strategy, there is a surveillance period T of the tumor size [28,31]. At the initial time of each surveillance cycle, the decision on whether the treatment should be triggered depends on the tumor size. That is, if the tumor size exceeds a threshold , the drug is used for the patients during the surveillance cycle. In addition, in line with the general framework in [31], the treatment window () can cover the full surveillance cycle () or just cover a partial surveillance cycle (). Correspondingly, there are two sub-strategies as defined below:

• Adaptive therapy with full surveillance cycle treatment (AT-FSC)
• Adaptive therapy with partial surveillance cycle treatment (AT-PSC)

Correspondingly, the treatment time intervals for AT-PSC and AT-FSC are

and

respectively. Note that the special adaptive therapy of AT-FSC corresponds to the strategy studied in [28], which is therefore taken as an independent strategy for comparison.

Reinforcement learning based adaptive therapy (AT-RL). We develop a deep reinforcement learning (DRL) framework for adaptive tumor therapy using Proximal Policy Optimization (PPO) [44] shown in Fig 1. The environment is based on the dynamic system described in model (1). The policy/value network takes the current normalized tumor burden as input and consists of a fully connected input layer of size 32, followed by a single-layer Long Short-term Memory recurrent neural network (LSTM) with hidden size 64 and a shared multilayer perceptron with hidden dimensions 128, 64, 32, 16, and 10, before branching into separate actor and critic heads. This architecture follows the same general scalar-input + LSTM + pyramidal dense-stack design as Gallagher et al. [27], but is adapted here to our PPO-based actor–critic setting and action-space design. As in [27], the dynamic model-driven reinforcement learning approach can still be interpreted as a form of single-threshold-guided adaptive therapy, so it is not necessary to impose an explicit threshold policy in the learning process.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic diagram of single-threshold guided adaptive therapy.

https://doi.org/10.1371/journal.pcbi.1014457.g001

Similar to the single-threshold-guided adaptive therapy outlined above, we assume that the DRL agent decides whether to initiate treatment at the beginning of each surveillance cycle. In addition, we may impose a fixed treatment window within each cycle, corresponding to in the AT-PSC strategy or in the AT-FSC strategy. Under these settings, the reinforcement-learning-based adaptive therapy provides a framework that is consistent with the mechanistic single-threshold modeling framework and is therefore directly comparable to AT-PSC and AT-FSC. For reference, the treatment schedules for these three strategies are illustrated in Fig 1. Alternatively, as a more flexible framework for learning optimal strategies, AT-RL allows the treatment window to be adaptively adjusted in each surveillance cycle. This flexibility makes AT-RL an attractive approach for potentially further prolonging the TTP for patients.

Parameter and simulation setting

To ensure biological plausibility and clinical relevance, we utilize a set of patient-specific parameters derived from the Phase II clinical trial of intermittent androgen deprivation therapy (IADT) conducted by Bruchovsky et al. [45]. Specifically, we directly adopt the parameter values estimated by Strobl et al. [26] (and subsequently used in [27]), which were obtained by fitting the Lotka-Volterra model (Eq. (1)) to longitudinal Prostate Specific Antigen (PSA) data. The complete parameter sets for the seven representative patients are listed in Table A in S1 Appendix. These include the initial populations of drug-sensitive cells () and drug-resistant cells (), their respective growth rates (), natural death rates (), and the carrying capacity (). Consistent with prior studies on prostate cancer competition dynamics [44], we assume that there is symmetric competition between the two subpopulations ().

In line with the central motivation of this study, we consider a clinically realistic surveillance cycle in which tumor burden is assessed and treatment decisions are updated only at discrete follow-up visits. Following [28], we set the surveillance cycle to days, so that treatment on/off decisions are made every 30 days. Accordingly, the treatment window for AT-FSC is also 30 days. For AT-PSC, the treatment window is a fixed duration that can be chosen at any time from 0 to 30 days, providing a family of clinically feasible strategies. For intermittent therapy (IT), we set both the treatment-on period and the treatment holiday to 30 days for comparison.

Similarly, for the reinforcement learning–based adaptive therapy, the treatment window can either be fixed to a duration between 0 and 30 days in any surveillance cycle or be adaptively adjusted (within 0–30 days) across surveillance cycles. Accordingly, we focus on the following specific AT-RL strategies.

• RL(0,30): Binary action days of treatment per cycle. The agent learns when to initiate a full 30-day treatment block (equivalent to data-driven AT-FSC).
• RL(0,15): Action space restricted to days. Moderate treatment durations can outperform full-cycle therapy in certain parameter regimes.
• RL(0,19): The action space is days, where days denotes the optimal fixed treatment window for Patient 25 under the AT-PSC strategy. This value was obtained by numerically optimizing the treatment window and treatment threshold to maximize TTP.
• RL(0,1,…,30): Full action space (31 discrete actions). The agent dynamically selects both whether and for how long to treat at each decision point, achieving full personalization.

As mentioned above, the simulation environment captures intra-tumoral population dynamics using the mathematical framework described in model 1. The reward function of the DRL employs a multi-objective balancing scheme that combines a baseline survival reward (0.1/day), a treatment-sparing reward (+0.05 per non-treatment day), and a progression penalty (-0.1):

thus jointly targeting prolonged survival, reduced treatment burden, and delayed progression. The reward design was adapted from the reward designs used in [27] and simplified for the present tumor-dynamics setting. The reward weights were chosen heuristically rather than fitted from data, so as to encode three clinically motivated objectives on comparable scales. The policy was trained for 100,000 iterations using PPO clipping to improve training stability. As the actions are sampled from the learned policy distribution at each decision point, repeated rollouts of the same trained policy can yield different treatment trajectories and TTP values, although the environment is governed by a deterministic model.

Given the presence of a fully resistant subpopulation, complete tumor eradication is biologically infeasible in this framework. Therefore, the primary objective of our proposed policy is not cure, but containment: specifically, to decelerate tumor growth and maximize the TTP. To define this endpoint, we draw inspiration from established clinical practice. The widely used Response Evaluation Criteria in Solid Tumors (RECIST), for instance, define progression using a fixed-percentage increase in tumor burden, specifically a 20% increase relative to the size at treatment initiation [36]. Although RECIST is based on the sum of lesion diameters rather than total tumor volume, we use an analogous 20% increase in total tumor volume as a simplified model-based surrogate for clinically meaningful loss of disease control. This fixed-percentage rule is also consistent with standard practice in mathematical oncology modeling [26,36,43]. Formally, the TTP is defined as the first time that t satisfies:

The maximum simulation horizon is set to 5000 days. If progression does not occur within this period, TTP is capped at 5000.

To quantify drug use under different treatment strategies, we report dose, defined as the cumulative abiraterone exposure delivered by a strategy relative to continuous standard of care dosing. Here, the standard of care corresponds to continuous therapy (CT), i.e., uninterrupted daily abiraterone administration until disease progression [11]. Since a constant daily dose is assumed throughout this study, cumulative drug use up to progression is proportional to the number of days on which treatment is actually given. Therefore, the dose can be computed by the following day-based ratio:

Strategy optimization

For all the threshold-guided adaptive therapies, the key control parameter is the treatment threshold . There can be an optimal treatment threshold condition for maximizing TTP, which is usually denoted as . Consequently, the optimized AT-FSC is denoted as AT-FSC(), the optimized AT50 is denoted as AT50(), and optimized AT-PSC(). In addition to the treatment threshold, the optimization of the treatment window should also be an important aspect in prolonging the TTP, which is denoted as . Therefore, there can be an optimized combination that yields the maximum TTP, i.e., the optimal AT-PSC strategy denoted by AT-PSC().

To determine the optimal patient-specific strategy, we performed a systematic grid search over the admissible treatment parameters while keeping all tumor-dynamics parameters fixed for each patient, as listed in Table A in S1 Appendix. For both AT-FSC and AT-PSC, the treatment threshold () was scanned over the interval [0.2,1.2] with an increment of 0.01. For AT-PSC, a two-dimensional grid search was further conducted over both the treatment threshold () and treatment windows days. The tumor dynamics were simulated up to 5000 days or until progression. The dynamical system was numerically integrated using SciPy’s odeint solver over each 1-day interval. To assess numerical robustness, we further repeated the analysis on refined grids and independently confirmed the main features, using a fixed-step RK4 solver (Fig A in S1 Appendix).

Sensitivity and robustness analysis

We present the settings used to evaluate both the biological sensitivity and the robustness of the proposed AT-PSC strategy. First, we carried out a model-parameter sensitivity analysis by varying the intrinsic growth rates of the drug-sensitive and drug-resistant subpopulations, denoted by and . These two parameters were selected because they directly govern the competitive growth dynamics of the sensitive and resistant cell populations in the present model and therefore provide a natural representation of inter-patient heterogeneity. Based on clinically relevant ranges reported in previous studies [26,27,46], we varied both and over the interval [0.02,0.03] using uniform grids of 11 values each. For each fixed combination and treatment window , we performed a global grid search over the treatment threshold on [0.2,1.2] to identify the optimal threshold that maximized TTP. The threshold-optimized AT-PSC schedule was then compared with the optimized AT-FSC schedule under the same tumor-growth parameters. In this part of the analysis, robustness was quantified by the extent to which AT-PSC maintained a positive TTP improvement over AT-FSC across a broad region of the sampled space.

Second, we performed a sensitivity analysis to assess the robustness of AT-PSC with respect to clinically relevant variations in surveillance timing, such as missed appointments and holiday-induced scheduling fluctuations or delays. Taking Patient 25 as an example, we introduced stochastic appointment delays into the 30-day surveillance cycle. Specifically, the -th surveillance interval was set to days, where was independently sampled from an exponential distribution with mean . The mean appointment delay was varied from 0 to 30 days. For each nonzero value of , we performed 100 Monte Carlo simulations and summarized the resulting TTP by its median and interquartile range. These sensitivity analyses are model-specific and should therefore be interpreted within the assumptions of the present tumor-dynamics framework.

Results of AT-PSC adaptive therapy

Effects of varying and T₁.

We initially present the simulation results in Fig 2 based on the parameters related to Patient 25, with parameter values listed in the methods section. In Fig 2A, we show how the TTP varies as the treatment threshold increases for AT-PSC (marked as the black solid line), where the treatment window is fixed at 15 days within a 30-day surveillance cycle. For comparison, the TTP values of MTD, IT, and the standard AT50 strategies are plotted as horizontal reference lines, as they do not depend on the variable threshold and thus yield constant TTP outcomes. Overall, AT-PSC achieves longer TTP than the other two threshold-guided adaptive therapy strategies, AT-FSC and AT50, over the threshold range before the sharp decline in TTP occurs.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Impact of treatment threshold () and treatment window () on AT-PSC performance and comparison with other strategies.

A, Curves of TTP as the treatment threshold increases under different strategies. Here, is fixed at 15 days for AT-PSC. B, Curves of TTP as the treatment window increases from 1 day to the full surveillance cycle of 30 days under different strategies. Here, is fixed at 0.9865. C, Heatmap of TTP under the AT-PSC strategy with respect to and . The optimal strategy for maximum TTP is achieved at and days, as marked by the yellow star. To better visualize the surface structure, the color scale is power-normalized and contour lines are overlaid. D, Relative improvement in TTP achieved by AT-PSC compared to AT-FSC for each patient. E, Dose intensity comparison across three treatment strategies for each patient.

https://doi.org/10.1371/journal.pcbi.1014457.g002

On the other hand, TTP increases as the treatment threshold increases before reaching the critical value (), beyond which TTP sharply declines. Therefore, can be regarded as the optimal threshold for AT-PSC when the treatment window within each surveillance cycle is fixed, yielding AT-PSC(). Similarly, optimal threshold values can also be identified for AT-FSC() and AT50(), respectively. These optimal points are marked in Fig 2A for clarity, and the corresponding optimal treatment threshold values for each strategy are reported in Table B in S1 Appendix. A detailed comparison shows that the TTP under AT-PSC() is extended by 190 days (7.4%) relative to AT-FSC(), by 639 days (30.2%) relative to AT50(), and particularly by 1679 days (156%) relative to continuous treatment under MTD. This supports the conclusion that, under adaptive therapy, further reducing treatment exposure by shortening the treatment duration within each surveillance cycle can effectively extend TTP and increase the survival probability of the patients.

It should be noted that this sharp decline occurs because, once the threshold is set too high, treatment is triggered too late and becomes insufficient to control the rapid growth of sensitive cells, thereby leading to rapid progression of the total tumor burden. As a result, progression occurs before the potential benefit of adaptive therapy can be realized. This is fundamentally different from the more gradual loss of control caused by competitive release and the eventual dominance of resistant cells under excessive treatment pressure. Moreover, as shown in Fig 2A, the TTP gaps between AT-PSC, AT-FSC, and AT50 are not constant across the threshold range; rather, they are threshold-dependent and protocol-dependent, reflecting the treatment structure of each protocol.

Fig 2B illustrates how the treatment window in AT-PSC affects treatment outcomes with the treatment threshold fixed at , which corresponds to the optimal treatment threshold for AT-FSC under the same set of patient parameters. Since AT-PSC reduces to AT-FSC when , fixing the value of allows a direct comparison of the additional benefit gained by optimizing in AT-PSC. As shown in Fig 2B, once the treatment window exceeds a certain level (approximately 9 days), the TTP achieved by AT-PSC consistently exceeds those obtained under MTD, IT, and AT50. Notably, when the treatment window is limited to a short duration (e.g., 10–20 days), the resulting TTP remains higher than that of AT-FSC. These findings additionally suggest that substantial therapeutic benefit can be achieved without continuous treatment over the entire surveillance cycle, indicating that partial-cycle treatment scheduling may effectively balance tumor control and drug exposure. In addition, there is also an optimal treatment window of AT-PSC, which is denoted by AT-PSC (), which has extended the TTP by 402 days (increased by 15.7%) compared to it under AT-FSC, 1123 days (increased by 60.9%) compared to it under AT50, and 1891 days (increased by 176%) compared to it under MTD.

Cross-patient comparison of treatment outcomes and drug exposure.

We further evaluated the therapeutic advantages of AT-PSC in prolonging TTP while reducing drug dosage across different patients, namely the additional six prostate cancer patients described in the methods section. For each patient, we identified the optimal combination of and , and summarized the corresponding TTPs under different therapeutic strategies in Table 2. To ensure a transparent comparison of the optimization effort across all strategies, we also report the optimization cost for each strategy, summarized in Table C in S1 Appendix. The results demonstrate that AT-PSC significantly extends the TTP for all seven patients compared with MTD, IT, and AT50. Moreover, with the exception of patient 12, the TTP achieved under AT-PSC is also longer than that under AT-FSC, as illustrated in Fig 2D. In particular, patient 25 exhibits the largest TTP extension, reaching 402 days. For patient 12, the optimal treatment window under AT-PSC coincides with the full surveillance cycle, such that AT-PSC reduces to AT-FSC in this case.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of TTP and dose across different treatment strategies.

https://doi.org/10.1371/journal.pcbi.1014457.t002

Importantly, the observed extensions in TTP are not achieved at the cost of increased drug exposure. On the contrary, drug dosage is consistently reduced under AT-PSC for all patients, as shown in Table 2 and Fig 2E. These results indicate that the proposed AT-PSC strategy, by shortening the treatment window within each surveillance cycle, can effectively prolong TTP while simultaneously reducing overall drug exposure. This desirable “high-efficacy, low-toxicity” characteristic of AT-PSC can be intuitively concluded from Fig 2D and 2E.

To better understand the underlying dynamical mechanism, Fig 3 presents the evolutionary trajectories of different tumor cell populations under various therapeutic schedules for patient 25, including the outcome corresponding to AT-PSC, as an illustrative mechanistic example. These trajectories provide a mechanistic explanation of how AT-PSC effectively prolongs the TTP for patient 25 by regulating the competitive dynamics between drug-sensitive and drug-resistant tumor cells. Under the AT-FSC strategy, the relative tumor size begins to drift upward during oscillations at an earlier stage, as longer treatment reduces sensitive cells excessively, weakening their competitive suppression of resistant cells and allowing the resistant population to expand. In contrast, the AT-PSC strategy maintains stable oscillations over a longer period, effectively delaying disease progression. Similarly, the evolutionary trajectories of different tumor cell populations for the remaining six patients are provided in Figs C–H in S1 Appendix.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Simulation curves of relative tumor size for patient 25 under five treatment strategies.

https://doi.org/10.1371/journal.pcbi.1014457.g003

These results indicate that the superiority of the AT-PSC strategy arises from its dual optimization mechanism: the threshold ensures precise timing of treatment initiation, while the parameter enables individualized adjustment of treatment duration. The synergistic effect of these two components effectively delays the expansion of the resistant cell population while maximally preserving the drug-sensitive cell population. Consequently, comparable or even superior therapeutic efficacy can be achieved with fewer treatment days, thereby substantially reducing the total drug burden on patients. This personalized treatment paradigm not only enhances efficacy but also improves treatment efficiency.

Sensitivity and robustness analysis

In this section, we perform sensitivity and robustness analyses by varying the tumor growth parameters and , which represent inter-patient heterogeneity in the proliferation rates of drug-sensitive and drug-resistant cancer cells. This analysis aims to assess the robustness and clinical effectiveness of the proposed treatment strategies in prolonging the TTP across a broad spectrum of patient-specific tumor dynamics. The results in the previous section demonstrate that the single-threshold-guided adaptive therapy proposed in [31] consistently outperforms other treatment strategies, including MTD, IT, and AT50. Building on these findings, we therefore focused on evaluating whether the AT-PSC strategy can further improve the TTP compared with the AT-FSC adaptive therapy previously studied in [28]. Unless otherwise specified, any improvement in TTP reported in this section refers to the increase in TTP achieved by AT-PSC relative to AT-FSC.

As shown in Fig 4A and 4B, we first present two representative slices of the , parameter space by fixing one growth parameter at the fitted value for Patient 25, while varying the other growth parameter and the treatment window. These heatmaps show that, across a broad range of parameter values, there consistently exists a clinically “safe” range of treatment windows in which AT-PSC prolongs TTP relative to AT-FSC. This indicates that the therapeutic benefit of AT-PSC does not rely on fine-tuning to a single precise schedule, but can be achieved over a relatively wide window of treatment durations, enhancing its potential clinical feasibility. For each specific tumor growth profile, however, there is an optimal treatment window that maximizes TTP. Importantly, this optimal window varies substantially with and , reflecting inter-patient heterogeneity in tumor growth dynamics. Moreover, we observe a clear trend that the achievable improvement in TTP diminishes as the growth rate of drug-resistant cells increases. Nevertheless, even in such cases, AT-PSC maintains a non-negligible therapeutic benefit over AT-FPC and of course the continuous MTD or AT50, supporting its potential value across diverse resistance landscapes.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Robustness and sensitivity analysis of the AT-PSC strategy across varying tumor growth parameters.

A, Heatmap of the improvement in TTP, varying the treatment window and the resistant cell growth rate (with fixed). B, Heatmap of the improvement in TTP, varying the treatment window and the sensitive cell growth rate (with fixed). C, Heatmap of the maximum achievable improvement in TTP across the parameter space, where the treatment window and treatment threshold are independently optimized for each growth rate combination. D, Heatmap of the TTP achieved by AT-PSC across the parameter space with a fixed treatment window of =13 days. E, Distribution of the improvement in TTP across different fixed treatment windows (=10-30 days). For each value of , the distribution is obtained by varying the sensitive cell growth rate and resistant cell growth rate across their respective parameter ranges.

https://doi.org/10.1371/journal.pcbi.1014457.g004

In Fig 4C, we further present a heatmap illustrating the improvement in TTP achieved by the optimal AT-PSC() relative to the optimal AT-FSC(), as both the growth rates of drug-resistant cells and drug-sensitive cells vary from 0.02 to 0.03. Within this clinically relevant parameter range, AT-PSC robustly improves TTP by shortening the treatment window relative to AT-FSC, with the maximal extension in TTP reaching as much as 1103 days. Similar to the setting in Fig 4, additional heatmaps for different slices of the , parameter space are provided in the Supplementary Materials (Figs I–K in S1 Appendix). These supplementary results are consistent with those shown in Fig 4 and lead to the same conclusion, indicating that AT-PSC can robustly extend TTP relative to AT-FSC over a broad range of growth-rate parameters for both drug-sensitive and drug-resistant cells.

Integrating the results shown in Fig 4A–4C, a consistent pattern emerges: AT-PSC is most effective in prolonging TTP when the growth rate of drug-resistant cells is relatively low. Clinically, this corresponds to tumors in which resistant subpopulations expand slowly and remain susceptible to ecological competition from drug-sensitive cells. In contrast, when approaches or exceeds , indicating aggressive resistance dynamics, the additional benefit of AT-PSC over AT-FSC is markedly reduced. This suggests that the therapeutic leverage of partial-window adaptive therapy diminishes as resistant clones gain a growth advantage.

This trend is further supported by simulations in which the treatment window is fixed at 13 days under AT-PSC, as shown in Fig 4D. As and vary over the same range, the resulting TTP spans a wide interval (from 1107 to 4042 days), reflecting substantial inter-patient variability. Notably, the longest TTP (4042 days) is achieved when both and are at their lower values (), reinforcing the clinical insight that tumors with slower growth and weaker resistance dynamics are most amenable to benefit from adaptive strategies that reduce treatment intensity.

Although the above results suggest that individualized therapeutic strategies should be designed to maximize the TTP for each patient, from a clinical perspective, it is also desirable to identify a unified treatment schedule that can deliver sustained benefits across a broad range of patients. That is, instead of relying solely on patient-specific optimization, we aim to identify a fixed treatment window of AT-PSC that can robustly improve the TTP for most patients. To this end, in Fig 4E, we plot the distribution of the improvement in TTP of AT-PSC (resulting from variations in and ) across different treatment windows, relative to AT-FSC, showing that the median improvement in TTP exhibits a non-monotonic dependence on the treatment window.

In particular, for =10–20 days the median TTP improvement consistently exceeds 120 days (approximately 123–183 days), peaking at 183 days when =13 days. Meanwhile, the interquartile ranges of the box plots are comparable across these windows, indicating limited sensitivity to moderate changes in the treatment window. On the other hand, Fig 4E shows that treatment windows in the range of approximately 12–16 days can robustly improve TTP relative to AT-FSC. In view of this robustness, together with the practical convenience of a two-week-on/two-week-off schedule within a 30-day surveillance cycle, we consider a constant AT-PSC schedule with a fixed 14-day treatment window (i.e., a “14-30” schedule) as a feasible and clinically interpretable candidate schedule within the present modeling framework.

To further examine whether this conclusion depends on treating all sampled parameter combinations equally, we repeated the analysis by assigning larger weights to parameter combinations closer to the fitted growth rates of the seven patients during sampling, as shown in Fig L in S1 Appendix. Under this setting, both the median and mean improvements in TTP were also maximized at days, consistent with the result shown in Fig 4E. Moreover, the results in Fig L in S1 Appendix further suggest that a “14-30” schedule may serve as a feasible and clinically interpretable candidate schedule. Therefore, the main conclusion drawn from Fig 4E is not dependent on uniform parameter sampling.

Beyond the above sensitivity analysis, we further examined the robustness of AT-PSC with respect to surveillance intervals and appointment delays in monitoring. First, we re-optimized the AT-PSC schedule under different surveillance intervals for each patient-specific parameter set, as shown in Fig M in S1 Appendix. Fig M in S1 Appendix indicates that TTP can be very sensitive to the surveillance interval depending on patient-specific parameters. For example, more frequent surveillance may yield a longer TTP for Patient 25. However, for many of the other patients, TTP changed only slightly as the surveillance interval varied from 1 to 60 days. For comparison, we further evaluated the optimized AT-FSC and AT-PSC strategies across the same set of surveillance intervals, and the corresponding TTP heatmaps are shown in Fig N in S1 Appendix. The results show that, across different surveillance intervals, AT-PSC generally yields longer TTP than AT-FSC for most patients.

We then examined the sensitivity of treatment outcomes to stochastic appointment delays under a 30-day surveillance cycle, as shown in Fig O in S1 Appendix. The results indicate that imperfect adherence to the planned monitoring schedule, represented here by appointment delays, generally reduces TTP, with the maximal TTP usually achieved in the absence of delays. In particular, when the treatment threshold was optimized under the no-delay setting, the resulting strategy was highly sensitive to appointment delays, leading to a rapid decrease in TTP as the mean delay increased. In contrast, lowering the treatment threshold improved robustness to moderate delays. When the mean delay was less than 7 days, TTP could remain close to that achieved by the no-delay AT-PSC strategy, although this robustness came at the cost of reducing the maximal TTP gain under ideal no-delay conditions.

Results of reinforcement learning based adaptive therapy

We first report the results of the reinforcement learning (RL)-based adaptive therapy with a fixed treatment window within each surveillance cycle, aiming to learn an effective adaptive treatment strategy for patient 25. Consistent with the setting of our mechanistic modeling framework, we consider three RL schedules: full-cycle treatment RL(0,30), and partial-cycle treatment with a fixed treatment window of 15 days RL(0,15) and 19 days RL(0,19), where 19 days corresponds to the patient-specific optimal treatment window identified for Patient 25 under the AT-PSC framework. The results are shown in Fig 5.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Impact of action space design on RL-based adaptive therapy performance under the parameter set of Patient 25.

A, Kaplan–Meier curves showing progression-free survival evaluated on a cohort of 100 stochastic simulations for four RL strategies with different action spaces: full action space (0-30 days) and binary action spaces (0/19, 0/15, 0/30 treatment days per cycle). B, Violin plots depicting the TTP distribution for each strategy across the same 100 simulations with the underlying individual data points overlaid. The accompanying table summarizes key statistics including median, mean, and 95% quantile range.

https://doi.org/10.1371/journal.pcbi.1014457.g005

The survival analysis in Fig 5A indicates that RL(0,30) tends to be less effective in prolonging survival (or TTP) than the partial-cycle treatment strategies. This finding is consistent with the results obtained under the rule-based protocols and further supports the clinical relevance of reducing the treatment window as a feasible way to de-escalate aggressive therapy while improving disease control. More specifically, RL(0,19) appears to outperform RL(0,15), yielding an additional improvement of approximately 234 days in TTP. Notably, the 19-day treatment window is also identified as optimal for Patient 25 under the AT-PSC framework, and the median TTP obtained by RL under this fixed 19-day constraint is 2934 days, which is comparable to the 2967 days achieved by the optimized AT-PSC strategy. Taken together, these results provide further evidence that our modeling framework can offer an effective mechanistic design of adaptive therapy schedules, yielding sustained improvements in TTP while maintaining clinical feasibility.

In Fig 5, we further present the results of the RL-based adaptive therapy in which the treatment window is adaptively adjusted from 1 day to 30 days, denoted as RL(0,1,...,30). The results show that the median TTP can be further extended by 213 days compared with RL(0,19). This advantage is also reflected in the violin plots, where RL(0,1,…,30) exhibits a higher median TTP and a narrower interquartile range, indicating more consistent performance across repeated rollouts. Note that the Kaplan–Meier curves in Fig 5A show a relatively steep decline in survival probability across all strategies. Fig 5 was generated from 100 stochastic simulations for a cohort of virtual patients sharing the same parameter set (Patient 25). In these simulations, the only source of variation is the stochastic action selection from the trained RL policy at each decision point, while the underlying tumor dynamics remain deterministic. The steep drop therefore indicates that progression times are relatively concentrated, with many virtual patients reaching disease progression within a narrow time window, rather than being broadly dispersed. This suggests that, under the learned RL strategy and the fixed cycle-based decision interval, the simulated treatment trajectories are relatively consistent across repeated rollouts. This pattern is consistent with the observation in [27].

In addition, we evaluated the performance of RL(0,1,…,30) against other strategies for the seven prostate cancer patients, with results summarized in Fig 6 (exact values are provided in Table D in S1 Appendix). The four strategies exhibit a clear performance ordering in terms of TTP: RL(0,1,…,30) performs best, followed by AT-PSC, AT-FSC, and AT50. The differences are visually apparent in Fig 6A, and RL(0,1,…,30) achieves longer TTP for all patients, with maximum gains of 180 days relative to AT-PSC, 582 days relative to AT-FSC, and 1303 days relative to AT50. RL may generate different treatment trajectories across independent runs; therefore, we report the median TTP for comparison. Importantly, the treatment schedule for each trajectory can be explicitly recorded. The treatment schedule corresponding to the highest TTP can then be substituted into the deterministic model to reproduce the same trajectory and outcome, as the RL training environment is constructed from the same deterministic model. This indicates that the RL-learned schedules, particularly the schedule achieving the longest TTP, are feasible within the proposed modeling framework. Moreover, as shown in Fig 6B, RL(0,1,…,30) not only prolongs TTP but also reduces cumulative drug use, with maximum dose reductions of 9.26% relative to AT-PSC, 14.81% relative to AT-FSC, and 30.37% relative to AT50.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6.

A, Time to progression (TTP) and B, dose under RL(0,1,...,30), AT-PSC, AT-FSC, and AT50. For RL(0,1,...,30), the plotted values are the medians over repeated stochastic simulations, and error bars indicate the 95% quantile range. C, Tumor trajectory for Patient 25 under the adaptive RL(0,1,…,30) strategy, shown for the rollout achieving the maximal TTP, together with the corresponding within-cycle treatment schedule. D, Action probability heatmap for Patient 25 under RL(0,1,…,30), illustrating the probabilities of selecting treatment windows at different levels of relative tumor size. E, Expected treatment window length under RL(0,1,…,30) across relative tumor sizes for the seven patients. Solid lines represent the smoothed expected value of the treatment window, and shaded bands show the corresponding smoothed dispersion (mean standard deviation) computed from the action-probability distribution. F, Treatment selection probability curves under the binary action RL policy (treat for a fixed window vs. no treatment) for the seven patients, plotted against relative tumor size. Here, denotes the patient-specific optimized treatment window reported in Table 2. Curves are smoothed for visual clarity. The implied switching threshold learned by the agent is highlighted in the white boxed region, and the reference threshold is indicated for comparison.

https://doi.org/10.1371/journal.pcbi.1014457.g006

Further, we explore the mechanism behind how the RL(0,1,…,30) adaptive therapy can achieve superior tumor control, compared with the treatment strategy with a fixed treatment window. To this end, we first show, for Patient 25, the evolution of the two tumor cell populations under RL(0,1,…,30) for the simulation run that achieves the maximal TTP (Fig 6C), together with the corresponding action selection probabilities across different tumor size states (Fig 6D). Note that the strategy learned from RL(0,1,…,30) extended the TTP by approximately 8.2% (3210 days vs. 2967 days) compared with the AT-PSC () strategy, and prolonged it by 234 days compared with RL(0,19).

From Fig 6D, when the relative tumor size is below 1, the action probabilities under RL(0,1,…,30) concentrate on very short treatment windows (days). This indicates that in the early, low-burden stage, RL(0,1,…,30) tends to use short window dosing to reduce treatment exposure while preserving the competitive advantage of drug-sensitive cells. As the tumor size increases to around 1.05, the preference for short windows rapidly diminishes and shifts to a moderate window (days), suggesting that stronger within-cycle intervention is required as the system enters a higher-risk state. When the tumor burden further rises into the high-risk range (approximately 1.1-1.2), RL(0,1,…,30) increasingly favors longer windows (days), and as the tumor approaches the progression boundary it further shifts toward days. Analogous policy heatmaps for all seven patients are provided in Fig P in S1 Appendix. This pattern reflects a more aggressive intensification of treatment in high-risk states to suppress tumor growth and avert rapid progression. Notably, this early de-escalation in treatment intensity is consistent with a delayed expansion of the drug-resistant subpopulation, as can be seen by comparing Fig 6C with 3E.

This phenomenon suggests that the RL strategy may imply a multi-threshold treatment approach: different treatment windows are set based on the tumor burden at various stages. When the tumor size exceeds a certain threshold, the treatment window increases to address higher-risk states. In the early stages, the RL strategy tends to use short treatment windows to avoid overtreatment; as the tumor progresses, the window gradually extends, allowing for more precise treatment control. This mechanistic interpretation is further supported by Fig 6E, where the RL agent adaptively increases the treatment window from a relatively short duration in the initial phase as the tumor size grows for the seven patients.

In contrast, under the fixed treatment window constraint, the strategy learned by RL primarily manifests as a single threshold decision rule, as illustrated in Fig 6F. Specifically, when facing the binary choice of at each surveillance cycle, the agent learns to make decisions based on relative tumor size: it tends to pause treatment when the tumor size is below a specific level, and triggers a treatment once the threshold is exceeded. We define the relative tumor size corresponding to a treatment probability exceeding 0.5 as the agent’s learned “treatment threshold.” Notably, these learned thresholds (labeled as “Learned”) align closely with the theoretical optimal threshold derived from the single-threshold AT-PSC framework. This suggests that, under identical fixed treatment window constraints, AT-PSC has achieved a near-optimal switching strategy. This consistency explains why the fixed window RL and the optimized AT-PSC scheme yield comparable TTP outcomes. Specifically, the RL approach with the fixed treatment window days achieves a median TTP of 2934 days (95% quantile range: [2859, 2976]; Fig 5B), which is close to the 2967 days for the optimized AT-PSC scheme (Fig 3E). To provide a complete comparison of the RL formulations, we further summarize their action-space definitions, network sizes, and benchmark runtimes in Table E in S1 Appendix.

Ethics statement

No ethics approval was required for this study, as all data used were obtained from publications for which ethical approval had been obtained.