Optimal dynamic control approach in a multi-objective therapeutic scenario: Application to drug delivery in the treatment of prostate cancer

Itziar Irurzun-Arana; Alvaro Janda; Sergio Ardanza-Trevijano; Iñaki F. Trocóniz

doi:10.1371/journal.pcbi.1006087

Abstract

Numerous problems encountered in computational biology can be formulated as optimization problems. In this context, optimization of drug release characteristics or dosing schedules for anticancer agents has become a prominent area not only for the development of new drugs, but also for established drugs. However, in complex systems, optimization of drug exposure is not a trivial task and cannot be efficiently addressed through trial-error simulation exercises. Finding a solution to those problems is a challenging task which requires more advanced strategies like optimal control theory. In this work, we perform an optimal control analysis on a previously developed computational model for the testosterone effects of triptorelin in prostate cancer patients with the goal of finding optimal drug-release characteristics. We demonstrate how numerical control optimization of non-linear models can be used to find better therapeutic approaches in order to improve the final outcome of the patients.

Author summary

Mathematical models of the disease processes are widely used in computational biology to quantitatively describe the time course of disease progression and are often linked to pharmacokinetic–pharmacodynamic models in order to evaluate the effect of drug treatment on disease. Once the models are built from observed information and/or literature data, they can predict the dynamics of the system under different conditions through computer simulations. However, simulation exercises are not always effective to obtain the desired objectives due to the complexity of these systems. In this work, we optimized the release characteristics of a synthetic gonadotropin-releasing hormone analog used to induce chemical castration by inhibiting the testosterone levels in prostate cancer patients. The therapeutic goals to achieve were to minimize the initial flare up of testosterone levels and the time to reach testosterone values below castration limit, while maximizing the castration period of the patients. Our methodology, based on control theory, introduces a manipulable variable into the system’s equations to drive the model towards the established goals. We demonstrated how drug-release properties can be improved with the implementation of optimal control strategies to enhance the outcome of cancer patients. These methods are extrapolable to other problems encountered in the field.

Citation: Irurzun-Arana I, Janda A, Ardanza-Trevijano S, Trocóniz IF (2018) Optimal dynamic control approach in a multi-objective therapeutic scenario: Application to drug delivery in the treatment of prostate cancer. PLoS Comput Biol 14(4): e1006087. https://doi.org/10.1371/journal.pcbi.1006087

Editor: James Gallo, Mount Sinai School of Medicine, UNITED STATES

Received: November 28, 2017; Accepted: March 12, 2018; Published: April 19, 2018

Copyright: © 2018 Irurzun-Arana et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This work has been supported by research funds from the University of Navarra (http://unav.edu/). SA was financed by Agencia Estatal de Investigación and FEDER funds (grant MTM2016-79422-P). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Optimizing delivery systems targeting constant levels of drug concentration represents always a challenge for chronic diseases requiring continuous treatment and especially in those cases where the relationship between drug exposure (represented generally as levels of drug concentration plasma measured longitudinally) and pharmacological response is complex and non-linear. The management of prostate cancer with sustained release formulations of triptorelin (TRP) injected every 3-6 months represents a good example [1].

For the case of the hormone-sensitive prostate tumors the therapeutic goal of any pharmacology treatment is to maintain as longer as possible the levels of testosterone (TST) below the castration limit (CT) which is set to the plasma concentration value of 0.5 ng/mL [2].

In recent past, we have developed a mechanistic computational model for the TST effects of the agonist TRP in prostate cancer patients using longitudinal pharmacokinetic (PK; drug concentration in plasma) and pharmacodynamics (PD; TST concentrations in plasma) data obtained from several clinical trials testing the efficacy of different sustained-release formulations (SR) [3]. Briefly, TRP exerts its action by increasing the fraction of activated receptors and therefore stimulating the production of TST. However, the prolonged exposure of TRP causes receptor down-regulation, resulting in a reduced synthesis of TST. The typical TST vs time profile after a single injection of TRP is represented in Fig 1. The schematic representation of the PKPD model developed for TST effects of TRP, excluding the absorption compartments of the original model, and the estimates of model parameters are shown in Fig 2.

Download:

Fig 1. Typical testosterone profile after administration of triptorelin.

TST_max refers to the maximal testosterone concentrations, t_cast indicates the time where testosterone levels fall below 0.5ng/ml (castration limit of prostate cancer patients, marked with an horizontal dashed line in the figure) and t_effect indicates the castration period of the patients.

https://doi.org/10.1371/journal.pcbi.1006087.g001

Download:

Fig 2. Schematic representation of the state variables and control input for the pharmakinetic-pharmacodynamic model of the testosterone effects of Triptorelin (left) and model parameter estimates (right).

C_TRP, serum concentrations of Triptorelin; CL, apparent total clearance; V_c, V_T1, and V_T2, apparent volumes of distribution of the central, shallow, and deep peripheral compartments, respectively; CL_D1 and CL_D2, distribution clearances between the central and peripheral compartments; R_T, total receptors; R_T₀, total receptors at baseline; TST₀, baseline testosterone level; K_D, receptor equilibrium dissociation constant of triptorelin; D_R, down-regulation process; D_{R_50}, the value that elicits a 50% maximal reduction in k_{S_R} for a given amount of total receptors; k_{S_R}, zero-order rate constants of receptor synthesis; k_{D_R}, first-order rate constants of degradation; k_{S_R}, zero-order rate constants of testosterone synthesis; k_{D_T}, first-order rate constants of testosterone degradation; k_in, zero-order rate production of testosterone independent from gonadotropins; AGN, ratio between the endogenous agonist concentration and its receptor equilibrium dissociation constant; FDB, feedback.

https://doi.org/10.1371/journal.pcbi.1006087.g002

As highlighted in Fig 1 there are three critical aspects to be taken into consideration at the time to develop an innovative delivery system of TRP for the treatment of prostate cancer: initial flare up, time to reach CT, and castration period. Ideally, such new formulation should release TRP at a rate eliciting levels of concentration in plasma minimizing both the initial flare up and the time to reach CT, as well as maximizing the castration period. Specifically, limitation in the TST flare-up (TST_max) to 50% increase with respect to baseline, minimize time to castration after first injection (t_cast) to values below 3 weeks, and extend the castration time after injection (t_effect) for at least 9 months.

Given the complex relationship between concentrations of TRP in plasma and response as represented in Figs 1 and 2, together with the requisite of maintaining the TST profiles within the constraints mentioned above, optimization of the rate of drug release is not a trivial task and cannot be efficiently addressed through an extensive trial & error simulation exercise.

In the current work we aimed to optimize the release profile of TRP from SR formulations matching the multi-objective therapeutic needs applying optimal control methodology [4]. The rationale behind the decision of focusing on the release process is based on the assumption that once the drug is absorbed and reaches systemic circulation (represented as part of the central compartment in Fig 2) it follows the same distribution and elimination characteristics regardless the type of formulation administered. The same is assumed with respect to the TST response, the rate of synthesis and degradation of TST and receptors, the dynamics of receptor occupation, and the down-regulation process. These mechanisms are independent from the absorption properties of the drug.

Despite we focused on a specific case, the workflow and methodology used can be readily translate to other therapeutic areas and scenarios such as dosing schedule optimization and personalized treatments.

Materials and methods

Having defined the therapeutic goals of the project, the analysis was divided in several steps: i) a population of virtual subjects were generated in order to have a representative population of the study; ii) the optimal TST profiles for each virtual patient were derived by means of optimal control methods; and finally iii) the empirical absorption profiles obtained in step ii were characterized using parametric models to assist biopharmaceutics at the time to develop and evaluate new SR formulations of triptorelin. A schematic representation of the workflow is given in Fig 3.

Download:

Fig 3. Principal steps implemented in our methodology.

https://doi.org/10.1371/journal.pcbi.1006087.g003

Generation of a virtual patient population

Values listed in the table inserted in Fig 2 include estimates of typical population parameters and between-subject variability (represented by BSV in the table and hereafter) obtained from [3] for a set of model parameters. In order to obtain the population of virtual patients, parameters were modelled as , where P_i and P_pop represent the i^th individual and typical population values of the P parameter, respectively, and η_{i_P} corresponds the deviation of P_i with respect the typical value P_pop; the set of individual η_{i_P} forms a random variable with mean value of 0 and variance following a normal distribution, whereas the distribution of individual parameters is log-normal. The magnitude of reflects the BSV associated to a specific model parameter, which in Fig 2 is expressed as coefficient of variation (CV%).

One thousand set of disposition (clearance and volume of distribution in the central compartment, represented as CL and V_c respectively), pharmacodynamics (receptor equilibrium dissociation constant of triptorelin (K_D)) and system (baseline TST levels (TST₀), zero-order rate of TST production independent from gonadotropins (k_in), zero-order rate constants of receptor synthesis (k_{S_R}) and the value that elicits a 50% maximal reduction in k_{S_R} for a given amount of total receptors (D_{R_50})) related parameters were generated using the typical population estimates and corresponding marginal distributions reported in the table of Fig 2. The parameter values for the virtual population were generated with NONMEM 7.2 [5].

Optimal control

An optimal control problem is a dynamic optimization problem in which the state of a system is linked in time to the application of a control function u(t), which drives the system towards a desirable outcome by minimizing a cost function J(u) subject to operating constraints [4, 6]. In other words, the control variable u(t) forces a system to have an optimal performance. The concrete control strategy will depend upon the criterion used to decide what is meant by “optimal”; in the current case TST_max < 1.5 ⋅ TST₀ ng/ml, minimize t_cast ≤ 3 weeks, and maximize t_effect ≥ 9 months.

Therefore any optimal control problem can be formulated to find the magnitude of u(t) over the time of study [from initial time t₀ to final time t_f] such that: (1) where J(u) is the cost function, u(t) is the control variable; x(t) the vector of state variables; x₀ the set of initial conditions of the state variables; h() the equality constraints; and g() the inequality constraints. The general form of the equation in J(u) is known as Bolza optimization problem [7], which is represented as the sum of a terminal cost functional (Mayer problem) and an integral function of the state and control from t₀ to t_f (Lagrange problem). For a more detailed information see S1 Text.

Fig 2 shows a schematic representation of the state variables and control input defined in this work. The state system is characterized by the variables that predict serum concentrations of triptorelin (C_TRP), concentrations of triptorelin in the shallow and deep peripheral compartments (C₁ and C₂ respectively), drug input profile (D), amount of total receptors (R_T), and optimal testosterone levels (TST), each of them represented by the corresponding ordinary differential equation as shown in Table 1. Note that in Fig 2 the terms resembling the 0^th and 1^st order absorption processes have been removed from the original model structure from [3] and have been replaced by the new control variable u(t). Therefore the expression associated to the rate of change of the levels of TRP in plasma () is: (2) An additional compartment D was defined, where the dose of TRP administered to the patients (10mg in this evaluation exercise) was placed as initial condition (D₀). The control variable u(t) leaves this compartment and enters to the central (systemic) compartment of TRP as follows: (3) Recall that, u(t) (ng/day) does not represent any particular mechanism of absorption (i.e., zero and/or first order kinetics), but a vector of different values that influence the system to behave in a pre-determined (optimal) way.

Download:

Table 1. Summary of the setup of the different components of the optimal control problem.

https://doi.org/10.1371/journal.pcbi.1006087.t001

The aim of this work was to find the time profile of u(t) (input function of TRP into the central compartment) that minimizes an objective (or cost) function and satisfies all constraints which represent the boundaries and therapeutic goals to be achieved (see Table 1). The choice of an objective function represents a critical aspect in optimal control problems [8]. Here, the problem is divided into two phases each represented by a different cost function and defined between: (i) 0 and t_cast, and (ii) t_cast and ≥ 280 + t_cast days, respectively.

During the first phase (from 0 to t_cast) the u(t) profile is optimized to transfer the system from an initial state TST₀(baseline testosterone level) to the final state of 0.5ng/ml (CT value) in the shortest possible time. To solve the first phase of the optimization problem, the following objective function and equality constraint were defined respectively: (4) (5) where t_cast is an static control variable for minimizing J_I. Here, we wished to obtain the minimum value of t_cast that causes TST levels to achieve the CT value. The final time t_cast was not known in advance, and that is the reason why the optimization problem was divided into two different phases.

Additionally, an inequality constraint was added to limit the initial flare-up of the testosterone below 50% increase with respect to baseline: (6) The second phase, covering the period between t_cast and 280+t_cast days, aims to maintain the TST levels below CT. If a second objective function or constraints were not incorporated into the optimization problem, values of TST rose above CT at times much earlier than 280 days. The approach used to overcome the above mentioned undesired effect and maintain TST predictions within the therapeutic goal led to the minimization of a second objective function of the form: (7) The rationale for formulating J_II using a quadratic function (TST(t)²) for minimizing testosterone levels, instead of TST(t), was because it offers relevant mathematical advantages in the context of optimization problems. In optimal control theory, one of the main necessary conditions for optimality is that control variables minimize a Hamiltonian function over u(t). The Hamiltonian becomes convex if quadratic forms are used for the objectives and thus the problem will have a unique minimizer [4]. See S1 Text for more information about the Hamiltonian matrix and the necessary and sufficient conditions for optimal control problems. Furthermore, using squared terms amplify the effects of large variations and de-emphasize the contributions of small fluctuations.

Continuity between the two phases of the optimization problem was ensured by imposing the initial conditions of the state variables at phase II to be equal to their final values at the end of the phase I (see Table 1).

Alternatively, other objective functions or constraints could have been defined. For example, an alternative approach to model the second phase of the optimal control problem is to only add the inequality constraint TST[t_cast: (280 + t_cast)] − 0.5 < 0, instead of a second objective function J_II. This approach resulted in TST levels closer to CT compared to the values obtained with the addition of J_II. However, in the work from [1, 9] suggested that a CT value lower than 0.2 ng/ml could be an even better target to maximize therapeutic outcomes of prostate cancer patients. Due to these variations in the definition of the most appropriate CT value, we prioritized the minimization of TST levels during the second phase using J_II because we obtained the lowest possible values of TST.

Table 1 summarizes the setup of the different components of the optimal control problem described above. There exists different methods to solve this type of problems [10, 11]. In our case, the dynamic optimization problem was solved numerically via direct methods with the IPOPT Solver (Interior Point OPTimizer) [12] which is freely available in the APMonitor Optimization Suite (http://apmonitor.com/) through MATLAB programming environment [13]. The results were evaluated by calculating the proportion of individuals that achieved the described therapeutic goals and constraints.

For more information about control theory see the works from [6, 8, 14] and for a more comprehensive overview of the role of optimal control in cancer research read the reviews from [4, 10, 15].

Mechanistic characterization of the optimal absorption profiles

During the optimal control exercise, values of TST in plasma were obtained approximately every 12h for the first phase and every 120h in the second phase. Given the fact that the disposition, pharmacodynamics, and system parameters were already known as they were randomly generated as described in Material and methods, the analyses of the TST profiles described in this section focused on the mechanistic/parametric characterization of the absorption process of TRP aiming to provide biopharmaceutics with metrics useful to guide the development of new sustained release formulations. Those metrics are the fractions of the total dose injected absorbed following 0^th and 1^st order processes, the cumulative drug release profiles over time, the percentage of the dose that should remain in the site of injection at t_cast and the time at which the different absorption mechanism are activated.

The absorption model used to estimate the corresponding absorption parameters allowing afterwards computation of metrics is represented in the work from [3] and comprises three non-simultaneous absorption mechanisms, two of them following 1^st order kinetics and the third one following a 0^th order process. This model is considered of a sufficient complexity to deal with almost any absorption profile that can take place after administration of SR formulations [16, 17]. A schematic representation of the structural model with the corresponding ordinary differential equations is provided in S1 Fig.

The analyses were performed with the NONMEM version 7.2 software [5], following a two stage approach in which the parameters of each subject are first obtained and summary statistics (median, and 95th confidence intervals) are then calculated. BSV in the absorption parameters was modelled exponentially as described in section for the rest of model parameters. TST concentrations obtained in step were logarithmically transformed for the analysis, and residual variability was modeled by using an additive error model on log-transformed data.

Results

Optimal pharmacodynamic profiles

Fig 4 (blue points) illustrates the optimal testosterone profiles for the 1000 hypothetical individuals that we obtained after applying the optimal control problem formulated in Table 1. The initial dose was considered to be 10mg. The code and data to reproduce these results in MATLAB can be found in the S1 Data. All of them achieved the 3 quantitative therapeutic goals (95% interval confidence between parenthesis) defined in the Introduction section: time to castration was minimized to 18.96 days (11.408—36.289), the increase of TST levels at the flare was always smaller than 50% with respect to baseline (36.8%-50.002%), and t_effect was greater than 280 for all the patients.

Download:

Fig 4. Optimal testosterone (TST) profiles for 1000 simulated individuals.

Solid circles represent optimal TST observations obtained after the optimal control approach, solid line represent the median tendency of the data and red dashed line indicates the castration limit (0.5ng/ml) of prostate cancer patients.

https://doi.org/10.1371/journal.pcbi.1006087.g004

These profiles were generated with the manipulable variable u(t) which could take any values in order to minimize the multi-objective problem. However if we looked to the TRP concentration vs time profiles that induced the optimal TST levels (data not shown), those profiles did not seem attainable by using simple first or zero order kinetics. That was the reason to directly approximate the TST levels with the PKPD model presented by [3] and estimate the most adequate absorption parameters.

Optimal release characteristics

The optimal release characteristics corresponding to the selected PKPD model from [3] are listed in Table 2. The final model adequately described the optimal TST profiles calculated in the previous section as shown in the individual profiles of Fig 5. A lag time was associated with one absorption compartment. The first order rate constant of absorption of the second depot compartment (K_A2) had a very low median value (0.003 day⁻¹), resulting in a slow decay of TRP in serum concentrations. The first order rate constant of the first depot compartment (K_A1), instead, had a higher value (0.25 day⁻¹) to allow for a rapid decay of the TST levels in the firsts days of treatment. Table 2 also indicates that most of the drug is released following 1^st order kinetics as the fraction of drug associated with the 0^th order absorption process (F_inf) is very small (4%). This result is also reflected in Fig 6, where the median tendency of the drug release following each of the absorption mechanisms for the 1000 individuals is shown. The values of the duration of the 0^th order process (D_inf) varied immensely between individuals (from hours to more than 200 days), thus the variability term was removed from this parameter.

Download:

Table 2. Population absorption parameters estimated for the optimal triptorelin profiles.

The median values and the 95% confidence intervals (CI%) are shown for a population of 1000 patients.

https://doi.org/10.1371/journal.pcbi.1006087.t002

Download:

Fig 5. Optimal testosterone profiles of the 1000 virtual patients.

Optimal testosterone observations (solid circles) with individual predictions (solid blue lines) of the pharmacokinetic/pharmacodynamic model and a red dashed line indicating the castration limit (0.5ng/ml) of prostate cancer patients.

https://doi.org/10.1371/journal.pcbi.1006087.g005

Download:

Fig 6. Optimal drug release characteristics following each of the absorption mechanisms.

https://doi.org/10.1371/journal.pcbi.1006087.g006

The therapeutic objectives obtained were compared to those from the optimal TST profiles (Fig 7) and the 95% confidence intervals were also calculated from the 1000 samples. Minimal time to achieve castration levels was 19.5 days (11.4-56.7) and the median percentage of drug consumed until that moment was 38.9% (16.55%-66.96%). In Fig 7A the distribution of t_cast values for the 1000 individuals can be appreciated. With the modeling approach 63.9% of the patients had a t_cast smaller than 21 days, whereas in the optimal TST profiles this value was equal to 70.7%. Regarding the second therapeutic goal, the initial peak in the TST levels had a median of 55% (17%-75%) increase with respect to baseline. This indicated that the second objective was not always achieved, contrary to the case of the TST profiles obtained by the optimal control problem where the flare up was much more controlled (Fig 7B). Nonetheless, the median value was very close to the optimal value of 50%, so we assumed that the modeling approach managed to achieve the second therapeutic goal as well. Finally, the long-term castration had a median value of 351 days (235.9—708), which was higher than expected (Fig 7C), but we again needed to take into account that a small fraction of individuals (7.8%) did not achieve a t_cast + t_effect above 9 months due to their specific physiological characteristics.

Download:

Fig 7. Comparison of the three therapeutic objectives between the optimal control strategy (salmon) and the pharmacokinetic/pharmacodynamic modeling approach (blue) for 1000 individuals.

A) Distribution of t_cast (time to obtain testosterone levels below castration limit) values. B) Distribution of the values of the testosterone flare-up (maximum testosterone level/baseline testosterone level). C) Distribution of t_cast + t_effect (castration time after injection of the drug) time values of the modelling approach.

https://doi.org/10.1371/journal.pcbi.1006087.g007

Discussion

In this paper, we have applied mathematical modeling and control theory to establish optimal drug input (release) profiles to support the development of new release formulation of triptorelin aiming to improve patient coverage. Optimal control (OC) has a long and successful history of applications in engineering [8, 18] and economics [19, 20] but also has become an important issue in biomedical research. Especially in clinical cancer research, a significant amount of effort has been devoted to developing mathematical models to identify the most effective chemotherapeutic administration regimens using OC methods [15, 21] and references therein. One of the earliest studies where chemotherapy treatment planning was defined as an OC problem was in the work from [22]. The authors applied OC to improve the treatment administration of the bone cancer IgG multiple myeloma. Understanding the dynamics of resistance mechanisms against chemotherapy and targeted drugs and emerging of adverse effects represent challenges that have also been addressed through these techniques as shown in the works from [23–26]. For example, in [24] the authors added the pharmacokinetics of the drug in the OC problem in order to provide chemotherapeutic protocols in qualitative terms. The injected drug concentration is used as the control variable and the minimization of the number of tumor cells at the end of the treatment is defined as the cost function of the problem. The results showed that the best strategy corresponds to the maximum rate of drug injection when growth rate is assumed to be constant, but not in other type of models.

Combination of different active compounds, are the rule rather than the exception in oncology and other therapeutic areas, but the potentially high number of different possible combinations (including different dosing schemes), makes drug selection an unaffordable task from an experimental trial and error perspective. Therefore any guide on how to administer these therapies to achieve the best possible responses is of great potential as shown by [27, 28].

In the area of infectious diseases, the works from [29] and [30] showed that using treatment regimens obtained from optimal control could lead to a substantial improvement in HIV patients outcome in comparison to the administration of constant-dose standard regimens. Anesthesia is another medical area where optimal control strategies are used to maintain patient response within the desired therapeutic window for the case of the Bispectral Index (BIS) as an indicator of sedation, or the degree of neuromuscular blockade [31, 32].

Despite the approach shown in the current evaluation is not novel in the drug delivery arena, it has been seldom used beyond optimizing drug exposure. Especially in the context of multi-objective optimization, the current example and others presented below indicate that the optimal control approach should be considered part of the computational modeling arsenal advocated by the FDA promoted critical pathway initiative under the Model Informed Drug Discovery and Development (MID3) paradigm [33].

Here we focused on the non-trivial problem of simultaneously achieving multiple therapeutic goals related to drug onset and offset in the context of clinical trials with a minimum duration of 9 months, which implies high cost and uncertainty regarding the final response outcome. Therefore the possibility of providing to pharmaceutical technology scientists guidance in the form of release/input (absorption) profiles represents a real added value to avoid failed clinical studies. In this context we performed a reverse engineering exercise interpreting the empirical input profiles from a mechanistic biopharmaceutic perspective and showing the practical application of our optimal control analysis, encouraging cooperation between computational and experimental/technology scientists.

In the current exercise the majority of the system accounting for the relationship between dose and response (drug disposition, receptor interaction, and down-regulation mechanisms) was already well characterized with the corresponding typical values and associated variability reported in [3]. In addition the subcutaneous route of administration represents a much simpler biological system compared for example with the oral route. Therefore the optimal control approach used here represents an appropriate choice even recognizing that in more complex situations the advantages offered by similar approaches like non-linear model predictive control [13, 34] could represent a better alternative.

In this context, the main critical aspect of the analysis is the choice of the appropriate structure of the cost function to be minimized and the constraints of the problem. As highlighted in Materials and methods section, we divided the problem into two phases because the minimum time to achieve CT values (t_cast) in TST levels was not known in advance. For the first phase, we implemented the Mayer form of optimization problems whereas for the second phase a Lagrange term was used (see Table 1). In this work, we focused on the resulting testosterone levels of the prostate cancer patients, but, as shown in the above paragraphs of the Discussion, for other therapeutic areas different objectives could have been established, like the minimization of the tumor cell population at the end of treatment, the maximization of the number of healthy immune system cells or the penalization of excessive application of therapeutic agents [35–37].

The concept of optimization is present at every stage of the drug development process. Optimal design methods, based on the D-optimality criteria which relies on the maximization of the determinant of the Fisher information matrix [38, 39] is becoming also popular to select the appropriate number of subjects in each cohort of the trials, the sampling times and the number of dose levels [40]. However, we must not confuse optimal design methods with optimal control techniques. The aim of the first is to simplify population trials but maintaining the same efficiency as the original studies. There, they do not alter the system equations nor the objective function of the algorithms and the focus is to search for similar results to the original study (identical PKPD parameters, similar concentration vs time profiles…). For example, in the work from [40], they used this method to optimize a population pharmacodynamic experiment of the effect of ivabradine on exercise-induced tachycardia. On the other side, in optimal control, we introduce what are known as control variables into the model equations in order to manipulate the system response towards the desired goal. Therefore, in this type of problems, system equations can be modified and objective functions and constraints defined to search for improved solutions compared to the ones of the original study. Still, both approaches have something in common; they avoid the use of intensive computer simulations when the optimal solutions of a problem are being explored.

Conclusion

Optimal control theory has been applied to a population pharmacokinetic/pharmacodynamic model to derive the optimal drug release profiles to achieve multiple therapeutic goals. The optimal control analysis is more relevant in physiological systems with complex dynamics where simple simulation tuning parameters exercises are not effective to obtain the optimal profiles. Moreover, the flexibility of the method allows to deal with multiple and tight therapeutic objectives performing real optimization. In this context the question of how to define the objective functions and how to quantify our therapeutic goals becomes crucial. Here, we focused on the resulting testosterone levels of the patients, however, within the oncology area, different therapeutic objectives can be established with the goal of improving drug combinations, help to lessen the side effects of cancer treatments, etc.

Finally, the optimal release characteristics have been described based on standard absorption PK models. Although there are some discrepancies between the resulting TST profiles from the optimal control strategy and the modeling approach (see Results), we note that the important aspect of this work was to find the optimal release characteristics for prostate cancer patients, not to perform an ideal PKPD modeling exercise as there was not real data to fit. We conclude that this objective is achieved and that the information summarized in this article could be very useful for the development of new formulations, since it provides insight into the desired absorption characteristics and could produce a broad benefit for future prostate cancer patients.

Supporting information

S1 Fig. Original pharmacokinetic model of triptorelin.

Original pharmacokinetic model of triptorelin from [3]. D_inf is the duration of the zero-order absorption process; K_A1 and K_A2 are the first order rate constants of the first and second depot compartments respectively; F₁, F₂ and F_inf represent the fraction of the drug associated with the first and second depot compartments and the zero-order absorption process respectively; t_lag1/t_lag2 is the lag time associated to the first/second absorption compartment; C_TRP is the serum concentrations of triptorelin; CL, the apparent total clearance; V_c, V_T1, and V_T2, apparent volumes of distribution of the central, shallow, and deep peripheral compartments respectively; and CL_D1 and CL_D2, distribution clearances between the central and peripheral compartments.

https://doi.org/10.1371/journal.pcbi.1006087.s001

(TIF)

S1 Text. Mathematics behind optimal control methods.

Summary of the mathematics involving necessary and sufficient conditions for optimality in dynamic systems.

https://doi.org/10.1371/journal.pcbi.1006087.s002

(PDF)

S1 Data. APMonitor MATLAB code and datasets.

APMonitor Optimization Suite code for MATLAB and datasets to reproduce the results of this work.

https://doi.org/10.1371/journal.pcbi.1006087.s003

(RAR)

References

1. Oefelein MG, Resnick MI. Effective testosterone suppression for patients with prostate cancer: is there a best castration? Urology. 2003;62(2):207–213. pmid:12893320
- View Article
- PubMed/NCBI
- Google Scholar
2. Scher HI, Halabi S, Tannock I, Morris M, Sternberg CN, Carducci MA, et al. Design and End Points of Clinical Trials for Patients With Progressive Prostate Cancer and Castrate Levels of Testosterone: Recommendations of the Prostate Cancer Clinical Trials Working Group. J Clin Oncol. 2008;26(7):1148–1159. pmid:18309951
- View Article
- PubMed/NCBI
- Google Scholar
3. Romero E, Vélez de Mendizabal N, Cendrós JM, Peraire C, Bascompta E, Obach R, et al. Pharmacokinetic/pharmacodynamic model of the testosterone effects of triptorelin administered in sustained release formulations in patients with prostate cancer. J Pharmacol Exp Ther. 2012;342(3):788–798. pmid:22691297
- View Article
- PubMed/NCBI
- Google Scholar
4. Schättler H, Ledzewicz U. Optimal Control for Mathematical Models of Cancer Therapies: An Application of Geometric Methods. Interdisciplinary Applied Mathematics. Springer New York; 2015.
5. Bauer R. NONMEM users guide introduction to NONMEM 7.2. 0. ICON Development Solutions Ellicott City, MD. 2011;.
6. Stengel RF. Optimal Control and Estimation. Courier Corporation; 2012.
7. Cesari L. Lagrange and Bolza Problems of Optimal Control and Other Problems. In: Optimization—Theory and Applications. Applications of Mathematics. Springer, New York, NY; 1983. p. 196–205.
8. Parkinson AR, Balling R, Hedengren JD. Optimization Methods for Engineering Design. Brigham Young University. 2013;5.
- View Article
- Google Scholar
9. Oefelein MG, Feng A, Scolieri MJ, Ricchiutti D, Resnick MI. Reassessment of the definition of castrate levels of testosterone: implications for clinical decision making. Urology. 2000;56(6):1021–1024. pmid:11113751
- View Article
- PubMed/NCBI
- Google Scholar
10. Shi J, Alagoz O, Erenay FS, Su Q. A survey of optimization models on cancer chemotherapy treatment planning. Ann Oper Res. 2014;221(1):331–356.
- View Article
- Google Scholar
11. Drag P, Styczeń K, Kwiatkowska M, Szczurek A. A Review on the Direct and Indirect Methods for Solving Optimal Control Problems with Differential-Algebraic Constraints. In: Recent Advances in Computational Optimization. Studies in Computational Intelligence. Springer, Cham; 2016. p. 91–105.
12. Wächter A, Biegler LT. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math Program. 2005;106(1):25–57.
- View Article
- Google Scholar
13. Hedengren JD, Shishavan RA, Powell KM, Edgar TF. Nonlinear modeling, estimation and predictive control in APMonitor. Comput Chem Eng. 2014;70:133–148.
- View Article
- Google Scholar
14. Bryson AE. Applied Optimal Control: Optimization, Estimation and Control. CRC Press; 1975.
15. Swan GW. Role of optimal control theory in cancer chemotherapy. Math Biosci. 1990;101(2):237–284. pmid:2134485
- View Article
- PubMed/NCBI
- Google Scholar
16. Tornøe CW, Agersø H, Senderovitz T, Nielsen HA, Madsen H, Karlsson MO, et al. Population pharmacokinetic/pharmacodynamic (PK/PD) modelling of the hypothalamic–pituitary–gonadal axis following treatment with GnRH analogues. Br J Clin Pharmacol. 2007;63(6):648–664. pmid:17096678
- View Article
- PubMed/NCBI
- Google Scholar
17. Buil-Bruna N, Garrido MJ, Dehez M, Manon A, Nguyen TXQ, Gomez-Panzani EL, et al. Population Pharmacokinetic Analysis of Lanreotide Autogel/Depot in the Treatment of Neuroendocrine Tumors: Pooled Analysis of Four Clinical Trials. Clin Pharmacokinet. 2016;55(4):461–473. pmid:26416534
- View Article
- PubMed/NCBI
- Google Scholar
18. Longuski JM, Guzmán JJ, Prussing JE. Optimal Control with Aerospace Applications; 2014.
19. Sethi SP, Thompson GL. Optimal Control Theory: Applications to Management Science and Economics. Springer Science & Business Media; 2006.
20. Black F, Scholes M. The Pricing of Options and Corporate Liabilities. J Polit Econ. 1973;81(3):637–654.
- View Article
- Google Scholar
21. Martin R, Teo KL. Optimal Control of Drug Administration in Cancer Chemotherapy. World Scientific; 1994.
22. Swan GW, Vincent TL. Optimal control analysis in the chemotherapy of IgG multiple myeloma. Bull Math Biol. 1977;39(3):317–337.
- View Article
- Google Scholar
23. Martin RB, Fisher ME, Minchin RF, Teo KL. Optimal control of tumor size used to maximize survival time when cells are resistant to chemotherapy. Math Biosci. 1992;110(2):201–219.
- View Article
- Google Scholar
24. Costa MI, Boldrini JL, Bassanezi RC. Drug kinetics and drug resistance in optimal chemotherapy. Math Biosci. 1995;125(2):191–209. pmid:7881194
- View Article
- PubMed/NCBI
- Google Scholar
25. Coldman AJ, Murray JM. Optimal control for a stochastic model of cancer chemotherapy. Math Biosci. 2000;168(2):187–200. pmid:11121565
- View Article
- PubMed/NCBI
- Google Scholar
26. Kimmel M, Swierniak A. Control Theory Approach to Cancer Chemotherapy: Benefiting from Phase Dependence and Overcoming Drug Resistance. In: Tutorials in Mathematical Biosciences III. Lecture Notes in Mathematics. Springer, Berlin, Heidelberg; 2006. p. 185–221.
27. de Pillis LG, Fister KR, Gu W, Head T, Maples K, Neal T, et al. Optimal Control of Mixed Immunotherapy and Chemotherapy of Tumors. J Biol Syst. 2008;16(01):51–80.
- View Article
- Google Scholar
28. Ledzewicz U, Moore H. Dynamical Systems Properties of a Mathematical Model for the Treatment of CML. NATO Adv Sci Inst Ser E Appl Sci. 2016;6(10):291.
- View Article
- Google Scholar
29. Gu W, Moore H. Optimal therapy regimens for treatment-resistant mutations of HIV; 2006.
30. Kirschner D, Lenhart S, Serbin S. Optimal control of the chemotherapy of HIV. J Math Biol. 1997;35(7):775–792. pmid:9269736
- View Article
- PubMed/NCBI
- Google Scholar
31. Almeida J, Mendonça T, Rocha P. A simplified control scheme for the Depth of Anesthesia. IFAC-PapersOnLine. 2016;49(5):230–235.
- View Article
- Google Scholar
32. Almeida J, Mendonça T, Rocha P. A Simplified Control Approach for the Neuromuscular Blockade Level. In: CONTROLO 2016. Lecture Notes in Electrical Engineering. Springer, Cham; 2017. p. 37–44.
33. Woodcock J, Woosley R. The FDA Critical Path Initiative and Its Influence on New Drug Development. Annu Rev Med. 2008;59(1):1–12. pmid:18186700
- View Article
- PubMed/NCBI
- Google Scholar
34. Grüne L, Pannek J. Nonlinear Model Predictive Control. In: Nonlinear Model Predictive Control. Communications and Control Engineering. Springer, London; 2011. p. 43–66.
35. Stengel RF, Ghigliazza RM, Kulkarni NV. Optimal enhancement of immune response. Bioinformatics. 2002;18(9):1227–1235. pmid:12217914
- View Article
- PubMed/NCBI
- Google Scholar
36. Engelhart M, Lebiedz D, Sager S. Optimal control for selected cancer chemotherapy ODE models: a view on the potential of optimal schedules and choice of objective function. Math Biosci. 2011;229(1):123–134. pmid:21129386
- View Article
- PubMed/NCBI
- Google Scholar
37. He Q, Zhu J, Dingli D, Foo J, Leder KZ. Optimized Treatment Schedules for Chronic Myeloid Leukemia. PLoS Comput Biol. 2016;12(10):e1005129. pmid:27764087
- View Article
- PubMed/NCBI
- Google Scholar
38. Fedorov VV. Theory of optimal experiments. Elsevier; 1972.
39. Mentré F, Mallet A, Baccar D. Optimal design in random-effects regression models. Biometrika. 1997;84(2):429–442.
- View Article
- Google Scholar
40. Duffull SB, Mentré F, Aarons L. Optimal design of a population pharmacodynamic experiment for ivabradine. Pharm Res. 2001;18(1):83–89. pmid:11336357
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Oefelein MG, Resnick MI. Effective testosterone suppression for patients with prostate cancer: is there a best castration? Urology. 2003;62(2):207–213. pmid:12893320
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Scher HI, Halabi S, Tannock I, Morris M, Sternberg CN, Carducci MA, et al. Design and End Points of Clinical Trials for Patients With Progressive Prostate Cancer and Castrate Levels of Testosterone: Recommendations of the Prostate Cancer Clinical Trials Working Group. J Clin Oncol. 2008;26(7):1148–1159. pmid:18309951
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Romero E, Vélez de Mendizabal N, Cendrós JM, Peraire C, Bascompta E, Obach R, et al. Pharmacokinetic/pharmacodynamic model of the testosterone effects of triptorelin administered in sustained release formulations in patients with prostate cancer. J Pharmacol Exp Ther. 2012;342(3):788–798. pmid:22691297
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Schättler H, Ledzewicz U. Optimal Control for Mathematical Models of Cancer Therapies: An Application of Geometric Methods. Interdisciplinary Applied Mathematics. Springer New York; 2015.

[ref5] 5. Bauer R. NONMEM users guide introduction to NONMEM 7.2. 0. ICON Development Solutions Ellicott City, MD. 2011;.

[ref6] 6. Stengel RF. Optimal Control and Estimation. Courier Corporation; 2012.

[ref7] 7. Cesari L. Lagrange and Bolza Problems of Optimal Control and Other Problems. In: Optimization—Theory and Applications. Applications of Mathematics. Springer, New York, NY; 1983. p. 196–205.

[ref8] 8. Parkinson AR, Balling R, Hedengren JD. Optimization Methods for Engineering Design. Brigham Young University. 2013;5.
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref9] 9. Oefelein MG, Feng A, Scolieri MJ, Ricchiutti D, Resnick MI. Reassessment of the definition of castrate levels of testosterone: implications for clinical decision making. Urology. 2000;56(6):1021–1024. pmid:11113751
View Article
PubMed/NCBI
Google Scholar

[21] View Article

[22] PubMed/NCBI

[23] Google Scholar

[ref10] 10. Shi J, Alagoz O, Erenay FS, Su Q. A survey of optimization models on cancer chemotherapy treatment planning. Ann Oper Res. 2014;221(1):331–356.
View Article
Google Scholar

[25] View Article

[26] Google Scholar

[ref11] 11. Drag P, Styczeń K, Kwiatkowska M, Szczurek A. A Review on the Direct and Indirect Methods for Solving Optimal Control Problems with Differential-Algebraic Constraints. In: Recent Advances in Computational Optimization. Studies in Computational Intelligence. Springer, Cham; 2016. p. 91–105.

[ref12] 12. Wächter A, Biegler LT. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math Program. 2005;106(1):25–57.
View Article
Google Scholar

[29] View Article

[30] Google Scholar

[ref13] 13. Hedengren JD, Shishavan RA, Powell KM, Edgar TF. Nonlinear modeling, estimation and predictive control in APMonitor. Comput Chem Eng. 2014;70:133–148.
View Article
Google Scholar

[32] View Article

[33] Google Scholar

[ref14] 14. Bryson AE. Applied Optimal Control: Optimization, Estimation and Control. CRC Press; 1975.

[ref15] 15. Swan GW. Role of optimal control theory in cancer chemotherapy. Math Biosci. 1990;101(2):237–284. pmid:2134485
View Article
PubMed/NCBI
Google Scholar

[36] View Article

[37] PubMed/NCBI

[38] Google Scholar

[ref16] 16. Tornøe CW, Agersø H, Senderovitz T, Nielsen HA, Madsen H, Karlsson MO, et al. Population pharmacokinetic/pharmacodynamic (PK/PD) modelling of the hypothalamic–pituitary–gonadal axis following treatment with GnRH analogues. Br J Clin Pharmacol. 2007;63(6):648–664. pmid:17096678
View Article
PubMed/NCBI
Google Scholar

[40] View Article

[41] PubMed/NCBI

[42] Google Scholar

[ref17] 17. Buil-Bruna N, Garrido MJ, Dehez M, Manon A, Nguyen TXQ, Gomez-Panzani EL, et al. Population Pharmacokinetic Analysis of Lanreotide Autogel/Depot in the Treatment of Neuroendocrine Tumors: Pooled Analysis of Four Clinical Trials. Clin Pharmacokinet. 2016;55(4):461–473. pmid:26416534
View Article
PubMed/NCBI
Google Scholar

[44] View Article

[45] PubMed/NCBI

[46] Google Scholar

[ref18] 18. Longuski JM, Guzmán JJ, Prussing JE. Optimal Control with Aerospace Applications; 2014.

[ref19] 19. Sethi SP, Thompson GL. Optimal Control Theory: Applications to Management Science and Economics. Springer Science & Business Media; 2006.

[ref20] 20. Black F, Scholes M. The Pricing of Options and Corporate Liabilities. J Polit Econ. 1973;81(3):637–654.
View Article
Google Scholar

[50] View Article

[51] Google Scholar

[ref21] 21. Martin R, Teo KL. Optimal Control of Drug Administration in Cancer Chemotherapy. World Scientific; 1994.

[ref22] 22. Swan GW, Vincent TL. Optimal control analysis in the chemotherapy of IgG multiple myeloma. Bull Math Biol. 1977;39(3):317–337.
View Article
Google Scholar

[54] View Article

[55] Google Scholar

[ref23] 23. Martin RB, Fisher ME, Minchin RF, Teo KL. Optimal control of tumor size used to maximize survival time when cells are resistant to chemotherapy. Math Biosci. 1992;110(2):201–219.
View Article
Google Scholar

[57] View Article

[58] Google Scholar

[ref24] 24. Costa MI, Boldrini JL, Bassanezi RC. Drug kinetics and drug resistance in optimal chemotherapy. Math Biosci. 1995;125(2):191–209. pmid:7881194
View Article
PubMed/NCBI
Google Scholar

[60] View Article

[61] PubMed/NCBI

[62] Google Scholar

[ref25] 25. Coldman AJ, Murray JM. Optimal control for a stochastic model of cancer chemotherapy. Math Biosci. 2000;168(2):187–200. pmid:11121565
View Article
PubMed/NCBI
Google Scholar

[64] View Article

[65] PubMed/NCBI

[66] Google Scholar

[ref26] 26. Kimmel M, Swierniak A. Control Theory Approach to Cancer Chemotherapy: Benefiting from Phase Dependence and Overcoming Drug Resistance. In: Tutorials in Mathematical Biosciences III. Lecture Notes in Mathematics. Springer, Berlin, Heidelberg; 2006. p. 185–221.

[ref27] 27. de Pillis LG, Fister KR, Gu W, Head T, Maples K, Neal T, et al. Optimal Control of Mixed Immunotherapy and Chemotherapy of Tumors. J Biol Syst. 2008;16(01):51–80.
View Article
Google Scholar

[69] View Article

[70] Google Scholar

[ref28] 28. Ledzewicz U, Moore H. Dynamical Systems Properties of a Mathematical Model for the Treatment of CML. NATO Adv Sci Inst Ser E Appl Sci. 2016;6(10):291.
View Article
Google Scholar

[72] View Article

[73] Google Scholar

[ref29] 29. Gu W, Moore H. Optimal therapy regimens for treatment-resistant mutations of HIV; 2006.

[ref30] 30. Kirschner D, Lenhart S, Serbin S. Optimal control of the chemotherapy of HIV. J Math Biol. 1997;35(7):775–792. pmid:9269736
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref31] 31. Almeida J, Mendonça T, Rocha P. A simplified control scheme for the Depth of Anesthesia. IFAC-PapersOnLine. 2016;49(5):230–235.
View Article
Google Scholar

[80] View Article

[81] Google Scholar

[ref32] 32. Almeida J, Mendonça T, Rocha P. A Simplified Control Approach for the Neuromuscular Blockade Level. In: CONTROLO 2016. Lecture Notes in Electrical Engineering. Springer, Cham; 2017. p. 37–44.

[ref33] 33. Woodcock J, Woosley R. The FDA Critical Path Initiative and Its Influence on New Drug Development. Annu Rev Med. 2008;59(1):1–12. pmid:18186700
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref34] 34. Grüne L, Pannek J. Nonlinear Model Predictive Control. In: Nonlinear Model Predictive Control. Communications and Control Engineering. Springer, London; 2011. p. 43–66.

[ref35] 35. Stengel RF, Ghigliazza RM, Kulkarni NV. Optimal enhancement of immune response. Bioinformatics. 2002;18(9):1227–1235. pmid:12217914
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

[ref36] 36. Engelhart M, Lebiedz D, Sager S. Optimal control for selected cancer chemotherapy ODE models: a view on the potential of optimal schedules and choice of objective function. Math Biosci. 2011;229(1):123–134. pmid:21129386
View Article
PubMed/NCBI
Google Scholar

[93] View Article

[94] PubMed/NCBI

[95] Google Scholar

[ref37] 37. He Q, Zhu J, Dingli D, Foo J, Leder KZ. Optimized Treatment Schedules for Chronic Myeloid Leukemia. PLoS Comput Biol. 2016;12(10):e1005129. pmid:27764087
View Article
PubMed/NCBI
Google Scholar

[97] View Article

[98] PubMed/NCBI

[99] Google Scholar

[ref38] 38. Fedorov VV. Theory of optimal experiments. Elsevier; 1972.

[ref39] 39. Mentré F, Mallet A, Baccar D. Optimal design in random-effects regression models. Biometrika. 1997;84(2):429–442.
View Article
Google Scholar

[102] View Article

[103] Google Scholar

[ref40] 40. Duffull SB, Mentré F, Aarons L. Optimal design of a population pharmacodynamic experiment for ivabradine. Pharm Res. 2001;18(1):83–89. pmid:11336357
View Article
PubMed/NCBI
Google Scholar

[105] View Article

[106] PubMed/NCBI

[107] Google Scholar

Figures

Abstract

Author summary

Introduction

Materials and methods

Generation of a virtual patient population

Optimal control

Mechanistic characterization of the optimal absorption profiles

Results

Optimal pharmacodynamic profiles

Optimal release characteristics

Discussion

Conclusion

Supporting information

S1 Fig. Original pharmacokinetic model of triptorelin.

S1 Text. Mathematics behind optimal control methods.

S1 Data. APMonitor MATLAB code and datasets.

References