^{1}

^{2}

^{3}

^{*}

Conceived and designed the experiments: ABZ SDV GB. Performed the experiments: ABZ GB. Analyzed the data: ABZ GB. Contributed reagents/materials/analysis tools: ABZ GB. Wrote the paper: ABZ SDV GB.

The authors have declared that no competing interests exist.

The hypothalamic-pituitary-adrenal (HPA) axis is a major system maintaining body homeostasis by regulating the neuroendocrine and sympathetic nervous systems as well modulating immune function. Recent work has shown that the complex dynamics of this system accommodate several stable steady states, one of which corresponds to the hypocortisol state observed in patients with chronic fatigue syndrome (CFS). At present these dynamics are not formally considered in the development of treatment strategies. Here we use model-based predictive control (MPC) methodology to estimate robust treatment courses for displacing the HPA axis from an abnormal hypocortisol steady state back to a healthy cortisol level. This approach was applied to a recent model of HPA axis dynamics incorporating glucocorticoid receptor kinetics. A candidate treatment that displays robust properties in the face of significant biological variability and measurement uncertainty requires that cortisol be further suppressed for a short period until adrenocorticotropic hormone levels exceed 30% of baseline. Treatment may then be discontinued, and the HPA axis will naturally progress to a stable attractor defined by normal hormone levels. Suppression of biologically available cortisol may be achieved through the use of binding proteins such as CBG and certain metabolizing enzymes, thus offering possible avenues for deployment in a clinical setting. Treatment strategies can therefore be designed that maximally exploit system dynamics to provide a robust response to treatment and ensure a positive outcome over a wide range of conditions. Perhaps most importantly, a treatment course involving further reduction in cortisol, even transient, is quite counterintuitive and challenges the conventional strategy of supplementing cortisol levels, an approach based on steady-state reasoning.

The hypothalamic-pituitary-adrenal (HPA) axis is one of the body's major control systems helping to regulate functions ranging from digestion to immune response to metabolism. Dysregulation of the HPA axis is associated with a number of neuroimmune disorders including chronic fatigue syndrome (CFS), depression, Gulf War illness (GWI), and posttraumatic stress disorder (PTSD). Objective diagnosis and targeted treatments of these disorders have proven challenging because they present no obvious lesion. However, the body's various components do not work in isolation, and it is important to consider exactly how their interactions might be altered by disease. Using a relatively simple mathematical description of the HPA axis, we show how the complex dynamical behavior of this system will readily accommodate multiple stable resting states, some of which may correspond to chronic loss of function. We propose that a well-directed push given at the right moment may encourage the axis to reset under its own volition. We use model-based predictive control theory to compute such a push. The result is counterintuitive and challenges the conventional time-invariant approach to disease and therapy. Indeed we demonstrate that in some cases it might be possible to exploit the natural dynamics of these physiological systems to stimulate recovery.

The hypothalamic-pituitary-adrenal (HPA) axis constitutes one of the major peripheral outflow systems of the brain, serving to maintain body homeostasis by adapting the organism to changes in the external and internal environments. It does this by regulating the neuroendocrine and sympathetic nervous systems as well modulating immune function

Many aspects of the organization and function of the HPA axis have been characterized in clinical and laboratory studies revealing a number of component feedback and feed forward signaling processes. Stress activates the release of corticotropin-releasing hormone (CRH) from the paraventricular nucleus (PVN) of the hypothalamus. The release of CRH into the hypophysial-portal circulation in turn acts in conjunction with arginine vasopressin on CRH-R1 receptors of the anterior pituitary stimulating the rapid release of adrenocorticotropic hormone (ACTH). ACTH then is released into the peripheral circulation and stimulates the release of the glucocorticoid cortisol from the adrenal cortex by acting on the receptor MC2-R (type 2 melanocortin receptor). Cortisol enters the cell and binds to the glucocorticoid receptor present in the cytoplasm of every nucleated cell; hence the widespread effects of glucocorticoids on practically every system of the body including endocrine, nervous, cardiovascular and immune systems.

To keep HPA axis activity in check, glucocorticoids also exert negative feedback at the hypothalamus and pituitary glands to inhibit the synthesis and secretion of CRH and ACTH, respectively. Moreover, glucocorticoid negative feedback causes a reduction in corticotroph receptor expression leading to a desensitization of the pituitary to the stimulatory effects of CRH on ACTH release. This negative feedback is also felt in the hippocampus where it exerts a negative influence on the PVN. A detailed review of the physiology and biochemistry of the HPA axis as well as it's know interactions with the immune system may be found in work by Silverman et al.

A number of chronic diseases have been characterized by abnormalities in HPA axis regulation. These include major depression and its subtypes, anxiety disorders such as post-traumatic stress disorder, panic disorder and cognitive disorders such as Alzheimer's disease and minimal cognitive impairment of aging

Though much is known about its components, one of the main difficulties in studying the behavior of the HPA axis has been in integrating the expansive body of published experimental information. Numerical models provide an ideal framework for such integration. Simple models of the HPA axis have been constructed using deterministic coupled ordinary differential equations

In this work we adopt the model proposed by Gupta et al.

A model of the HPA axis which includes glucocorticoid receptor and the dynamics of glucocorticoid receptor-cortisol interactions have been proposed by Gupta et al.

The system states are given as x = [x_{1}; x_{2}; x_{3}; x_{4}]^{T} and are described in _{i1}; k_{cd}; k_{ad}; k_{i2}; k_{cr}; k_{rd}; k]^{T}. Nominal values for the system parameters are listed in

State | Description | Stable Rest Points |

x_{1} |
CRH concentration | (0.6261, 0.6610) |

x_{2} |
ACTH concentration | (0.0597, 0.0513) |

x_{3} |
Free GR concentration | (0.0809, 0.5629) |

x_{4} |
Cortisol concentration | (0.0597, 0.0513) |

Parameter | Description | Value |

k_{i1} |
Inhibition constant for CRH synthesis | 0.100 |

k_{cd} |
CRH degradation constant | 1.000 |

k_{i2} |
Inhibition constant for ACTH synthesis | 0.100 |

k_{ad} |
ACTH degradation constant | 10.000 |

k_{cr} |
GR synthesis constant | 0.050 |

k_{rd} |
GR degradation constant | 0.900 |

k | Inhibition constant for GR synthesis | 0.001 |

In this first analysis the HPA axis system is considered under idealized conditions where all parameters are assumed constant and precisely known. In addition, the states x are assumed known as a function of time with no measurement error and the control action is implemented perfectly. The approach taken for choosing an optimal control is based on the Model Predictive Control (MPC) framework

In this work it is assumed that the variable to be manipulated for treatment is the rate of addition or removal of cortisol from circulation. To model this control action, System _{4}) (Eq. 2). Note that System

To avoid dangerous destabilization of the HPA axis by the application of control action _{2}) and cortisol (x_{4}) levels even though we purposely manipulate circulating cortisol to perturb the system _{u}

Where t_{0} and t_{f} are the start and end time of the optimization horizon, λ is a tuning parameter taking values from zero to one and x_{2}^{*} and x_{4}^{*} are the healthy steady-state concentrations of ACTH and cortisol, respectively. _{2} and x_{4} are the only measured states.

The resulting cost function can be written as:

The parameter λ is used to penalize excessive imbalance of the other hormones (x_{1}, x_{2}, x_{3}) in response to the control action applied to cortisol (x_{4}). In this case, the objective of the controller is to bring the cortisol concentration to set point while minimizing the impact of the treatment on the other three states of the HPA axis. Any change in CRH (x_{1}) or the glucocorticoid receptor (GR; x_{3}) will be reflected in the concentration of ACTH (x_{2}) by virtue of the coupled dynamics described by System H_{u}. The tuning parameter λ can be selected to match the intensity of the desired treatment. A λ value of near zero will lead to a more intense treatment while a value of λ near one will lead to very conservative treatment. For proof of concept, a more direct treatment was favored in this work and a λ value of 0.01 was used throughout. Note that x_{2}^{*} and x_{4}^{*} correspond to the stable steady state of the unperturbed system (i.e., when

Typically, a treatment or control action is applied at discrete intervals. As a result, the objective function in Equation 3 was optimized with respect to a piece-wise constant input signal _{4}(u(t))_{c} of all piecewise constant functions on

The initial condition, x(t = t_{0}) is the steady state of the unperturbed system with d_{0} = 0. The optimal control,

The steady-state solutions for HPA axis model described above as System _{1}; x_{2}; x_{3}; x_{4}}. Under this framework, the disturbance variable, d, is assumed to take on a constant value

The above is a set of polynomials in x, with real coefficients, and maximum total degree of five. Equations 5 to 8 can be simplified using the theory of polynomial ideals

Therein f_{3} is a polynomial in x_{3} of degree seven, and f_{1}, f_{2} and f_{4} are functions only of x_{3} and d_{0}. The functions f_{1} to f_{4} can be computed using a symbolic algebra package such as Maple. For the nominal parameter values proposed in Gupta et al. _{3} and these correspond to the roots of f_{3}. Each root is a steady-state value for x_{3} and can be used to generate the corresponding values of x_{1}, x_{2} and x_{4} given Equations 10 to 12. Note that at steady state x_{2} = x_{4} (Eq. 8). A plot of the steady-state values of x_{1}, x_{2} and x_{3} as a function of d_{0} is shown in

Steady-state concentration of CRH (x_{1}), ACTH (x_{2}), GR (x_{3}) and cortisol (x4) as a function of the external stressor

In this model of HPA axis dynamics a chronically stressed individual would occupy the stable steady state associated with a depressed cortisol concentration (∼0.05) at rest or at d_{0} = 0. If a healthy person were subjected to extreme stress (i.e., d_{0}>0.168) for an extended period of time their body would reach the only steady state available locally that is one corresponding to chronic stress. In other words, for values of d_{0} greater than 0.168, Equation (9) dictates that there is only one steady-state solution for free GR (x_{3}) concentration as opposed to the 3 solutions available for 0≤d_{0}<0.168. By virtue of Equation (12) this results in only one steady-state solution being available for cortisol (x_{4}) for d_{0}>0.168. When the stress is removed (i.e., d_{0} = 0), the body will stay at this new depressed steady-state value of cortisol concentration. This process is shown graphically in

Concentration of circulating cortisol plotted as a function of the external stressor

As one might expect the assumptions of ideal control do not correspond to a physically realizable system. However, the analysis of the system under idealized conditions allows the study of possible treatments. Any practical treatment would then be a suboptimal solution as compared to the treatment under idealized conditions. This allows proposed treatments to be benchmarked and compared. In addition, the solution obtained under idealized conditions can serve as a qualitative guideline for the creation of a practical, although suboptimal, treatment. In engineering terms the objective of treatment is to succeed in bring the subject to the healthy steady-state target while exerting the smallest disturbance possible to the HPA axis. For example, even though we intend to manipulate circulating cortisol concentration it should not be allowed to decrease excessively because of the important role cortisol plays in regulating a number of cellular and physiological functions. To avoid such excess perturbations the concentration of ACTH has been included in the objective function of Equation 3.This concentration is more readily measured than that of either CRH or GR making ACTH a good candidate for monitoring the progress of a treatment.

The optimal control solution that minimizes disruption of HPA axis function (Eq. 3) is shown in

Concentrations of CRH (x_{1}), ACTH (x_{2}), GR (x_{3}) and cortisol (x4) as a function of time in response to an ideal externally applied perturbation in cortisol

In this section a suboptimal control strategy is proposed for the HPA axis system. The goal of this strategy is to mimic the qualitative results of the MPC solution while being realizable in a clinical setting. The MPC solution suggests that manipulating cortisol concentration is a plausible strategy for redirecting the HPA axis to a healthy steady state. The key difficulty in applying this approach is determining when the cortisol concentration has been sufficiently lowered with regard to the other state variables to allow the system to return to a healthy equilibrium. That is, one must identify an observable event (corresponding to a measurable variable) which signals that the steady state of the system has shifted. In a clinical setting only ACTH and cortisol concentrations, corresponding to x_{2} and x_{4}, respectively, can be readily measured. The availability of cortisol analogues makes it possible to manipulate x_{4} directly. Therefore as postulated previously (Eq. 3–4) ACTH (x_{2}) can be used to determine when a change in available steady state or attractor has occurred. Under the MPC framework, most of the control action is expended near the initial time. In _{2}) are both plotted as a function of time. The value of x_{2} increases by about 30% as the system moves from the cusp of multiple candidate steady states to the basin of a single steady state. The following treatment is therefore proposed:

_{2}) have increased by more than 30% relative to the initial condition. Once this signal is observed, the system's own natural feedback control action should restore cortisol levels to normal.

Simulation results for Treatment 1 are shown in

Concentrations of CRH (x_{1}), ACTH (x_{2}), GR (x_{3}) and cortisol (x4) as a function of time in response to a suboptimal but more realistic externally applied perturbation in cortisol

Diagram of the minimal perturbation in normalized circulating cortisol

The results for Treatment 1 shown in _{0}) is examined in this section. A direct computational evaluation of robustness of Treatment 1 is difficult to implement. There are four initial conditions (x_{1}(0); x_{2}(0); x_{3}(0); x_{4}(0)), seven parameters, and one disturbance variable (d_{0}). A simulation study where each variable (initial condition, parameter and disturbance) is evaluated at a nominal, high and low values, would require, at a minimum 3^{12} = 531,441 simulations. Even if these simulations were completed, the choice of high, low and nominal value for each variable would be difficult to justify using available data. An alternative approach analyzing robustness analysis is to study the asymptotic behavior of System _{u}_{4}) be manipulated so that the product of cortisiol and GR concentrations (x_{3}x_{4}) is constant. Under these conditions, the asymptotic value of glucocorticoid receptor concentration GR (x_{3}) is obtained from Eq. 7 as:

The asymptotic value or GR concentration x_{3}∞ has a minimum as a function of cortisol concentration (x_{4}) at x_{4} = 0. That is, if one were to lower the cortisol concentration to zero one would obtain the lowest possible steady-state value for GR and this value would be:

At the steady-state point given by x_{4}∞ = 0 and x_{3}∞ = k_{cr}/k_{rd} the unique asymptotic solution for CRH (x_{1}) and ACTH (x_{2}) is given by

It should be noted that the values for GR, CRH and ACTH identified in Equations 14–15 represent an asymptotic minimum for the externally controlled system (System _{u}_{4}→0 and that this state corresponds to a stable set of non-zero real-valued concentrations of CRH, ACTH and GR. This result confirms that reducing the cortisol concentration to a small enough positive value can indeed take the system to a single stable condition. This is regardless of the value of d_{0}, parameters, or initial conditions. This condition will correspond to a healthy equilibrium value when treatment is administered in the absence of elevated levels of external stress d_{0}. At high levels of the stressor d_{0} the success of the treatment would be short lived as we would simply be immediately re-administering the same insult originally responsible for the illness state. This is true regardless of whether the idealized or the suboptimal treatment approach is used.

Patients with CFS have been found to exhibit decreased adrenal response to ACTH stimulation and lower daily cortisol levels in plasma, urine and saliva

Cortisol output of the HPA axis can in reality be manipulated either directly or indirectly through several interventions. The most direct approaches involve (1) inhibition of cortisol synthesis at the level of the adrenal gland or (2) inhibition of CRH induced ACTH synthesis by the pituitary. Inhibitors of cortisol synthesis include pharmaceutical agents such as ketoconazole that have been used in limited human trials

Indirect approaches to cortisol suppression focus on modulation of the biochemical feedback returning to the higher HPA axis from the immune system and the adrenal gland. Inflammatory events exert a positive immune system feedback to the HPA axis that is conducted via a number of pro-inflammatory cytokines for which several components of the HPA axis have receptors. Supported by immune, epidemiological and small-scale gene expression data

Unfortunately as in strategies involving the direct inhibition of CRH, a reduction of positive feedback to the hypothalamus also leads to a reduction in ACTH synthesis by the pituitary. Recall that the proposed treatment requires the inhibition of the negative cortisol feedback without the removal of positive stimulation of ACTH production. This could be achieved by temporarily reducing the bioavailability of cortisol itself. Binding proteins and metabolizing enzymes have been identified for cortisol. Corticosteroid-binding globulin (CBG) regulates the concentration of free or active cortisol

It should be noted that although the model of HPA axis dynamics used in this work is currently the most credible model, it remains in many ways incomplete. For example, there is mounting data including observations of moderate hypocortisolism in depressed patients undergoing IFN-α therapy suggesting that GR receptor function not only affects the release of cytokines but is itself affected by these same cytokines

It is important to note however that while the specific treatment solution identified using MPC is model-dependent the general MPC framework is not. Therefore as more detailed models become available these can easily be exploited to improve a treatment course. Putting aside issues of model fidelity and completeness, the proposed MPC framework could still be exploited in a two-step treatment approach. In a first step data obtained from a standard dexamethasone test could serve to calibrate a simple lumped-parameter model capturing the overall HPA dynamics for a given subject. The calibrated model could then be used within the proposed MPC framework to estimate the most appropriate combination of dosage and duration of treatment for that same patient. Ultimately even if a given model is not entirely correct our robustness analysis shows that the desired outcome may be obtained reliably over a wide range of parameter values. This will be true as long as the structure of the model is valid.

In conclusion we have demonstrated in this work the use of model-based predictive control methodology in the estimation of robust treatment courses for displacing the HPA axis from an abnormal hypocortisol steady state back to a normal function. Using this approach on a numerical model of the HPA axis proposed by Gupta et al.

Special thanks to the Dr. William C. Reeves and the staff of the Chronic Viral Diseases Branch at the Centers for Disease Control and Prevention for many helpful discussions.