Optimization of light exposure and sleep schedule for circadian rhythm entrainment

The circadian rhythm, called Process C, regulates a wide range of biological processes in humans including sleep, metabolism, body temperature, and hormone secretion. Light is the dominant synchronizer of the circadian rhythm—it has been used to regulate the circadian phase to cope with jet-lag, shift work, and sleep disorder. The homeostatic oscillation of the sleep drive is called Process S. Process C and Process S together determine the sleep-wake cycle in what is known as the two-process model. This paper addresses the regulation of both Process C and Process S by scheduling light exposure and sleep based on numerical simulations of circadian rhythm and sleep mathematical models. This is a significant step beyond the existing literature that only considers the entrainment of Process C. Regulation of the two-process model poses several unique features and challenges: 1. Process S is non-smooth, i.e., the homeostatic dynamics are different in the sleep and wake regimes; 2. Light only indirectly affects Process S through Process C; 3. Light does not affect Process C during sleep. We consider two scenarios: optimizing light intensity as the control input with spontaneous (i.e., unscheduled) sleep/wake times and jointly optimizing the light intensity and the sleep/wake times, which allows limited delayed sleep and early waking as part of the decision variables. We solve the time-optimal entrainment problem for the two-process model for both scenarios using an extension of the gradient descent algorithm to non-smooth systems. To illustrate the efficacy of our time-optimal entrainment strategies, we consider two common use cases: transmeridian travelers and shift workers. For transmeridian travelers, joint optimization of the two-process model avoids the unrealistic long wake duration when only Process C is considered. The entrainment time also decreases when both the light input and the sleep schedule are optimized compared to when only the light input is optimized. For shift workers, we show that the entrainment time is significantly shortened by optimizing the night shift working light.


Introduction
Sleep plays a significant role in our lives as it allows the body to recuperate and regenerate cells. It is tied to neurobehavioral performance: sleep deprivation and disorder have been empirically linked to degeneration of neurocognitive function [1,2] and disruption of the circadian rhythm [3,4]. The sleep-wake cycle is modeled with two separate processes: Process S models the sleep drive generated by the accumulation of sleep-inducing substances in the brain and Process C represents the sleep pressure generated by circadian rhythm [5,6]. Process S may be modeled by the exponential growth and decay of the sleep drive during the waking and sleep periods [7], based on the similar pattern of the EEG power. Light (in the blue spectrum) is the dominant influence on Process C, though other factors such as exercise and meals also play a role. Empirical nonlinear forced oscillator models have been proposed for Process C based on the oscillation of circadian rhythm markers such as the core body temperature [8][9][10]. Experimental results in [6,11] have demonstrated that the spontaneous sleep/wake times are jointly determined by Process S and Process C.
As light is the dominant synchronizer of the circadian rhythm, it has been used for regulating the circadian rhythm against issues such as jet-lag, shift work, and sleep disorder. Such a problem may be posed as a minimum-time optimal control problem: finding the lighting schedule to drive the circadian rhythm to a reference pattern in the shortest time possible, subject to the light intensity constraints. In [12], the problem is solved with a switching light input (bang-off control) for the nonlinear oscillator model in [10]. The time points between the light-on and light-off intervals are updated with a gradient descent algorithm. In [13], an optimal light control strategy has been proposed for the Drosophila circadian phase alignment using a reduced order phase dynamics model. Such formulation has been extended to phase synchronization of complex oscillator networks [14]. In our earlier work [15], we have used the Pontryagin Minimum Principle to efficiently compute the optimal light input for both the nonlinear oscillator model and phase reduced model.
In this paper, we solve a larger problem beyond the current literature of light-based circadian rhythm entrainment: how to entrain both Process C and Process S using the light input, and possibly the sleep scheduling. We first consider the spontaneous sleep/wake case, with the light intensity as the input. We then include the sleep/wake times as additional input variables. The combined circadian rhythm and sleep entrainment problem poses several challenges: 1. Process S is non-smooth, as the homeostatic dynamics depend on the sleep state; 2. Light input only indirectly affects sleep, through circadian rhythm; 3. Light input is decoupled from the system during the sleep phase. To address these challenges, we apply optimal control for a hybrid dynamical system that contains two modes: sleep and wake. In [16], the problem is solved using optimization of the mode-switching points. We apply variational calculus to evaluate the co-state discontinuity around the switching times and use the gradient descent algorithm to determine the optimal light intensity. In [17], we report our initial results for the spontaneous sleep/wake case. Here we include sleep scheduling as additional input variables. As in our work on Process C regulation [15], several reduced order models are also considered to understand the impact of model reduction on the optimal entrainment time. To reduce the dependence on model parameters, we use the optimal control solution to train a feedback controller that adjusts the light intensity and sleep schedule based on the states of Process C and Process S. To our knowledge, our work is the first optimal control results involving the combined Process C and Process S dynamics.
To illustrate the application of the optimal entrainment algorithms, we consider two common but challenging cases: jet-lag adjustment for travelers and recovery of shift workers. For transmeridian travelers with only Process C considered, the optimal lighting requires excessive waking time (longer than 20 hours in some cases) which leads to suppressed amplitude of the circadian oscillation [12,15]. When Process S is included in the optimization, such long waking times and amplitude suppression are avoided. The optimal Process C entrainment also exhibits an east-west symmetry, i.e., the entrainment times are almost identical for the same amount of delay (going west) or advance (going east) [12,15]. This is no longer true with the introduction of Process S. The optimal solution for the combined processes shows a preference towards delay, i.e., longer entrainment times traveling eastward. This observation agrees previously reported results in [18], where the authors consider entrainment of Process C under periodic light cycles, and in [19], where the authors consider the entrainment of a network of the brain's Suprachiasmatic Nuclei (SCN) also under periodic light cycles. Note that in both forementioned papers, the authors do not consider optimal entrainment or the sleep-wake process. Further, the addition of sleep scheduling significantly reduces the impact of jet-lag, e.g., for the 12-hour shift case, the entrainment time is decreased by 27% by optimizing both light and sleep schedule. For night shift workers, the optimal control solution shows that optimization of light input and wearing goggles to block part of the blue spectrum during shift work result in significantly lower entrainment time to return to the regular schedule.

Two-process modeling approach
Circadian rhythm regulation is performed under the dynamics of the circadian rhythm with the sleep homeostatic process. These two processes together form a model of the sleep-wake cycle, called the two-process model. The mathematical circadian rhythm model used in this paper is the Jewett-Forger-Kronauer (JFK) model in [12], which is a 3rd-order nonlinear differential equation and formulated based on the dynamics of the core body temperature. We will also discuss a 2nd-order circadian rhythm model and a 1st-order phase-reduced model to show the impacts of model reduction on the entrainment time and optimal solution. We model the sleep homeostatic process and sleep-wake cycle using Achermann's two-process model [20]. Note that the circadian rhythm of the two-process model in [20] has no dynamics; it is represented as a pre-determined skewed sine wave that ignores the lighting impact. Therefore, this model has not been studied for light-based minimum-time entrainment. The JFK model in [12] has been updated from previous models to fit the latest empirical data and incorporate the lighting impacts on core body temperature. The minimum-time entrainment problem of this model has been studied in [12,15]. However, this model, as well as the corresponding entrainment studies, ignores the sleep-wake cycle and sleep homeostatic process. As shown in Table 1, we formulate a two-process model in this paper by combining the JFK model with the homeostatic process and study the minimum-time entrainment based on this model.
We note that there have been earlier work that provided more comprehensive models of the sleep-wake dynamic, e.g., those that consider rapid eye movement (REM) and non-rapid eye movement (NREM) sleep stages. Diniz Behn and Booth [21] proposed a two-time scale model consisting of fast synaptic-based interactions between neuronal populations and slow homeostatic and circadian processes for the dynamics of REM sleep. Later, they used the same mathematical tool (fast-slow decomposition) to analyze several physiologically-based mathematical models of sleep-wake and sleep stage regulation [22]. In this paper, we primarily focus on the daily dynamics of the sleep-wake process and the entrainment of these circadian and sleep-wake rhythms. Therefore, we choose to use the two-process model for our study.
2.1 Mathematical models for Process C 2.1.1 Full (third-order) JFK circadian rhythm model. Based on the whole process through which light stimuli work on the hypothalamus and affect the core body temperature (CBT) [9], the JFK circadian rhythm model in [12] is separated into two parts: the first part is a light processor that models the transduction process on the retina, which transfers light energy into neuron stimulus. This part is called Process L and its dynamics are expressed as a 1storder ODE, given as Eq (1a)-(1c): where the parameter values are set as α 0 = 0.05 h −1 , γ = 0.0075 h −1 , I 0 = 9500 lux, G = 33.75, p = 0.5. The unit of time t in this paper is hour (h). Light intensity is denoted by I in lux. The variable u represents the circadian drive that works on the hypothalamus. The terms n and 1 − n represent the fractions of retinal cells in used state and ready state, respectively. Used state means the retinal cells have been stimulated by light and emitted action potentials, while ready state means the retinal cells are ready to accept the light signal and generate a neuron stimulus. The second part of the model simulates the whole process where the neuron stimulus works on the hypothalamus and affects the running of the core body temperature, which is called Process P in the JFK model. This part is expressed as the 2nd-order nonlinear differential equation: Here x(t) is the state of core body temperature, x c (t) is a complementary state with a unit of h −1 . The circadian drive u(t) is the input of the core body temperature oscillator, downstream from Process L. The values of the parameters in this part are given as We should note that the JFK model was formulated to simulate the dynamics of core body temperature under white light [8,9]. Previous studies suggest that the melatonin secretion suppression [24,25], circadian gene expression [26], sleepiness, and alertness [27] demonstrate spectral sensitivity to light, i.e., they are more sensitive to short-wavelength light. We represent the circadian spectral sensitivity to light wavelength as R(λ), where λ is the light wavelength. If experimental data about light spectral sensitivity of humans' core body temperature are available in the future, we can formulate the term R(λ) and incorporate it into the JFK model by rewriting the Eq (1b) in Process L as where I(λ) represents the spectral density of the light intensity. The experimental studies in [28,29] demonstrated that the melatonin, cortisol, circadian gene (per2 and Bmal1 mRNA), and subjective alertness of subjects staying under the filtered fluorescent light (filtering short wavelength spectrum less than 480 nm) are similar to those of subjects staying under darkness. Therefore, R(λ) � 0 for λ > 480 nm.
In the subsequent subsections, we review two reduced-order versions of the full JFK model. The purpose of including these models is to evaluate whether optimal entrainment strategies that are computed for these models can be effectively used on the full-order model.
2.1.2 Second-order circadian rhythm model. We observe from the Process L model in Eq (1a)-(1c) that when the light is off, I(t) = 0 and dn dt < 0, the used state transfers to the ready state and n gradually converges to 0. When the light is on, the light input drives the ready state to used state gradually. The dynamics of n reach an equilibrium if α(I)(1 − n) = γn, u(t) and n(t) also reach steady states with values given as The dynamics of the state n in Process L have a fast time scale compared to that of the circadian oscillation. We assume that Process L is always at its steady state, and the values of n and u are given as (3) when the light is on.   [30]. This function defines the effect of a light pulse on a circadian rhythm and reduces a high-order model into a one-dimensional phase-based model. In this paper, we define the circadian phase θ based on Process P states as where θ is expressed in rad. The dynamics of the circadian phase are given as where ω 0 is the so-called free-running frequency, expressed in rad/h. The state and phase without light input (u � 0) are called the free-running state and free-running phase, respectively. The free-running period of the core body temperature oscillator is about 24.2 hours [9]. Therefore, we set ω 0 = 2π/24.2 rad/h. The function f(θ) is the phase response function or phase response curve (PRC) of the circadian rhythm model. Four different methods for the measurement of a phase response function were summarized in [31]. In this paper, the phase response curve f(θ) is estimated by applying a 30-min light pulse with intensity u = 0.1731 (corresponding to I = 1000 lux) on the 2nd-order model in Section 2.1.2 and calculating the difference between the circadian phase having been stimulated by the light pulse and the free-running phase [15].

Mathematical model for Process S
The sleep homeostasis H(t) represents the accumulation of a substance that generates the sleep drive. It works like an internal timer that describes the tendency to fall asleep if the subject is awake and the tendency to wake up if the subject is asleep. The dynamics of H(t) during sleep and wake cycles are modeled respectively as The variable β(t) 2 {0, 1} indicates whether the subject is asleep (β(t) = 1) or awake (β(t) = 0) at time t. Eq (6) implies H(t) ! 0 as t ! 1 when β � 1 and H(t)!1 as t ! 1 when β � 0. The parameters τ r = 18.2 h and τ d = 4.2 h define the time scales of these dynamics, which are estimated based on experimental data from a study with 8 healthy human subjects [32]. These parameters are within the standard deviation of the parameters reported in a more recent study in [33].

Two-process model
Sleepiness, denoted as B(t), quantitatively represents a subject's tendency to fall asleep. It is affected by both the sleep homeostasis and the circadian state. The work in [34] defined the value of sleepiness by the JFK circadian rhythm model in Section 2.1.1 and sleep homeostasis as and showed that the values of sleepiness predicted by the model were similar to the empirical data, where A c = 0.1333. If Process C is represented by the 2nd-order model in Section 2.1.2, the sleepiness value is still defined by Eq (7). When we use the phase reduced model in Section 2.1.3 to represent Process C, based on the approximation that the periodic orbit of the JFK model is very close to a unit circle [15], we have x � cos ðyÞ: The sleepiness value of the circadian phase-sleep two-process model is given as This model can be further generalized by replacing A c cos(θ) with other 2π-periodic functions of θ [34]. Based on the model presented in [20], if a subject follows the spontaneous sleep schedule, the sleep state β(t) is completely determined by the value of B and the sleep state in the previous time. When B is equal to an upper threshold of H m = 0.67, the subject feels tired enough to sleep. The subject wakes up spontaneously when B reaches a lower threshold of L m = 0.17. The spontaneous wake time T wake and sleep time T sleep are defined as The discrete dynamics of the sleep state based on sleepiness are demonstrated in Fig 2 and explicitly expressed as where β(t − ) = lim τ!t − β(τ), t − represents the time just before t and y represents all continuous states in the two-process model. To incorporate the coupling impact of sleep state on the light and circadian rhythm, we replace the light-dependent term in Eq (1b) with This equation implies that when β(t) = 1, we have α(I) = 0 and u(t) = 0. Thus, light has no impact on the Process L and the whole Process C when the subject is asleep.
In the following sections of this paper, the three two-process models are called S+C3 model, S+C2 model, and S+C1 model, respectively, with details listed in Table 2.

Entrainment strategies and problem formulation
When a periodic light input is used on the two-process model with the spontaneous sleep schedule, the two-process state converges to a stable trajectory with the same period. This phenomenon is called entrainment. Phase misalignment happens when the circadian and sleep rhythms of a subject deviate from the entrained rhythms corresponding to the ambient lightdark cycle, for example, in the case of jet-lag. In this paper, we look at three different reentrainment strategies: open-loop entrainment with spontaneous sleep, minimum-time entrainment with spontaneous sleep, and minimum-time entrainment with controllable sleep.

Open-loop entrainment with spontaneous sleep schedule
As mentioned in Section 2.3, we denote the continuous state in the two-process model as y. For the S+C3 model, the state is expressed as y = [n, x, x c , H] T . Specifically, we consider a 24-hour periodic light input as: which approximates the natural light-dark cycle on earth. The sunrise time t = 0 corresponds to 6 am and daily light is present between 6 am and 10 pm. Note that the natural light-dark duty cycle constantly changes depending on the latitude. We use the light pattern in Eq (11) as a simple approximation of the daily light pattern. The optimization methodology that we present in this paper does not depend on the particular reference light pattern. In terms of Eqs (1c) and (3), when I = 1000 lux, the corresponding steady value of u is 0.1731. For S+C2 and S+C1 models, the corresponding 24-hour periodic light input is represented as The stable periodic solution of Eqs (1-2b), (6) with the spontaneous sleep schedule in Eq (9) and the periodic light input in Eq (11) is the entrained state for the S+C3 model, represented circadian rhythm synchronized with the periodic light input in Eq (11). The corresponding periodic sleep state is represented as β REF (t). In this paper, the reference circadian rhythm that the entraining subject wants to reach is defined as where Δ init represents the difference between the reference time at the beginning of entrainment (t 0 = 0) and 6 am. The existence of a periodic solution for the two-process model has been previously studied, e.g., in [35], where the authors presented a one dimensional function that maps the timing of sleep onset on one day to the timing of sleep onset on the following day. The fixed points of such function represent periodic solutions. In [23], the authors constructed a similar map for a more detailed sleep-wake regulation model that includes REM and NREM sleep stages.
One straightforward approach for entrainment is applying the spontaneous sleep schedule and reference light-dark cycle on the entraining subject directly, where the reference lightdark cycle is expressed as As the light input is just a function of time, we call this entrainment the open-loop entrainment. The entrainment process is finished once some terminal conditions for each model are given in Table 3, where the final tolerance tol is a small positive scalar, t f is the final entrainment time, and k�k 2 represents the Euclidean norm. The state n is ignored in the terminal condition of the S+C3 model as it has a very fast time scale. The value of tol used in this paper will be discussed in S1 Appendix in details. The entrainment time of the open-loop entrainment is treated as a baseline for comparison.

Minimum-time entrainment with spontaneous sleep schedule
In this case, the subject still falls asleep and wakes up spontaneously during the entrainment process. We want to find the optimal light input I � (t) 2 [0, I max ] ≜ O I to drive y(t) to reach the terminal condition of entrainment in minimum time, where I max is the maximum light intensity we can use during the entrainment process and O I represents the feasible set of the light input. Given the initial condition y(0) = y 0 , the state equation the sleep dynamics in Eq (9), and the reference state in (13), the minimum-time entrainment Table 3. Entrainment formulation for two-process models.

Model
Continuous state y State equation where I(t) is the optimization variable, the state y, state Eq (15), and terminal conditions of the three two-process models are listed explicitly in Table 3. We denote the minimizing I(t) as I � (t). (A detailed solution algorithm of the minimum-time entrainment problem with spontaneous sleep schedule is provided in the supplemental S1 Appendix).

Minimum-time entrainment with controllable sleep schedule
Here we consider an alternative case. The subject could be woken up by an alarm clock or other means before waking up spontaneously, and when the subject begins to feel sleepy, he may remain awake. Thus, the value of B(t) at sleep and wake times could be higher than the thresholds H m and L m . The dynamics of the sleep state are then given as where T i sleep and T j wake represent the i-th sleep time and j-th wake time. The parameters N f and M f are total times of falling asleep and waking up during entrainment. Although the sleep schedule is controllable, we assume that it still should follow some constraints during entrainment: the entraining subject can only fall asleep when the value of sleepiness is high enough, i.e., BðT i sleep Þ � 0:67, and cannot remain sleeping if the sleepiness is less than 0.17. The values of BðT i sleep Þ and BðT j wake Þ should also not be too high. Higher BðT i sleep Þ means the subject falls asleep later and higher BðT j wake Þ means the subject wakes up earlier, both of which could lead to sleep deprivation. Thus, the constraints for BðT i sleep Þ and BðT j wake Þ are given as shown as Fig 3. In this case, we assume that the subject should remain awake for no longer than 18 hours every day. Hereafter, the upper bound for sleepiness at the sleep time is chosen as B 1max = 0.77, which is close to the value of sleepiness when the subject has stayed awake for about 2 hours after B = 0.67. The upper bound for sleepiness at the wake time is set as B 2max = 0.27, which corresponds to the sleepiness about 2 hours before B = 0.17. For this entrainment strategy, we want to find not only the light input but also the sleep schedule to entrain the circadian rhythm in minimum time. Given the initial condition y(0) = y 0 , the state equation in Eq (15), sleep dynamics in Eq (16), and reference state in Eq (13), the minimum-time entrainment problem with controllable sleep schedule is written as follows: subject to the light constraint IðtÞ 2 ½0; I max �;

Application scenarios of circadian rhythm entrainment optimization
To demonstrate the application of our solution strategies (in S1 Appendix) in circadian rhythm entrainment optimization, we solve several entrainment cases of transmeridian travelers and shift workers in this section.

Entrainment of transmeridian travelers
In the case of transmeridian travelers, the reference state that travelers want to reach is given as y ref (t) = Y REF (t + Δ init ) with Δ init = 1, which means the entrainment process begins at 7 am at the destination. The initial state of travelers is set as y(0) = y ref (Δ shift ), where Δ shift 2 [0, 24] represents the time shift between home and the destination (home in advance/westward travel). These results imply that during the minimum-time entrainment process, the circadian rhythm and sleep schedule are delayed by delaying the sleep times but keeping the spontaneous wake times, while advanced by advancing the wake times but keeping the spontaneous sleep times. These phenomena are explained by the phase response curve and reference sleep state shown in Fig 6: In the periodic spontaneous sleep-wake cycle, the daily wake period starts from 7:30 am to 11:10 pm. The phase response value is positive in the time region between 7:30 am and 5 pm, and negative between 5 pm and 11:10 pm. The subject tends to fall asleep in the phase-delay region and wake up in the phase-advance part. Fig 6 also shows the scheduled sleep periods by delaying the sleep time (left) and advancing the wake time (right), respectively. Delaying the sleep time increases the phase-delay region and decreases the phase-advance region in the wake period while advancing the wake time results in a larger phase-advance region in the wake period compared with the spontaneous wake period. Therefore, using optimized sleep/wake times with optimal light input accelerates the entrainment processes and takes shorter entrainment time than only using optimal light input with the spontaneous sleep schedule. Fig 7. During the interval between 6 am and 7:30 am, the subject is sleeping

Fig 5. Sleepiness comparison of the minimum-time entrainment with spontaneous sleep (blue curves) and with optimal (controllable) sleep (black curves).
In this figure, we only plot the sleepiness value during the entrainment process (i.e., before reaching the terminal condition in Eq (14)). The black curves terminate earlier than the blue curves, emphasizing that the entrainment times with optimal sleep are shorter than those with spontaneous sleep. https://doi.org/10.1371/journal.pone.0251478.g005

PLOS ONE
Optimization of light exposure and sleep schedule for circadian rhythm entrainment but the light is present. Therefore, the actual light-dark cycle that the reference subject receives is 14.5-9.5-hour. The periodic solution of the full-order JFK model under the 14.5-9.5-hour light-dark cycle without Process S and sleep-wake cycle is plotted in the lower panel in Fig 7, which is equal to the reference value [x ref (t), x cref (t)]. The minimum-time entrainment problem on the JFK model (Process C) without Process S and sleep has been discussed in [12,15]. We treat the periodic solution in the lower panel in Fig 7 as the reference trajectory and calculate the optimal light input for the minimum-time entrainment of JFK circadian rhythm model by the method in [15]. The minimum entrainment time is plotted as the green dashed lines in Fig 4, where the maximum entrainment time in minimum-time entrainment cases occurs at around 12-hour shift. However, the maximum entrainment time of the S+C3 model in minimum-time entrainment cases occurs around 14-16 hours shift, shown on the blue and red curves in Fig 4. These results imply that after taking Process S and the sleep-wake cycle into account, the minimum-time circadian rhythm entrainment becomes asymmetrical, in which recovery from traveling eastward tends to be more time-consuming than from traveling westward.
Denote the optimal light in the case of only entraining Process C as I � C ðtÞ. We apply I � C ðtÞ on the S+C3 model by IðtÞ ¼ I � C ðtÞ. To guarantee that the subject receives all light input during the entrainment process, the sleep schedule is tuned to fit the optimal light schedule as follows: The left subfigures (a), (c), (e) and (g) in Fig 8 show four entrainment cases with IðtÞ ¼ I � C ðtÞ and the tuned sleep schedule in (18) for 8, 12, 16 hours shift with I max = 10000 lux and 12 hours shift with I max = 1000 lux. The amplitudes of the core body temperature oscillators are all quenched in these cases. This phenomenon is called the minimum path shifting in [12], which is more likely to happen in entrainment using bright light. The maximum value of sleepiness during the wake periods reaches up to 0.81 in these cases, and the wake periods are longer than 20 hours in some days during entrainment. For several days, the subject only sleeps for 4-5 hours. Short sleep durations, especially those less than 6 hours, have been associated with several health problems [36]. Even though the total entrainment time could be shorter, the optimal light of the JFK circadian rhythm model without consideration of Process S and the sleep-wake cycle is impractical for the entraining subject. The sleep constraints in (17) should be taken into account in entrainment solution strategies. We perform the solution algorithm in S1 Appendix for the S+C3 model and solve the optimal light and sleep schedule for the minimum-time entrainment of these cases, shown in right subfigures (b), (d), (f) and The minimum-time entrainment problem of the two simplified two-process models are studied, and the optimal light input and sleep schedule of these two models are represented as u 2nd (t), β 2nd (t), u 1st (t), β 1st (t), respectively. We adapt and apply these optimal solutions to the S+C3 model as follows As u 1st (t) and u 2nd ( simplified as a reference phase, defined as The feedback entrainment controller is defined in the form of (21), where the functions F wake and F sleep represent the feedback entrainment strategy during wake and sleep periods. The inputs of the feedback controller are entraining and reference phase, H(t) and current sleep state. The state n(t) is ignored in the feedback implementation as Process L is a transient process and has very small effects on the entrainment strategy. The function F wake outputs light I(t) and sleep schedule Γ sleep , which represents how much time the subject has before he should fall asleep; the function F sleep only outputs a wake-up schedule Γ wake , i.e., how many hours the subject has before he should wake up.
The procedure of formulating this feedback entrainment controller is listed as follows: Step  (21) are similar to the optimal light and sleep in the minimum-time entrainment of these cases. The feedback entrainment controller generated from the procedure we propose above functions robustly on entrainment cases with various initial conditions.

Entrainment of shift workers
Here we consider the entrainment problem of shift workers based on the S+C3 model. The experiments in [28,29] asked the shift workers to stay awake between 8 pm and 8 am in one night under a constant artificial light with intensity I shift . In Section 4.2.1, the entrainment processes of shift workers in [28] are simulated under three values of I shift : 0 lux, 100 lux (working under a dim light) and 1000 lux (working under a bright light with the same intensity as the reference daylight). Note that 0 lux light exposure can be achieved, practically, by wearing a pair of circadian-light blocking goggles to filter all light with wavelengths less than 480 nm that could suppress the melatonin secretion. The open-loop entrainment and minimum-time entrainment with the controllable sleep schedule (after night shifts) of shift workers are studied and discussed. In Section 4.2.2, we assume the light input during the night shift is also controllable. The optimal light input during and after the night shift and the sleep schedule after the night shift are calculated for minimum-time entrainment of the shift workers.  Fig 13. Working with goggles during the night shift results in the shortest entrainment time among these three night shift lighting conditions. Experimental results in [28,29] demonstrated that compared with the night shift in complete darkness or wearing the circadian-light blocking goggles, the night shift in bright light with 1000 lux suppressed the melatonin production and disturbed circadian rhythm more. The simulation results in this part indicate that using the goggles during the night shift shortens the entrainment time following the night shift. Note that even if the shift worker's Process C maintains synchrony with that of the reference at the end of the shift, his Process S will lose synchronization due to sleep deprivation. A recent work by Piltz et al [38,39] showed that recovery from sleep deprivation under a spontaneous schedule can take a long time.
The light input and sleep schedule after the night shift are optimized and the results of minimum-time entrainment with controllable sleep and various I shift and I max are plotted in   Fig 13 shows the reference subject gets the reference light from 8 pm-10 pm (phase-delay region with negative PRC value) and 8 am-12:30 pm (phase-advance region with positive PRC value) while the entraining shift worker keeps away from light input during these two intervals. The phase-advancing impact of the reference light from 8 am-12:30 pm (approximately the first wake time of shift workers) is slightly larger than the phase-delay impact of the reference light from 8 pm-10 pm. If the constant light intensity during the night shift from 8 pm to 8 am is I shift = 100 or 1000 lux, the light between 8 pm-8 am delays the phase of circadian state, drives the entraining state away from the reference state and, as a result, prolongs the entrainment process following the night shift. Thus the case with I shift = 0 lux is less time-consuming compared with the other two cases. While in the case with a night shift between 10 pm and 8 am in Fig 15, the reference light between 8 am and 12:30 pm on the first day after the night shift advances the reference circadian phase while the shift worker is sleeping during this time. To compensate for the loss of the phase-advancing light in this interval, the dim light I shift = 100 lux during 10 pm and 8 am is used to shorten the entrainment process. Increasing or decreasing I shift leads to growth in the entrainment time.
4.2.2 Minimum-time entrainment of shift workers with optimal light during night shift. As discussed in Section 4.2.1, the light during the night shift has significant impacts on the entrainment time. If the light during the night shift is also controllable, as illustrated in Fig 17, we optimize the light input both during and after the night shift as well as the sleep schedule after the night shift to minimize the entrainment time.
The optimal light input during the night shift from 8 pm to 8 am for minimum-time entrainment is given in Fig 18. In the cases with maximum light intensity I max = 1000 lux and I max = 10000 lux, the entrainment time is shortened to 3.28 hours (entrainment ends at around 11:17 am). In the minimum-time entrainment case with I max = 1000 lux, the shift worker remains under 1000 lux light without goggles for about 30 minutes towards the end of the night shift, while in the case with I max = 10000 lux, the shift worker remains under 10000 lux light without goggles for about 17 minutes. As mentioned in Section 4.2.1, the reference light from 8 pm to 10 pm and 8 am to 11:17 am slightly advances the reference circadian phase. Therefore, to compensate for the small phase-advance impact of the reference light on reference subject, the shift worker should get light input in a very short interval with positive PRC value during the minimum-time entrainment, as shown in Fig 18. The minimum-time entrainment results with optimal light during the night shift from 10 pm to 8 am are shown in Fig 19, in which the shift worker receives 1000 lux light for about 98 minutes or 10000 lux light

Discussion and conclusions
This paper solves the minimum-time entrainment problem on the two-process model that combines the circadian rhythm model (the JFK model) with the sleep process model. Using the calculus of variations and a gradient descent method, we determine the optimal light input and sleep schedule for minimum-time entrainment of transmeridian travelers and shift workers under different maximum entrainment light intensities, time shifts, and night shifts. Different from previous works, we take Process S and sleep-wake cycle into circadian rhythm entrainment and treat the controllable sleep schedule as an optimization variable.
The simulations of the entrainment cases of transmeridian travelers in this paper demonstrate that: (a) Compared with the open-loop entrainment by natural (reference) light, the optimal lighting strategy significantly decreases the entrainment time. The entrainment time is further shortened by optimizing both the light input and the sleep schedule; (b) The daily (optimized) sleep period is about 6-8 hours in the minimum-time entrainment results with the controllable sleep schedule. In contrast, the optimal entrainment results in [12,15] ignore Process S and the sleep-wake cycle, resulting in sleep periods of only 4-5 hours in several cases, which could result in excessive sleepiness and make the entrainment process impractical; (c) Applying the optimal light and sleep schedule from the simplified two-process models on the full two-process model directly is ineffective, resulting in longer entrainment times. However, we also propose a feedback controller that is learned from the optimal entrainment results of these simplified models, and demonstrate that it performs well and it is robust against the error resulting from model simplification in various entrainment cases.
The simulations of the entrainment cases of night shift workers in this paper demonstrate that: (a) The open-loop entrainment time of night shift workers wearing circadian-light blocking goggles during 8 pm and 8 am is shorter than that of workers in bright light. This result is related to the conclusion in [28] that wearing these goggles during night shifts helps maintain the circadian rhythm; (b) The entrainment time of shift workers may be shortened by optimizing the light input and sleep schedule after the night shift; (c)The lighting condition during the night shift also has significant impacts on the entrainment time. Appropriately setting the light intensity (or wearing goggles) during the night shift also shortens the entrainment time of shift workers.
The amplitude suppression in the core body temperature oscillator is closely connected with the minimum-time entrainment with bright light (with intensity higher than 1000 lux) in previous literature [12,15,40]. However, the phenomenon of CBT amplitude suppression may also be attributed to some health-related issues: the experimental results in [41] demonstrated the CBT oscillator tends to flatten with increasing age. The work in [42] showed that the quenched CBT amplitude is related to sleep disruption. Simulation results in Section 4.1.2 and Fig 8 in this paper demonstrate that the optimal light solution without consideration of sleep could lead to quenched CBT amplitude, excessive sleepiness (B(t) � 0.8), and short sleep duration (shorter than 5 hours), while the optimal light and sleep schedule from the two-process model avoid quenched amplitude and ensure the daily sleep duration is not shorter than 6 hours and maximum sleepiness (B(t)) is not larger than 0.77. These simulation results also indicate the quenched amplitude has a close connection with the sleep disruption during entrainment.
Real-time application of the results reported in this paper requires the ability to compute the optimal light and sleep schedules in real-time. To this end, we have developed a feedbackbased implementation of the optimal schedules, which does not require executing the functional gradient descent optimization in real-time. The feasibility of the feedback-based implementation is supported by two assumptions. First, we have the two-process model that accurately represents the subject. Second, the states of the two-process models can be measured or observed. Regarding the first assumption, we note that both the JFK circadian rhythm model and Achermann's two-process model used in this paper were formulated based on previous experimental data from human subjects. However, the sleeping habit and biological features vary between subjects and therefore mathematical models with fixed parameters may not match individual circadian rhythms. As for the second assumption, the states of the two-precess model are typically not directly observable. However, we envision designing state observers [43] that can provide state estimations based on some observables. These observables can be provided by wearable sensors, such as Actiwatch and portable core body temperature monitor [44], which have been developed to measure the circadian and sleep features. Further, they can also facilitate the efforts to personalize the models. The measurement data from these sensors may be used in model calibration and design of personal light and sleep schedule for entrainment.