Control Strategies for COVID-19 Epidemic with Vaccination, Shield Immunity and Quarantine: A Metric Temporal Logic Approach

Ever since the outbreak of the COVID-19 epidemic, various public health control strategies have been proposed and tested against the coronavirus SARS-CoV-2. We study three specific COVID-19 epidemic control models: the susceptible, exposed, infectious, recovered (SEIR) model with vaccination control; the SEIR model with shield immunity control; and the susceptible, un-quarantined infected, quarantined infected, confirmed infected (SUQC) model with quarantine control. We express the control requirement in metric temporal logic (MTL) formulas (a type of formal specification languages) which can specify the expected control outcomes such as"the deaths from the infection should never exceed one thousand per day within the next three months"or"the population immune from the disease should eventually exceed 200 thousand within the next 100 to 120 days". We then develop methods for synthesizing control strategies with MTL specifications. To the best of our knowledge, this is the first paper to systematically synthesize control strategies based on the COVID-19 epidemic models with formal specifications. We provide simulation results in three different case studies: vaccination control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; shield immunity control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; and quarantine control for the COVID-19 epidemic with model parameters estimated from data in Wuhan, China. The results show that the proposed synthesis approach can generate control inputs such that the time-varying numbers of individuals in each category (e.g., infectious, immune) satisfy the MTL specifications. The results also show that early intervention is essential in mitigating the spread of COVID-19, and more control effort is needed for more stringent MTL specifications.

Abstract-Ever since the outbreak of the COVID-19 epidemic, various public health control strategies have been proposed and tested against the coronavirus SARS-CoV-2. We study three specific COVID-19 epidemic control models: the susceptible, exposed, infectious, recovered (SEIR) model with vaccination control; the SEIR model with shield immunity control; and the susceptible, un-quarantined infected, quarantined infected, confirmed infected (SUQC) model with quarantine control. We express the control requirement in metric temporal logic (MTL) formulas (a type of formal specification languages) which can specify the expected control outcomes such as "the deaths from the infection should never exceed one thousand per day within the next three months" or "the population immune from the disease should eventually exceed 200 thousand within the next 100 to 120 days". We then develop methods for synthesizing control strategies with MTL specifications. To the best of our knowledge, this is the first paper to systematically synthesize control strategies based on the COVID-19 epidemic models with formal specifications. We provide simulation results in three different case studies: vaccination control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; shield immunity control for the COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; and quarantine control for the COVID-19 epidemic with model parameters estimated from data in Wuhan, China. The results show that the proposed synthesis approach can generate control inputs such that the time-varying numbers of individuals in each category (e.g., infectious, immune) satisfy the MTL specifications. The results also show that early intervention is essential in mitigating the spread of COVID-19, and more control effort is needed for more stringent MTL specifications. For example, based on the model in Lombardy, Italy, achieving less than 100 deaths per day and 10000 total deaths within 100 days requires 441.7% more vaccination control effort than achieving less than 1000 deaths per day and 50000 total deaths within 100 days.

I. INTRODUCTION
The COVID-19 pandemic [1] has caused over 20 million confirmed cases and over 0.74 million deaths globally as of August 12, 2020. Ever since the outbreak of COVID-19, various public health control strategies have been proposed and tested against the coronavirus SARS-CoV-2 [2].
Currently, over 90 vaccines are being developed against SARS-CoV-2 by research teams across the world [3]. Besides Zhe Xu and  vaccination, other strategies have also been proposed to control the spread of SARS-CoV-2. In [4], the authors proposed shield immunity to protect the susceptible people from getting infected with SARS-CoV-2. Specifically, shield immunity works by first identifying and deploying recovered individuals who have protective antibodies to SARS-CoV-2, and then increasing the proportion of interactions with recovered individuals as opposed to other individuals. In [5], the authors analyzed how quarantine has mitigated the spread of SARS-CoV-2 based on a model that differentiates quarantined infected individuals and un-quarantined infected individuals.
Despite the fact that various promising control strategies have been proposed against SARS-CoV-2, such control strategies still suffer from several limitations. (a) The control strategies against SARS-CoV-2 often treat the control inputs (e.g., the shield strength in shield immunity and the quarantine rate in quarantine control) as parameters that stay constant during one stage of time, while in reality such parameters may change on a daily basis with more fine-tuned control. (b) The control inputs in the literature are often tuned manually through trial-and-error instead of being synthesized systematically. (c) There is a lack of specific and formal specifications for the expected effects and outcomes of the control strategies.
To address these limitations, we propose a systematic control synthesis approach for three control strategies against SARS-CoV-2. We use metric temporal logic (MTL) formulas to specify the expected control outcomes such as "the deaths from the infection should never exceed one thousand per day within the next three months" or "the population immune from the disease should eventually exceed 200 thousand within the next 100 to 120 days". Such temporal logic formulas have been used as high-level knowledge or specifications in many applications in artificial intelligence [6], robotic control [7], power systems [8], etc.
The proposed control synthesis approach is based on three specific COVID-19 epidemic mitigation models: the susceptible, exposed, infectious, recovered (SEIR) model with vaccination control; the SEIR model with shield immunity control; and the susceptible, un-quarantined infected, quarantined infected, confirmed infected (SUQC) model with quarantine control. We develop methods for synthesizing control strategies based on the three specific COVID-19 epidemic models with MTL specifications. Specifically, we convert the synthesis problem into mixed-integer bi-linear programming or mixed-integer fractional constrained programming problems, and solve the optimization problems using highly efficient solvers [9].
We provide simulation results in three different case studies: vaccination control for COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; shield immunity control for COVID-19 epidemic with model parameters estimated from data in Lombardy, Italy; and quarantine control for COVID-19 epidemic with model parameters estimated from data in Wuhan, China. The proposed synthesis approach can generate control inputs such that the time-varying numbers of individuals in each category (e.g., infectious, immune) satisfy the MTL specifications.
Based on the simulation results, we observe that early control is essential in mitigating the spread of COVID-19, and more control effort is needed for more stringent MTL specifications. For example, based on the model in Lombardy, Italy, achieving less than 100 deaths per day and 10000 total deaths within 100 days requires 441.7% more vaccination control effort than achieving less than 1000 deaths per day and 50000 total deaths within 100 days. As the control inputs are generated on a daily basis, the proposed approach can be used to assist and provide quantitative guidelines in public health control strategies to achieve specific specifications for mitigating the spread of COVID-19.

II. RELATED WORK
COVID-19 epidemic modeling and control strategies: Ever since the outbreak of COVID-19, there has been numerous research focusing on the modeling of COVID-19 epidemic based on data collected from both the epicenters and other places [10], [11], [12], [13]. Among the various models, compartmental models such as SEIR and SUQC models have been used frequently for the analysis of COVID-19. There has also been work in analyzing or predicting the spread of COVID-19 using artificial intelligence models [14], stochastic intensity models [11], etc. The models we use in this paper are based on the SEIR (both the standard and with shield immunity) and SUQC models, but we have replaced some essential parameters (e.g., the shield strength in shield immunity, the quarantine rate in quarantine control) with control inputs which can be synthesized to vary on a daily basis. Optimal control of epidemic models: There exist work in designing vaccination control for the SEIR or SIR models of epidemics [15], [16]. However, such methods have not been applied in the setting of COVID-19. Besides, there has been no work in optimal control of epidemic models with formal specifications (e.g., expressed in temporal logic formulas). Control synthesis with temporal logic specifications: There are three main categories of approaches to designing controllers that meet temporal logic specifications [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27]. The first category of approaches abstract the system as a transition system and transform the control syntheses problem into a series of constrained reachability problems [28], [29], [30]. The second category of approaches mainly focus on linear dynamical systems and they convert the control synthesis problem into a mixed-integer linear programming (MILP) problem [31], [32], [33], [34], [35], [36] which can be solved efficiently by MILP solvers. The third category of approaches substitute the temporal logic constraint into the objective function of the optimization problem and apply a functional gradient descent algorithm on the resulting unconstrained problem [37], [38], [8], [39]. The control synthesis approach in this paper is based on the second category of approaches, but we have extended the method to non-linear dynamical systems to fit the epidemic models for COVID-19.

III. METRIC TEMPORAL LOGIC (MTL)
In this section, we briefly review metric temporal logic (MTL) [40] interpreted over discrete-time trajectories. The state x (e.g., representing the susceptible, exposed, infectious, recovered population of a certain region) belongs to the domain X ⊂ R n . The time set is T = R ≥0 . The domain B = {True, False} is the Boolean domain, and the time index x(t[k]) to denote the value of x at time t[k]. We use ξ to denote a trajectory as a function from T to X . A set AP is a set of atomic propositions, each mapping X to B. The syntax of MTL is defined recursively as follows: where stands for the Boolean constant True, π ∈ AP is an atomic proposition, ¬ (negation), ∧ (conjunction), ∨ (disjunction) are standard Boolean connectives, U is a temporal operator representing "until", I is a time index interval of the form We can also derive two useful temporal operators from "until" (U), which are "eventually" ♦ I ϕ = U I ϕ and "always" I ϕ = ¬♦ I ¬ϕ.
For example, the MTL formula [0,100] (DeathsP erDay ≤ 0.001) ∧ ♦ [40,60] (Recovered ≥ 6) means "the deaths from infection should never exceed 0.001 million (one thousand) per day within the next 100 days, and the immune population should eventually exceed 6 million after 40 to 60 days" (we assume that the unit in π is million and the unit in I is day in this paper, unless otherwise indicated).
We define the set of states that satisfy the atomic proposition π as O(π) ∈ X . We denote ϕ (ξ, k) = if the trajectory ξ satisfies the formula ϕ at discrete-time instants t[k] (k ∈ I). Then the Boolean semantics of MTL are defined recursively as follows [41]:

IV. COVID-19 MODELS WITH CONTROL STRATEGIES
In this section, we study three models for COVID-19 epidemic [12], [4], [5] and introduce the corresponding models with vaccination control, shield immunity control and quarantine control.

A. COVID-19 SEIR Model with Vaccination Control
The susceptible, exposed, infectious, recovered (SEIR) model has been frequently used in epidemic analyses. As shown in Fig. 1, the total population is divided into five subgroups: • The susceptible population S: everyone is susceptible to the disease by birth since immunity is not hereditary; • The exposed population E: the individuals who have been exposed to the disease, but are still not infectious; • The infectious population I: the individuals who are infectious; • The immune (recovered) population R: the individuals who are vaccinated or recovered from the disease, i.e., the population who are immune to the disease; • The dead population D: the dead individuals from the disease.
where the control input V is the number of vaccinated individuals per day, N = S + E + I + R ≤ N 0 is the total population in the region (N 0 is the initial total population in the region), S, E, I, R and D are the number of susceptible, exposed, infectious and recovered population in the region, respectively, and D is the number of deaths from SARS-CoV-2 in the region. For the parameters, λ denotes the percapita birth rate, µ is the per-capita natural death rate (death rate from causes unrelated to SARS-CoV-2), α is the SARS-CoV-2 virus-induced average fatality rate, β is the probability of disease transmission per contact (dimensionless) times the number of contacts per unit time, is the rate of progression from exposed to infectious (the reciprocal is the incubation period), and γ is the recovery rate of infectious individuals (the reciprocal is the infectious period). Note that in (1), holds as we have assumed that the birth rate and the natural death rate are the same for the population we are investigating, i.e., λ = µ.
Remark 1. Note that one difference between this model and the vaccination control model in [15] is that we control V as the number of vaccinated individuals per day (constrained to be less than the susceptible population S), while in [15] the control input is the ratio of the vaccinated individuals per day to the average born population per day. We found it more convenient this way for computational convenience in the control synthesis in later sections.

B. COVID-19 SEIR Model with Shield Immunity Control
Shield immunity is a strategy recently proposed in [4] to limit the transmission of SARS-CoV-2. The basic idea of this strategy is to increase the proportion of interactions with recovered individuals as opposed to the other individuals in the population. The effectiveness of this strategy is based on the assumption that recovered individuals (virus-negative and antibody-positive) can safely interact with both susceptible and infectious individuals without getting infected with the disease.
As the model used in [4] is modified from an SIR model, we consider a corresponding SEIR model with shield immunity control as follows (see Fig. 2 as an illustration).
where the states and parameters are the same as in (1), while χ(·) is the shield strength [4] as control input to be synthesized for the recovered population to substitute the contact for the susceptible population.

C. COVID-19 SUQC Model with Quarantine Control
The susceptible, un-quarantined infected, quarantined infected, confirmed infected (SUQC) model was recently pro- posed in [5] based on the COVID-19 data in Wuhan, China. As shown in Fig. 3, we consider four subgroups in the population: • The susceptible population S: everyone is susceptible to the disease by birth since immunity is not hereditary; • The un-quarantined infected population U : the individuals who are infected and un-quarantined, and they can be either asymptomatic or symptomatic; • The quarantined infected population Q: the individuals who are infectious and quarantined (the un-quarantined infected become quarantined infected by isolation or hospitalization, and the quarantined infected lose the ability of infecting the susceptible individuals); • The confirmed infected population C: the individuals who are confirmed to be infected with the disease (i.e., the positive cases). We consider the SUQC model with quarantine control as follows.Ṡ = −β 0 U S/N ; where q is the quarantine rate (for an un-quarantined infected to be quarantined) as control input to be synthesized, S, U , Q and C are the number of susceptible, un-quarantined infected, quarantined infected and confirmed infected population in the region, respectively, β 0 is the infection rate (i.e., the mean number of new infected caused by an un-quarantined infected per day), γ 2 is the confirmation rate of Q (i.e., the probability that the quarantined infected are identified to be confirmed cases through conventional methods such as laboratory diagnosis), σ is the subsequent confirmation rate of those infected that are not confirmed by the conventional methods, but confirmed with additional tests.

V. CONTROL SYNTHESIS OF COVID-19 EPIDEMIC WITH METRIC TEMPORAL LOGIC SPECIFICATIONS
In this section, we present the control synthesis methods for the three COVID-19 epidemic models in Section IV with vaccination control, shield immunity control and quarantine control, respectively.
Vaccination control: For the COVID-19 SEIR model with vaccination control, we discretize the model in (1) as follows.
where T s is the sampling period. We also use ∆D The vaccination control synthesis problem can be formulated as a constrained optimization problem as follows.
The above optimization problem is generally a mixedinteger non-linear programming problem. We refer the readers to [34] for a detailed description of how the constraint ϕ V (ξ V ·;x init V ,V , 0) = is encoded to satisfy an MTL specification ϕ V . The integer variables are introduced when a big-M formulation [42] is needed to satisfy MTL specifications such as ♦ [0,10] ϕ (ϕ should hold true for at least one day during the first 10 days) or ϕ 1 ∨ ϕ 2 (at least one of the MTL formulas ϕ 1 , ϕ 2 should hold true). As the change of total population is relatively small compared to the multiplication of the susceptible population and the infectious population, we approximate the term T s βS[k]I[k]/N [k] with T s βS[k]I[k]/N 0 . With such an approximation, the optimization problem becomes a mixed-integer bi-linear programming problem, which can be more efficiently solved using techniques such as McCormick's relaxation [43], [44]. Furthermore, if the MTL specification ϕ consists of only conjunctions (∧) and the always operator ( ), the integers in the optimization problem can be eliminated [34] and the problem becomes a bi-linear programming problem.
Shield immunity control: For the COVID-19 SEIR model with shield immunity control, we discretize the model in (2) as follows.
where T s is the sampling period.
Following the notations in Section III, we use x S = [I, E, S, R, D] to denote the state of (5)  Problem 2 (Shield immunity control). Given the SEIR model with shield immunity control in (5) and an MTL specification ϕ S , compute the control input χ[·] that minimizes the shield immunity control efforts χ[·] while satisfying ϕ S (ξ S ·;x init S ,χ , 0) = , i.e., the trajectory ξ S ·;x init S ,χ satisfies the MTL specification ϕ S .
The shield immunity control synthesis problem can be formulated as a constrained optimization problem as follows.
,χ , 0) = , where T ∈ I is the maximal time index we consider, and χ max is the maximal shield strength.
The above optimization problem is generally a mixedinteger fractional constrained programming problem. If the MTL specification ϕ consists of only conjunctions (∧) and the always operator ( ), the integers in the optimization problem can be eliminated [34] and the problem becomes a fractional constrained programming problem.
Quarantine control: For the COVID-19 SUQC model with quarantine control, we discretize the model in (3) as follows.
where T s is the sampling period. We also use ∆C The quarantine control synthesis problem can be formulated as a constrained optimization problem as follows.
where T ∈ I is the maximal time index we consider, and q max is the maximal quarantine rate. The above optimization problem is generally a mixedinteger non-linear programming problem. As the change of total population is relatively small compared to the multiplication of the susceptible population and the unquarantined infectious population, we approximate the term /N 0 (we useN 0 to denote the initial population in the region in the scenario with quarantine control). With such an approximation, the optimization problem becomes a mixed-integer bi-linear programming problem, which can be more efficiently solved using techniques such as McCormick's relaxation [43], [44]. Furthermore, if the MTL specification ϕ consists of only conjunctions (∧) and the always operator ( ), the integers in the optimization problem can be eliminated [34] and the problem becomes a bi-linear programming problem.   In this section, we implement the proposed control synthesis approach in the three different control models as introduced in Section IV.

A. COVID-19 SEIR Model with Vaccination Control
The parameters of the COVID-19 SEIR model are shown in Table I. They were estimated in [12] from the data in the early days (from February 23 to March 16, 2020) in Lombardy, Italy with no isolation measures. The start time for the simulations in this subsection are February 23, 2020. We consider three MTL specifications as shown in Table II Fig. 4 shows the simulation results without any vaccination. It can be seen that the three MTL specifications ϕ 1 V , ϕ 2 V and ϕ 3 V are all violated in such a situation. Note that as isolation measures (i.e., home isolation, social distancing and partial national lockdown) were taken since March 16 in Lombardy, Italy, the real situation was better than those shown in Fig.  4. Now we investigate the hypothetical scenario where the isolation measures are replaced by vaccination.
We use the solver GEKKO [9] to solve the optimization problems formulated in Section V. Fig. 5 and Table II show the simulation results for vaccination control of COVID-19 SEIR model with MTL specifications ϕ 1 V , ϕ 2 V and ϕ 3 V , respectively. The results show that the MTL specifications ϕ 1 V , ϕ 2 V and ϕ 3 V are satisfied with the synthesized vaccination control inputs respectively. It can be seen that vaccination within the first 40 days after the outbreak can mitigate the spread of SARS-CoV-2 in the most efficient manner. The results also show that the control effort for satisfying ϕ 1 V is less than that for satisfying ϕ 2 V , which is still less than that for satisfying ϕ 3 V . This is consistent with the fact that ϕ 2 V implies ϕ 1 V , and ϕ 3 V implies both ϕ 1 V and ϕ 2 V . For all three specifications, the computations are completed within 4 seconds on a MacBook Laptop with 1.40-GHz Core i5 CPU and 16-GB RAM.

B. COVID-19 SEIR Model with Shield Immunity Control
We use the same parameters of the COVID-19 SEIR model as shown in Table I Table III) to be less stringent than the MTL specifications with the vaccination control, as shield immunity is generally less effective than vaccination. It can be shown that without any control strategies the three MTL specifications ϕ 1 S , ϕ 2 S and ϕ 3 S are all violated. Now we investigate the hypothetical scenario where the isolation measures are replaced by shield immunity control. Fig. 6 and Table III show the simulation results for shield immunity control of the COVID-19 SEIR model with MTL specifications ϕ 1 S , ϕ 2 S and ϕ 3 S , respectively. The results show that the MTL specifications ϕ 1 S , ϕ 2 S and ϕ 3 S are satisfied with the synthesized shield immunity control inputs respectively.  We observe that with the three MTL specifications, the synthesized shield immunity control inputs all increase to a peak after approximately 20 to 40 days and then gradually decrease. These observations indicate that shield immunity at early days of COVID-19 is more efficient than shield immunity at later days. The results also show that the control effort for satisfying ϕ 1 S is less than that for satisfying ϕ 2 S , which is still less than that for satisfying ϕ 3 S . This is consistent with the fact that ϕ 2 S implies ϕ 1 S , and ϕ 3 S implies both ϕ 1 S and ϕ 2 S .

C. COVID-19 SUQC Model with Quarantine Control
The parameters of the COVID-19 SUQC model are shown in Table IV. They were estimated in [5] from the data in  Fig. 7 shows the simulation results for the COVID-19 SUQC model estimated from data in Stage I (January 20 to January 30, 2020) of Wuhan, China. It can be seen that the three MTL specifications ϕ 1 Q , ϕ 2 Q and ϕ 3 Q are all violated in such a situation (with quarantine rate being always 0.063). Now we investigate the scenario where the quarantine rate can be controlled to satisfy the MTL specifications. Fig. 8 and Table V show the simulation results for quarantine control of the COVID-19 SUQC model with MTL specifications ϕ 1 Q , ϕ 2 Q and ϕ 3 Q , respectively. The results show that the MTL specifications ϕ 1 Q , ϕ 2 Q and ϕ 3 Q are satisfied with the synthesized quarantine control inputs respectively. The results also show that the control effort for satisfying ϕ 1 Q is less than that for satisfying ϕ 2 Q , which is still less than that for satisfying ϕ 3 Q . This is consistent with the fact that ϕ 2 Q implies ϕ 1 Q , and ϕ 3 Q implies both ϕ 1 Q and ϕ 2 Q . We observe that with ϕ 1 Q , the synthesized quarantine control inputs first increase to a peak at approximately 90 days and then gradually decrease; with ϕ 2 Q , the synthesized quarantine control inputs first increase to a peak at approximately 50 days and then  parameter value parameter value β 0 0.2967 γ 2 0.05 N 0 8.9 million σ 0.001 T s 1 day gradually decrease; and with ϕ 3 Q , the synthesized quarantine control inputs are at a peak from the beginning and gradually decrease. These observations indicate that quarantine in the early days of COVID-19 can reduce the number of confirmed infected cases more efficiently than quarantine in the later days, and more stringent control specifications require stronger quarantine measures to be implemented.

VII. CONCLUSION
In this paper, we proposed a systematic control synthesis approach for mitigating the COVID-19 epidemic based on three control models with vaccination, shield immunity and quarantine, respectively. We used metric temporal logic (MTL) formulas to formally specify the required performance of the control strategies. The proposed approach can synthesize control inputs that lead to satisfaction of the MTL specifications.
The work in this paper opens the door to the formal synthesis of control strategies based on epidemic models. We list several future directions as follows. First, we will investigate  the effects of model uncertainties and parameter uncertainties in the control synthesis, and explore robust control synthesis methods in the presence of such uncertainties. Second, we will extend this work to online control synthesis so that  control inputs can be generated in real-time based on the latest information (e.g., using online parameter identification and receding horizon control). Finally, as we investigated the three control strategies separately in this paper, we will study the benefits and costs of joint control of different control strategies (vaccination, shield immunity and quarantine) so that the specifications can be satisfied with coordinated efforts.