https://orcid.org/0000-0002-4217-0935
Figures
Abstract
Optimal intervention planning is a critical part of epidemiological control, which is difficult to attain in real life situations. Ordinary differential equation (ODE) models can be used to optimize control but the results can not be easily translated to interventions in highly complex real life environments. Agent-based methods on the other hand allow detailed modeling of the environment but optimization is precluded by the large number of parameters. Our goal was to combine the advantages of both approaches, i.e., to allow control optimization in complex environments. The epidemic control objectives are expressed as a time-dependent reference for the number of infected people. To track this reference, a model predictive controller (MPC) is designed with a compartmental ODE prediction model to compute the optimal level of stringency of interventions, which are later translated to specific actions such as mobility restriction, quarantine policy, masking rules, school closure. The effects of interventions on the transmission rate of the pathogen, and hence their stringency, are computed using PanSim, an agent-based epidemic simulator that contains a detailed model of the environment. The realism and practical applicability of the method is demonstrated by the wide range of discrete level measures that can be taken into account. Moreover, the change between measures applied during consecutive planning intervals is also minimized. We found that such a combined intervention planning strategy is able to efficiently control a COVID-19-like epidemic process, in terms of incidence, virulence, and infectiousness with surprisingly sparse (e.g. 21 day) intervention regimes. At the same time, the approach proved to be robust even in scenarios with significant model uncertainties, such as unknown transmission rate, uncertain time and probability constants. The high performance of the computation allows a large number of test cases to be run. The proposed computational framework can be reused for epidemic management of unexpected pandemic events and can be customized to the needs of any country.
Author summary
Pandemic control is a complex task that requires balancing the impact of interventions like lockdowns or closures of institutions with the need to limit the spread of disease. In this study, we developed a new tool that combines two powerful approaches: mathematical modeling to predict the spread of the disease and detailed simulations that reflect real-world conditions. By combining these methods, we were able to create a strategy that efficiently manages an epidemic similar to COVID-19. Our approach was not only effective in controlling the outbreak but also worked well even when there were uncertainties about how fast the virus spreads. We believe that the proposed method of combining agent-based and ODE models, will inspire research communities working in other scientific fields involving agents, Cell Biology, Transportation, or Logistics.
Citation: Polcz P, Reguly IZ, Tornai K, Juhász J, Pongor S, Csikász-Nagy A, et al. (2025) Smart epidemic control: A hybrid model blending ODEs and agent-based simulations for optimal, real-world intervention planning. PLoS Comput Biol 21(5): e1013028. https://doi.org/10.1371/journal.pcbi.1013028
Editor: Alejandro Fernández Villaverde, Universidade de Vigo, SPAIN
Received: October 21, 2024; Accepted: April 7, 2025; Published: May 8, 2025
Copyright: © 2025 Polcz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Package PanSim and the additional Matlab code are available in the public repository https://github.com/khbence/pansim. Otherwise, this research includes solely synthetic data, which can be produced using PanSim and the supplementary Matlab scripts executing PanSim.
Funding: The study was funded by the National Research, Development, and Innovation (NRDI) Office in Hungary (RRF-2.3.1-21-2022-00006) and the Ministry of Culture and Innovation of Hungary from the NRDI Fund through the grants PD-145902 (P.P.), K-145934 (G.Sz.), and FK-145931 (I.Z.R.) financed under the OTKA Research Thematic Call 2023. We acknowledge the support of Ministry of Culture and Innovation of Hungary through the New National Excellence Program scholarship ÚNKP-23-4-II-PPKE-27 (P.P.). Moreover, projects no. TKP2021-NKTA-66 (P.P.) and TKP2021-NVA-26 (P.P.) have been implemented with the support provided by the Ministry of Culture and Innovation of Hungary from the NRDI Fund, financed under the TKP2021-NKTA and TKP2021-NVA funding schemes, respectively. This work was also supported by the Hungarian Academy of Sciences through the Grant POST-COVID2021-64 (I.Z.R.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: We have read the journal’s policy and the authors of this manuscript have the following competing interests: BK-H and AC-N reporting that they have financial and business interests in Cytocast Kft. that may be affected by the research reported in the enclosed paper.
Introduction
Designing appropriate measures to meet often conflicting goals and constraints during the COVID-19 pandemic was challenging for most countries, especially in 2020 [1]. Moreover, we have seen that any delay in intervention could cause serious damage [2–4]. Beyond varying degrees of success in containing the epidemic, there was a palpable sense of panic among the population, which was even worsened by a lack of accurate information about the local situation. This emergency demonstrated that local and national governments need to be equipped with a comprehensive pandemic management system with monitoring, prediction, and decision support capabilities to keep epidemic spread under control at multiple levels of the community, e.g., in settlements [5], but also at a national and international level [6]. Such an epidemic surveillance and modeling framework has to be able to cope with uncertainties coming from the lack of data, unmodeled phenomena, or the appearance of yet unknown pathogen variants.
Over the years, analysis, estimation, and planning based on dynamic models have become essential parts of epidemic management [7]. Moreover, the recent pandemic provided unprecedented momentum to computational epidemiology worldwide (see, e.g., the review papers [8, 9] and references therein). Notably, important dynamic analysis results are available for epidemic processes [10–12]. Furthermore, numerous epidemic surveillance tools were developed to solve particular problems related to the COVID-19 pandemic, such as state and parameter estimation [13, 14], or the retrospective assessment of certain non-pharmaceutical interventions (NPI) [15]. There is also an extensive body of literature on predicting the future evolution of disease spread, as demonstrated in the review paper [16]. Although several machine learning approaches are available for prediction [17] and intervention planning [18], compartmental ordinary difference equation (ODE) models are still widely used and remain highly popular for managing pandemics at the population level [8]. These models make it possible to design robust and/or optimal control strategies to mitigate the impact of the pandemic (see, e.g., [19–21]). In addition, model predictive control (MPC) strategies allow the formulation of complex control objectives [22–24] through the minimization of a cost function, which can reflect many aspects of the future evolution of a spread, e.g., meeting hospital capacity, minimizing the stringency of interventions, etc.
The majority of the model-based control strategies are designed for SEIR-type compartmental models. A SEIR model describes the dynamics of susceptible, exposed, infected, and recovered people. Basic SEIR models can be enhanced in several ways, e.g., they can be augmented with additional phases or states of the disease, and compartments can be split by age groups, vaccination status or by locations. Although compartmental models can be extended to include a high level of detail, they still present several issues and limitations, two of which are critical to our work. First, model-based controllers provide control actions such as lockdown [18] or quarantine measures [23, 25]. These are often combined with screening measures [21, 24], or vaccination intensity [20]. Since these measures take values in continuous ranges, it can be difficult to translate them into realistic interventions. Secondly, low-dimensional compartmental models have limited prediction accuracy in heterogeneous communities [26]. On the other hand, detailed models require a large number of parameters and state variables, which are often difficult to determine from available measurements.
Another large group of results in pandemic management is related to agent-based models (ABMs) [27–29], which enable a fine-grained study of disease spread. Although statistical [30], compartmental [31, 32], and other model-based [33] techniques have been used to assess the impact of interventions, ABMs have proven particularly effective in capturing the nuanced impact of NPIs [34–36].
Some ABMs, such as a soil-crop irrigation model in [37], can be expressed in closed-form dynamic equations, making them amenable to advanced control techniques, including model predictive control (MPC). However, agent-based epidemic models that simulate real-world NPIs [38] are significantly more complex, making direct integration with model-based control algorithms challenging. As a result, most agent-based decision support systems lack fully automated intervention planning.
To deal with the complexity of agent-based simulators, surrogate models are often used to approximate their behavior, making statistical inference [39], calibration [40], or sensitivity analysis [41] easier and more efficient. One of the early works to recognize the potential of surrogates in predictive control design for agent-based models (ABMs) is presented by An et al. [42]. It proposed a novel framework for the predictive control of agent-based simulators. By approximating the simulators behavior with mathematically more tractable models, the study laid the groundwork for the multi-level optimization-based decision-making in complex real-world processes with agents. This concept has been further developed by Niemann et al. [43, 44], who proposed gradient-based optimization methods for the multi-level predictive control for agent-based epidemic models.
For agent-based epidemic spread simulators, it is natural to choose a SEIR-type ODE surrogate, however, fitting a model to a general ABM is often challenging. A data-driven approach is proposed in [43], where Koopman operators are trained using simulation data to fit linear models to ABM. Instead, Fonseca et al. [45] have considered a few classes of nonlinear system (e.g., S-systems [46]), and investigate, how they can be fit to the targeted ABM.
Despite the rapid development and spotlight on computational epidemiology, there are existing open problems to solve. In addition to the challenges already mentioned, Alamo et al. [28] highlighted the difficult multi-criteria nature of real and practically implementable intervention planning. The available ABMs and new results in MPC design, as well as the gaps in the state of the art, motivated this research on the intelligent combination of different modeling approaches to support complex decision-making.
As the key contribution of the paper, we propose a computationally efficient framework that computes real-word implementable NPIs in terms of six measures, mask-wearing rules, quarantine policies, testing intensity, curfew, school closure, or closing public venues. The stringency of these measures belong to a discrete set of values (“low”, “medium”, “high”), which correspond to the actually implemented interventions in Hungary or other countries during the COVID-19 pandemic between 2020 and 2022. Computing optimal combinations of these measures allow us to achieve a wide variety of epidemic prevention or mitigation targets, even following a targeted epidemic reference curve. To achieve these results, we solve an important research gap in epidemic management by constructing a statistical model from synthetic data to map the targeted transmission rate to a set of combinations of measures. Synthetic data was generated using PanSim, a GPU-accelerated, open-source micro-simulation model developed by our research group [5]. PanSim provides a realistic, age-stratified, georeferenced simulation of 180,000 agents across 81,000 locations, operating with a relatively small, a 10-minute time step. The statistical model, combined with the ODE-based robust MPC framework and the fine-grained yet accelerated PanSim, enables an effective integration of model-based control with a highly detailed agent-based simulation. This fusion leverages the strengths of both approaches: a mathematically rigorous control strategy capable of optimizing diverse cost functions and a high-resolution model that accurately represents spatial and temporal population dynamics, facilitating the direct implementation of NPIs.
The paper is organized in three main sections. First, we present the main findings in section Results, which are illustrated through seven case studies labeled by capital letters A– G. In section Discussion, we summarize the advantages and limitations of our approach and suggest possible research directions for the future work. We present the MPC methodology in detail in section Methods.
Results
For pandemic control through non-pharmaceutical intervention planning, we consider two distinct models, an agent-based and a compartmental ODE model, each characterized by unique properties and limitations. The ABM boasts high fidelity, rendering it capable of accurately simulating the effects of specific NPIs. However, its drawback lies in its non-closed form nature, posing challenges in planning control inputs based on this model. Conversely, an ODE model is endowed with a closed-form structure, facilitating model-based control computations. Nevertheless, it lacks the ability to simulate the effects of specific interventions. To harness the strengths of both models, an interface between the two models can be devised to translate control inputs intended for the ODE model into practically implementable control strategies for ABM.
We make use of a high fidelity fine-grained microsimulation model, to which we propose MPC strategies. The framework computes a targeted transmission rate () for the pathogen, then, it translates to an appropriate combination of NPIs, which is simulated on the ABM. Possibly, the simulator shows a significantly different transmission rate than the targeted one. In that case, the mapping from the transmission rates to NPIs (
) can be updated optionally.
We examine the combinations of six predefined policy measures, which are described in Table 1. A combination of the six policy measures is considered a non-pharmaceutical intervention (NPI). For instance, the combination (TP-medium, PL-high, CF-low, SC-high, QU-medium, MA-high) is one of the possible NPIs. During the intervention planning, we allowed any possible combinations of these policy measures, having different effects on the transmission rate. Possibly not all the 216 NPIs are reasonable or practically useful, but those NPIs can be sorted out at any stage of the workflow.
Note that the effects of the six interventions in a combination are generally neither additive nor multiplicative, hence, it is difficult to quantify the overall effect of an NPI on the transmission rate. For this reason, a lookup-table will be introduced later to model the effects of NPIs.
PanSim, a detailed agent-based simulator
The development of PanSim [5] started shortly after the creation of the Hungarian COVID-19 Mathematical Modelling and Epidemiological Analysis Task Force at the beginning of the pandemic in March 2020. The goal was to complement ODE-based models and generate fine resolution high quality synthetic data [47] by capturing finer details of the spread of the epidemic in realistic, age-stratified populations, and thereby allow the direct evaluation of the effects of non-pharmaceutical interventions. PanSim allows for the direct, realistic implementations of the previously defined interventions because it directly simulates the movement of agents to various locations and allows adding rules to modify these behaviors. Therefore, the effect of interventions on the spread of the infection (the reproduction rate) can be simply calculated from the simulation output; this is in contrast to compartmental models, where the change in the reproduction rate due to a given intervention is captured in a time-varying parameter which is difficult to quantify.
PanSim was set up to simulate a mid-sized Hungarian town (Szeged) using realistic statistics on the population, the points of interest the agents can visit (schools, offices, shops, hospitals, and more), as well as their daily movements. This setup includes 180k agents and 81k locations. The simulation has a 10-minute time step (i.e., state of each agent is updated in every 10 minutes) and probabilistically infects agents at their current location, depending on the number of infectious agents there. Agents follow a probabilistic daily schedule, which is potentially altered by interventions/restrictions currently active - these can be updated daily. The simulator calculates various statistics at midnight of each simulated day, including the number of agents in different stages of the disease by age group and the number of tests, their positivity, the number of agents in quarantine, daily vaccinations, and many more.
PanSim was initially implemented and parameterized for the wild-type SARS-CoV-2. Later, it was easily adapted to the Alpha, Delta, and Omicron variants by updating disease-specific parameters stored in JSON and spreadsheet files. The model can also be extended to other respiratory pathogens, such as influenza virus, respiratory syncytial virus (RSV), or measles virus. PanSim was successfully used extensively during 2021 and 2022 to forecast the spread of different variants and to evaluate the efficacy of various interventions. The hospital load was calculated with less than 10% error 3–4 weeks in advance, which greatly helped intervention planning. Although hospitalizations typically lag infections, this delay does not necessarily degrade the quality of the prediction. The time evolution from infection to hospitalization is inherently captured through the time constants governing compartmental transitions, which will be introduced in the next section. While alternative indicators, such as test positivity rates [24], could also provide valuable predictive insights, hospital data remained the most reliable source throughout the first two years of the COVID-19 pandemic in Hungary. From 2022 onward, the integration of nationwide wastewater analysis further improved the accuracy and robustness of epidemic predictions [48].
Compartments of the epidemic models
We reuse the compartmental description proposed by [23], wherein the population of individuals is partitioned into eight distinct groups. The susceptible population (
) encompasses individuals who have never been infected. The recovered individuals (
) consist of those who have acquired immunity through recovery. The infected population is further subdivided based on various phases of the disease and the potential outcomes.
Specifically, individuals are categorized into the latent phase (), presymptomatic phase (
), and the main sequence of the disease. It is noteworthy that the incubation period corresponds to the sum of duration of the latent and presymptomatic phases. The distinction between
and
is justified by the fact that individuals in the latent phase are not yet infectious, whereas those in the presymptomatic phase may transmit the infection.
To depict the potential disease outcomes during the main sequence of the disease, we delineate four severity scenarios, each parameterized by three probability coefficients. Firstly, a subset of individuals remains asymptomatic (), while others exhibit symptoms (
) with a probability
. Individuals displaying symptoms may require hospitalization (
) with a probability
, and among hospitalized patients, there is a possibility of mortality (
) with a probability
.
The average residence time in compartments ,
,
,
,
is denoted by the reciprocals of the time constants
,
,
,
,
, respectively.
To make the agent-based model suitable for clinical purposes, severe cases are treated separately by the simulator. Therefore, the compartments ,
,
in the simulator are each split into two disjoint compartments
,
, and
, respectively, where
and
denotes the infected and hospitalized people with mild symptoms, whereas,
and
denote those who show severe symptoms. Severe cases remain in hospital (
) for complications or observation even after recovery, others recover and are sent home (
).
The transitions between the eight compartments of the ODE model are illustrated in purple box of Fig 1, whereas, the gray box illustrates the transitions between the eleven compartments of the ABM. Fig 1 also shows that the compartmental model behind the ABM is a more detailed version of the ODE.
The thick purple arrow illustrates that the ODE model is used as a prediction model in the MPC. The blue arrow shows that selected interventions are simulated on the agent-based model (or applied to a real pandemic event) over a period of days, called a cycle. The red arrow illustrates the fact that the state and the transmission rate of the simulator (or a real epidemic) is estimated using the available data. The green arrows indicate that the lookup table (LUT) is updated whenever the reconstructed transmission rate is significantly higher than as it was expected for the selected NPI.
To synchronize the states of the agent-based and the ODE models, we use as a primary measure. This fact is illustrated in Fig 1 by the red link between compartments
the ABM (purple box) and
of the ODE model (gray box)
In a real pandemic situation, the accurate number of infected people with symptoms in the main phase of the disease () is typically unknown. However, previous work demonstrated that the number of active cases can be inferred from the hospital load and the admission rate [49–51] possibly in combination with wastewater data [48]. Using
alone to match the two models may compromise accuracy as
have longer and more distributed delays and lower values or fluctuations. We have decided, therefore, to link the two models through
, which is sufficiently high and representative during the course of a pandemic wave, such that the stochastic nature of the simulation (and a real epidemic event) does not significantly affect
even for a smaller population like Szeged.
Assumptions
In this section, we introduce a few considerations to simplify epidemic modeling. First, as reported by [52, 53], individuals who are asymptomatic during the main phase of infection exhibit lower infectiousness compared to those in the presymptomatic phase () or individuals symptomatic during the main phase (
). Additionally, we do not exclude the possibility of transmitting the disease in hospitals. Accordingly, we assume that only a fraction
of asymptomatic individuals and
of hospitalized patients are infectious. Therefore, the number of infectious people at time k is
Secondly, we assume that the effects of the non-pharmaceutical interventions are time-invariant. Namely, a NPI will result in approximately the same transmission rate drop of the pathogen at any phase of an outbreak (e.g., rise, peak, or fall) independently of the virus variant. This assumption is reasonable as the social distancing measures reduce the number of contact events by a given scaling factor regardless of virus mutations or epidemic phase. We also note that time-invariance for PanSim was demonstrated in [54] when the spread is caused by a single virus strain.
Thirdly, we assume that a realistic simulator is already set up and ready to use. Namely, this paper does not deal with the model calibration of an agent-based model, as the parameter calibration of PanSim was already addressed in [5, 55].
Finally, we assume that only patients in hospitals with severe symptoms are at risk of dying from the disease. This assumption implies, naturally, that severely ill patients are admitted to hospital. We do not model the natural deaths and births in this relatively short period.
It is reasonable to assume that the average residence times ,
,
,
in compartments
,
,
,
, the probability of symptomatic infection (
), and the relative transmission rate of asymptomatic individuals (
) depend primarily on the properties (e.g., virulence and infectiousness) of the pathogen, making them approximately constant during the dominance of a single variant. On the other hand, parameters such as the average length of hospitalization (
), the relative infectiousness of hospitalized patients (
), the probability of hospitalization (
), and the probability of fatal outcomes (
) could be influenced by external factors like hospitalization policies or masking rules in hospitals. However, in this study, these factors are not varied. For instance, masking rules (MA) in both low and high policy levels always assume mask-wearing in hospitals. Thus, we consider
,
,
, and
to remain constant during the dominance of a single virus variant.
Preparing steps of the workflow
Although, the workflow is described in detail in Section Methods, we think useful to present the major building blocks of our control strategy in brief.
The ODE-based epidemic controllers (see, e.g., [23]) usually compute an action in terms of a single scalar value, the desired transmission rate , that should be implemented through interventions. The question arises, which interventions are to be selected to achieve the target rate
? Obviously, the simulator was specifically designed to quantify the effects of NPIs even in terms of the transmission rate
, therefore, the mapping
is simple to approximate from the simulation data. However, the inverse mapping is more challenging to describe.
In this work, we demonstrate that even a simple statistical model in the form of a lookup table (LUT) is sufficient to capture the mapping from to
. To construct the LUT, we generate synthetic data by simulating the effects of every possible NPIs in different phases of the disease as proposed in [55]. Then, the LUT is filled in with the averages and the standard deviations of the resulting transmission rates. In this way, a small amount of data and expert knowledge about a specific pathogen are sufficient for the fine-grained simulator to generate useful synthetic data, considering various possible scenarios that have not been tested in real life. It is also worth mentioning that the use of synthetic data in epidemiology was already discussed in the literature, and it was portrayed as a promising approach [47].
Control loop
When designing the controller, we must take into account that it is not feasible to introduce new measures on a daily basis. Therefore, the controller is forced to plan a sequence of transmission rates, which does not change within an intervention planning cycle having a length of h days. In this way, the targeted transmission rate function is a piecewise constant function, which can be directly integrated into a model predictive controller synthesis. An intervention planning cycle is a period of or 30 days, during which the measures to suppress the spread cannot be changed.
We consider a target epidemic curve, which we want to achieve using the interventions. The target curve is an important element of the control design as it allows to shape the time evolution of the spread. Through the target curve we can prescribe, e.g., how much we want to flatten the curve of the spread compared to the free spread.
In the controller loop (Fig 1), first, we presume an initial state, in which the majority of the population is susceptible and only a few people are infected. A model predictive controller with an ODE prediction model computes a sequence of transmission rates, one value for each future intervention planning cycle. Using the LUT, we select an NPI, which is expected to result in a rate close to the desired . We remark that a combination of measures achieving a given effect is not unique. However, this flexibility is not a limitation; rather, it allows for selecting combinations based on heuristics or specific objectives. For instance, criteria can be introduced to minimize changes in interventions across the intervening cycles.
Then, we simulate the selected NPI for the next cycle (i.e., for or 30 days) and collect the simulation’s data. By the end of the cycle, we reconstruct the evolution of the spread in the simulator to match the ODE model. The reconstructed data includes the actual state and the rate
that we actually achieved with the selected NPI. Closing the loop, the new state is fed back and considered during the next MPC execution.
As an optional step by the end of an intervention planning cycle, we are allowed to update the LUT if it seem reasonable. E.g., when the ongoing epidemic is not well-characterized by the simulator, the transmission rate values in the LUT may not be precise enough. It may also happen that a new variant emerges with a higher transmission rate. In these situations, we provide the opportunity to scale up the transmission rate values in the LUT. Later, we will illustrate the effectiveness of this tuning knob.
We will use the simulator to demonstrate the adaptivity of our approach in possible unexpected scenarios. However, if the situation requires so, the proposed control loop can also be applied to a real epidemic situation. In that case, the unknown sequence for can be computed by our wastewater-based reconstruction method presented in [48], which was successfully applied during the outbreaks caused by the Delta and the Omicron variants in 2022 and 2023.
Case studies of expected scenarios
In this and the following sections, we analyze multiple scenarios and test the controller algorithm with different pandemic management strategies.
We considered multiple epidemic control goals to evaluate the controller when the pathogen does not go under mutations. In these cases, the controller is well prepared for the emergent pathogen, its transmission rate is precisely estimated and the disease course dynamics is well modeled. We considered a pathogen, which resembles the original SARS-CoV-2 virus strain in both transmissibility and virulence.
When the virus is free to spread without interventions, the epidemic peaks at 1950 in a population of 100k individuals. The obtained curves for the free spread and their deviation are shown in Fig 2. Considering the epidemic curve of a free spread, which is illustrated by green curves in Fig 2, we defined four possible strategies to mitigate the impact of the outbreak:
The four cases are distinguished by labels A, B, C, and D. The first plots () in all cases illustrate the achieved curves obtained through multiple executions compared to the targeted curve and the free spread. In each case, we highlight in black the obtained curve, which is closest to the mean of all executions. This curve was obtained using the interventions presented in the second plots. The third plots illustrate the average strictness of interventions computed through the multiple MPC executions. The fourth plots illustrate the achieved transmission rates. Labels TP, , MA on the y axes of the second and third plots correspond to the six policy measures explained in Table 1. The mean and standard deviation of each curve were computed from 20 separate executions.
- A. flatten the curve to avoid hospital overload ( A1– A4 in Fig 2),
- B. delay the outbreak to increase hospital capacity ( B1– B4),
- C. minimize economic impact ( C1– C4),
- D. allow higher mobility for special occasions ( D1– D4).
We want to achieve these control objectives with an intervention planning cycle of length at least h = 21 days.
In all case studies ( A– D), we executed PanSim 20 times both when the virus was allowed to spread freely and when the outbreak was controlled by MPC. In Fig 2, all the 20 curves are presented for all cases in gray. The 95% confidence intervals for all curves are illustrated by the colored shaded areas: orange for controlled spread, green for the free spread, and blue for the partially free spread.
Although in PanSim 180k agents are considered, the computed numbers are all normalized to a 100k population. In all figures, we illustrate the normalized curves, i.e., the number of individuals in a 100k population.
In all studies, we designed a target curve for the numbers of active cases (), which is meant to be tracked using NPIs. However, our framework allows to formulate multi-objective strategies as well. For example, instead of following a target curve, we can minimize both the active cases and the impact of the applied interventions simultaneously. In the simplest case, the impact of the an NPI can be modeled by the resulting transmission rate drop, otherwise, we can define an economic/social impact value for each NPI.
The following four titles labeled by capital letters A– D denote sub-subsections of the current subsection, each corresponding to a case study.
A. Flattening the curve.
First, we would like to delay the peak by 7 weeks with a maximum value of 836 (out of 100,000), namely, we want to “flatten the curve” as presented in A1 of Fig 2. The results suggest that the targeted curve can be achieved even with monthly intervention planning cycles (h = 30). One possible monthly sequence of NPIs is presented in A2 of Fig 2. Multiple executions show that mild interventions in the first four months are enough to achieve the desired epidemic curve, then, in the decreasing phase of the epidemic, interventions can be gradually released.
B. We give more time.
Secondly, the outbreak is intended to be drastically suppressed in the first three months, to give more time to hospitals for scaling up there capacity. As illustrated in B3 of Fig 2, this can be achieved by a lock-down in the first 9 weeks, after which the interventions can be released for a short period. Then, when the outbreak emerges, the interventions should be slowly relaunched, to achieve the peak at 1114 (out of 100,000) by the end of week 22. This means that the peak is 12 weeks delayed compared to the peak of the free spread. This strategy does not consider the impact on the economy but is focused only on gaining more time to develop the infrastructure in hospitals.
C. Almost free but delayed.
When the impact on the economy is also a crucial aspect, one may plan to delay the peak e.g., by 8 weeks but allow the pathogen to spread almost freely. This strategy requires a lock-down in only the first 3 weeks, and mild relaunch after the recognition of the exponential phase (10.–12. weeks) of the outbreak. Otherwise the interventions can be completely released (see C1– C4 in Fig 2).
D. Two waves.
Finally, the fourth strategy will allow the population to celebrate a winter holiday without extreme measures, as the epidemic has already been suppressed by previous restrictions. This strategy also a makes it possible to organize public events later by releasing all mobility restrictions in the first half of February (see D3 in Fig 2). On the other hand, if we split the epidemic into two waves, each of the two peaks will be smaller than in the first case, “Flatten the curve”. This can be a good strategy for a limited number of hospital/ICU beds. Admittedly, decision makers need to keep making closures over the whole period, but they are not very strict. If the second wave is completely released, the peak will not be higher than in the third case, “We give more time”.
Scenarios with significant model uncertainty
Next, we analyze how the controller performs in unexpected situations, e.g., when the emerging pathogen is not well characterized, or when a new variant emerges during a pandemic. We analyze three situations. First, we simulate that a new variant appears in the 11th week, which has similar characteristics to the Alpha variant of SARS-CoV-2 [56]. Secondly, a new variant is simulated appearing in the same week but characterized as an Omicron BA.1-like variant [57]. In the third study, we analyze how the controller can cope with an Omicron BA.1-like pathogen appearing in a fully susceptible population. In all three case studies, we consider a prediction ODE model with parameters calibrated to the wild strain of SARS-CoV-2 virus and prescribe a flattened epidemic curve (as illustrated in A1 (Fig 2). In these studies, we consider multiple cycle lengths days, and analyze how they affect the robustness of the controller, when the model is not well-calibrated. In these three studies, we executed PanSim 50 times for each cycle length. The following three titles labeled by capital letters E– G denote sub-subsections.
E. Alpha variant emerges.
In silico, we reproduce the events when the Alpha variant appeared in Hungary. In the decreasing phase of an outbreak caused by the wild strain, a more virulent variant called the Alpha appeared with even higher transmission rate. In a free spread, the new variant appearing in week 11 does not seem to be more impactful than the wild mutant. Therefore, we try reusing the interventions in A2, which successfully flattened the epidemic curve for the wild mutant (see A1 in Fig 2). However, these interventions are insufficient to contain the spread for the emergent new variant, and, as it is illustrated in E1 in Fig 3, the outbreak peaks 1.5 times higher than the peak of the free spread of the wild mutant.
The vertical line with the label “ higher than expected” illustrates the first cycle, after which the reconstructed transmission rate was at least 20% higher than the average registered in the LUT for the selected and applied NPI. This anomaly suggests that the current pathogen is more infectious than expected.
Then, we considered an MPC strategy, in which the values in the LUT are scaled up at the end of every NPI planning cycle if the estimated
is 20% higher than it was preliminarily expected for the selected NPI. The threshold of 20% is about two times higher than the standard deviation of the obtained transmission rates for an arbitrary fixed NPI. Therefore, a more than 20% higher transmission rate lies outside of the 95% confidence interval of the expected rate. The results obtained for
days cycle length are summarized in Fig 3 in E2, E3, E4, and E5, respectively. In each of these plots, the first vertical line (“Alpha appeared”) shows the date when the Alpha variant appeared. With the second vertical line (“
higher than expected”), we illustrate the first time when the computed
was as least 20% higher then it was targeted by the applied NPI. In this case study, the controller detects the increase in
about 50 days after its appearance, regardless of the length of the planning cycles.
We found that a two weeks long intervention planning cycle is necessary to provide a plausible adaptivity of the controller strategy. When the cycle is days long, the controlled spread peaks more than two times higher than it was prescribed by the reference curve. The dates and heights of the peaks as well as the percentage of the population who has been infected in the three unexpected scenarios and control strategies applied are summarized in Table 2.
The third column called cumulative infected (CI) indicates the percentage of population who have been infected or reinfected.
F. Omicron variant emerges.
We assume that the spread is started on day 1 by the wild SARS-CoV-2 strain, then, a new variant like Omicron BA.1 appears in the 11th week. Again, we analyze the free and controlled spread of two variants. The results of the case study are summarized in the five subplots of Fig 4, which are labeled by F1– F5.
The vertical line with the label “ higher than expected” illustrates the first cycle, after which the achieved transmission rate was at least 20% higher than it was targeted.
When the spread is not controlled at all, the outbreak peaks in week 10, then, 5 weeks after the appearance of the Omicron variant, rebounds in the middle of the week 16. Up till this time, 60% of the population is infected. One month after the rebound, the second wave peaks at 81% of the altitude of the first peak in week 26. After 210 days, 110% of the population is infected or reinfected.
If we reuse the interventions in A2 (Fig 2), the spread of the wild strain is suppressed, but the new variant causes a more then twice higher peak compared to the free spread of the wild strain ( F1). About 93% of the population is infected in this case.
Similarly to the previous case study ( E), in Fig 4, we illustrated the dates when the algorithm first computed a more than 20% higher than it was targeted. This study reveals that the adaptivity of the control strategy degrades with the increasing length of the intervention planning cycles. Using
days long cycles the appearance of a new variant is detected 4, 4, 5, and 7 weeks later, respectively, and the outbreak peaks 1.5, 2, 4 and 5 times higher than as prescribed. We can observe that shorter cycles allows to detect and handle unexpected increase earlier.
G. Omicron appears in a fully susceptible population.
In this case, we considered a new pathogen with a transmission rate and disease characteristics similar to the Omicron variant of SARS-CoV-2. Namely, this is the case when it turns out that both the control model (ODE) and the simulation model (ABM) are highly uncertain, as we do not know precisely its transmission rate nor its time () or probability (
) constants. We assumed that this pathogen emerges in a fully susceptible population. When the spread is not controlled, almost the full population is infected in 6 weeks. For this pathogen, the controller is not able to flatten the curve to the desired level even with an h = 7 days long intervention planning cycles. In all cases, the higher transmissibility is detected after the first cycle, after which the controller imposes a full closure to achieve the desired targets. Although a full closure has not been able to reduce the spread of the epidemic significantly, an h = 7 days cycle length allows to reduce the number of cases from 97% to 88% of the population. These results are illustrated in Fig 5.
Discussion
The main motivation of this work is to make epidemic intervention-planning more explicit. For this, we address the challenge of integrating advanced model-based control and high-fidelity, realistic agent-based simulation. To handle the various constraints on the state and input variables, we use a model predictive control (MPC) strategy. The controller uses an 8-compartment ODE-based model which is discretized in time. The manipulable input of the model is the spreading rate of the disease which can be influenced between certain limits by appropriate measures. MPC can take into consideration a main practical limitation, namely that the strictness level of measures can only be changed after an interval of several days or even weeks. The cornerstone of the realistic nature of the study is the PanSim simulator [5], which is a state of the art agent-based model capable of tracking the everyday behaviour of hundreds of thousands of agents in an urban environment. Several workplaces, institutions, public places and living districts are implemented in PanSim which allows the detailed tracking of infection chains and also the simulation of various measures and quarantine policies. The attributes and epidemic states of the agents can also be set and simulated in detail.
In contrast to the closed-form ABM in [37], PanSim can be controlled by an MPC only through an interface built on three key pillars: calibration, reconstruction, and input translation. The first step involves calibrating the parameters of the control model to align with PanSim by minimizing the squared error between the simulated and observed number of infected individuals in the main phase of the disease (). Next, we use an optimization-based approach to reconstruct the simulator’s state evolution in terms of the control model’s compartments. Since most of the control model’s states are not directly measurable in practice, but are essential for effective control, this reconstruction estimates the missing states based on available data. Additionally, this process enables the computation of the actual daily transmission rate of the pathogen. Finally, the spreading rate prescribed by the controller is translated into specific measures using a lookup table generated from numerous stochastic simulations. This table captures the potentially complex and non-additive effects of applying multiple interventions in parallel. By integrating these three components, the interface effectively closes the loop between mathematical control and agent-based simulation.
Several simulation scenarios are presented to illustrate the capabilities of the proposed methodology. The initial step of the design is the definition of a time-varying reference for the infected people which is to be followed by the controlled system. Together with other constraints on the input and states, this allows the prescription of different goals of various computational complexity such as flattening the epidemic curve or timing the peak of the epidemic wave at a given level. The effect of significant model uncertainties is also studied in the context of the appearance of new, not precisely modeled virus variants with a higher spreading rate than the initial pathogen. The obtained simulation results underline the importance of early response and that of the selected intervention period. The results clearly show that the tracking performance is good and the variance of the obtained infected curves is acceptably small if the mathematical model used in the controller is precise enough. In such cases, even a large intervention period of one month can be enough for successful operation. However, the possibility of the frequent change of measures becomes important when model uncertainties appear. Otherwise, keeping the infected level close to the prescribed value may become infeasible, and the variance of the output may grow unacceptably large. If the transmission rate is very high (like in the case of the Omicron variant in a susceptible population) then it is not possible to reach the control goals even with the application of the strictest modeled measures (for similar practical observations, see e.g., [58, 59]).
In the following, we compare our methodology to the most closely related solutions in the literature. Niemann et al. [43, 44] implemented an optimization-based predictive controller using a gradient method, in which both the ABM and its surrogate is used as a prediction model. In contrast, our approach does not include the ABM in the control optimization loop, instead, the ODE-based MPC is computed repeatedly (with a gradually decreasing prediction horizon) considering the “updated” state after each simulation cycle ( or 30 days). Moreover, we put a major focus on considering realistic measures (Table 1), which are easy to implement, whereas, Niemann et al. considered two measures: the proportion of children not attending school and the proportion of adults working home. These two measures belong to a continuous range [0,1], which makes it straightforward to map the control input to an intervention in the ABM, however, in a real-word situation these continuous-range interventions are not practical as it requires selecting a group of people not attending school or work. Moreover, the compartments in both of our ODE and ABM models are more refined compared to the ABM [29] and its SIR-based surrogates used in [43, 44], e.g., infected people are split to 8 compartments in [5] and only 4 compartments in [29] according to the phase and severity of the disease. On the other hand, the method proposed in [43, 44], relies on a georeferenced demographic agent-based simulator (GERDA) [29], which alongside its higher flexibility compared to PanSim has over an order of magnitude lower simulation throughput. First, compared to the hourly schedules in GERDA, PanSim uses a shorter, 10 minutes long time step, which facilitates capturing the short-duration activities and infection dynamics, as potentially risky durations of exposure can be as short as 15-20 minutes [60]. Secondly, a 2000 hours (83 days) long simulation on a mid-range Desktop PC in PanSim with 180k agents and 81k locations takes about 27 seconds. This allows a relatively large number of simulation runs which is clearly useful for uncertainty evaluation. For comparison, the execution time of GERDA for 14k agents and 5k locations is about 30 minutes.
To facilitate traditional (static, linear, and in a quadratic sense optimal) state or output feedback design, the central question of [45] is how to construct and parameterize an ODE-based surrogate model. Our methodology is different in that it bypasses this challenge since we already have an ODE model (SEIR-type) that exhibits sufficiently analogous behavior to our ABM. Instead, we focus on designing a robust and predictive control algorithm for intervention planning. Our approach builds on this by integrating ODE-based control with an ABM, bridging the gap between mathematical tractability and real-world intervention strategies.
The main limitations of the approach are the following. We assume constant time periods between the possible changes of measures although there might be a need for the adaptive change of these intervals depending on the actual state of the epidemic. (We remark that the applied control design method technically allows such an adaptive solution.) Similarly, a finer resolution of the levels of certain measures might provide more precise reference tracking. Additionally, the currently used cost functions do not take into account the socio-economical impact of the infections in different age groups or that of the suggested interventions. However, these factors can be included in the optimization using, e.g., statistical data [4, 61] or dynamical model [62]. Moreover, the recursive feasibility of the MPC approach was not analyzed in this paper, although we have to add that infeasible runs or runs with too high variance are also informative, and suggest that the objectives cannot be met with high probability and/or more precise information is needed to handle the epidemic wave.
In summary, our approach is an important step in making mathematical model based epidemic control more realistic. Firstly, it is shown through various scenarios that an appropriately designed feedback can efficiently reduce the effect of model uncertainties which are inevitable especially in the initial phases of outbreaks. Secondly, the practical problems of given intervention periods of possibly several weeks and quantized input can be handled by the applied MPC methodology. Thirdly, the mapping of the controlled input to specific measures is also solved. Moreover, it is simple to re-parameterize the proposed computation and control design scheme for a different disease or virus variant.
Future work will be focused on the further differentiation and extension of the model, and on the optimization of the combinations of possible interventions based on their economic and societal impact.
Methods
State equations
The transition dynamics of the epidemic are characterized by the following discrete-time dynamical equations:
For simplicity, let the epidemic state and the parameter vector be denoted by
The transmission rate of the pathogen can be considered as the control input, which can be manipulated by the non-pharmaceutical interventions. The number of infected people
in the main phase of the disease is considered as an output.
To use the compartment model (2) as a control model for the ABM, we need to face the following challenges:
- CL1. Model parameters in
should be estimated as they do not appear explicitly in PanSim, but through intervals or probability distributions. Moreover, the transitions between the subcompartments of
,
,
in the ABM makes it even more difficult to infer the transition rates and probabilities of the compartments in the ODE model.
- CL2. The stochastic nature of the simulation makes it hard to approximate the average transmission rate of the pathogen.
In the following two sections, we address these two challenges. First, we address Challenge [Cl:Params]CL1, and propose a method to fit the ODE model to the ABM.
Model matching
To align the ODE model with PanSim, we calibrate the model coefficients of (2) such that the solutions best match the simulation data. For this, we first collected simulation data, considering an exhausting number of combinations of interventions in the different phases of the outbreak. A total of 1000 simulations were performed, each lasting 168 days. During each execution, we simulated six different intervention packages, with each package applied for h = 28 days. Based on the authors’ experience, a 4-week period is adequate to reveal the specific effects of an NPI.
Then, the model calibration is performed such that the results of PanSim for is considered as the “primary reference”. Then, we consider another measure h(xk) as a reference to estimate one single model constant
from the vector of constants
. Constant
is estimated such that the state vector xk should satisfy the recursion (2) for a given initial state x0 and with constant parameters
(except for
), furthermore,
and h(xk) should match the data obtained from the simulator as much as possible in a squared error sense. The estimation of
can be formalized as follows.
Problem 1 (Parameter estimation). Consider a discrete-time epidemic processes model (2) within the time horizon . Presume an initial epidemic state x0 at k = 0. Let
denote one distinguished parameter from the vector of parameters
, such that all the parameters from
are fixed except
. Furthermore, consider the following two sequences:
- R1.
as a reference for the number of people in the main phase of the disease having symptoms,
- R2.
as a reference for a function h(xk) of the epidemic state,
where . We are looking for
- UV1. the daily epidemic states in terms of the cardinality of the eight compartments xk,
,
- UV2. the daily transmission rates
, where
,
- UV3. the value of parameter
,
such that the state equations (2) are satisfied and the solution minimizes the following cost function:
where ,
,
are the weight parameters of the multi-objective optimization. (End of Problem 1)
The unknown variables of the optimization are declared in [pest:UV:state]UV1, [pest:UV:beta]UV2 and [pest:UV:param]UV3 of Problem [prb:pest]1. For the weight parameters we used ,
due to the multiple orders of magnitude difference between
and the slope of
.
The advantageous topology of the compartmental model (having no cycles in it), allows us to estimate the model constants one by one, such that changing a parameter in a later step does not change the effect of a parameter
in an earlier step significantly.
The parameter calibration relies on the observation that the relative cardinality of every compartments depends on the average residence time in that compartment. For example, a compartment with a shorter residence time empties faster than another compartment with a longer residence time. Consequently, a higher residence time results in a more populated compartment. On the other hand, the relative cardinality of pairs
,
,
also depend on the probability constants
,
,
.
Preliminarily, we fix the average length of the main phase of the disease based on serological analysis. We furthermore assume that
and
are known and fixed. Based on the clinical data we fix the average hospitalization time
.
Using these considerations and the preliminarily fixed constants, the parameters of the ODE model (2) are calibrated in the following steps:
- Step1. Align
, obtain
. The proportion between
and
is practically affected only by
and
. Parameters
and
affect the phase shift between
and
but not their proportion. Therefore, to estimate
, we consider the simulated number of susceptibles
as a secondary reference and solve the optimization in Problem 1. We note that the time-dependent variable
, which determines the shape of the epidemic curves is also searched alongside the parameter
.
- Step2. Align
, obtain
. After fixing
, we consider
as a secondary reference to estimate
.
- Step3. Align
, obtain
. After fixing
and
, we consider
as a secondary reference to estimate
.
- Step4. Align
, obtain
. Constant
, which strongly influences the ratio of
to
, is already calculated and fixed, hence,
is calibrated by fitting
to the simulation results.
- Step5. Align
, obtain
. The proportion between
and
is affected both by
and
. However,
can be well estimated from the clinical data, therefore,
can be estimated from the simulated results for
.
- Step6. Align
, obtain
. Finally, the simulated number of deceased people
is used to estimated
.
We considered the following values: ,
,
,
. For
we reused the values considered in [48] for the wild strain as initial guess. Then, through the parameter calibration, we obtained
,
,
,
,
,
.
Unknown input and state reconstruction
In order to simulate a real epidemic situation, we assume that the transmission rate and the number of infected individuals not showing symptoms (i.e., those in compartments ,
,
) are unknown. However, we presume the knowledge of the exact number of symptomatic patients in the main phase of the disease (
). As we already discussed, this assumption is not unrealistic, but it facilitates the presentation of the essence of the control approach presented in this paper.
In the following problem formulation, we propose a dynamical model-based data assimilation technique to compute average transmission rates per intervention planning cycle, and simultaneously estimate the daily epidemic state xk.
Problem 2. Presume an initial epidemic state x0 at k = 0. It is furthermore given a reference sequence for the number of people in the main phase of the disease having symptoms, where
. Assume that the time horizon [0,T] is split into p intervention planning cycles (
) of length h, i.e., ph = T. We are looking for
- UV1. the daily epidemic state in terms of the cardinality of the eight compartments xk,
,
- UV2. the average transmission rate
in each cycle
,
such that the epidemic state equations (2) are satisfied with
and the solution minimizes the following cost function:
where (wk) is a sequence of weights. Furthermore, is predefined sparse matrix, in which sum of each column is 1. (End of Problem 2)
Note that in Problem 1, the transmission rate was searched as a slowly variable daily series , where
. In contrast, in Problem 2, the transmission rate is computed as a piecewise constant staircase function, such that we are searching for the mean transmission rate
for each cycle
. E.g., when h = 7 days,
denotes the weekly mean transmission rate. This mean value is assumed to be a good estimate for all days in the corresponding cycle, therefore, in Problem 2, we are looking for the same
rate for every day
of each cycle j. This consideration is formalized in (12), where the kth column of matrix M determines, which value of
,
has to be used on day k. When the interventions are presumed to take effect promptly, matrix M is defined as follows:
However, this matrix allows us to model a gradual transition in between two interventions, e.g., a 2-day long transition can be described by the following matrix:
The uncertainty analysis of the reconstruction is presented in Sect 4 of S1 Text.
Construct the lookup table (LUT)
To construct a LUT, we identify how the distinct NPIs affect the average transmission rate in an h = 28 days long cycle. Through an unknown input state reconstruction (Problem 2), we approximate the average transmission rates resulted by the several NPIs simulated through the 1000 random executions. In the LUT, an average transmission rate and its standard deviation are assigned to each NPI. During the average computations, we omit those simulated days where less than 0.5% of the population is infected. The constructed LUT can be considered an input translation between the agent-based and compartmental ODE models or as a mapping between the relative transmission rate and the NPIs.
The standard deviations of the computed transmission rates obtained for the different NPIs are between 5% and 25%. This means that the effects of some NPIs are more uncertain and result in larger variations in the than other NPIs whose effects can be predicted more accurately. However, the standard deviations in
computed for the 216 NPIs has a median of 10%. Therefore, we chose 20% as the confidence bound, in the sense that if the (retrospectively) calculated
is more than 20% higher than the average value in the LUT, we attribute this to changes in pathogen properties. Otherwise, the anomaly in
can be credited to the stochastic nature of the process. The uncertainty of the entries in the lookup table is quantified in the supplementary material (S1 Table) and present in Sect 3 of S1 Text.
Control loop
The control algorithm has three major steps and one optional:
- I. Intervention planning for the future cycles.
- II. Apply the selected NPI for the next cycle.
- III. Unknown input reconstruction and state observation for the past cycles.
- IV. Update the mapping
.
The steps of the control loop are illustrated in Fig 1.
Step I. Intervention planning.
Let denote the current time at the end of an intervention planning cycle. Let
denote the next future cycle,
. We presume the knowledge of the current state xc. Then, we solve Problem 2 from the initial state xc within the time frame
, such that
constitutes the target curve for compartment
to be achieved. The program computes a sequence of transmission rates
for each upcoming cycles of the epidemics
. The first computed value for the transmission rates (
) is used as the targeted value for the next cycle. Utilizing the statistical model (i.e., the LUT), we select multiple NPIs that is expected to achieve a transmission rate close to the target through different strategies. This offers an additional degree of freedom in the control design, allowing for flexibility in choosing the most appropriate intervention. To ensure a minimal deviation between NPIs from cycle-to-cycle, we first identify NPIs whose (e.g.) the
confidence interval (mean
std) for
contains the target rate
.
Note that Problem 2 was formulated as an unknown input state reconstruction, nevertheless, it can be reused as an MPC program with the declared modifications.
Step II. Apply interventions.
The selected NPI is applied in the following cycle on days . When the control strategy is executed as an experiment, the NPI is simulated on the agent-based model using PanSim and the values for compartment
are collected as a measurement.
We remark that the MPC has computed transmission rates for the whole control horizon (for multiple future cycles), yet only the first control action is applied in the first future cycle. The remaining values will be recalculated in the upcoming intervention planning steps.
Step III. Reconstruction.
Using the measured values for as reference
, we solve the program in Problem 2 in time frame
to compute the new state xc + h and the actual transmission rate
achieved in the recent cycle q.
In real pandemic event, we collect the number of patients in hospitals, the number of daily hospital admissions, and the weekly viral loads measured in the wastewater treatment plants. These data allow a detailed reconstruction of the spread according to our enhance reconstruction method presented in [48]. This method computes not only ,
but also xc + h, and
. Considering the computed
as a measurement, we propose to execute a second reconstruction but now according to Problem (2). The second reconstruction may improve the accuracy of the predictions in the MPC for the following reasons. The epidemic model used in [48] is slightly different from (2) not including the waning immunity nor the viral load. Moreover, we found that a precise and detailed epidemic model gives bad predictions if the present state is not well-estimated. Similarly, we demonstrated in [48] that a bad epidemic model can produce good predictions if the present state is estimated according to the same model.
Step IV. Update the LUT.
If the transmission rate computed retrospectively is 20% higher than its expectation
(according to its actual mean value in the LUT), the transmission rate value in each entry of the LUT is increased proportionally by the one third of the measured rate increment. Namely, the transmission rate values in the LUT are multiplied by
, where K = 3. (This corresponds to an exponential forgetting with factor
.)
Overview and summary
It is noteworthy that the control input computation, intervention selection, reconstruction, and LUT update are executed only once in a cycle. But the time frame, in which the consecutive steps are executed is different. To clarify the time schedule of the algorithm in a single intervention planning cycle, we present a short summary of the time frames in the following equation:
The vertical lines in (16) separate the p intervention plannings cycles, the thick vertical line illustrates the current time, at which we plan a control action (I) and apply during cycle q (II). When cycle q comes to its end, we reconstruct the evolution of the spread (III), and scale the transmission rate values in the LUT if necessary (IV).
The time schedule suggests that the overall time frame of an epidemic control mission is fixed to T = ph = 210 days, such that the horizon of the MPC is decreasing from cycle to cycle since the end date of the horizon is not advanced with the time. We often call this implementation a decreasing horizon MPC. However, this algorithm can be easily modified to a so-called receding horizon MPC, if the horizon is advanced by h days from cycle to cycle.
Also note that the reconstruction is implemented in an increasing horizon fashion as the starting date of the reconstruction is not advanced in time. This is reasonable until an outbreak is evolving, however, when the number of infected people decreases below a given threshold, it may be useful to reset the starting date of the reconstruction to that date.
Throughout the paper, we described how we connected a robust controller to the complex microsimulation model, PanSim. The interface between the MPC and the simulator was implemented at three levels: parameter calibration, state reconstruction, and control action translation. First, we calibrated a compartmental ODE model to match the simulator’s dynamics by minimizing the squared output error, as detailed in Problem 1. Second, at each cycle, we reconstructed the simulator’s entire history and current state using measurements from a single observable quantity. Third, the computed control actions were translated into specific combinations of non-pharmaceutical interventions (NPIs) using a lookup table, updated with each cycle.
Supporting information
S1 Text. Alongside the hardware information of our framework, the supplement includes multiple figures illustrating the computed mean and variances in the lookup table, and the uncertainty of the reconstruction, and finally, it presents a control execution of Scenario A (“Flattening the curve”) in more pictures
https://doi.org/10.1371/journal.pcbi.1013028.s001
(PDF)
S1 Table. This supplement provides the numerical values of the statistical information in the lookup table, which maps a transmission rate to a combination of interventions.
https://doi.org/10.1371/journal.pcbi.1013028.s002
(XLSX)
References
- 1. Tartaglia R, La Regina M, Tanzini M, Pomare C, Urwin R, Ellis LA, et al. International survey of COVID-19 management strategies. Int J Qual Health Care. 2021;33(1):mzaa139. pmid:33219683
- 2. Perico N, Fagiuoli S, Di Marco F, Laghi A, Cosentini R, Rizzi M, et al. Bergamo and Covid-19: How the Dark Can Turn to Light. Front Med (Lausanne). 2021;8:609440. pmid:33681246
- 3. Montcho Y, Klingler P, Lokonon BE, Tovissodé CF, Glèlè Kakaï R, Wolkewitz M. Intensity and lag-time of non-pharmaceutical interventions on COVID-19 dynamics in German hospitals. Front Public Health. 2023;11:1087580. pmid:36950092
- 4. Demirgüç‐Kunt A, Lokshin M, Torre I. The sooner, the better: The economic impact of non‐pharmaceutical interventions during the early stage of the COVID‐19 pandemic. Econ of Transit and Inst Chang. 2021;29(4):551–73.
- 5. Reguly IZ, Csercsik D, Juhász J, Tornai K, Bujtár Z, Horváth G, et al. Microsimulation based quantitative analysis of COVID-19 management strategies. PLoS Comput Biol. 2022;18(1):e1009693. pmid:34982766
- 6. Chen K, Jiang X, Li Y, Zhou R. A stochastic agent-based model to evaluate COVID-19 transmission influenced by human mobility. Nonl Dyn. 2023;111(13):12639–12655. pmid:37361002
- 7. Chowell G, Sattenspiel L, Bansal S, Viboud C. Mathematical models to characterize early epidemic growth: A review. Phys Life Rev. 2016;18:66–97. pmid:27451336
- 8. Kong L, Duan M, Shi J, Hong J, Chang Z, Zhang Z. Compartmental structures used in modeling COVID-19: a scoping review. Infect Dis Poverty. 2022;11(1):72. pmid:35729655
- 9. Cao L, Liu Q, Hou W. COVID-19 modeling: a review. arXiv preprint 2021. https://arxiv.org/abs/2104.12556
- 10. Berestycki H, Desjardins B, Heintz B, Oury J-M. Plateaus, rebounds and the effects of individual behaviours in epidemics. Sci Rep. 2021;11(1):18339. pmid:34526528
- 11. Mangiarotti S, Peyre M, Zhang Y, Huc M, Roger F, Kerr Y. Chaos theory applied to the outbreak of COVID-19: an ancillary approach to decision making in pandemic context. Epidemiol Infect. 2020;148:e95. pmid:32381148
- 12. Hernández G, Martín del Rey A. Community-distributed compartmental models. Phys A: Statist Mech Appl. 2022;596:127092.
- 13. Silva DM, Secchi AR. Recursive state and parameter estimation of COVID-19 circulating variants dynamics. Sci Rep. 2022;12(1):15879. pmid:36151226
- 14. Hadley E, Rhea S, Jones K, Li L, Stoner M, Bobashev G. Enhancing the prediction of hospitalization from a COVID-19 agent-based model: A Bayesian method for model parameter estimation. PLoS One. 2022;17(3):e0264704. pmid:35231066
- 15. Ryu H, Abulali A, Lee S. Assessing the Effectiveness of Isolation and Contact-Tracing Interventions for Early Transmission Dynamics of COVID-19 in South Korea. IEEE Access. 2021;9:41456–67. pmid:36281327
- 16. Ghafouri-Fard S, Mohammad-Rahimi H, Motie P, Minabi MAS, Taheri M, Nateghinia S. Application of machine learning in the prediction of COVID-19 daily new cases: A scoping review. Heliyon. 2021;7(10):e08143. pmid:34660935
- 17. Lalmuanawma S, Hussain J, Chhakchhuak L. Applications of machine learning and artificial intelligence for Covid-19 (SARS-CoV-2) pandemic: A review. Chaos Solitons Fractals. 2020;139:110059. pmid:32834612
- 18. Li Y, Jia W, Wang J, Guo J, Liu Q, Li X, et al. ALeRT-COVID: Attentive Lockdown-awaRe Transfer Learning for Predicting COVID-19 Pandemics in Different Countries. J Healthc Inform Res. 2021;5(1):98–113. pmid:33426422
- 19. Perkins TA, España G. Optimal control of the COVID-19 pandemic with non-pharmaceutical Interventions. Bull Math Biol. 2020;82(9):118. pmid:32888118
- 20. Oke SI, Ekum MI, Akintande OJ, Adeniyi MO, Adekiya TA, Achadu OJ, et al. Optimal control of the coronavirus pandemic with both pharmaceutical and non-pharmaceutical interventions. Int J Dyn Control. 2023;11(5):2295–2319. pmid:36743263
- 21. Zhang G, Li Z, Din A, Chen T. Dynamic analysis and optimal control of a stochastic COVID-19 model. Mathematics and Computers in Simulation. 2024;215:498–517.
- 22.
Köhler J, Schwenkel L, Koch A, Berberich J, Pauli P, Allgöwer F. Robust and optimal predictive control of the COVID-19 outbreak; 2020.
- 23. Péni T, Csutak B, Szederkényi G, Röst G. Nonlinear model predictive control with logic constraints for COVID-19 management. Nonlinear Dyn. 2020;102(4):1965–86. pmid:33281298
- 24. Peni T, Csutak B, Bartha FA, Rost G, Szederkenyi G. Optimizing symptom based testing strategies for pandemic mitigation. IEEE Access. 2022;10:84934–45.
- 25. Howerton E, Ferrari MJ, Bjørnstad ON, Bogich TL, Borchering RK, Jewell CP, et al. Synergistic interventions to control COVID-19: Mass testing and isolation mitigates reliance on distancing. PLoS Comput Biol. 2021;17(10):e1009518. pmid:34710096
- 26. Pastor-Satorras R, Castellano C, Van Mieghem P, Vespignani A. Epidemic processes in complex networks. Rev Mod Phys. 2015;87(3):925–79.
- 27. Sulis E, Mariani S, Montagna S. A survey on agents applications in healthcare: Opportunities, challenges and trends. Comput Methods Programs Biomed. 2023;236:107525. pmid:37084529
- 28. Alamo T, Millán P, Reina DG, Preciado VM, Giordano G. Challenges and future directions in pandemic control. IEEE Control Syst Lett. 2022;6:722–7.
- 29. Goldenbogen B, Adler SO, Bodeit O, Wodke JAH, Escalera-Fanjul X, Korman A, et al. Control of COVID-19 Outbreaks under Stochastic Community Dynamics, Bimodality, or Limited Vaccination. Adv Sci (Weinh). 2022;9(23):e2200088. pmid:35607290
- 30. Snoeijer BT, Burger M, Sun S, Dobson RJB, Folarin AA. Measuring the effect of Non-Pharmaceutical Interventions (NPIs) on mobility during the COVID-19 pandemic using global mobility data. NPJ Digit Med. 2021;4(1):81. pmid:33986465
- 31. Sturniolo S, Waites W, Colbourn T, Manheim D, Panovska-Griffiths J. Testing, tracing and isolation in compartmental models. PLoS Comput Biol. 2021;17(3):e1008633. pmid:33661888
- 32. McCombs A, Kadelka C. A model-based evaluation of the efficacy of COVID-19 social distancing, testing and hospital triage policies. PLoS Comput Biol. 2020;16(10):e1008388. pmid:33057438
- 33. Shearer FM, McCaw JM, Ryan GE, Hao T, Tierney NJ, Lydeamore MJ, et al. Estimating the impact of test-trace-isolate-quarantine systems on SARS-CoV-2 transmission in Australia. Epidemics. 2024;47:100764. pmid:38552550
- 34. Abueg M, Hinch R, Wu N, Liu L, Probert W, Wu A, et al. Modeling the effect of exposure notification and non-pharmaceutical interventions on COVID-19 transmission in Washington state. NPJ Digit Med. 2021;4(1):49. pmid:33712693
- 35. Kerr CC, Stuart RM, Mistry D, Abeysuriya RG, Rosenfeld K, Hart GR, et al. Covasim: An agent-based model of COVID-19 dynamics and interventions. PLoS Comput Biol. 2021;17(7):e1009149. pmid:34310589
- 36. Hinch R, Probert WJM, Nurtay A, Kendall M, Wymant C, Hall M, et al. OpenABM-Covid19-An agent-based model for non-pharmaceutical interventions against COVID-19 including contact tracing. PLoS Comput Biol. 2021;17(7):e1009146. pmid:34252083
- 37. Lopez-Jimenez J, Quijano N, Dewasme L, Vande Wouwer A. Agent-based model predictive control of soil–crop irrigation with topographical information. Control Engineering Practice. 2024;150:106012.
- 38. Reguly IZ, Csercsik D, Juhász J, Tornai K, Bujtár Z, Horváth G, et al. Microsimulation based quantitative analysis of COVID-19 management strategies. 2021.
- 39. Jørgensen ACS, Ghosh A, Sturrock M, Shahrezaei V. Efficient Bayesian inference for stochastic agent-based models. PLoS Comput Biol. 2022;18(10):e1009508. pmid:36197919
- 40. Cavany SM, España G, Lloyd AL, Vazquez-Prokopec GM, Astete H, Waller LA, et al. Fusing an agent-based model of mosquito population dynamics with a statistical reconstruction of spatio-temporal abundance patterns. PLoS Comput Biol. 2023;19(4):e1010424. pmid:37104528
- 41. Angione C, Silverman E, Yaneske E. Using machine learning as a surrogate model for agent-based simulations. PLoS One. 2022;17(2):e0263150. pmid:35143521
- 42. An G, Fitzpatrick BG, Christley S, Federico P, Kanarek A, Neilan RM, et al. Optimization and control of agent-based models in biology: a perspective. Bull Math Biol. 2017;79(1):63–87. pmid:27826879
- 43. Niemann J-H, Klus S, Conrad ND, Schütte C. Koopman-based surrogate models for multi-objective optimization of agent-based systems. Phys D: Nonl Phenomena. 2024;460:134052.
- 44. Niemann J-H, Uram S, Wolf S, Djurdjevac Conrad N, Weiser M. Multilevel optimization for policy design with agent-based epidemic models. J Comput Sci. 2024;77:102242.
- 45. Fonseca LL, Böttcher L, Mehrad B, Laubenbacher RC. Optimal control of agent-based models via surrogate modeling. PLoS Comput Biol. 2025;21(1):e1012138. pmid:39808665
- 46. Savageau MA, Voit EO. Recasting nonlinear differential equations as S-systems: a canonical nonlinear form. Math Biosci. 1987;87(1):83–115.
- 47. Giuffrè M, Shung DL. Harnessing the power of synthetic data in healthcare: innovation, application, and privacy. NPJ Digit Med. 2023;6(1):186. pmid:37813960
- 48. Polcz P, Tornai K, Juhász J, Cserey G, Surján G, Pándics T, et al. Wastewater-based modeling, reconstruction, and prediction for COVID-19 outbreaks in Hungary caused by highly immune evasive variants. Water Res. 2023;241:120098. pmid:37295226
- 49.
Csutak B, Polcz P, Szederkenyi G. Computation of COVID-19 epidemiological data in Hungary using dynamic model inversion. In: 2021 IEEE 15th International Symposium on Applied Computational Intelligence and Informatics (SACI). 2021:91–6. https://doi.org/10.1109/saci51354.2021.9465563
- 50.
Csutak B, Polcz P, Szederkényi G. Model-based epidemic data reconstruction using feedback linearization. In: 2022 International Conference on Electrical, Computer and Energy Technologies (ICECET). 2022. 1–6.
- 51. Polcz P, Csutak B, Szederkényi G. Reconstruction of epidemiological data in hungary using stochastic model predictive control. Appl Sci. 2022;12(3):1113.
- 52. McEvoy D, McAloon C, Collins A, Hunt K, Butler F, Byrne A, et al. Relative infectiousness of asymptomatic SARS-CoV-2 infected persons compared with symptomatic individuals: a rapid scoping review. BMJ Open. 2021;11(5):e042354. pmid:33947725
- 53. Nakajo K, Nishiura H. Transmissibility of asymptomatic COVID-19: Data from Japanese clusters. Int J Infect Dis. 2021;105:236–8. pmid:33618004
- 54.
Horváth G, Szederkényi G, Reguly IZ. Quantifying and comparing the impact of combinations of non-pharmaceutical interventions on the spread of COVID-19. In: 2023 31st Mediterranean Conference on Control and Automation (MED). IEEE; 2023.
- 55.
Keomley-Horvath B, Horvath G, Polcz P, Siklosi B, Tornai K, Juhasz J. The design and utilisation of pansim, a portable pandemic simulator. In: 2022 First Combined International Workshop on Interactive Urgent Supercomputing (CIW-IUS). IEEE. 2022.
- 56. Volz E, Mishra S, Chand M, Barrett JC, Johnson R, Geidelberg L, et al. Assessing transmissibility of SARS-CoV-2 lineage B.1.1.7 in England. Nature. 2021;593(7858):266–9. pmid:33767447
- 57. Lyngse FP, Mortensen LH, Denwood MJ, Christiansen LE, Møller CH, Skov RL, et al. SARS-CoV-2 Omicron VOC transmission in Danish households. 2021.
- 58. Chen Z, Deng X, Fang L, Sun K, Wu Y, Che T, et al. Epidemiological characteristics and transmission dynamics of the outbreak caused by the SARS-CoV-2 Omicron variant in Shanghai, China: A descriptive study. Lancet Reg Health West Pac. 2022;29:100592. pmid:36090701
- 59. Ye L, Li WF, Shao J, Xu Z, Ju J, Xu H. Fighting Omicron epidemic in China: Real-world big data from Fangcang shelter hospital during the outbreak in Shanghai 2022. J Infect. 2022;85(4):436–80. pmid:35835412
- 60.
European Centre for Disease Prevention and Control. 2021. Contact tracing: public health management of persons, including healthcare workers, who have had contact with COVID-19 cases in the European Union – third update. https://www.ecdc.europa.eu/sites/default/files/documents/covid-19-contact-tracing-public-health-management-third-update.pdf
- 61. Correia S, Luck S, Verner E. Pandemics depress the economy, public health interventions do not: evidence from the 1918 flu. J Econ Hist. 2022;82(4):917–57.
- 62. El Jai M, Zhar M, Ouazar D, Akhrif I, Saidou N. Socio-economic analysis of short-term trends of COVID-19: modeling and data analytics. BMC Public Health. 2022;22(1):1633. pmid:36038843