Rationale

Epidemic outbreaks present complex decision-making challenges that require adaptive interventions [1]. Traditional modeling and control strategies, such as rule-based systems or fixed policy heuristics, often struggle to capture the dynamics of evolving epidemic environments or to optimize long-term outcomes under uncertainty [2]. Reinforcement learning (RL) mitigates this limitation by iteratively refining intervention strategies within simulated environments, where real‑world experimentation would be impractical or unethical [3,4].

Interest in RL intensified during the COVID‑19 pandemic, when researchers used it to design non‑pharmaceutical interventions such as mask mandates, mobility restrictions, social distancing measures, etc. This approach has demonstrated that RL can balance competing objectives such as infection control and economic preservation. Agent-based modeling (ABM) further enhances this potential by simulating the behavior and interactions of heterogeneous individuals, yielding more realistic and granular simulations of epidemic dynamics and policy effects [5].

Nevertheless, the field remains fragmented. Studies differ in algorithmic choices, modeling assumptions, and evaluation metrics, complicating cross‑study comparison. A comprehensive mapping of how RL methods are currently employed for epidemic policymaking is lacking. This scoping review addresses this gap by analyzing the state of research at the intersection of epidemic control and RL.

Objectives

This review aims to map the current research landscape on the use of RL methods for epidemic control policymaking. Specifically, the objectives are:

1. To identify and categorize studies that apply RL algorithms to NPI design in epidemic settings.
2. To analyze how these studies model epidemic dynamics, agent behaviors, policy environments, and what model validation techniques are used.
3. To characterize the RL methods employed, including algorithmic types, reward design, and state and action spaces.
4. To identify methodological patterns, research gaps, and potential directions for future work in this domain.

Eligibility criteria

Study design.

We included only studies that employ deep RL techniques and, preferably but not necessarily, ABM. We excluded studies that rely exclusively on statistical modeling, rule-based decision-making, or other non-RL methods.

Search strategy

The search was conducted on December 23, 2025, in computer science and multidisciplinary databases, specifically IEEE Xplore, ACM Digital Library, ScienceDirect, and Scopus.

Three main categories were identified, each with corresponding keywords:

1. Artificial Intelligence: reinforcement learning.
2. Agent-Based Modeling: multi-agent systems, agent modeling, agent-based modeling, intelligent agents, autonomous systems.
3. Epidemics: epidemics, pandemics, outbreak, infectious disease, public health.

For Scopus, IEEE Xplore, and ACM Digital Library, the search string was the following:

“reinforcement learning” AND (“multi-agent systems” OR “agent modeling” OR “agent-based modeling” OR “intelligent agents” OR “autonomous systems”) AND (epidemics OR pandemics OR outbreak OR “infectious disease” OR “public health”).

For Science Direct, due to search string length restrictions, it was shortened to the following:

“reinforcement learning” AND (“multi-agent systems” OR “agent modeling” OR “agent-based modeling”) AND (epidemics OR pandemics OR “infectious disease” OR “public health”).

Keywords were searched in the Title, Abstract (Summary), and Keywords fields.

Study selection

The selection process was conducted using the Covidence information system. Both authors independently screened the titles and abstracts of eligible studies, followed by a full-text review of those that passed the initial screening. Any discrepancies were resolved through discussion.

Data extraction

The following key components were extracted from all included studies: bibliographic information, methods, experiments, results, and discussion. Data items for each component were defined as follows:

1. Bibliographic details: title, authors, publication year, and journal.
2. Methods: reinforcement learning algorithm, epidemic modeling approach, and software for simulation.
3. Experiments: RL-specific characteristics, experimental design, and model validation techniques.
4. Results: epidemic mitigation outcomes and RL-related findings.
5. Discussion: stated limitations and suggested future research directions.

Data extraction was conducted by OB and independently verified by DC. Disagreements were resolved through discussion.

Synthesis of results

Comparative tables were created to summarize study characteristics, including epidemic models, RL algorithms, intervention types, etc. The synthesis is focused on identifying common trends, differences in RL methodologies, and potential research gaps. Findings were analyzed qualitatively and quantitatively, highlighting strengths and limitations across studies.

Use of artificial Intelligence tools

We used ChatGPT (model o3) exclusively to improve the readability and stylistic clarity of this manuscript. All AI-generated suggestions were carefully reviewed, revised, and validated by the authors to ensure factual accuracy. The final wording, scientific interpretations, conclusions, and limitations reflect the authors’ own ideas and intellectual contributions. The authors bear full responsibility for the integrity and originality of the content presented.

Study selection

Fig 1 presents the PRISMA flow diagram. The keyword search identified 512 studies, with 17 duplicates removed. During the screening phase, 460 papers were excluded as irrelevant. Of the 35 papers assessed in the full-text review, 25 were excluded for the following reasons: no policymaking epidemic control (n = 17), no deep learning (n = 2), and no RL (n = 6). Additionally, three studies were included through reference list scanning.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flow diagram.

https://doi.org/10.1371/journal.pone.0351176.g001

Study characteristics

Table 1 contains key characteristics of the selected studies, including bibliographic details (title and authors), the RL algorithm, and the epidemic dynamics model.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. General characteristics of the studies.

https://doi.org/10.1371/journal.pone.0351176.t001

Epidemic dynamics modeling.

The reviewed studies utilize two primary epidemic modeling approaches: compartmental models and agent-based models. While these are often considered distinct methodologies, some studies integrate both into hybrid models, combining the strengths of each approach.

Compartmental models divide the population into distinct health states and describe transitions between these states using differential equations, assuming a homogeneous population [21,22]. These models are particularly useful for large-scale epidemic forecasting and intervention planning, as they estimate infection trends, hospitalization needs, and intervention effects at the population level [23]. Among the reviewed studies, variations of the SIR (Susceptible-Infected-Removed) and SEIR (Susceptible-Exposed-Infected-Removed) models were commonly used. Study [9] followed the standard SEIR formulation, while others expanded it with additional compartments. Study [7] employed the SEIHRD model, introducing a hospitalization state and splitting the “removed” compartment into “recovered” and “deceased.” Study [12] took a slightly different approach, using an SIQHRD model, where Q represents a quarantined state. Study [13] modified the standard SEIR model into MID-SEIR, incorporating external intervention effects. Study [15] proposed a stochastic generalized SIR (GSIR) model in which intervention levels directly affect the infection rate. Study [18] utilized a networked SIS model to represent infection dynamics without permanent immunity. Study [19] expanded the SIR framework with a Dynamic SIHR model, including a hospitalization state and online parameter updates.

Unlike compartment models, ABM captures population heterogeneity, simulating epidemic spread at the individual level, where agents represent people with unique characteristics, behaviors, and movement patterns. Transmission occurs through direct agent interactions, making ABM useful for modeling mobility, social behavior, and localized outbreaks [5]. Six studies, [8,10,11,14,16,17], used a purely agent-based approach. In these studies, both transmission and disease progression were modeled at the individual level, with infected agents transitioning through disease states based on probabilistic state changes, rather than differential equations applied to the population.

One study [7] combined ABM with a compartmental model, creating a hybrid model that balances realistic agent interactions with structured epidemic progression. In this case, ABM governed agent interactions and exposure, while the compartmental model dictated infection progression rate.

Table 2 summarizes the key RL components across studies, including algorithm type, state representation, action space, and reward formulation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. RL characteristics of the studies.

https://doi.org/10.1371/journal.pone.0351176.t002

Epidemiological validity.

The epidemiological validity of reinforcement learning policies depends on the accurate parameterization of disease transmission and progression models, ensuring that the model accurately reflects the underlying biological reality. Across the reviewed studies, researchers handle the basic reproductive number R₀ using three primary approaches: as an explicit input, an emergent simulation property, or a dynamically estimated variable. Studies [9,11,13,17,18] represent the first approach and define explicit R₀ values to characterize specific pathogens or variants. The second strategy, where R₀ serves as an emergent property of individual interactions tuned to match the historical trends, is used in studies [8,10,14,16]. Finally, the third approach focuses on real-time estimation from empirical data [7,12,15,19].

Biological parameters defining disease progression, such as incubation periods, latent periods, and fatality ratios, are mostly based on clinical reports and peer-reviewed medical literature. Studies [7,8,10,14,17,19] use COVID-19 clinical investigations to justify disease progression parameters and health-state transitions. Study [15] relied on estimates from the SARS pandemic to set priors for disease features. Study [11] utilized variant-specific data from medical journals regarding vaccine effectiveness. Study [18] relies on assumed parameters for a synthetic networked SIS model to demonstrate algorithmic performance without specific clinical grounding. Studies [9,10,17] used data from national statistical, census, and demographic databases to tailor simulation environments to specific geographic contexts.

The justification of contact rates and social mixing patterns relies on either empirical behavioral data or structural assumptions. Study [9] integrated age-structured contact matrices from an internet-based social contact survey to capture social mixing patterns within a meta-population framework. Studies [17,19] utilize large-scale empirical mobility data from mobile phones to parameterize contact patterns through observed population movement. Studies [12,13,18] approximate interaction frequencies through administrative boundary analysis or synthetic network topologies, where transmission parameters are assumed rather than empirically sourced.

Researchers used various methods to make their models match real epidemics. Most studies used COVID-19 data from regions like China, Sweden, Canada, Italy, and the USA. The calibration rigor differs across studies: some studies relied on literature-based assumptions and manual alignment with historical peaks, while others used statistical methods to derive parameters from health surveillance data. Validation was primarily achieved by reproducing historical outbreak trajectories or comparing simulation outputs against compartmental models. Almost all authors conducted sensitivity analysis to see how changes in key epidemiological parameters affect policy performance or epidemic dynamics. Table 3 summarizes validation and calibration methods used in the reviewed studies.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Calibration and validation methods.

https://doi.org/10.1371/journal.pone.0351176.t003

Individual study findings

Table 4 summarizes the results of the studies included in the review. Each study is analyzed regarding its simulation setup, epidemic control outcomes, and policy evaluation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Results of the studies.

https://doi.org/10.1371/journal.pone.0351176.t004

The study by Shuvo et al [7] simulates COVID-19 dynamics over 50-week episodes, with the RL agent selecting among four intervention options: full, partial, minimal lockdowns, and no action. While the model simulates different lockdown intensities and their societal impacts, the study does not provide direct epidemic outcome metrics such as infection rates or mortality reduction. Instead, outcomes are evaluated using a normalized reward function balancing health burden and intervention severity. Among the algorithms tested, N-step Deep Q-Learning (N = 3) consistently outperforms Deep Q-Learning, Double DQ-Learning, Deep SARSA, and Dueling DQ-Learning in terms of both normalized total reward and stability. Although epidemic-specific outcomes remain implicit, the improved reward suggests greater intervention efficiency. Additionally, the model was compared‌‌ against deterministic and random agents to demonstrate its effectiveness.

Zhang et al [8] conduct the simulation in a NetLogo environment with a 64 × 64 grid world, where 500 agents navigate daily routines, including homes, workplaces, and hospitals. The simulation spans 50 days, with the RL agent making decisions every 5 days starting from the tenth day. The sensitivity analysis indicated that mask-wearing (0.94 sensitivity), staying home (0.93), and limiting gatherings (0.92) were the most effective measures. Area lockdown (0.60) and isolation of infected individuals (0.78) were less impactful but still contributed to epidemic control. The analysis of the prioritization of various factors in the reward function shows that when prioritizing economic or health factors, the model favors strict lockdowns until infections decline to a safe level, whereas emphasizing psychological factors leads to a preference for minimal restrictions. Balancing all factors results in a more lenient approach, maintaining some restrictions while avoiding excessive lockdown measures.

Libin et al [9] present a simulation of pandemic influenza spread across Great Britain using a meta-population model of 379 interconnected administrative districts, each represented by an age-structured stochastic SEIR model. The RL agent controls weekly school closures with a limited budget. Simulations were run over 43 weeks. Two kinds of experiments were conducted. In the first one, a single-district setting was used to establish a ground truth via an exhaustive search in a deterministic version of the model. For ten demographically diverse districts and two reproductive numbers (R₀ = 1.8 and 2.4), the learned policies closely matched the ground truth in reducing the proportion of the population infected under various school closure budgets (2, 4, 6 weeks), confirming that the model converges to near-optimal solutions. In the second experiment, multi-agent coordination was evaluated by learning a joint policy across 11 highly interconnected districts in the Cornwall-Devon region and comparing it to independent policies learned separately for each district and applied simultaneously. While both approaches reduced infections, the joint policy consistently achieved lower infection levels, particularly under moderate transmission (R₀ = 1.8), demonstrating a clear collaborative advantage.

Zong and Luo [10] propose the Multi-Agent Recurrent Attention Actor-Critic (MARAAC) algorithm, an RL approach designed to handle time-varying epidemic environments, and apply it to simulate the spread of COVID-19 across four US states, California, Arizona, Nevada, and Utah, each with distinct demographic and economic characteristics. The simulation spans 20 weeks, with weekly updates to lockdown policies. The environment simulates daily population movement and interactions across various locations. Two experimental settings were used: in the first, each state was assigned an RL agent; in the second, RL agents managed specific location types within each state. Results show that the RL-based policy adapts effectively to state-specific conditions: California implemented prolonged strict lockdowns due to its large population, Nevada applied strict initial measures reflecting its tourism-driven exposure, while Utah maintained minimal interventions and still achieved the lowest death toll, benefiting from a small, young population. The learned policies effectively controlled infection levels and minimized deaths in all cases. The proposed algorithm outperformed two other multi-agent algorithms (MADDPG and MAAC) in terms of training stability, convergence speed, and final reward values. It also showed robust transferability when scaled to an environment with 100,000 agents.

Guo et al [11] introduce PaCAR, a pandemic control decision-making framework integrating large-scale ABM. The simulation replicates epidemic dynamics in King County, USA, and Beijing, China, utilizing over 10 million agents and runs for a maximum of 200 days per trial, with the RL agent taking action every three days. The model incorporates vaccines and non-pharmaceutical interventions. To evaluate the framework, the authors designed benchmark policies, three representing different levels of restrictions and two others simulating real-world governments’ approaches: China’s relatively strict policy and Sweden’s open policy. PaCAR framework demonstrates superior epidemic control compared to them. In King County (population of 2.23 million), without vaccines, the epidemic duration was reduced to 41 days, requiring only 6 days of lockdown, whereas other policies lasted 118–141 days with longer lockdowns. Infection peaks remained under 30 cases, far lower than the 1447–1857 in non-adaptive approaches. These numbers are slightly higher than those under strict policies, but PaCAR achieves lower economic losses. With vaccines, PaCAR controlled the Alpha variant in 41 days without lockdowns and mitigated the Delta variant in 47 days, requiring only 18 days of lockdown compared to 37–45 under other policies. In Beijing (population of 21 million), it outperformed all benchmarks, containing the Delta outbreak in 87 days with just 6 days of lockdown, whereas the strictest alternative required 59 days. PaCAR leverages DQN augmented with a Transformer-based self-attention mechanism, enabling the agent to make decisions based on historical data trends rather than single-day statistics. The analysis demonstrated that the best performance was achieved with 1–2 transformer layers and 20-day input. Shorter sequences (5 days) lacked sufficient temporal context, while longer ones (40 days) diluted relevant information.

Hu et al [12] introduce the decentralized graph-based RL using reward machines (DGRM) algorithm to simulate COVID-19 spread across Italy’s 20 regions, each modeled as a separate agent within a graph-based MDP framework. The regions interact based on fluxes of movement between them. The simulation lasts 28 days with daily discretization, and each region selects from four actions: no intervention, social distancing, inter-regional flux control, or full lockdown. Initial conditions are set such that all regions start in a severe epidemic state. The control objective is that each region would not be in a severe situation for two consecutive weeks, and avoid a long lockdown of two consecutive weeks. Epidemic outcomes show that DGRM with k = 1, meaning each agent’s policy and Q-function are based on local information within its 1-hop neighborhood, enables 19 of 20 regions to avoid sustained lockdowns and hospital saturation. The method outperforms the bang-bang baseline by improving the global accumulated reward by 119%, and only one region failed to meet control goals, compared to 13 under the baseline. DGRM with k = 1 improves global discounted reward by 218% compared to k = 0. Increasing k beyond 2 showed no significant gain due to higher learning complexity.

Du et al [13] present a simulation based on COVID-19 data from two Chinese cities, Changchun and Shanghai, which were selected due to their contrasting population densities and sizes. The simulation spans one month in each city, corresponding to real outbreaks in March-April 2022, with the RL agent making intervention decisions daily. The effectiveness of the proposed algorithm (HRL4EC) was compared to several baselines: ground truth (official intervention policies), no intervention, random, and DRL-EC (an algorithm that implements only a city lockdown). The study reports that HRL4EC consistently outperforms all baselines, achieving the lowest cumulative infection rates in both cities. Key insights include the effectiveness of moderate mobility constraints, greater temporary medical resource demand in more densely populated areas, and the benefit of low-level, sustained necessities supply to maintain compliance. Multi-mode interventions are generally preferred, but the agent sometimes switches to single-mode interventions in early or low-density outbreak conditions. In general, HRL4EC demonstrates generalizability by adapting policies appropriately across cities with different structural and demographic attributes. A sensitivity analysis reveals the model’s robustness across various reproductive numbers (R₀ = 7.5 to 8.5), budget levels (80–100%), and economic-health trade-off weights.

Kompella et al [14] introduce an agent-based simulation platform modeling fine-grained human interactions across various locations in a 1,000-agent community. Each episode simulates 120 days, with the RL agent selecting one of five restriction levels daily and observing only the imperfect testing information and number of hospitalizations. The study evaluates multiple benchmark policies, including three handcrafted rule-based interventions and models of Swedish and Italian governmental strategies. Without intervention, the entire population becomes infected, and hospital capacity is exceeded early. Static policies show expected trends: higher restriction levels reduce infection peaks and deaths but increase the economic burden. The RL-optimized policy adapts to infection dynamics, keeping critical cases below capacity while minimizing restrictions. It consistently outperforms all baselines across epidemic and economic metrics, including infection peak, death rate, and cumulative reward. The policy generalizes well when tested in a larger 10,000-agent population and under reduced action frequency (every 3 or 7 days).

Wan et al [15] developed a multi-objective, model‐based framework for real-time infectious disease control, demonstrated through simulations of COVID-19 dynamics in six Chinese cities over 120 days. The RL agent selects among three intervention levels, ranging from minimal action to stringent lockdown, every seven days. The Pareto‐optimal policies derived via grid search and a DQN-based method consistently outperform baseline policies in lowering both infection counts and economic losses. The framework’s robustness is further validated through sensitivity analyses involving varying hyperparameters, employing a generalized SEIR model, and testing the model using less infectious H1N1 virus parameters. Additionally, the model produces prediction bands that reliably capture the observed epidemic curves and cost trajectories, with cross-validation yielding an average error ratio of 18%.

Ohi et al [16] present a virtual environment simulating a COVID-19 pandemic. The simulation is executed within a 2D grid world of 10,000 agents and lasts until the disease is fully mitigated. Economic contribution depends on individuals’ mobility and health, meaning that infectious and deceased individuals do not contribute to the economy. The RL agent selects movement restrictions daily across three levels: 0%, 25%, and 75% movement restrictions. Comparing agents with different memory lengths, the best-performing one with a 30-day memory achieves minimal deaths and infections while securing better daily and total economic gains. It applies initial strict lockdowns to curb the first wave and follows with cyclic and short-term lockdowns to suppress resurgence. The learned policy outperforms traditional cyclic lockdown strategies like the 7-work-7-lockdown rule in flattening infection curves.

Luo et al [17] simulate the spread of respiratory infections using an agent-based model of Shenzhen, China. The simulation involves 1.6 million agents distributed across 673 administrative communities over 150-day episodes. The study introduces the Hi-RICE framework, where RL agents make daily regional decisions to set the intensity levels for isolation, home quarantine, and community confinement. To manage the massive scale, the authors use a multi-agent reinforcement learning algorithm (MAPPO) with centralized training and decentralized execution, using LSTM layers to process temporal data. Individual-level interventions are then assigned based on a risk assessment model (I-RAM) that considers both movement history and social contact density. The framework was tested against ten baselines, including expert heuristics and state-of-the-art machine learning methods, across low, medium, and high R₀ scenarios. Results show that Hi-RICE significantly outperforms all benchmarks in balancing infection control and economic costs. The agents demonstrate proactive behavior, applying strict measures in the early stages of an outbreak and shifting to targeted, less restrictive control as cases decline. Sensitivity analyses further confirm the model’s robustness to R₀ uncertainty and varying levels of public compliance.

Samadi et al [18] apply a networked SIS model to simulate disease spread on weighted Erdős–Rényi graphs with 50, 75, and 100 nodes. They use DDPG algorithm to manage a continuous action space. Decentralized agents control specific subgraphs by adjusting connection weights, which represents the intensity of social distancing. Each agent operates under partial observability and a strict budget constraint. The study compares trained DDPG agents against untrained baselines across varying agent counts. Results indicate that in a 50-node network with five agents, the average infection probability was reduced from 0.728 to 0.488, while in a 100-node scenario, it decreased from 0.821 to 0.780. Sensitivity analysis shows that increasing the number of agents significantly enhances epidemic control, although the framework’s effectiveness diminishes as the network size scales up without a proportional increase in the number of agents.

Luo et al [19] use the H2-MARL framework, applied to COVID-19 control across four Chinese cities (Guangzhou, Wuxi, Chongqing, and Ezhou) using township-level meta-population models. In this environment, agents manage daily mobility quotas to balance hospital capacity against movement restrictions. A key technical innovation is the use of the Entropy Weight Method to dynamically weight these competing objectives, which helps the policy to adapt to environmental uncertainty. Furthermore, the model incorporates accumulated mobility losses with a decay factor into the state representation; this serves a role similar to sequence modeling techniques in other studies by providing the agent with long-term context regarding intervention fatigue and past policy impacts. Results showed that H2-MARL significantly outperformed heuristic and standard RL baselines, such as PPO and DQN. In Guangzhou, the framework maintained approximately 63% mobility while reducing hospital strain by 25.9% compared to existing graph-based RL approaches. The learned policies also exhibited spatial adaptability, imposing stricter restrictions on high-density urban centers while allowing greater flexibility in sparsely populated districts.

Synthesis of results

This section synthesizes the key methodological patterns and design choices identified across the reviewed studies regarding the unique challenges of applying RL to epidemic control, such as time delays, population diversity, and data uncertainty.

Summary

The reviewed studies exhibit recurring methodological patterns and design choices that reflect how RL is currently applied in epidemic policymaking. Key trends relate to algorithm selection, epidemic modeling strategies, temporal data processing, and reward design.

All studies showed that RL-based policies outperform heuristic policies and real-world governmental strategies across key epidemiological and economic metrics. In those studies where adaptability was explicitly evaluated, the proposed models demonstrated the ability to (1) maintain performance under varying epidemic parameters and environmental conditions [8–13,15,17,19], and (2) scale effectively to larger populations and more complex environments in agent-based simulations [10,11,14]. These findings indicate that RL methods offer superior performance in epidemic mitigation and show strong generalization and robustness. This can be explained by the key strength of RL in this domain, which is its ability to navigate high-dimensional and stochastic state spaces to discover non-obvious, adaptive NPI strategies that human experts might overlook. This enables context-sensitive approaches where interventions are dynamically tuned to the specific velocity and scale of an outbreak. Conversely, a critical limitation is the sensitivity to the “sim-to-real gap,” where policies trained on idealized models may lack robustness when faced with the unpredictable behavioral shifts and structural complexities of actual populations.

This gap is directly linked to the data reliability, which determines the accuracy of the simulated environment and the resulting effectiveness of RL-trained policies. It is important since the reviewed studies utilize public health surveillance records from the COVID-19, 2009 H1N1 influenza, and SARS pandemics. Poor data quality, specifically low reporting accuracy and temporal lags, limits the agent’s ability to identify the true epidemic state. When data is sparse or delayed, the agent develops policies based on an inaccurate perception of the epidemic dynamics. This leads to miscalculated intervention timing or intensity, which can cause hospital overcapacity or prolonged socio-economic disruption in real-world applications. High data availability allows for more granular calibration of age-, demographic-, or region-specific parameters.

The introduction of sequence modeling methods is an important step towards overcoming these data-related challenges and the non-Markovian nature of epidemic control. These methods specifically address the time lags between an intervention and its epidemiological outcome, which otherwise complicate the reward attribution process. Studies using sequence-aware models report improved policy performance, suggesting that incorporating temporal structure into the state representation is a beneficial design choice. On the other hand, approaches that still rely on short-term reward signals can lead to sub-optimal decisions due to the agent’s inability to correctly relate current interventions to the delayed epidemiological effect. Future research should focus on methods related to the Credit Assignment Problem to avoid the “myopia” of policies in real conditions.

One of the most evident methodological patterns is the frequent use of Deep Q-Networks. Five out of thirteen studies employed DQN or its variants as the primary algorithm for epidemic control. Several factors may explain this preference. DQN was originally designed for environments with a limited set of discrete actions, such as the Atari games [24], making it well-suited for epidemic policymaking tasks that involve selecting from a limited set of intervention strategies. DQN offers reasonable sample efficiency, which is beneficial in computationally intensive simulations. Its relative simplicity and ease of implementation make it particularly accessible for researchers from healthcare and epidemiology. The explicit assignment of Q-values to actions enhances interpretability, which is advantageous when communicating decisions to policymakers. Finally, DQN’s prominence as one of the earliest and most widely adopted deep RL algorithms may contribute to its continued popularity. In contrast, policy-based and hybrid algorithms, like PPO and SAC, offer advantages in stability or robustness under partial observability, but often at the cost of increased complexity and training instability in small discrete settings.

Another critical design choice concerns how epidemic dynamics are modeled. The reviewed studies employ three epidemic modeling approaches: compartmental models, agent-based models, and hybrid combinations. Compartmental models offer mathematical tractability for population-level forecasting, but rely on the assumption that all individuals within a compartment have an equal probability of being infected. This assumption represents a significant limitation for fine-grained policy design, as it fails to capture localized transmission hotspots or the influence of individual mobility patterns on transmission. ABM is generally preferable for epidemic control, as it enables the representation of individual-level heterogeneity, behavioral patterns, and spatially structured interactions. By simulating these micro-level details, ABM allows RL agents to evaluate the impact of granular measures, such as localized lockdowns or specific gathering restrictions, which population-level models cannot adequately resolve. However, ABM is computationally intensive and often challenging to scale. Nevertheless, Guo et al [11] developed a simulation framework involving over 10 million agents, demonstrating that large-scale, detailed ABM is feasible for RL-based epidemic control.

The reward function is a central component in RL. It directly influences the trade-offs the agent learns to make. In eleven out of thirteen studies, the reward function is constructed to balance epidemiological outcomes with social or economic costs, reflecting the trade-offs inherent in real-world policymaking. Despite the prevalence of this design, the representation of economic loss is generally simplified. In most cases, it is incorporated indirectly by penalizing the duration or intensity of lockdowns, serving as a proxy for economic damage. Less commonly, economic loss is linked to epidemic severity, where infections or deaths reduce the agent’s reward to reflect assumed productivity loss. While economic trade-offs are central to the policy objective, their modeling remains limited in fidelity and often lacks grounding in real economic indicators.

The transition from theoretical modeling to practical deployment demands a fundamental shift beyond purely epidemiological metrics. Although RL agents demonstrate high technical performance, their long-term viability depends on how effectively they navigate social dynamics and ethical constraints. A critical gap remains in the modeling of recursive feedback loops between policy execution and public sentiment. Neglecting these behavioral dynamics, particularly the phenomenon of behavioral fatigue, risks producing mathematically optimal policies that are unsustainable in real-world contexts.

Similarly, ethical challenges are typically reduced to balancing weights in a multi-objective reward function. Beyond acknowledging mental health consequences [8] or privacy risks [11], the reviewed studies provide few mechanisms for ensuring the accountability of autonomous decision-making. Transitioning to practical deployment requires risk-sensitive architectures that treat social compliance as a dynamic state and incorporate explicit ethical constraints. RL agents should function as transparent decision-support tools rather than autonomous enforcers to maintain public trust.

Limitations

This scoping review has several limitations that may influence the interpretation and generalizability of its findings. First, the search strategy was limited to four major academic databases and used a predefined set of keywords. Although these databases are comprehensive, relevant studies indexed elsewhere or using alternative terminology might have been omitted.

Second, the eligibility criteria focused exclusively on studies where a RL agent explicitly acts as a policymaker for non-pharmaceutical interventions. While this strict focus ensures conceptual coherence, it also excludes related applications of RL in epidemic control, such as resource allocation or crowd avoidance, that might offer transferable insights. For instance, studies focused strictly on testing might offer more sophisticated methodologies for state estimation, and pure resource allocation research could offer insights into logistical constraints and multi-objective trade-offs that are often simplified in NPI-centric works.

Third, the search strategy primarily focused on the intersection of RL and agent-based modeling. Therefore, this study is subject to a selection bias: research applying RL with compartmental models was likely omitted if the text did not include ABM-related terms. While some studies relying only on compartmental dynamics were captured, the results mainly represent literature that aligns with both reinforcement learning and agent-based modeling.

Fourth, this review did not conduct a formal critical appraisal of the included studies. Given that scoping reviews aim to map the extent and nature of evidence rather than evaluate methodological quality, this omission aligns with standard practice. However, it prevents any conclusions regarding the internal validity or empirical robustness of the included models.

Finally, heterogeneity in simulation environments, epidemic models, evaluation metrics, and inconsistencies in reporting key experimental details complicate direct comparisons. Although common trends were identified, differences in modeling assumptions and experimental setups limit the ability to generalize performance claims or compare algorithmic efficiency under unified conditions.

Conclusions

This scoping review mapped how RL is currently employed to design non‑pharmaceutical intervention policies for epidemic control. Across thirteen studies, RL algorithms consistently surpassed heuristic strategies in reducing infections, deaths, or lockdown duration while mitigating socio‑economic disruption. Agent‑based simulations enabled granular modeling of population heterogeneity; compartmental or hybrid models offered tractability. Sequence‑aware state representations improved adaptability to dynamic epidemic conditions. Findings support the feasibility of RL‑driven policymaking frameworks when real‑world experimentation is impractical or unethical.

Nevertheless, methodological diversity limits cross‑study comparability. Economic costs are often simplified, epidemic models differ in calibration rigor, and evaluation metrics lack standardization. Few studies quantify uncertainty, assess policy robustness to data shifts, or verify transferability beyond the simulated setting.

Several research gaps should be addressed to move RL from simulation studies to practical epidemic-response applications. First, common benchmark environments and reporting standards are required to make comparative evaluations rigorous and reproducible. Second, empirically grounded economic and behavioral models should be integrated to capture broader societal trade‑offs. Third, uncertainty‑aware and probabilistic RL frameworks need to be adopted to enhance robustness to data noise and partial observability. Fourth, the more sophisticated action spaces should be developed by applying hierarchical and hybrid (discrete‑plus‑continuous) methods. These methods should decompose composite restriction levels into individually adjustable interventions and support spatially targeted interventions. Fifth, implementing counterfactual methods allows for what-if assessments of historical outbreaks to evaluate the effectiveness of specific interventions. Sixth, RL models should incorporate traditional ensemble modeling techniques to ensure that interventions remain robust across various scenarios. Finally, learned policies must be validated prospectively by deploying RL‑based decision‑support tools during live outbreaks, in collaboration with epidemiologists and public‑health agencies, to assess performance under data lags, behavioral shifts, and operational constraints.