https://orcid.org/0000-0002-0449-1324
Figures
Abstract
Since the outbreak of the COVID-19 pandemic, many healthcare facilities have suffered from shortages in medical resources, particularly in Personal Protective Equipment (PPE). In this paper, we propose a game-theoretic approach to schedule PPE orders among healthcare facilities. In this PPE game, each independent healthcare facility optimises its own storage utilisation in order to keep its PPE cost at a minimum. Such a model can reduce peak demand considerably when applied to a variable PPE consumption profile. Experiments conducted for NHS England regions using actual data confirm that the challenge of securing PPE supply during disasters such as COVID-19 can be eased if proper stock management procedures are adopted. These procedures can include early stockpiling, increasing storage capacities and implementing measures that can prolong the time period between successive infection waves, such as social distancing measures. Simulation results suggest that the provision of PPE dedicated storage space can be a viable solution to avoid straining PPE supply chains in case a second wave of COVID-19 infections occurs.
Citation: Abedrabboh K, Pilz M, Al-Fagih Z, Al-Fagih OS, Nebel J-C, Al-Fagih L (2021) Game theory to enhance stock management of Personal Protective Equipment (PPE) during the COVID-19 outbreak. PLoS ONE 16(2): e0246110. https://doi.org/10.1371/journal.pone.0246110
Editor: Yong-Hong Kuo, University of Hong Kong, HONG KONG
Received: September 25, 2020; Accepted: January 14, 2021; Published: February 1, 2021
Copyright: © 2021 Abedrabboh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This paper uses occupied hospital beds data, for 7 NHS England regions. All data used in this paper is published online by the UK government. The files are available from the NHS database can be accessed at: https://coronavirus.data.gov.uk/healthcare?areaType=nation{&}areaName=England. The dataset and code can be accessed at: https://doi.org/10.6084/m9.figshare.13642640.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Novel infectious diseases pose a serious challenge to policy makers and healthcare systems. Emerging from Wuhan, China, the ongoing Coronavirus Disease 2019 (COVID-19) pandemic, caused by the Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has wreaked havoc globally. This is due to its rapid rate of transmission, its virulence and the inability of most countries to adequately prepare for such a disease [1]. Identified in December 2019, the disease now has a global distribution with over 30 million confirmed cases and almost one million confirmed deaths as of September 2020 according to the World Health Organisation (WHO) [2]. COVID-19 is primarily transmitted through respiratory droplets and the WHO has identified two principal routes through which these are carried between people. The first mode of transmission involves a person being in direct, close contact with someone who carries the virus (within one metre) where they become directly exposed to potentially infectious respiratory droplets. The second mode of transmission involves contact with fomites in the immediate vicinity of the infected person [3]. Nguyen et al. estimated that frontline healthcare professionals (HCPs) had a 3.4 times higher risk than non-healthcare workers of contracting COVID-19, even when adjusting for the probability of being tested [4]. Indeed, approximately 10% of the confirmed cases in China [5] and up to 9% of all cases in Italy have been among healthcare professionals [6] as of the date of publication of these studies. This increased risk does not only pose a problem for the HCPs themselves, but also poses a major threat to the elderly and vulnerable populations they care for, since outbreaks within healthcare settings are important amplifiers of infection [7]. One of the most crucial ways by which transmission of this virus (and other infectious diseases) is reduced is the use of proper Personal Protective Equipment (PPE) [7]: the WHO recommends a surgical mask, goggles, or face shield, gown, and gloves to be worn as PPE in their COVID-19 PPE guidelines. If an aerosol-generating procedure is performed, the surgical mask is replaced with an N95 or FFP2/3 respirator which provides a greater level of filtration than surgical masks [8]. These guidelines are replicated globally with minor differences. In the United Kingdom (the case study reported in this paper), the most recent Public Health England guidelines are essentially the same [9]. Globally, medical resources, particularly PPE have come under unprecedented demand. This has led to shortages in many countries with some rationing their use of PPE and in some cases reusing disposable material [10]. The natural consequence of this has been the disproportionately high rate of infection amongst HCPs which in turn contributes to disease spread. This may not necessarily result from a national shortage of PPE but rather local shortages resulting from inefficient distribution of resources in timely manner [11].
In spite of the limited applicability of resource allocation methods, as choices have to be made, they can aid in making the decisions that achieve the best health outcomes (see [12] for a review on the use of such methods in epidemic control). Single- and multi-objective optimisation methods have been used frequently to address problems of resource allocation and scheduling of purchase orders for medical supplies (see [13] for a comprehensive survey and taxonomy). Such methods are centralised in the sense that a central planner seeks to optimally coordinate supply activities for the entire system, e.g. minimise overall costs for the central planner. In contrast to this, game theory is a tool that allows decentralised decision-making [14, 15]. That means, different entities in the system can make decisions based on their individual preferences. For instance, they can schedule their orders to minimise their own costs. The decentralised approach leaves the actors with more freedom and is the more applicable direction for our scenario.
Game theory has been applied to a variety of subject areas such as biology [16], economics [17], computer science [18] and energy [19, 20]. In general, game theory is used to mathematically model systems of competing agents. Usually these individuals act in a selfish and rational manner. In this context, selfish can be understood as being only interested in their own good, i.e. an agent strives to maximise its outcome irrespective of the outcome of others [14]. Furthermore, rational means that there is a clear logical reasoning behind every decision [15]. The most widely used solution concept for a non-cooperative game is the Nash equilibrium [21]. It is achieved when none of the players has an incentive to change their strategy unilaterally.
Despite its various applications in supply chain management (cf. [22] and the references therein), only few studies have applied game theory to the management of medical supplies. To the best of our knowledge, a 2008 study [23] (an extended version was later published in [24]) was the first to develop a game theoretic approach for stockpiling of critical medical items. In their model, the authors propose a non-cooperative game where hospitals stockpile critical medical resources in preparation for disasters. It is proposed that these hospitals determine their individual stockpile levels strategically in a way that minimises their expected total spend. Their model uses a cost function that consists of the cost of ordering, borrowing, storage and a penalty cost in the case of shortages. Although their model proves to give some breathing space to the medical supply chain at the onset of an epidemic, it is only intended as a preparation scheme, and therefore may suffer from inapplicability if an epidemic lasts longer than the planning period considered. The model assumes a given likelihood of the occurrence and severity of a pandemic, which is, in practice, heavily unpredictable. Their work was further developed in [25] by introducing network constraints, thus giving realistic hospital sharing policies. The authors find that deficits between stockpiles and demand can be reduced through central stockpiling and through increasing penalties for deficits.
Game theory has also been applied to the problem of drug allocation [26], where countries behave selfishly to minimise their expected number of infections. Unlike in [27] where the authors propose a similar resource competition among countries, their model imitates the stochasticity of infection transmission parameters. The resulting comparison between this selfish allocation scheme and a utilitarian division scheme, where a central planner (e.g. WHO) makes all of the allocation decisions, shows that having a central planner reduces the total number of worldwide infections considerably. Although their proposed model can aid in understanding how countries, acting in their own self interest, would behave in an epidemic, it does not address the problem of resource distribution within a given country. [28] compares selfish vaccination coverage, which the authors call the ‘Nash vaccination‘, with group optimal coverage, which they call the ‘utilitarian vaccination’. The authors find that the cost of vaccination would be a pivotal factor in determining the effectiveness of Nash vs. utilitarian coverage. However, this can result in different outcomes depending on the level of disease severity according to age, such as in the case of chickenpox.
Most recently, Nagurney et al. proposed a novel decentralised model for medical supply chain with multiple supply and demand points [29]. In their model, selfish consumers who have stochastic demands make purchasing decisions that minimise their total expenditure. The authors assume that the disutility function of a consumer consists of a linear cost of demand, a quadratic cost of transportation and a penalty for shortages/surpluses. Although the quantification of the shortage/surplus penalty can be debated, the authors conclude by suggesting that shortages in supply can be avoided by redirecting global production efforts to instead adding or increasing local production capabilities.
In this paper, we propose a game theoretic approach for managing PPE supplies during a pandemic. We take inspiration from the electricity storage scheduling game developed by the co-authors in [30], where a decentralised system of individually owned home energy systems served by the same utility company schedule their day-ahead battery usage over a full year. This decentralised system resulted in improved energy efficiency for the utility company and cost savings for the participating users.
This work proposes a centralised-decentralised approach to the PPE supply chain (cf. Section 2). In the proposed architecture, healthcare facilities report their demand to a central entity. This central entity is assumed to have the commitment power to fulfill its orders and set the costs for PPE. Given the actions of all game participants, healthcare facilities optimise their PPE orders by making stockpiling decisions individually. Our approach is centralised in the sense that cost and supply is controlled by a central entity. It is also decentralised because healthcare facilities make their stockpiling decisions independently. By adapting the model developed in [30] and applying it to COVID-19 related PPE demand in England, we study the effects of early stockpiling as well as increasing storage capacities on PPE supply in challenging circumstances. Additionally, we examine the impact of a putative second wave on PPE supply and investigate whether delaying this putative second wave can ease the challenge of fulfilling the PPE demand of healthcare facilities. Finally, insights and conclusions from this study are drawn and suggestions regarding PPE supply in disaster management are discussed. The contributions of this work can be summarised as follows:
- A game theory-based model designed to enhance stock management of PPE supply.
- A study of PPE supply management in England during the current COVID-19 pandemic demonstrating the benefits of the centralised-decentralised allocation approach advocated by the proposed model.
- A detailed analysis of how key factors, i.e. stockpiling start date, storage capacity and the date of a putative second wave, impact PPE supply in England.
- Insights and suggestions regarding the handling of a putative second wave in terms of PPE supply management.
The remainder of this paper is organised as follows. In Section 2 we provide an overview of the system architecture, explain the chosen cost function and give details on the game formulation. Section 3 contains information on the experimental setup and Section 4 presents an analysis of the results with simulations of different scenarios. This is followed by a discussion and recommendations in Section 5 and finally, Section 6 concludes the paper and points out future research directions.
2 System model
During the COVID-19 pandemic, most countries have experienced strained healthcare resources and even shortages. The efficient management of personal protective equipment (PPE) supply has proved vital in limiting the spread of the virus and in keeping the healthcare professionals protected. In this paper, after designing a game theory-based model to enhance stock management of PPE supply, we investigate as a case study PPE provision in hospitals of the English National Health Service (NHS). England was selected as not only has it been one of the first and most severely hit countries, but it has also released COVID-19 data in a transparent and timely manner. The datasets and assumptions that were used in this case study are described in Section 3. Data on occupied hospital beds with illness related to COVID-19 [31] was used as a proxy to generate PPE demand profiles.
2.1 System architecture
Healthcare facilities in England are run by the NHS (NHS England). The NHS is made up of organisational units named Trusts which mainly serve geographic areas, but can also serve specialised functions. For a detailed description of the NHS structure in England, please see [32]. Prior to the outbreak of COVID-19 in England, procurement of medical supplies to NHS Trusts, including PPE, was centralised where each Trust would ‘order’ supplies via NHS Supply Chain [33]. However, in the initial period of the pandemic, the NHS was unable to fulfil Trusts’ demand centrally leading to a chaotic albeit temporary decentralised supply chain; some of which was unconventional [34]. On 01 May, NHS Supply Chain introduced a dedicated PPE supplies channel separate to other medical supplies to address this issue [35].
The NHS openly expresses its commitment to improve efficiency [36] and, via its NHS Improvement department, it has advocated the use of modelling and novel ideas to facilitate this [37]. In line with favoured practice by the NHS, we propose a centralised-decentralised PPE supply chain architecture, shown in Fig 1, where a central entity (controlled by the NHS) defines the pricing function based on the market price of the sourced PPE. Thus, for a given day, all NHS Trusts will pay the same price per PPE item.
It shows the centralised-decentralised approach to PPE supply chain.
Similar to the decentralised healthcare resource allocation game proposed in [27], in this architecture, each NHS Trust is modelled as an independent entity that is selfishly concerned with their own interests of satisfying their demand and minimising their cost. Thus, NHS Trusts individually manage the use of their storage capabilities so that their PPE cost is minimised. After optimising their storage schedules independently, NHS Trusts then report their optimal PPE orders to the central entity, which in turn fulfils their demand and charges them their share of PPE cost. An important feature of our model is that the daily PPE demand is always fulfilled. Therefore, patients are always prioritised over costs and thus the stock management strategy does not have any public health implications.
2.2 Cost function
In order to ensure security of supply and avoid burdening the supply chain, sudden surges in demand should be eliminated or at least planned for. It is therefore beneficial for the central entity to aim at having a relatively flat demand profile. This can be done by incentivising Trusts to utilise their storage in an optimal manner so that their overall demand is level. Accordingly, the following assumptions are made regarding the cost function of PPE, C(Q), where Q is the total quantity of PPE consumption:
- The cost function of PPE C(Q) is strictly increasing in consumption Q,
(1) i.e. the higher the consumption the higher the cost.
- The marginal cost of PPE is strictly increasing in consumption,
(2) i.e. the rate of rise of cost increases when consumption increases. This assumption can be justified by considering that an increase in demand will likely require new supply routes or new production facilities be established.
- Zero consumption yields (incurs) zero cost
(3)
While many functions satisfy the above assumptions, this work adopts a quadratic cost function where
(4)
and where a > 0, b ≥ 0 are constant cost parameters, as is the case in [29], where the authors use a quadratic cost function for transportation of PPE along with a linear cost function for PPE supply. The use of quadratic cost function of PPE is further supported by the documented rise in PPE costs during peak COVID-19 cases in England. In some instances, a 1000% increase in prices was reported by NHS Trusts [38].
2.3 Game formulation
The notations used in this formulation are defined in Table 1. Within our model the consumption Q (as introduced in the previous section) consists of two separate quantities. On the one hand, there is the demand d for PPE according to the number of patients that are currently treated. This number cannot directly be influenced within the game formulation. On the other hand, there is the number of PPE kits a that are put into the storage (or taken from the storage), which is our decision to make. Thus we have:
(5)
While d will always be larger than zero, a can take values in a range from max(−d, −s) to smax−s, where s is the amount of PPE currently available in storage and smax is the storage capacity. Thus, the largest amount of PPE that can be taken from storage is limited by either the current demand or the amount of available stored PPE, whichever is lower. Additionally, the maximum amount of PPE that can be stored is bounded by the storage capacity. Formally, this can be summarised by the following constraint:
(6)
The chosen action a then directly affects the stored PPE leading to the following transition equation:
(7)
Similar to [30], we propose a discrete time dynamic game, where the decisions of the players (the Trusts) of how much to put/take in/from the PPE storage are performed sequentially in stages. These stages (also called intervals) are defined according to the actual demand variations, i.e. if the demand changes on a daily basis, each interval would cover a one day period. Furthermore we introduce the state of the game (for each interval) and how it interacts with the decisions of the players. Overall the goal of the players is to minimise their own costs of PPE, i.e. their utility function which is closely related to the cost function discussed in the previous section (see Eq 4). To summarise, the game consists of:
- A set of players (Trusts)
, where N is the total number of participants. All players are assumed to be selfish and rational.
- A set of intervals (typically days),
where T is the number of intervals that comprise the intended time cycle of the game.
- Scalar state variables
denoting the amount of stored PPE of the nth player at stage
. Collectively, we denote the state variables of all players at stage t by
. In the open-loop information structure it is assumed that the initial state s0 is known to all players
.
- Scalar decision variables
(for definition of
see item (5)) denoting the usage of the stored PPE of the nth player at time
. Collectively, we denote the decision variables of all players at stage t by
Furthermore we define the schedule of PPE usage of an individual player
as a collection of all its decisions in the stages of the game by
. A strategy profile is denoted by a ≔ [a1, …, aN].
- A set of admissible decisions
for the nth player. The function
has been defined in Eq 6, capturing the restrictions posed by the storage facilities. We denote
- A state transition equation
(8) governing the state variables
. The function
is the discretised version of the transition equation (Eq 7), showing how a decision of the player influences the state of its PPE storage for the upcoming stage.
- A stage additive utility function
(9) for the nth player, where a−n ≔ [a1, …, an−1, an+1, …, aN] denotes the decisions of all other players. The function
fulfils the assumptions as denoted in Eq 4 capturing the costs to the nth player at the tth stage. Note that the utility function depends only on the initial state variable
, since the subsequent states
are determined by Eq 8. The function
(10) can be interpreted as a penalty for the nth Trust that is incurred by ending up in state
, i.e. its overbought PPE capacity, at the end of the scheduling period.
The objective of rational players is to maximise their total utility, i.e. minimise their overall costs, over the complete time cycle T. We represent the decision problem Gn of the nth player (given the actions a−n of all the other players) as the following optimisation problem:
(11)
Moreover, the game is referred to as {G1, …, GN}, which denotes the simultaneous (and linked) decision problems for all the Trusts. Within this game formulation we can formally define the Nash equilibrium (cf. Section 1 or [14, 15]) by:
A strategy profile
is a Nash equilibrium for the game {G1, …, GN} if and only if for all players
we have
(12)
3 Materials and methods
In this paper, we propose a game-theoretic approach for the supply of PPE to healthcare facilities. We consider the case study of England as it was one of the countries severely impacted by the COVID-19 pandemic and where shortages in medical resources, especially PPE, were reported [10]. The players in this game structure are assumed to report their demand and are committed to paying their respective invoices. The players are also assumed to have the financial independence that inspires their selfish behaviour. Although this game is intended for independent healthcare providers, such as NHS Trusts in England, only region-level COVID-19 data are publicly available. Therefore, in this experiment, we assume that each of the seven NHS England regions can act selfishly in a way that represents the interests of its accommodated Trusts. We believe this has no effect on the insights drawn from this experiment since a region is merely a group of Trusts.
3.1 Demand profiles until 01 Aug
The region demand profiles used in this experiment were originated from daily COVID-19 occupied hospital beds data, which is available for the seven NHS England regions and published by the UK government [31]. Fig 2 shows the peak and total (up to 01 Aug) COVID-19 occupied hospital beds for each of the seven NHS England regions. We assume this information can represent the extent of the regions’ response to the pandemic. Therefore, we assume that the PPE consumption that is related to all COVID-19 activities within these regions can be estimated from their daily COVID-19 occupied hospital beds data. As the collection of those daily data is already standard procedure in NHS hospitals, usage of the proposed model does not put any additional burden on NHS staff.
Data to produce the background map in Fig 2 is from the UK Office of National Statistics, 2019 licensed under the Open Government License [39].
On 15 Apr, the UK government published its estimated PPE deliveries in England since the start of the outbreak in the country: “Since 25 February 2020, at least 654 million items of PPE have been supplied in this way” [40]. As this number counts a pair of hand gloves as two items and includes consumables that are not considered for this study as mentioned in Section 1, e.g. body bags and swabs, we used the number of delivered aprons (135 M) as our reference for consumed PPE kits. However, only 86% of these items were delivered to the NHS Trusts, whereas the rest were distributed to primary care providers (GPs), pharmacies, dentists and social care providers, as well as to other sectors [40]. Additionally, although these deliveries took place during the period between 25 Feb to 15 Apr, the first COVID-19 hospitalization only took place on 20 Mar. Also, given that PPE supply chains were heavily burdened at the beginning of the outbreak, the UK government was led to change PPE usage guidance that health and social care workers should use in different settings when caring for people with COVID-19. The most notable of these was on 02 Apr when a single kit of PPE was advised to be used per session rather than per patient [41]. This change of guidance along with the fact that our model is only concerned with COVID-19 related PPE consumption have led us to assume that 50% of the PPE kits delivered to NHS Trusts between 25 Feb to 15 Apr were consumed for COVID-19 related activities in the period between 20 Mar and 15 Apr. This results in an estimated PPE consumption of 58 M kits for COVID-19 related hospitalisation cases. Dividing this number by the aggregated daily COVID-19 occupied beds in England for the aforementioned period results in a rough estimate of 195 PPE kits daily consumption per occupied hospital bed. Taking this into consideration, along with the mentioned change in guidance, we were able to extrapolate the publicly available daily COVID-19 occupied beds region data into PPE consumption by using a factor randomised between 210 and 240 kits per bed until 02 Apr, and between 150 and 180 from then onwards.
3.2 Storage capacity
According to the experience of our physician co-authors who have worked in several NHS hospitals, it can be estimated that each set of 20 hospital beds is supported by a PPE storage space of three shelves, the dimensions of each are 4.0m × 0.6m × 0.8m. Our extrapolation in m3 to the storage capacity of the seven NHS England regions is listed in Table 2. Moreover, using the volumes supplied by PPE suppliers for bulk orders [42–45], the volume of a thousand of PPE kits, each consisting of a face shield (visor), a face mask, an apron and a pair of gloves, can be approximated to 1.35 m3. Consequently, the storage capacities in terms of PPE kits can be estimated for each region as seen in Table 2.
Although we are aware that our estimates are very coarse, we believe that they represent a reasonable attempt at quantifying those largely unpublished quantities. In any case, they are only used as a baseline for our modelling as four other values of storage capacities are also considered for each region, i.e. the baseline storage multiplied by a factor of 5, 10, 15 and 20.
3.3 Second wave
Given that several countries have already experienced a second wave of COVID-19 infections (e.g. Spain and France) [46] and several studies [47–49] predict that this is most likely to occur in England as well, our model is used to investigate how PPE demand should be handled in such a situation. Our experiments are informed by the model proposed by [47], where the authors predict that a second surge of COVID-19 cases happening in England can be of a lesser magnitude than the first wave, depending on the control measures in place (e.g. lockdown and social distancing measures), but will most likely last longer than the first wave.
Therefore, a second surge in PPE demand was introduced to the regions’ demand profiles from 01 Aug. Inspired by [47], we modelled the second wave with a lower peak and taking place over a longer period. The second wave PPE demand data does not reflect results from a prediction model built for this study, but is merely a useful depiction for evaluating the performance of our model in case a putative second wave occurs. Fig 3 shows an example of a second wave PPE demand profile expected to peak in mid October 2020. As the time of occurrence of a putative second wave is largely unknown, we considered in our experiments waves peaking at five different dates, i.e. mid October, mid November, mid December 2020 and mid January and mid February 2021. As can be shown from [47], it is highly unlikely that a putative second wave would last longer than 100 days after peaking. This has led us to assume that PPE demand for the second wave would come to an end 100 days after the second peak.
Here, stockpiling started on 28 Feb, standard storage capacity was multiplied by five and the peak of the second wave is expected to happen in mid October. Stored PPE sets for each region is stacked on top of each other in order to show the cumulative PPE stock of all regions to illustrate the game mechanism. Note that the areas are scaled down by a factor of five in order to make the figure more readable.
3.4 Stockpiling starting dates
In this study, we investigate whether shortages in PPE could have been avoided if the stockpiling game had been initiated at an earlier stage. Five different starting dates are considered as each of these had some importance regarding the development of the COVID-19 pandemic in the UK. Consequently, each could have triggered the start of the PPE stockpiling process:
- 20 March 2020: data on COVID-19 cases and occupied hospital beds in England start to be released to the public [31].
- 11 March 2020: the WHO declared COVID-19 a pandemic [50].
- 28 February 2020: the European Union proposed to the UK a scheme to bulk-buy PPE [51].
- 07 February 2020: WHO warned of PPE shortages [52].
- 31 January 2020: the first COVID-19 case was confirmed in the UK [53].
3.5 Cost parameters.
As mentioned in Section 2, considerable rises in PPE costs were reported during peak COVID-19 cases in England [38]. This led us to use a quadratic cost function in our model to express cost as a function of demand (see Eq 4). Since our proposed cost function does not have any cost terms other than the cost of ordering PPE, scheduling decisions are fairly independent of the cost parameters. In fact, the authors in [30] showed that the sensitivity of similar scheduling games in relation to cost parameters is quite low. In the experiments conducted in this study, we use 8 × 10−6 as the quadratic term parameter and 1 × 10−2 as the linear term parameter. Usage of these parameters has resulted in a 300% increase in PPE cost between peak and average demand. We believe this is in line with what has been reported in the literature [38], where the costs of certain PPE items increased by up to 1000% between pre-COVID and peak COVID periods.
3.6 Experiment implementation.
In order to show the outcome of the game and whether it has the capability to ease the challenge of securing PPE during peak demand, this experiment was implemented in two stages. In the first stage, we considered the overall period using five different stockpiling start dates (as stated above), to the end of a putative second wave, which also has five different peak dates. Additionally, we used five different storage capacities for each region in order to analyse the effects of adding extra PPE storage space. This means that 125 games were played in the first stage, the results of which will be discussed in the following sections. In the second stage of this experiment, we only examined the effects of running the game for the duration of the second wave; with the simulation starting from 01 Aug, (i.e. the cut-off date of the data used to simulate demand), and ending 100 days after the peak of the second wave. We have simulated five different dates for the peak of the second wave. Furthermore, we used the same five storage capacities for this stage as already employed before. The results of the 25 games that were simulated in the second stage will also be discussed in the following sections. In both stages, we assume that regions start the scheduling game with an empty store. We should specify that, since NHS hospitals received daily PPE orders during the considered period, the game was performed sequentially using a 24-hour interval to provide automated daily decisions fulfilling the PPE requirements of each Trust.
A Python code was written to find the Nash equilibrium of this game, where players sequentially update their PPE orders to minimise their costs. The optimisation package scipy.optimize was used for each player to find their optimal scheduling decisions at each game iteration. The game converges once the mean squared error (MSE) between successive game iterations drop below a threshold. Given that peak PPE demand is in the order of millions of sets, we used a maximum value of 10 for the MSE to obtain accurate results.
4 Results
4.1 Analysis of outputs generated by the game
In order to illustrate the outputs generated by the game, we show results from running one of the games in Fig 3. The outcome of the game is a series of daily decisions that each English NHS region should take regarding their PPE requirements: order PPE, use PPE from their storage or stockpile PPE in their storage. The result of those decisions can be visualised by monitoring the status of their storage levels. Fig 3 displays the amount of PPE sets available in the storage for each region (coloured areas), the PPE demand (dotted line) and PPE orders (bold line). From the stockpiling date, a constant set of PPE is ordered daily, leading to storage reaching full capacity at the start of the first wave that triggers PPE demand. However, as storage capacity becomes rapidly insufficient to meet the daily demand, available (stored) PPE decreases rapidly and daily ordering increases until reaching a plateau around the period of the peak of the first wave. Eventually, storage becomes depleted and the daily order equals the daily demand of a receding wave. Finally, when the daily demand is sufficiently low, storage is repleted allowing English regions to face the putative second wave with full PPE storage capacity. From then, patterns observed in terms of order and storage levels are similar to that already described for the first wave.
As this model assumes that the cost of ordering PPE is quadratic with respect to the aggregated PPE order, the game aims at producing a level of orders as constant and low as the conditions permit. Deviations from this ideal scenario generate additional costs which corresponds in the proposed model to an increased challenge for the NHS at delivering required PPE. This challenge is visible in Fig 4 where a comparison between three different scenarios is shown. The first scenario can be considered as a reference scenario where NHS regions report their PPE demand to NHS without scheduling their storage usage. The second and third scenarios are outcomes from the game when regions start stockpiling on 11 Mar and 07 Feb, respectively. One should also note that storage capacity in the third scenario is multiplied by 10. In this figure, the cumulative cost for the second and third scenarios is shown with reference to the first. As shown, a 9% saving resulted from the second scenario at the end of the scheduling period while the third scenario resulted in a 38% saving. Since a higher cost saving means that less fluctuations occur in demand, these figures can be directly used as a method to quantify the challenge of securing PPE. In Fig 4, as in the remaining figures, colours have been used to illustrate that level of challenge. The associated colour grading goes from dark red to dark green, where dark red expresses a high challenge and dark green indicates a relatively low challenge.
This shows that cumulative cost saving can represent the challenge level of securing PPE supply. A colour grading is applied to represent the level of this challenge, where dark red is used for the highest challenge level and green for the lowest.
4.2 Scenario simulations
Using the proposed model, a range of different scenarios were considered to assess how the challenge in terms of fulfilling PPE demand varies according to the amount of available storage capacity, the stockpiling starting date and the peak date of a putative second wave of COVID-19. In total, 125 scenarios were conducted by considering five different values for each of the three parameters (see subsections above). Fig 5 summarises the outcomes of these experiments. The figure reveals that in a two-wave pandemic, the two most critical parameters are the stockpiling date and storage capacity. With the peak of the first wave being estimated at 08 Apr [54], and the second wave taking place at least six months after the first one, the timing of the second wave thereafter would have only a moderate impact on easing the PPE challenge. On one hand, Fig 5 highlights that if the storage capacity is low, an early stockpiling starting date hardly alleviates the PPE challenge and has minimal, if not negligible, cost savings (leftmost column). On the other hand, a high storage capacity only slightly mitigates a late stockpiling date (bottom row). Indeed, only early stockpiling and a sizeable increase in storage (top right area) would provide the right conditions to lower the PPE challenge overall and indeed would result in considerable cost savings.
In order to focus on the future, an additional 25 scenarios were simulated focusing on the PPE challenges associated with the available storage capacity and the peak date of a putative second wave. Using the last date of bed occupancy data (01 Aug) as the simulation starting date, five different values were considered for the two parameters (see subsections above). The results of these 25 scenarios are shown in Fig 6. As shown, while the storage capacity remains a key parameter, the impact of the date of the peak of the second wave has become apparent. Indeed, as the period of study is shorter than in the previous set of simulations and only a single wave of infection is considered, its timing substantially affects the PPE challenge. Consequently, even if a large amount of storage is available (two rightmost columns), only a late 2020 peak or later would be handled well in terms of cost savings.
5 Discussion and recommendations
The model we have constructed to analyse the provision of PPE has important implications for the management of the pandemic hitherto. Firstly, we have shown objectively that early stockpiling and increasing storage capacity would have helped to massively reduce the costs associated with the containment of the pandemic. Secondly, the early, sufficient and cheap provision of PPE would have had a substantial impact on the ability to contain the pandemic, and protect both the most vulnerable patients and the healthcare professionals that care for them [7].
Based on these findings and the mathematics underlying our approach, we have identified the key elements that govern the aggregated demand (and therefore, the cost) of PPE in our scenario. The two key parameters involved in saving costs by reducing the demand for PPE in the context of a pandemic that were identified using this approach are: (i) the storage capacity (for PPE) available to the health system. We have shown that increasing the storage capacity enhances the ability to stockpile which in turn helps flatten the demand curve (making it easier for suppliers to meet this demand in a cost-effective way); and (ii) the date when stockpiling of PPE commences. We have shown that earlier stockpiling would have saved costs and ensured adequate provision of PPE for the first peak of the pandemic. These two factors will also prove crucial when planning for a second wave. Of secondary importance is the ability to predict the date of the second wave. This is because, providing the second wave occurs after the new year, the degree of cost saving that occurs diminishes to negligible levels beyond this point in time. Using our game-theoretic approach, we have found that the NHS would be able to achieve all the cost-saving advantages if the second wave was to occur in 2021. A practical implication of this is that policymakers may need to keep some of the restricting measures that are in place to control the pandemic until beyond this point in time.
In order to highlight the significance of having dedicated and sufficient storage for PPE in preparation for a putative second wave, we show a comparison between different storage capacities in Fig 7. In this figure, the aggregate demand profile for all NHS England regions is shown both with and without storage scheduling for a second wave of PPE demand that peaks in mid November. As shown, there is a substantial improvement in cost savings between the different storage capacities, especially when it is amplified by a factor of five and 10, resulting in an improvement in cost saving of 15.9% and 7.7% respectively. This means that the challenge of securing PPE during a second wave can be eased by a considerable amount when additional storage space is provided for stockpiling PPE. The figure also shows that further storage enhancement beyond this would result in less cost-saving improvements and would therefore have little impact on the challenge of securing sufficient PPE. This is especially visible when storage is expanded from 15 times to 20 times the standard storage space. Indeed, enhancing the PPE storage capacity by 15 times, perhaps by using temporary storage facilities as an interim solution, results in a relatively steady demand profile as shown in Fig 7.
These results are for a second wave where PPE demand peaks in mid November.
6 Conclusion
In this paper, we developed a game-theoretic model for scheduling PPE supply for healthcare facilities. In this discrete time dynamic game, healthcare facilities make stockpiling decisions that minimise their PPE ordering cost. Our model adopts a centralised-decentralised approach to the PPE supply chain, where a central entity controls the PPE pricing formula, yet is committed to fulfilling the orders independently placed by healthcare providers. Based on publicly available COVID-19 hospitalisation data for NHS England regions, we performed simulations to investigate the impact of three key factors on PPE security of supply. These factors comprise: (i) the stockpiling start date, (ii) the time of the peak of a putative second wave, and (iii) the amount of storage available for medical PPE. The two most critical parameters were found to be the storage capacity and the stockpiling start date, while the timing of the second wave has only had a moderate impact on the challenge of securing PPE. Within our model we observe that enhancing PPE storage capacities by a factor of 15 is sufficient to considerably lower the peak demand at any given day and effectively minimises the strain on the health care system. (This does not imply that dealing with the pandemic in this scenario is rendered trivial. It rather represents the most favourable way to act during the crisis).
While the shortage of PPE supplies for care home workers during the first peak in the UK was a topic of great concern, since those shortages were blamed for care home outbreaks which led to a large number of lost lives [55], we explicitly excluded this from our study due to the lack of publicly available data. Access to such data would provide further insights into demand patterns which can potentially influence the game. Extensions to our work can include introducing supply constraints where demand cannot always be fulfilled, perhaps by adding a shortage penalty to the cost function. Also, our model assumes that healthcare facilities can predict their demand without forecasting errors. This can be enhanced by adding stochastic elements that capture the uncertainty in demand [30]. The model can also be extended by investigating scenarios where hospitals can share their resources by having mutual agreements or using the selfish sharing approach proposed in [56]. This could be via using a scheme to incentivise matching scheduled regional demand with the production of the supplier. Another possible extension of this research is to propose a model that finds the optimal storage capacity for each game player. Although the above refinements may give greater insight, our model already has the potential to be a useful tool that can help governments and policy makers in decision-making in relation to critical PPE supplies. Finally, if one wishes to extend this work to incorporate the global picture, this could be achieved by introducing another layer of competition modelling the orders of the different nations. Here, a Leader-Follower structure would be required and most suitably captured by a Stackelberg game [14].
Acknowledgments
The authors dedicate this work to healthcare workers that have sadly lost their lives during this pandemic, and would like to thank all of those working on the front-lines, whether it is in hospital wards, care homes or research laboratories.
References
- 1. Madabhavi I, Sarkar M, Kadakol N. CoviD-19: A review. Monaldi Archives for Chest Disease. 2020;90(2):248–258. https://doi.org/10.4081/monaldi.2020.1298 pmid:32498503
- 2.
World Health Organization (WHO). WHO Coronavirus Disease (COVID-19) Dashboard; 2020. Available from: https://covid19.who.int. Accessed 22 September 2020.
- 3. Ong SWX, Tan YK, Chia PY, Lee TH, Ng OT, Wong MSY, et al. Air, Surface Environmental, and Personal Protective Equipment Contamination by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) from a Symptomatic Patient. JAMA—Journal of the American Medical Association. 2020;323(16):1610–1612. https://doi.org/10.1001/jama.2020.3227
- 4. Nguyen LH, Drew DA, Graham MS, Joshi AD, Guo CG, Ma W, et al. Risk of COVID-19 among front-line health-care workers and the general community: a prospective cohort study. The Lancet Public Health. 2020. https://doi.org/10.1016/s2468-2667(20)30164-x pmid:32745512
- 5.
World Health Organization (WHO). Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19); 2020. Available from: https://www.who.int/docs/defaultsource/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf. Accessed 01 March 2020.
- 6.
Istituto Superiore di Sanità (ISS). Sorveglianza Integrata COVID-19 in Italia; 2020. Available from: https://www.epicentro.iss.it/coronavirus/bollettino/Infografica_26marzoITA.pdf. Accessed 22 September 2020.
- 7.
European Centre for Disease Prevention and Control (ECDC). Infection prevention and controland preparednessfor COVID-19 in healthcare settings; 2020. Available from: https://www.ecdc.europa.eu/sites/default/files/documents/Infection-prevention-and-control-in-healthcare-settings-COVID-19_4th_update.pdf. Accessed 22 September 2020.
- 8.
World Health Organization (WHO). Rational use of personal protective equipment for coronavirus disease 2019 (COVID-19); 2020. Available from: https://apps.who.int/iris/bitstream/handle/10665/331215/WHO-2019-nCov-IPCPPE_use-2020.1-eng.pdf. Accessed 22 September 2020.
- 9.
Public Health England (PHE). COVID-19: infection prevention and control (IPC); 2020. Available from: https://www.gov.uk/government/publications/wuhan-novel-coronavirus-infection-prevention-and-control. Accessed 22 September 2020.
- 10. Tabah A, Ramanan M, Laupland KB, Buetti N, Cortegiani A, Mellinghoff J, et al. Personal protective equipment and intensive care unit healthcare worker safety in the COVID-19 era (PPE-SAFE): An international survey. Journal of Critical Care. 2020;59:70–75. https://doi.org/10.1016/j.jcrc.2020.06.005 pmid:32570052
- 11.
Reuters. UK has enough protective equipment but there may be local shortages: health official; 2020. Available from: https://uk.reuters.com/article/us-health-coronavirus-britain-equipment/uk-has-enough-protective-equipment-but-there-may-be-local-shortages-health-official-idUSKCN2242SL. Accessed 22 September 2020.
- 12.
Brandeau ML. In: Brandeau ML, Sainfort F, Pierskalla WP, editors. Allocating Resources to Control Infectious Diseases. Boston, MA: Springer US; 2004. p. 443–464. https://doi.org/10.1007/1-4020-8066-2_17.
- 13.
Tüzün S, Topcu YI. In: Kahraman C, Topcu YI, editors. A Taxonomy of Operations Research Studies in Healthcare Management. Cham: Springer International Publishing; 2018. p. 3–21. https://doi.org/10.1007/978-3-319-65455-3_1.
- 14.
Shoham Y, Leyton-Brown K. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press; 2008.
- 15.
von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. Princeton University Press; 1944.
- 16.
Hammerstein P, Selten R. Chapter 28 Game theory and evolutionary biology. vol. 2 of Handbook of Game Theory with Economic Applications. Elsevier; 1994. p. 929—993. https://doi.org/10.1016/S1574-0005(05)80060-8.
- 17.
Cournot A. Researches Into the Mathematical Principles of the Theory of Wealth; 1838.
- 18. Shoham Y. Computer Science and Game Theory. Commun ACM. 2008;51(8):74–79. https://doi.org/10.1145/1378704.1378721
- 19. Pilz M, Al-Fagih L. Recent Advances in Local Energy Trading in the Smart Grid Based on Game–Theoretic Approaches. IEEE Transactions on Smart Grid. 2019;10(2):1363–1371. https://doi.org/10.1109/TSG.2017.2764275
- 20. Pilz M, Naeini FB, Grammont K, Smagghe C, Davis M, Nebel JC, et al. Security attacks on smart grid scheduling and their defences: a game-theoretic approach. International Journal of Information Security. 2020;19(4):427–443. https://doi.org/10.1007/s10207-019-00460-z
- 21. Nash J. Non-Cooperative Games. Annals of Mathematics. 1951;54(2):286–295. https://doi.org/10.2307/1969529
- 22. Vasnani NN, Chua FLS, Ocampo LA, Pacio LBM. Game theory in supply chain management: Current trends and applications. International Journal of Applied Decision Sciences. 2019;12(1):56–97. https://doi.org/10.1504/IJADS.2019.096552
- 23. DeLaurentis PC, Adida E, Lawley M. A game theoretical approach for hospital stockpile in preparation for pandemics. hospital. 2008;101:1.
- 24. Adida E, DeLaurentis PCC, Lawley MA. Hospital stockpiling for disaster planning. IIE Transactions. 2011;43(5):348–362. https://doi.org/10.1080/0740817X.2010.540639
- 25.
Lofgren E, Vullikanti A. Hospital stockpiling problems with inventory sharing. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI’16. AAAI Press; 2016. p. 35–41.
- 26. Sun P, Yang L, De Véricourt F. Selfish drug allocation for containing an international influenza pandemic at the onset. Operations Research. 2009;57(6):1320–1332. https://doi.org/10.1287/opre.1090.0762
- 27. Wang S, de Véricourt F, Sun P. Decentralized resource allocation to control an epidemic: A game theoretic approach. Mathematical Biosciences. 2009;222(1):1–12. https://doi.org/10.1016/j.mbs.2009.08.002 pmid:19683014
- 28. Liu J, Kochin BF, Tekle YI, Galvani AP. Epidemiological game-theory dynamics of chickenpox vaccination in the USA and Israel. Journal of the Royal Society Interface. 2012;9(66):68–76. https://doi.org/10.1098/rsif.2011.0001 pmid:21632611
- 29.
Nagurney A, Salarpour M, Dong J, Dutta P. Competition for Medical Supplies Under Stochastic Demand in the Covid-19 Pandemic: A Generalized Nash Equilibrium Framework. Forthcoming in: Nonlinear Analysis and Global Optimization, Themistocles M. Rassias, and Panos M. Pardalos, Editors, Springer Nature Switzerland AG. 2020; Available from: https://ssrn.com/abstract=3657994.
- 30.
Pilz M, Al-Fagih L. A dynamic game approach for demand-side management: scheduling energy storage with forecasting errors. Dynamic Games and Applications. 2019; p. 1–33. https://doi.org/10.1007/s13235-019-00309-z.
- 31.
Public Health England (PHE). Coronavirus (COVID-19) in the UK: Healthcare in England; 2020. Available from: https://coronavirus.data.gov.uk/healthcare. Accessed 22 September 2020.
- 32.
National Health Service (NHS) England. Understanding the New NHS—NHS England; 2014. Available from: https://www.england.nhs.uk/2014/06/understanding-nhs. Accessed 22 September 2020.
- 33.
Supply Chain Coordination Limited (SCCL). NHS Supply Chain—Supply Chain Coordination Limited (SCCL) Timeline;. Available from: https://www.supplychain.nhs.uk/about-us/sccl-timeline. Accessed 22 September 2020.
- 34.
Milmo C, Kirby D. PPE chaos: Desperate NHS trusts’ race to secure lifesaving PPE from unproven suppliers amid catastrophic distribution failures; 2020. Available from: https://inews.co.uk/news/long-reads/ppe-chaos-investigation-desperate-nhs-trust-race-secure-lifesaving-ppe-unproven-suppliers-revealed-571094. Accessed 22 September 2020.
- 35.
Department of Health & Social Care (DHSC). Letter to procurement leads centralising procurement of PPE;. Available from: https://www.england.nhs.uk/coronavirus/wp-content/uploads/sites/52/2020/05/C0423-letter-to-procurement-leads-centralising-procurement-of-ppe-other-supplies-1-may-2020.pdf. Accessed 22 September 2020.
- 36.
National Health Service (NHS) Improvement. What we do | NHS Improvement;. Available from: https://improvement.nhs.uk/about-us/what-we-do. Accessed 22 September 2020.
- 37.
National Health Service (NHS) Improvement. Quality, Service Improvement and Redesign Tools: Modelling and simulation;. Available from: https://improvement.nhs.uk/documents/2136/modelling-simulation.pdf. Accessed 22 September 2020.
- 38.
Hardy J. Protective equipment companies under fire for’1,000% price increase’; 2020. Available from: https://www.telegraph.co.uk/news/2020/04/01/protective-equipment-companies-fire-1000-price-increase. Accessed 22 September 2020.
- 39.
UK Office of National Statistics. Open Government License v3.0; 2019. Available from: https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3. Accessed 22 September 2020.
- 40.
Department of Health & Social Care (DHSC). COVID-19: personal protective equipment (PPE) plan—GOV.UK; 2020. Available from: https://www.gov.uk/government/publications/coronavirus-covid-19-personal-protective-equipment-ppe-plan. Accessed 22 September 2020.
- 41.
National Audit Office (NAO). Readying the NHS and adult social care in England for COVID-19. Department of Health & Social Care (DHSC); 2020. Available from: https://www.nao.org.uk/wp-content/uploads/2020/06/Readying-the-NHS-and-adult-social-care-in-England-for-COVID-19.pdf. Accessed 22 September 2020.
- 42. Amazon.com. Branden Disposable Non-sterile Surgical Face Mask. Available from: https://www.amazon.co.uk/Branden-Disposable-Non-sterile-Surgical-3-ply/dp/B089DLW1DS. Accessed 22 September 2020.
- 43. Amazon.com. INTCO 100Pcs Disposable Nitrile Glove Powder Free Examination Gloves. Available from: https://www.amazon.co.uk/INTCO-Disposable-Examination-Protective-Laboratory/dp/B085L5NJTL. Accessed 22 September 2020.
- 44. Amazon.com. PPE Visor Face Shield. Available from: https://www.amazon.co.uk/Shield-Reusable-Adjustable-Visibility-Approved/dp/B08BTSCT97. Accessed 22 September 2020.
- 45. Amazon.com. 100x Disposable White Polythene Aprons. Available from: https://www.amazon.co.uk/Disposable-Polythene-Delivery-Manufacturer-delivery/dp/B089DLK3G2. Accessed 22 September 2020.
- 46. Worldometer. Coronavirus Cases; 2020. Available from: https://www.worldometers.info/coronavirus. Accessed 22 September 2020.
- 47. Keeling MJ, Hill E, Gorsich E, Penman B, Guyver-Fletcher G, Holmes A, et al. Predictions of COVID-19 dynamics in the UK: short-term forecasting and analysis of potential exit strategies. medRxiv. 2020; https://doi.org/10.1101/2020.05.10.20083683 pmid:32817970
- 48. Panovska-Griffiths J, Kerr CC, Stuart RM, Mistry D, Klein DJ, Viner RM, et al. Determining the optimal strategy for reopening schools, the impact of test and trace interventions, and the risk of occurrence of a second COVID-19 epidemic wave in the UK: a modelling study. The Lancet Child & Adolescent Health. 2020; https://doi.org/10.1016/S2352-4642(20)30250-9 pmid:32758453
- 49. Wise J. Covid-19: Risk of second wave is very real, say researchers. BMJ. 2020;369. https://doi.org/10.1136/bmj.m2294. pmid:32518177
- 50.
World Health Organization (WHO). WHO announces COVID-19 outbreak a pandemic; 2020. Available from: https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/3/who-announces-covid-19-outbreak-a-pandemic. Accessed 22 September 2020.
- 51.
Boffey D, Booth R. UK missed three chances to join EU scheme to bulk-buy PPE; 2020. Available from: https://www.theguardian.com/world/2020/apr/13/uk-missed-three-chances-to-join-eu-scheme-to-bulk-buy-ppe. Accessed 22 September 2020.
- 52.
World Health Organization (WHO). WHO Director-General’s opening remarks at the media briefing on 2019 novel coronavirus—7 February 2020; 2020. Available from: https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-2019-novel-coronavirus---7-february-2020. Accessed 22 September 2020.
- 53.
Bowden E. Coronavirus in the UK—a timeline; 2020. Available from: https://www.independent.co.uk/news/uk/home-news/coronavirus-uk-timeline-health-london-singapore-italy-scotland-a9367706.html. Accessed 22 September 2020.
- 54.
Leon DA, Jarvis CI, Johnson AM, Smeeth L, Shkolnikov VM. What can trends in hospital deaths from COVID-19 tell us about the progress and peak of the pandemic? An analysis of death counts from England announced up to 25 April 2020. medRxiv. 2020; https://doi.org/10.1101/2020.04.21.20073049.
- 55. Brainard JS, Rushton S, Winters T, Hunter PR. Introduction to and spread of COVID-19 in care homes in Norfolk, UK. medRxiv. 2020; https://doi.org/10.1101/2020.06.17.20133629. pmid:33367852
- 56.
Pilz M, Al-Fagih L. Selfish Energy Sharing in Prosumer Communities: A Demand-Side Management Concept. In: 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm); 2019. p. 1–6. https://doi.org/10.1109/SmartGridComm.2019.8909791.