10 Dec 2013: Haidari LA, Connor DL, Wateska AR, Brown ST, Mueller LE, et al. (2013) Correction: Augmenting Transport versus Increasing Cold Storage to Improve Vaccine Supply Chains. PLOS ONE 8(12): 10.1371/annotation/b428dad2-e829-4289-acae-c44369d55a80. https://doi.org/10.1371/annotation/b428dad2-e829-4289-acae-c44369d55a80 View correction
When addressing the urgent task of improving vaccine supply chains, especially to accommodate the introduction of new vaccines, there is often a heavy emphasis on stationary storage. Currently, donations to vaccine supply chains occur largely in the form of storage equipment.
This study utilized a HERMES-generated detailed, dynamic, discrete event simulation model of the Niger vaccine supply chain to compare the impacts on vaccine availability of adding stationary cold storage versus transport capacity at different levels and to determine whether adding stationary storage capacity alone would be enough to relieve potential bottlenecks when pneumococcal and rotavirus vaccines are introduced by 2015.
Relieving regional level storage bottlenecks increased vaccine availability (by 4%) more than relieving storage bottlenecks at the district (1% increase), central (no change), and clinic (no change) levels alone. Increasing transport frequency (or capacity) yielded far greater gains (e.g., 15% increase in vaccine availability when doubling transport frequency to the district level and 18% when tripling). In fact, relieving all stationary storage constraints could only increase vaccine availability by 11%, whereas doubling the transport frequency throughout the system led to a 26% increase and tripling the frequency led to a 30% increase. Increasing transport frequency also reduced the amount of stationary storage space needed in the supply chain. The supply chain required an additional 61,269L of storage to relieve constraints with the current transport frequency, 55,255L with transport frequency doubled, and 51,791L with transport frequency tripled.
When evaluating vaccine supply chains, it is important to understand the interplay between stationary storage and transport. The HERMES-generated dynamic simulation model showed how augmenting transport can result in greater gains than only augmenting stationary storage and can reduce stationary storage needs.
Citation: Haidari LA, Connor DL, Wateska AR, Brown ST, Mueller LE, Norman BA, et al. (2013) Augmenting Transport versus Increasing Cold Storage to Improve Vaccine Supply Chains. PLoS ONE 8(5): e64303. https://doi.org/10.1371/journal.pone.0064303
Editor: Matteo Convertino, University of Florida, United States of America
Received: January 14, 2013; Accepted: April 14, 2013; Published: May 22, 2013
Copyright: © 2013 Haidari et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the Vaccine Modeling Initiative (VMI), funded by the Bill and Melinda Gates Foundation (http://www.gatesfoundation.org/Pages/home.aspx) and the National Institute of General Medical Sciences Models of Infectious Disease Agent Study (MIDAS) (http://www.nigms.nih.gov/Initiatives/MIDAS/) grant 1U54GM088491-0109. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. No other financial disclosures were reported by the authors of this paper.
Competing interests: The authors have declared that no competing interests exist.
When addressing the urgent task of improving vaccine supply chains, there is often a heavy emphasis on stationary storage . To guide the assessment and improvement of vaccine supply chains, the World Health Organization (WHO) created the Effective Vaccine Management (EVM) tool and PATH collaborated with the WHO and the United Nations Children's Fund (UNICEF) to develop the Cold Chain Equipment Manager (CCEM) tool , . While both tools are helpful in assessing a country's supply chain, both focus more on stationary storage rather than transport aspects of supply chains. International donors have also responded to the growing needs of restricted cold chains by donating equipment, often in the form of stationary storage devices –. The recent and impending introductions of new vaccines have prompted many countries to examine their vaccine supply chains (i.e., the system and series of steps required to get vaccines from the manufacturers to the people). For example, the introduction of pneumococcal vaccine caused the storage space requirement in the Turkey vaccine supply chain to quadruple .
However, transport is a major component of vaccine supply chains and can be a source of bottlenecks. Rotavirus vaccine introduction overwhelmed both storage and transport capacities of vaccine supply chains in several Latin American countries in 2006 and 2007 . Many low- and middle-income countries face difficulties in maintaining efficient vaccine supply chains with the current routine immunization regimen, let alone with any of the 12 new, bulkier vaccines that are proposed for introduction by 2019 and are expected to create bottlenecks in both storage and transport aspects of vaccine supply chains . The recent availability of new vaccines, such as pneumococcal and rotavirus, can relieve the burden of disease for millions of infants worldwide, but if vaccine cold supply chains cannot accommodate the increased volume required to ensure adequate supply, populations will not receive these benefits. As new vaccines are introduced, transport constraints will become an increasing issue in vaccine supply chains.
It remains unclear whether adding cold storage or augmenting transport elicits a greater improvement in vaccine availability. It is imperative to evaluate vaccine supply chains as soon as possible to prevent disruptions, as many vaccine supply chains are already restricted by limited capacity in either transport or storage at certain levels . Determining the effects of augmenting transport can be challenging without using a dynamic analysis, such as a simulation model. Therefore, this study utilized a dynamic computational model of Niger's vaccine supply chain to compare the effects of introducing cold storage and altering transport frequency at various levels. The objective was to determine whether adding cold storage devices or increasing transport frequency would have a greater impact and to identify the locations where these additions would be most beneficial, as measured by vaccine availability.
This study utilized HERMES, the Highly Extensible Resource for Modeling Supply Chains, a software package developed by the HERMES Modeling Team, to generate a discrete event simulation model of the Niger vaccine supply chain. The HERMES-generated model represents all logistical components of a supply chain, including the number and size of transport and storage devices, shipment policies, delivery and order frequencies, packaged vaccine size within the supply chain and vaccine storage temperatures, the number of vaccines traveling through the system, and the routes of the transport vehicles. Previous publications have described this model in detail –.
Niger Vaccine Supply Chain
Figure 1 depicts the structure of the entire Niger vaccine supply chain. Data to construct this network came from direct field observations in Niger and personal communications with members of the following organizations: the WHO in both Geneva and the Niger country office in Niamey, the Niger Ministry of Health (MOH), the UNICEF Niger country office, the Niger National Geographic Institute (NGI), and the Expanded Program on Immunization (EPI) in Niger. The vaccine supply chain consists of four levels, whose functional units include one central depot in the capital city of Niamey, seven regional depots, 42 district depots, and 644 integrated health centers (IHCs) throughout the country. Vaccine administration occurs daily and only at the IHC level. The number of IHCs per district ranges from 5 to 36. Most of the IHCs are in the south of Niger, where the majority of the population resides.
The central depot receives vaccines from the manufacturers via UNICEF twice per year. The regional level receives vaccines from the central depot. All but one regional depot receive shipments by cold trucks on a fixed, quarterly schedule. The remaining regional depot, in Niamey, retrieves vaccines by pick-up truck as often as once per month, as needed, due to its close proximity to the central depot. The district level picks up vaccines from the regional level, except for six district locations which pick up vaccines directly from the nearby central depot. The districts retrieve vaccines by pick-up truck as often as once per month, as needed. IHCs equipped with cold storage capacity retrieve vaccines monthly using up to two 2.5L vaccine carriers but can pick up vaccines more often – up to once a week, as needed. Districts and IHCs order enough vaccines to meet average monthly demand, including an allowance for open vial waste as well as an additional 25% buffer.
Based on a previously performed inventory, each vaccine storage location has a number of refrigerators (2°C to 8°C) and freezers (−15°C to −25°C) dedicated to EPI operations, with pre-defined storage capacities based on the make and model of the registered unit. Walk-in refrigerators and freezers are utilized at the central depot and three of the regional level depots while the other regional, district, and IHC locations operate with conventional upright or supine refrigerators and freezers. Table 1 summarizes the devices that store and transport EPI vaccines in Niger.
HERMES-generated Simulation Model of Niger Supply Chain
Net capacity of each cold device was determined by its size and utilization (i.e., the percentage of gross physical space that can actually be used after accounting for space occupied by shelving and the inability to pack vaccines with no space between items). The data gathered from in-country personnel helped construct the vaccine shipping policies with realistic delivery occurrences between locations. The simulated shipments do not contain more vaccines than the cold transport device can hold. A location requesting vaccines in excess of the amount contained in one shipment must wait for the next shipment to fulfill the vaccine delivery request.
The model represents each vaccine vial with a computational entity and the flow of all WHO-EPI vaccines flowing through the supply chain simultaneously, including the six current EPI vaccines [Bacille Calmette-Guérin (BCG), diphtheria-tetanus-pertussis-hepatitis B-Haemophilus influenza type B (DTP-HepB-Hib), oral polio (OPV), measles (M), tetanus toxoid (TT), and yellow fever (YF)] and the new Prevnar 13 pneumococcal (PCV) and Rotarix rotavirus (RV) vaccines, which have been approved for funding to become regular EPI vaccines in Niger in 2013 and 2014, respectively . Under the regimen of one BCG dose at 1.9 cm3 (packed volume, including diluent), three DTP-HepB-Hib doses each at 16.8 cm3, four OPV doses each at 1.0 cm3, one M dose at 2.6 cm3, two TT doses each at 3.0 cm3, and one YF dose at 8.5 cm3, the current EPI schedule requires 73.4 cm3 to fully immunize one child –. With the addition of three PCV doses each at 12 cm3 and two RV doses each at 17.1 cm3 to the EPI, this volume will increase to 143.6 cm3 .
The model uses a projected population demand for 2015, based on district-level census data collected in 2005 which was inflated to conform to estimated pregnant woman, newborn, and surviving infant cohorts in the Comprehensive Multiyear Plan (cMYP) for Niger . Each district population was evenly distributed among the IHCs to determine the number of individuals in the specified demographic groups arriving at IHCs for vaccines. Target vaccination coverage rates were used as outlined in the cMYP: 95% for BCG, DTP-HepB-Hib, OPV, M, TT, YF, and PCV; and 70% for RV . The resulting median number of doses required to meet these coverage rates at an IHC location in a given month was 2088 with a maximum of 4358.
Supply Chain Performance Measure
The overall objective of this investigation was to maximize the number of vaccines available for the demand across all locations, time periods, and vaccine types. Vaccine availability, the primary output measure, expresses the percentage of patients that could be vaccinated based on the number of vaccines available at the IHCs. The vaccine availability is calculated as follows:Missed vaccination opportunities occur when individuals arrive for vaccines but the appropriate vaccines are not available, resulting in vaccine availability of less than 100%.
Each simulation represented one year of vaccine supply chain operations. The first set of experiments added additional storage capacity to locations in the Niger vaccine supply chain. This entailed identifying the maximum amount of storage space required at each location based on current ordering and shipping policies. A location experiences storage constraints if the maximum storage volume it might require is greater than the volume it has at baseline. The experiments compared the effects on vaccine availability of relieving storage constraints in different supply chain levels while maintaining current transport policies. The experiments also alleviated all storage constraints at every location in the supply chain. The resulting vaccine availabilities indicated at which locations alleviating storage constraints would provide the greatest benefit to the supply chain as well as the maximum effect that added storage can achieve.
The second set of experiments used existing storage capacities and increased in-country transport frequencies between various levels. Doubling the number of possible trips between the central and regional levels required altering the fixed schedule to deliver vaccines to the regional depots eight, rather than four, times per year and allowed the Niamey regional depot to retrieve vaccines from the central depot up to twice per month. Doubling the number of possible trips delivering to the district and IHC levels allowed the district depots to retrieve vaccines up to twice per month and allowed the IHCs to retrieve vaccines twice per month and as often as twice per week. The experiments initially doubled the frequency of vaccine delivery to one level at a time. The experiments also studied the effects of doubling the frequency between all levels of the supply chain at once.
The subsequent transport experiments used tripled transport frequencies, such that the Niamey regional depot could retrieve vaccines up to three times per month and all other regional depots would receive vaccines once per month. District depots could retrieve vaccines up to three times per month, and IHCs could retrieve vaccines up to three times per week. These experiments provided vaccine availabilities that identified where along the supply chain additional transport would most improve the flow of vaccines to the IHCs and allowed a comparison of the benefits of doubling versus tripling transport frequency. This study also compared the magnitude of improvement in vaccine availability between added storage and added transport.
Locations receiving vaccines at an increased frequency required fewer vaccines to supply them until the following shipment, so the sizes of affected shipments reduced accordingly. The experiments identified and alleviated storage constraints under doubled and tripled transport frequencies to measure the degree to which increasing transport frequency affects the amount of stationary storage required. The vaccine availability at IHCs and the additional storage volume used across all locations in the supply chain were compared under baseline, doubled, and tripled transport frequencies.
Adding Stationary Storage Capacity Alone
Figure 2 displays the resulting vaccine availabilities after storage or transport was added at each level. At baseline, for a 2015 Niger population with PCV and RV introductions, the vaccine availability across all IHCs was 39%. The central depot was highly constrained, needing more than 99% of its available 36,000L of net refrigerated storage capacity. An additional 52,948L of net storage relieved storage constraints at this location. Three of seven regional depots did not have a cold room at baseline and experienced storage constraints, each of which required an additional 1,155L to 4,186L. At the district level, 19 of the 42 depots required an additional 3L to 319L to relieve storage constraints. Only eight of the functional IHCs experienced storage constraints at baseline, requiring an additional 1L to 5L. Relieving storage constraints at upper levels allowed more vaccines to flow to lower levels, creating additional bottlenecks. Therefore, relieving storage constraints throughout the supply chain required more added capacity than the sum of the capacities needed at each individual level.
Relieving storage constraints only at the regional level increased vaccine availability by 4%. Adding storage to relieve only constrained district depots increased vaccine availability by 1%. Bottlenecks in storage and transport at other levels of the supply chain prevented the addition of any amount of storage to the central or IHC levels alone from having any effect on vaccine availability. Adding enough cold storage capacity to relieve storage constraints for the entire supply chain increased vaccine availability by 11%.
Increasing Transport Frequency Alone
Doubling the frequency of scheduled trips delivering vaccines from the central depot to the regional depots from four times to eight times per year, while also doubling the frequency at which the Niamey regional depot is able to retrieve vaccines from the central depot from once to twice per month, increased vaccine availability by 5%. Doubling the frequency at which districts were able to retrieve vaccines from once per month to twice per month increased vaccine availability by 15%. Allowing IHCs to retrieve vaccines from districts up to twice per week, rather than once per week, increased vaccine availability by 4%. Doubling the shipping frequency across the entire supply chain increased vaccine availability by 26%.
Tripling the frequency of trips delivering from the central depot to the regional level increased vaccine availability by 10% as compared to baseline. Tripling the frequency at which districts were able to retrieve vaccines increased vaccine availability by 18%, only a slightly higher increase than was achieved by doubling the frequency. Allowing IHCs to retrieve vaccines from districts up to three times per week increased vaccine availability by 4%, the same benefit achieved under doubled shipping frequency to the IHC level. Tripling the shipping frequency across the entire supply chain increased vaccine availability by 30%.
While adding cold storage capacity to the regional level can increase vaccine availability by up to 4%, doubling the possible number of trips delivering to the district level can increase vaccine availability by up to 15%. If no equipment is added to the current vaccine supply chain by 2015, the vaccine supply chain will have the ability to supply only 39% of the needed vaccinations (current EPI with PCV and RV introduced). Based on cMYP  population projections, increasing vaccine availability by 1% means that >135,000 more vaccinations could be provided in 2015. A 5% increase translates to >677,000 more vaccinations.
How Increasing Transport Affects Storage Capacity Requirements
As Figure 3 shows, increasing transport frequency not only resulted in higher vaccine availability but also reduced the amount of stationary storage required. With no added transport, the entire supply chain required an additional 61,269L of storage to relieve constraints. Doubling transport frequency reduced this storage need to 55,255L. Tripling transport frequency further reduced it to 51,791L. Relieving storage constraints while simultaneously doubling transport frequency increased vaccine availability by 42%. Tripling transport frequency while relieving storage constraints increased vaccine availability by 48%, thus providing 98% of the vaccinations requested at IHCs.
Although relieving storage constraints may be important, increasing shipping frequency to the district level alone resulted in a far greater improvement in vaccine availability than the addition of any amount of storage to all levels of the supply chain. Adding storage capacity at the central or IHC levels without adding transportation had little to no impact on Niger's overall vaccine availability. Both storage and transport capacity constraints, particularly at the regional and district levels of the vaccine supply chain, created significant bottlenecks that prevented the effective flow of vaccines to the IHC level. Identifying these areas of restriction allows for the most effective distribution of required new cold capacity. The most effective addition of storage capacity occurred at the regional level. Though the central level requires a far greater amount of cold volume in order to become unconstrained, adding a fraction of that space to the regional level will bring more relief to the currently overburdened supply chain. These results suggest that optimal space allocation is not easily determined by the area with the greatest deficit but is instead strongly influenced by dynamic downstream interactions.
Overlooking transport may be ignoring a vital component of vaccine supply chains. In Niger, there appears to be a ceiling effect for vaccine availability when only adding cold storage. In fact, augmenting transport would substantially decrease storage requirements and may not require further capital investment. If vehicles are already available, then this may instead involve additional fuel and labor or different vehicle and labor allocation strategies. If more trips for each vehicle are not feasible, then replacing existing vehicles with larger ones (e.g. substituting trucks for motorbikes) or adding more of the same vehicles could be considered, although this may depend on how accessible locations may be by large vehicles. Alternatively, implementing (or redesigning) delivery loops, where each truck would deliver vaccines to multiple locations in a single trip could be feasible.
Understanding the role and impact of transport can be difficult without dynamic simulation modeling –. While static models can analyze storage aspects of a supply chain, they may struggle to represent the complex interplay between storage and transport. Relieving some storage or transport constraints can exacerbate or alleviate bottlenecks at other locations in the supply chain. When adding capacity to highly constrained locations, particularly those at the upper levels of the supply chain, one must consider the consequences for facilities that receive vaccines from these locations. Adding storage capacity can, in some cases, alleviate transport constraints, as less frequent trips are necessary when a location can store a longer-term supply of vaccines. Conversely, increasing transport frequency between two levels can alleviate storage constraints at both levels, as fewer vaccines must be stored at these locations at any one time. When planning to increase the cold capacity of a vaccine supply chain, allocating new resources to have the greatest impact seems justified. Dynamic modeling can provide insight into where these resources can save the most lives.
Some organizations have emphasized the importance of transport; Transaid, Riders for Health, and VillageReach improve and expand transport systems to increase access to healthcare and other essential services in low- and middle-income countries. Transaid develops vehicle management and maintenance systems in areas such as vaccine cold chain transport logistics . Riders for Health trains drivers and provides preventative maintenance on vehicles commissioned by health organizations . VillageReach addresses distribution systems for health resources, including vaccine cold chain transport infrastructure . These organizations address the many aspects of transport that affect vaccine availability, including vehicle availability and road safety. Efforts such as these are essential for new and existing vaccines to reach the people they are intended to protect.
All computer models make simplifying assumptions and cannot represent all possible factors or outcomes –. For this analysis, while model assumptions and data inputs were drawn from extensive review of the literature and data collection, the sources may vary in quality and model parameters may not hold under all conditions. The findings of this study suggest that when seeking to improve the vaccine supply chain for any country, transport should be a consideration. While this study examined the effects of augmenting only in-country transport, increasing the frequency of shipments from vaccine manufacturers to the central depot would also likely have positive implications for vaccine availability. However, not all countries are the same in their supply chain needs and population demand. Some may benefit more or less from changes in transport. It is possible that a given country may only have storage constraints. Future research can study constraints and alleviation strategies for vaccine supply chains in other countries.
Cold capacity in country vaccine supply chains may need to expand to meet increasing demands due to growing populations, new vaccine introductions, and larger packaging. While there has been an emphasis on donating stationary cold storage, augmenting transport may be just as crucial. In fact, as this study has found in Niger, increasing transport may have a far greater impact on vaccine availability than adding only stationary storage capacity. Furthermore, increasing transport could substantially decrease stationary storage requirements. Dynamic simulation modeling of vaccine supply chains can elucidate the complex interplay between storage and transport and guide donor priorities.
The HERMES Project team consists of (in alphabetical order): Tina-Marie Assi, PhD, Shawn T. Brown, PhD (Technical Lead), Brigid E. Cakouros, MPH, Sheng-I Chen, PhD, Diana L. Connor, MPH (Co-Coordinator), Erin G. Claypool, PhD, Leila A. Haidari, MPH, Veena Karir, PharmD, MS, Bruce Y. Lee, MD, MBA (Scientific Lead), Jim Leonard, Leslie E. Mueller, MPH, Bryan A. Norman, PhD, Proma Paul, MHS, Roslyn J. Phillips, MPH, Jayant Rajgopal, PhD, Michelle M. Schmitz, BA, Rachel B. Slayton, PhD, Angela R. Wateska, MPH (Co-Coordinator), Joel S. Welling, PhD, and Yu-Ting Weng, MS. For further questions regarding HERMES, please contact B. Lee, MD, MBA (BYL1@pitt.edu) or S. Brown, PhD (firstname.lastname@example.org). For more information, please visit: hermes.psc.edu.
Conceived and designed the experiments: LAH DLC ARW STB LEM BAN MMS PP JR JSW JL SC BYL. Performed the experiments: LAH DLC ARW STB LEM BAN MMS PP JR JSW JL SC BYL. Analyzed the data: LAH DLC ARW STB LEM BAN MMS PP JR JSW JL SC BYL. Contributed reagents/materials/analysis tools: LAH DLC ARW STB LEM BAN MMS PP JR JSW JL SC BYL. Wrote the paper: LAH DLC ARW STB LEM BAN MMS PP JR JSW JL SC BYL.
- 1. Kaufmann JR, Miller R, Cheyne J (2011) Vaccine supply chains need to be better funded and strengthened, or lives will be at risk. Health Aff (Millwood) 30: 1113–1121.
- 2. WHO (2010) EVM Assistant Tool.
- 3. PATH (2012) CCEM 2.
- 4. Isselmou BO (2011) UNICEF and government of Japan boost ‘cold chain’ for immunization in Mauritania. UNICEF. Nouakchott, Mauritania.
- 5. Rinzin YC (2012) 16 kerosene coolers for ‘power’less BHUs. Kuensel Online. Phuentsholing, Bhutan: Kuensel Corporation.
- 6. Maternal and Child Health Integrated Program (2011) Impact of new vaccine introduction on developing country immunization programs: a review of the grey literature. USAID Bureau for Global Health.
- 7. Humphreys G (2011) Vaccination: rattling the supply chain. Bull World Health Organ 89: 324–325.
- 8. de Oliveria LH, Danovaro-Holliday MC, Matus CR, Andrus JK (2008) Rotavirus vaccine introduction in the Americas: progress and lessons learned. Expert Rev Vaccines 7: 345–353.
- 9. Assi TM, Brown ST, Djibo A, Norman BA, Rajgopal J, et al. (2011) Impact of changing the measles vaccine vial size on Niger's vaccine supply chain: a computational model. BMC Public Health 11: 425.
- 10. Lee BY, Assi TM, Rajgopal J, Norman BA, Chen SI, et al. (2012 Feb) Impact of introducing the pneumococcal and rotavirus vaccines into the routine immunization program in Niger. Am J Public Health 102: 269–276.
- 11. Lee BY, Cakouros BE, Assi TM, Connor DL, Welling J, et al. (2012) The impact of making vaccines thermostable in Niger's vaccine supply chain. Vaccine
- 12. GAVI Alliance (2012) Niger country hub.
- 13. WHO (2010) Immunization profile - Niger.
- 14. WHO (2010) WHO prequalified vaccines.
- 15. WHO (2009) Vaccine volume calculator.
- 16. WHO (2011) Niger comprehensive multiyear plan 2011–2015.
- 17. Assi TM, Rookkapan K, Rajgopal J, Sornsrivichai V, Brown ST, et al. (2012) How influenza vaccination policy may affect vaccine logistics. Vaccine 30: 4517–4523.
- 18. Lee BY, Assi TM, Rookkapan K, Connor DL, Rajgopal J, et al. (2011) Replacing the measles ten-dose vaccine presentation with the single-dose presentation in Thailand. Vaccine 29: 3811–3817.
- 19. Lee BY, Assi TM, Rookkapan K, Wateska AR, Rajgopal J, et al. (2011) Maintaining vaccine delivery following the introduction of the rotavirus and pneumococcal vaccines in Thailand. PLoS One 6: e24673.
- 20. TransAid (2012) TransAid - transport for life.
- 21. Riders for Health (2012) What Riders for Health does.
- 22. VillageReach (2012) VillageReach: last mile challenges.
- 23. Lee BY (2008) Digital decision making: computer models and antibiotic prescribing in the twenty-first century. Clin Infect Dis 46: 1139–1141.
- 24. Lee BY, Biggerstaff BJ (2006) Screening the United States blood supply for West Nile Virus: a question of blood, dollars, and sense. PLoS Med 3: 168–169.
- 25. Lee BY, Wiringa AE (2011) The 2009 H1N1 influenza pandemic: a case study of how modeling can assist all stages of vaccine decision-making. Hum Vaccines 7: 115–119.