http://orcid.org/0000-0002-2116-0324
Figures
Abstract
Introduction
Protecting vaccines from freeze damage is considered one of the most poorly addressed problems in vaccine management. Freezing may impair the potency especially of adsorbed vaccines. The Keep Cool study aims at ensuring optimal vaccine storage conditions in general practices. This publication analyses the baseline data using standardised temperature recordings.
Methods
This prospective study in German general practices analysed 7-day temperature recordings of refrigerators used for vaccine storage. Temperatures were recorded continuously using a standardised data logger with an accuracy of ±0.4 °C. The prevalence rates of refrigerators within the target range (2 to 8 °C) and of those reaching critically low temperatures (≤0 °C) were calculated. In addition, the cumulative time and the duration of single episodes beyond the target range were computed. To assess for structural deficits, the prevalence of refrigerators with a cycling of >5 °C was determined. Generalised linear mixed models were applied to analyse correlating factors between the dependent variables ‘within temperature range’ and ‘reaching critically low temperatures’ with practice characteristics.
Results
The study included 64 of 168 practices (38.1% response rate) with 75 refrigerators. The prevalence of refrigerators with temperatures within the target range was 32.0% (n = 24), and 14.7% (n = 11) reached critically low temperatures <0 °C. 44.0% of refrigerators (n = 33) showed temperatures >8 °C and 28.0% (n = 21) <2 °C. Of the 168 hours recorded per refrigerator, the average cumulative time >8 °C was 49 hours, <2 °C 75 hours and ≤0 °C 74 hours. The longest consecutive period of critically low temperatures was 168 hours (mean: 39±53). The prevalence of refrigerators with a cycling range of >5 °C was 29.3%.
Citation: Thielmann A, Puth M-T, Kersting C, Porz J, Weltermann B (2019) Vaccine cold chain in general practices: A prospective study in 75 refrigerators (Keep Cool study). PLoS ONE 14(11): e0224972. https://doi.org/10.1371/journal.pone.0224972
Editor: Sanjai Kumar, Food and Drug Administration, UNITED STATES
Received: January 23, 2019; Accepted: October 8, 2019; Published: November 19, 2019
Copyright: © 2019 Thielmann et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The manuscript’s data cannot be shared publicly because of ethical restrictions as our data set includes potentially identifying information. Data are available from the Ethics Committee of the Medical Faculty of the University of Duisburg-Essen (ethikkommission@uk-essen.de), Germany, for researchers who meet the criteria for access to confidential data.
Funding: The study received grants from Stiftung Universitätsmedizin Essen (http://www.universitaetsmedizin.de/) to buy thermometers. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Immunisations are among the most effective and cost-effective public health strategies worldwide [1]. However, their effectiveness depends on adequate vaccine storage conditions. Maintaining the cold chain, i.e. a temperature range of 2 °C to 8 °C, is crucial to ensure vaccine potency [2] and tolerability [3]. In the past, cold chain breaches were suspected of causing disease outbreaks, but confirming this suspicion is difficult [4–8]. Preventing freezing is especially important to maintain the potency of adsorbed vaccines (e. g. hepatitis A, hepatitis B, tetanus, diphtheria, pertussis, pneumococcal disease) [2]. Adsorbed hepatitis B vaccines are considered the most sensitive vaccines, with a freezing threshold of -0.5 °C [2]. At this temperature, irreversible precipitates of aluminium-containing adsorbents begin to form which decrease the potency of the vaccines. Also, these may induce local irritation upon injection [2,3]. Furthermore, all vaccines are at risk of contamination when exposed to freezing temperatures, as hairline cracks in the pre-filled syringe can develop which are not necessarily noticeable to the human eye [9]. The World Health Organization (WHO) considers protecting vaccines from freeze damage “one of the most poorly addressed problems in vaccine management” that requires attention in order not to jeopardise disease-prevention goals. [10].
Research of the American National Institute of Standards and Technology showed that refrigerators’ suitability to maintain the cold chain varies drastically depending on the type of refrigerator [11,12]. Relevant parameters are ‘temperature control stability, air circulation patterns, defrost cycles, and long-term drift of the temperature set point’ [11]. A crucial aspect is the technical design of the cooling compressor and its regulation based on on-off mechanisms. Purpose-built refrigerators for the storage of medical products, so-called pharmaceutical refrigerators, have several advantages compared to household models: enhanced temperature set point control, a better ventilation system, and a narrower temperature range [12]. Many household models are designed to allow different temperature zones required in food storage [13] and allow for freezing temperatures, e. g. -5 °C [11].
According to a systematic literature review, freezing temperature exposure occurred in approximately 33.3% of refrigerators used for vaccine storage in ten wealthier countries [14]. In our prior cross-sectional questionnaire study, 16% of German general practices self-reported experiencing cold chain breaches either as an error or near-error, and 49% lacked adequate monitoring and documentation [15,16]. The Keep Cool study aims at ensuring optimal vaccine storage conditions: after visual inspections of refrigerators used to store vaccines and a baseline temperature survey of seven days, general practices with temperature violations are offered access to a tailored online learning program [17,18].
This publication presents the baseline data of the Keep Cool study: standardised, continuous 7-day temperature data are analysed for cold chain breaches in general practices. First, we aimed to identify the prevalence of refrigerators with temperatures within the target range (2–8 °C). Second, we determined the prevalence of critically low temperatures (≤0 °C) and analysed the temperature cycling ranges of refrigerators in order to assess their capacity to maintain the cold chain. Third, associations between practice characteristics and temperature were analysed.
Methods
Study design
Details on this prospective intervention study with two temperature measurement periods have been reported elsewhere [17]. Briefly, this publication describes the baseline of the Keep Cool study, which was developed by two researchers (A.T., B.W.), formerly Institute for General Medicine, University of Duisburg-Essen, now: Institute for Family Medicine and General Practice, University of Bonn, Germany. We report about the quality of the vaccine cold chain in general practices with temperature readings over a 7-day monitoring period. Details on the quality of vaccine refrigerator management (e.g. temperature monitoring frequency, presence of a thermometer, placement of temperature probe) based on visual inspections of the refrigerators studied have been reported elsewhere [18].
Ethical approval was obtained from the Ethic Commission of the Medical Faculty of the University of Duisburg-Essen (14-6118-BO). Participants provided written informed consent.
Study population and recruitment procedure
The study was conducted in general practices affiliated with the University of Duisburg-Essen (N = 185) as teaching practices. Practices (n = 17) involved in study pre-tests were excluded. Recruitment followed a structured approach: Practices were contacted by phone and fax up to three times or until they responded. To estimate participation bias, non-participants received a short questionnaire by fax asking them to provide details on: 1) their reason for non-participation, 2) the number of refrigerators in practices including those in the recreation room, 3) the use of a thermometer, and 4) whether the temperature is monitored twice daily.
Temperature monitoring
Temperatures were measured with a data logger (testo 175T), which has an accuracy of ±0.4 °C within the operating range -5 °C to +10 °C (calibrated under a DIN EN ISO 9001:2008 certified quality assurance system). The device was equipped with a standard probe which measures the ambient air temperatures inside the refrigerator and the effects of door openings on refrigerator temperatures. We used continuous measurements over seven days with a logging interval of one reading per minute. Similar logging rates were used in prior studies [19–23].
In preparation for this study, we developed a protocol for the set-up of the data logger which had been piloted in a sample of 17 general practices with 21 refrigerators. In line with standards [11–13,24], the data logger was positioned in the centre of the refrigerator and placed in a plastic bin (see Fig 1). During the recording, the display of the data logger was turned off and access to its memory was locked.
Practice and physician characteristics
The following practice characteristics were obtained by questionnaire: type of practice (solo/group), number of practice team members by professional groups, patients per quarter (caseload), number of treatment rooms, thermometer in each vaccine refrigerator, vaccine spectrum offered, and selected services offered (tropical medicine and/or yellow fever, travel medicine, adolescent preventive services, paediatric preventive services and/or adolescent medicine), percentage of patients with statutory health insurance, and certified quality management. Data on the type of refrigerator used for vaccine storage were collected while setting up the data logger (pharmaceutical-grade/household refrigerator; location and insulation of ice compartment, if any).
Statistical analysis
Descriptive statistics were used to describe participating and non-participating practices for practice and physician characteristics. For the non-participant analysis, the χ2 test was used for categorical data, and Student’s t-test was used for continuous data.
The temperature readings of the 7-day monitoring period (10,080 minutes) were analysed. The first 120 minutes after each data logger set-up were excluded from analyses to allow the probe to acclimatise to the temperature of the refrigerator.
The primary outcome was the prevalence of refrigerators with temperatures within the target range (2 °C to 8 °C) for seven days. Secondary outcomes included reaching different cut-offs based on data on temperature sensitivity of the WHO [2], personal manufacturer information (GlaxoSmithKline) and systematic reviews [14,25]: <2 °C and >8 °C, >8 °C, <2 °C, ≤1 °C, ≤0 °C. In order to provide a better indication of unacceptable temperature exposure, we calculated the cumulative and consecutive time (in hours) outside the target range and beyond different cut-offs. Analyses were performed for each individual refrigerator and for the total sample using mean, standard deviation (SD) and range.
A further secondary outcome addressed the refrigerators’ capacity to keep temperatures within the target range. A temperature cycling range >5.0 °C was considered unacceptable. Temperature ranges were analysed for each refrigerator, for the total sample and stratified by household and pharmaceutical-grade refrigerators as well as by refrigerators considered acceptable and unacceptable for vaccine storage. Acceptable refrigerators included pharmaceutical-grade, freezerless refrigerators and full-size dual-zone refrigerators/freezers with separate exterior doors, while unacceptable refrigerators included any mini refrigerators and refrigerators with an internal ice compartment [11–13].
To estimate the relationship between practice characteristics and the two dependent variables ‘within temperature range (2–8 °C) versus outside target range’ and ‘reaching critically low temperatures (0 °C) versus within target range’, we used hierarchical generalised linear mixed models (GLMM) for binomial responses with random practice-specific intercepts (to account for practices with more than one refrigerator). Independent characteristics were: type of practice (solo/group), number of patients in practice (≤1,750/ >1,750), percentage of patients with statutory health insurance, yellow fever licence (yes/no), physician trainee in practice (yes/no), certified quality management (yes/no), and the provision of paediatric preventive services and/or adolescent medicine (yes/no).
Statistical analyses were performed using IBM SPSS Statistics for Windows, version 24 (Armonk, NY: IBM Corp.) and R, version 3.5.1. Percentages and mean values are reported for valid cases.
The trial is registered with the German Clinical Trials Register (DRKS00006561).
Results
Practice characteristics
Of the 168 practices contacted, 64 agreed to participate (response rate: 38.1%). The mean practice size was 2.1 general practitioners (±1.2) and 5.3 medical assistants (±3.3); 59.4% (n = 38) were group practices. 51.9% (n = 27) of the practices provided medical care to up to 1,750 patients per quarter (caseload). In total, 75 refrigerators were included in this study. 14.1% (n = 9) had more than one refrigerator for the storage of vaccines. See Table 1 for details.
73 of the 104 non-participating practices provided a reason for non-participation. The most frequent reasons were (multiple responses): no time (37.0%, n = 27), no interest in topic/study participation (35.6%, n = 26), no need (11.0%, n = 8), other (∑7, 23.3%, n = 17). The non-participant analysis showed no differences regarding key practice characteristics, except that participating practices were more likely to provide care to ≤1,750 patients per quarter (48.1% versus 68.1%, p = 0.029). See S1 Table for details.
Refrigerators and temperature recordings
Of the 75 refrigerators included, 88.0% (n = 66) were household refrigerators and 12.0% (n = 9) were pharmaceutical-grade models (see Table 1 for details). In 24 of 75 included refrigerators (32.0%), temperatures were within the target range of 2 °C to 8 °C (see Fig 2). This corresponds to 74.8% of the total measurement time. The mean temperature was 5.3 °C (±2.9), with readings ranging between -6.7 and +12.2 °C.
Based on their ability to maintain the target temperature range, refrigerators were categorised into six exclusive groups (Table 2). Refrigerators that were within the target range but had at least one temperature breach >8 °C (n = 28) had a mean temperature of 7.3 °C (±0.7) and were outside the target range 25.3% of the time. Refrigerators within the target range that had at least one temperature breach <2 °C (n = 17) had a mean temperature of 1.8 °C (±1.5) and were outside the target range 45.5% of the time. Separate data on individual refrigerators is presented in S2 Table.
Critically low temperatures ≤0 °C were recorded at least once in 14.7% (n = 11) of all refrigerators. This corresponds to 5.8% of the temperature recording time, i.e. the total time based on all refrigerators (mean cumulative time: 74.1 hours ±56.1). The longest consecutive time ≤0 °C was 39.1 hours on average (±52.9: 0.6–168.0): one refrigerator was below zero at all times (168 hours), for the other refrigerators the average cumulative time was 64.7 hours). Temperatures <2 °C were recorded in 28.0% (n = 21) of refrigerators, corresponding to 12.4% of the complete temperature recording time. These refrigerators had a mean of 31.6 episodes (±36.3: 1–142) below <2 °C with an average duration of 18.7 hours (±41.0: 12 min to 168 hours). Further temperature cut-offs are presented in Table 3. On average, the target temperature range (8 °C) was exceeded for 48.6 hours (±51.4). This accounts for 12.7% of the total study. See Table 3 for details.
Structural characteristics: Cycling range of refrigerators
In 29.3% (n = 22) of refrigerators, temperature cycling was >5 °C. Cycling ranges ≤3.0 °C were recorded in 45.3% (n = 34), >3.0 to ≤4.0 °C in 17.3% (n = 13), >4 °C in 37.3% (n = 28) and >6 °C in 17.3% (n = 13). Of the 22 refrigerators with cycling ranges >5 °C, only one refrigerator maintained the cold chain. In comparison, in refrigerators with cycling ranges ≤5 °C, 43.4% (n = 23) maintained the cold chain and 56.6% (n = 30) had cold chain breaches. Refrigerators varied with regard to the range of temperature cycling during the day and over the course of the 7-day monitoring period. Fig 3 shows typical temperature curves encountered. The overall mean temperature range was 4.0 °C (±2.5), with readings ranging between 0.7 to 10.6 °C.
All data were recorded at 1-minute intervals over 7 days. The graphs illustrate common problems regarding temperature cycling range and mean temperature: A: an adequate cycling range of 3°C with a too-high mean temperature at approx. 7°C; B: a cycling range of 5.5°C with a too-low mean temperature at approx. 3.5°C; C: a too-high cycling range of 10°C and a too-low mean temperature at approx. 2°C.
The mean temperature in pharmaceutical refrigerators (n = 9) was 5.3 °C (±1.1: 4.3 to 7.9 °C) with a mean temperature range of 3.1 (±1.5: 0.7 to 5.9 °C). In comparison, the mean temperature in household refrigerators (n = 66) was 5.3 °C (±2.8: -1.8 to 10.7 °C) with a mean temperature range of 4.1 (±2.6: 1.0 to 10.6 °C).
Discussion
Of the 75 refrigerators analysed, only 32% maintained the vaccine cold chain. However, 68% were beyond the target range and 15% reached a critically low temperature of 0 °C. We found that continuous freezing temperature exposure lasted longer than one day on average (39 hours) with a longest episode of seven days recorded. These data suggest that freeze damage likely occurred.
In line with the systematic review of freezing temperatures by Hanson [14] to assess whether freezing remains an ongoing issue, we cannot answer the question regarding the number of vaccines that were actually damaged or had reduced immunogenicity. Nevertheless, there is a link between disease outbreak and temperature excursions below the freezing threshold for hepatitis B [26] and pertussis [4]. In our practice sample, more than 98% store hepatitis B vaccines. Thus, until thermostable vaccines are available or freeze-free technologies are used across all practices, freeze prevention requires close attention.
To ensure the cold chain, two components need to be fulfilled: 1) structural component with a cycling range below 5 °C and 2) continuous refrigerator management targeting for a mean temperature of +5 °C. In 29% of refrigerators, cycling ranges exceeded the cut-off of 5 °C and thus constituted a major barrier for successful cold-chain maintenance. Interestingly, even in refrigerators with narrower temperature ranges, about 60% failed to maintain the cold chain, indicating procedural deficits.
In line with prior studies, our observations in the practices during the study conduct shed an interesting light on key influencing factors. Practices did not have adequate temperature monitoring rigor (i. e., a suitable thermometer, monitoring at least twice daily), which is a significant predictor of noticing critical temperatures [18,25]. Overall, knowledge and problem awareness deficits prevailed, which is known from other countries [19–21,27]. For instance, physicians and medical assistants expressed astonishment with regard to the encountered temperature ranges when shown the temperature curves of their refrigerators. With the exception of door opening times, most participants expected rather stable temperatures, as they were unaware of the construction-based cycling of refrigerators. In practices affected by freezing temperature exposure, we even encountered disbelief (‘your thermometer is broken’). There was a general belief that freezing temperatures would be noticed in the form of frozen vaccines. Frequently, constant fluctuations between freezing and thawing were never considered before. Misshapen cardboard packaging due to thawing ice (two practices) and thick ice walls (one practice) went unnoticed (for details see Thielmann et al.) [18].
Limitation
All participating practices are members of a teaching practice network. A potential selection bias can be excluded as we showed in a prior study that the practice sample is representative for general practices in Germany [28]. In order to assess participation bias, we conducted a thorough non-responder analysis. For financial reasons, we used a standard air probe to measure temperature. In contrast to that, a slow-reacting glycol probe resembles the temperature changes of the vaccine vials. Given the accuracy, all measured temperature values are within ±0.4 °C of the true value.
Conclusion
The prevalence of refrigerators with cold chain breaches and critically low temperatures is high, which emphasises the need for an intervention aimed at adequate vaccine storage. Risk communication should address the dangers associated with too cold temperatures and refrigerators’ temperature cycling issues. Furthermore, greater attention needs to be paid to structural and procedural best practices in vaccine storage that are used as a safeguard against temperature excursions.
Supporting information
S1 Table. Characteristics of participating practices (N = 64).
https://doi.org/10.1371/journal.pone.0224972.s001
(DOCX)
S2 Table. Temperature recordings per refrigerator (N = 75).
https://doi.org/10.1371/journal.pone.0224972.s002
(DOCX)
Acknowledgments
We would like to thank all general practices of the University of Duisburg-Essen who participated in the study. We also wish to thank Kim Steiner who conducted practice visits and helped with the data entry.
References
- 1.
World Health Organization, Unicef. Global Immunization Data; 2013. http://www.who.int/immunization_monitoring/Global_Immunization_Data_v2.pdf. Accessed 17 January 2014.
- 2.
World Health Organization. Temperature sensitivity of vaccines. Department of Immunization, Vaccines and Biologicals, World Health Organization. 2006: 1–62.
- 3.
Paul-Ehrlich-Institut. Ausflückungen in Adsorbatimpfstoffen. Bulletin zur Arzneimittelsicherheit—Informationen aus BfArM und PEI; 2012: 12–16.
- 4. McColloster P, Vallbona C. Graphic-output temperature data loggers for monitoring vaccine refrigeration: implications for pertussis. Am J Public Health. 2011; 101: 46–47. pmid:21088272
- 5. Lerman SJ, Gold E. Measles in children previously vaccinated against measles. JAMA. 1971; 216: 1311–1314. pmid:4102807
- 6. Onoja AL, Adu FD, Tomori O. Evaluation of measles vaccination programme conducted in two separate health centres. Vaccine. 1992; 10: 49–52. pmid:1539460
- 7. Boros CA, Hanlon M, Gold MS, Roberton DM. Storage at− 3 C for 24 h alters the immunogenicity of pertussis vaccines. Vaccine. 2001; 19: 3537–3542. pmid:11348721
- 8. McIntyre RC, Preblud SR, Polloi A, Korean M. Measles and measles vaccine efficacy in a remote island population. Bull World Health Organ. 1982; 60: 767. pmid:6983927
- 9.
Salisbury D, Ramsay M, Noakes K. Immunisation against infectious disease. 3rd ed. London: TSO; 2006.
- 10.
World Health Organization. Aide mémoire for prevention of freeze damage to vaccines. WHO/IVB/07.09. Geneva; 2007.
- 11.
Chojnacky M, Miller W, Ripple D, Strouse G. Thermal Analysis of Refrigeration Systems Used for Vaccine Storage (NISTIR 7656); 2009.
- 12.
Chojnacky M, Miller W, Strouse G. Thermal Analysis of Refrigeration Systems Used for Vaccine Storage (NISTIR 7753). Report on Pharmaceutical Grade Refrigerator and Household Refrigerator/Freezer; 2010.
- 13.
Center for Disease Control. National Center for Immunization and Respiratory Diseases. Vaccine Storage & Handling Toolkit: Centers for Disease Control and Prevention; 2018.
- 14. Hanson CM, George AM, Sawadogo A, Schreiber B. Is freezing in the vaccine cold chain an ongoing issue? A literature review. Vaccine. 2017; 35: 2127–2133. pmid:28364920
- 15. Thielmann A, Sikora M, Schnell U, Gesenhues S, Weltermann B. Impfkühlschrank-und Impfstoffmanagement in Hausarztpraxen: Eine repräsentative, Web-basierte Umfrage unter Hausärzten (Keep Cool I). Das Gesundheitswesen. 2015; 38: 1–6.
- 16. Weltermann BM, Markic M, Thielmann A, Gesenhues S, Hermann M. Vaccination Management and Vaccination Errors: A Representative Online-Survey among Primary Care Physicians. PloS one. 2014; 9: e105119. pmid:25118779
- 17. Thielmann A, Viehmann A, Weltermann BM. Effectiveness of a web-based education program to improve vaccine storage conditions in primary care (Keep Cool): study protocol for a randomized controlled trial. Trials. 2015; 16. pmid:26169675
- 18.
Thielmann A, Puth M-T, Weltermann B. Visual inspection of vaccine storage conditions in general practices:. A cross-sectional study of 75 vaccine refrigerators (Keep Cool study). PloS one: in Revision.
- 19. Gazmararian JA, Oster NV, Green DC, Schuessler L, Howell K, Davis J, et al. Vaccine storage practices in primary care physician offices: assessment and intervention. Am J Prev Med. 2002; 23: 246–253. pmid:12406478
- 20. Bell KN, Hogue C, Manning C, Kendal AP. Risk factors for improper vaccine storage and handling in private provider offices. Pediatrics. 2001; 107: e100. pmid:11389298
- 21. Page SL, Earnest A, Birden H, Deaker R, Clark C. Improving vaccination cold chain in the general practice setting. Aust Fam Physician. 2008; 37: 892. pmid:19002316
- 22. Gold MS, Martin L, Nayda CL, Kempe . Electronic temperature monitoring and feedback to correct adverse vaccine storage in general practice. MJA. 1999; 171: 83–84. pmid:10474583
- 23. Lewis PR, Reimer RF, Dixon AJ. Evaluating the efficacy of vaccine storage in the general practice setting. Aust N Z J Public Health. 2001; 25: 547–550. pmid:11824993
- 24.
Australian Government. National vaccine storage guidelines. Strive for 5. 2nd ed. Canberra: Dept. of Health and Ageing; 2013.
- 25. Matthias DM, Robertson J, Garrison MM, Newland S, Nelson C. Freezing temperatures in the vaccine cold chain: a systematic literature review. Vaccine. 2007; 25: 3980–3986. pmid:17382434
- 26. Davaalkham D, Ojima T, Wiersma S, Lkhagvasuren T, Nymadawa P, Uehara R, et al. Administration of hepatitis B vaccine in winter as a significant predictor of the poor effectiveness of vaccination in rural Mongolia: evidence from a nationwide survey. J Epidemiol Community Health. 2007; 61: 578–584. pmid:17568048
- 27. Lee S, Lim H-S, Kim O, Nam J, Kim Y, Woo H, et al. Vaccine Storage Practices and the Effects of Education in Some Private Medical Institutions. J Prev Med Public Health. 2012; 45: 78–89. pmid:22509448
- 28. Viehmann A, Thielmann A, Gesenhues S, Weltermann BM. Repräsentieren akademische Hausarztpraxen die hausärztliche Regelversorgung. Eine methodische Annäherung. Do Academic Family Practices Reflect Routine Primary Care? A Methodological Approach. Z Allg Med. 2014; 90: 354–359.