Vaccination Management and Vaccination Errors: A Representative Online-Survey among Primary Care Physicians

Background Effective immunizations require a thorough, multi-step process, yet few studies comprehensively addressed issues around vaccination management. Objectives To assess variations in vaccination management and vaccination errors in primary care. Methods A cross sectional, web-based questionnaire survey was performed among 1157 primary physicians from North Rhine-Westphalia, Germany: a representative 10% random sample of general practitioners (n = 946) and all teaching physicians from the University Duisburg-Essen (n = 211). Four quality aspects with three items each were included: patient-related quality (patient information, patient consent, strategies to increase immunization rates), vaccine-related quality (practice vaccine spectrum, vaccine pre-selection, vaccination documentation), personnel-related quality (recommendation of vaccinations, vaccine application, personnel qualification) and storage-related quality (storage device, temperature log, vaccine storage control). For each of the four quality aspects, “good quality” was reached if all three criteria per quality aspect were fulfilled. Good vaccination management was defined as fulfilling all twelve items. Additionally, physicians’ experiences with errors and nearby-errors in vaccination management were obtained. Results More than 20% of the physicians participated in the survey. Good vaccination management was reached by 19% of the practices. Patient-related quality was good in 69% of the practices, vaccine-related quality in 73%, personnel-related quality in 59% and storage-related quality in 41% of the practices. No predictors for error reporting and good vaccination management were identified. Conclusions We identified good results for vaccine- and patient-related quality but need to improve issues that revolve around vaccine storage.


Introduction
Effective immunizations require a thorough, multi-step process on behalf of the physicians and the practice teams in charge. In many Western countries, detailed recommendations address not only the medical indications for the various vaccinations, but also structural and procedural aspects of the vaccination management. Although millions of vaccinations are performed each year [1], relatively few studies address the every-day challenges of practices' vaccination management. Unsystematic observations in our region suggest marked differences between practices with regard to patient information, how to obtain patient consent, the involvement of practice assistants in vaccinations and the vaccine handling. Aiming to understand variations in vaccination management in primary care, we performed a web-based question-naire survey among primary care physicians in Germany's largest federal state North Rhine-Westphalia.

Ethics statement
Ethical approval was obtained from the ethic commission of the medical faculty of the University of Duisburg-Essen.

Survey design and study populations
We performed a cross-sectional, web-based questionnaire survey among a random and a convenience sample of primary care physicians in North Rhine-Westphalia, Germany's largest federal state. The survey included general practitioners (with and without board certification) and general internists, paediatricians were excluded. Primary care physicians in North Rhine-Westpha-lia are organized in the two regional Statutory Health Care Administrations North Rhine and Westphalia-Lippe. We draw a 10% random sample with a total of 946 of 9533 primary care physicians: 576 of 5757 physicians in North Rhine and 378 of 3776 physicians in Westphalia-Lippe. In addition, all primary care teaching physicians of the University of Duisburg-Essen, Essen (n = 211), were invited to participate in the survey.

Survey instrument
The online questionnaire survey was constructed using Lime-Survey (www.limesurvey.org). The questionnaire consisted of 30 questions addressing immunization aspects relevant for daily primary care. The items were developed and selected based on three practice visits and a systematic comparison of four national guidelines (United Kingdom [2], United States of America [3,4], Australia [5,6], Germany [7][8][9][10]). The first part of the questionnaire addressed issues of vaccination management, the second part errors and near-errors while the third part requested social demographic data. The questionnaire was pre-tested on the research team and three external academic primary care physicians.
Quality indicators. For the first part of the questionnaire a modified Delphi technique with five primary care teaching physicians was used to identify relevant quality aspects. The four selected items were: patient-related quality, vaccine-related quality, personnel-related and storage-related quality. In each of these four categories, three items were selected as relevant quality indicators: N Patient-related quality: -Patient information: in four case studies physicians provided all patients with information -Patient consent: consensus obtained always verbally and/or written -Three strategies to increase immunization rates were used: the physician often or always controls the immunization status of new patients, patients in risk groups and provides patients with a follow-up appointment for the next vaccination N Vaccine-related quality:

Errors and near-errors in vaccination management
In the second part of the questionnaire, physicians' experiences with 15 vaccination errors were addressed. Items were based on the anonymous German critical incident reporting system ''Jeder Fehler Zaehlt'' (ie, ''every error counts''; www.jeder-fehler-zaehlt. de). Answer options for each item were: not experienced, experienced as near-error or experienced as actual error.

Data collection
The survey was hosted on the LimeSurvey platform. Participants were invited by email or -if email address was missingletter and provided with login information. Participants were asked to complete the anonymous questionnaire within six weeks and received a reminder after three weeks.

Statistical analysis
Analyses were conducted using SPSS Statistics, version 20. For the descriptive statistics, the frequencies of various answers were calculated. The quality of vaccination management was assessed using one quality indicator each for patient-, vaccine-and personnel and storage-related aspects. Each indicator consisted of the above mentioned three items and scoring positive on all these served as cut-off point for good quality. Also, a summarizing quality indicator called ''good vaccination management'' was developed through the summation of scores above the cut-off points on the four indicators; scoring positive on all 12 indicators was considered good quality. Chi-square tests were used to analyse if independent parameters (practice and physician characteristics) influenced the frequencies of scoring on the four quality indicators and the overall quality indicator for good vaccination management. A statistical significance was assigned at a level of p,0.05. Logistic regression analysis was used to determine predictors for good vaccination management. Factors associated with good vaccination management quality in bivariate analysis were included in the final model.

Study participation
A total of 247 of 1157 physicians participated in the survey (response rate: 21%). The sub-analyses showed that 127 of 211 teaching physicians and 120 of 946 physicians in the random sample participated. The questionnaire was completed by 74% of the random sample and 65% of the university sample. Formal comparison showed that completion of the questionnaire was positively correlated with a larger vaccine spectrum in the random sample (Spearman r s = 0.191, p = 0.037), in the teaching sample no correlation was identified. Analyses included completed questionnaires only (n = 172). The two samples showed no significant differences in their structural characteristics except that the teaching practice sample showed more practices with two or more physicians (65% vs. 46%, p,0.01), more practices with more than 1500 patients (59% vs. 34%, p,0.01) and more physicians with palliative care training (29 vs. 16%, p = 0.04). For details see Table 1 and Table S1.

Physician and practice characteristics
The majority of the physicians was male with an average age of 51 years. 63% were GPs, 30% held a specialty degree in general internal medicine, and 5% were practitioners without degree. Almost half of the physicians (44%) were working in a solo practice. The majority (77%) had practices with more than 1000 patients quarterly. For details see Table 1 and Table S1.

Vaccination management
The indicators were met as follows: 69% met the patient-related quality indicator, 73% the vaccine-related quality indicator, 59% the personnel-related quality indicator and 41% the storagerelated quality indicator. The overall quality indicator for good vaccination management was reached by 19%. Comparison of the quality indicators in the two samples showed no significant difference except that the percentage of practices fulfilling the patient consent quality indicator was 96% in the random sample and 87% among the teaching physicians (p = 0.02). For details see Table S2. Potential for improvement was documented for all storage-related aspects and single other items: only 79% used a separate refrigerator for vaccine storage, 51% kept a storage temperature log, 92% performed a regular storage control (wrapping, temperature and expiration date), 77% had a complete chart documentation (86% the charge number and dose, 84% the trade name), 66% recommended vaccinations according to current recommendations. Practice strategies to increase vaccination coverage were used often or routinely by 76% of practices.
For details see Table 2.
Various additional characteristics of practice vaccination management not included in the quality indicators were obtained. Of the 20 vaccines offered, the practices used on average 16.6 vaccines (8-20, SD: 2.28), 95% offered all of the five standard vaccinations, 6% offered all 15 indication vaccinations (on average 11.7 vaccines). The following pattern of responsibilities within the practice team was identified: in most practices physicians were responsible for the vaccine pre-selection (79%) and providing patient information (83%), whereas in the majority of practices medical assistants were designated as solely responsible for vaccine ordering (73%), stock control (82%) and storage control (60%). All practices (99%) reported that either the physician or the medical assistant is responsible for the storage control. In 20% of the practices vaccines were administered by the physician only, while the decision who inoculates was case-specific in 56% of the practices, depending either on the vaccine or the patient. The participation rate in vaccination-related CME within the last two years was 77% in physicians and 70% in medical assistants. A recall to all patients was offered by 41% of practices. Regular practice software for the development of patients' lists was used by 35%, additional immunization specific software by 4%. Results on additional characteristics of practice vaccination management are presented in Table 3.

Error and near-error reporting
At least one error or near-error was reported by 89% of the participants. On average, a total of 3.4 errors and near-errors had occurred: 2.3 errors and 1.2 near-errors. The five most frequently reported types of errors were: double vaccination due to missing documentation in 40%, intramuscular injection of a patient on anticoagulants therapy in 39%, vaccination despite missing indication in 36%, wrong vaccine in 19%, and the vaccination of a 14-year-old without parental approval in 16%. The three most frequently reported near-errors were: vaccination despite of acute disease in 13%, intramuscular injection of a patient on anticoagulants therapy in 13%, and expired vaccine in 12%. For details on the sum of all 15 errors and near-errors see Table 4 and  Table S3.

Physicians' opinions about the survey
The survey was considered helpful by 30%, triggered a reconsideration of practice standards in 38% and caused the wish to participate in CME in 16%. Taking immediate action was reported by 10% of physicians, 69% voiced an interest in the results.

Predictors for good vaccination management quality and error reporting
One fifth (19%) of all practices showed ''good vaccination management''. Chi-square tests for the four quality indicators and the overall quality indicator for good vaccination management did  Logistic regression did not identify any predictors for good vaccination management.

Discussion
To our knowledge this is the first comprehensive study among primary care physicians about vaccination management: using twelve quality indicators we showed that 19% of practices had good practice vaccination management, while a marked variance between practices and the need to improve storage conditions became apparent.

Patient-related process quality
In the medical literature, three groups of interventions were studied as strategies to increase immunization rates in children and adults: patient-oriented interventions (e.g. written reminders), provider interventions (e.g. preventive services flow sheets in patient charts) and system interventions (e.g. public immunization campaigns) [11]. A Cochrane review comparing RCTs with patient-oriented interventions showed that postcards, letters, autodialers, and person-to-patient phone calls increased immunization rates: phone reminders were most effective but also most costly, yet increased immunization rates up to thirty percent [11]. In 2000, a survey among 316 US primary care physicians showed that 23% of practices are using mail or phone reminders [12]. This is comparable to our results which showed that 31% of practices are using phone, and up to 14% written reminders, e.g. sms, email, or letter. Provider interventions were more frequent than patient-oriented interventions both in the US survey and our study, yet the measures used differ. In decreasing order, US physicians used the following three strategies most frequently: preventive service flow sheets in patient charts (71%), walk-in immunization service (67%), and a policy to assess vaccination status at each visit [12]. In comparison, our survey showed that physicians apply the strategy to control risk groups (95%), provide appointment for next vaccination (90%) and use routine consultations to optimize immunization levels (90%). These differences may be due to health care system factors, physician education, and individual preferences. System differences also play a role with regard to the mode of patient consent and information: while verbal information and verbal patient consent is sufficient by German law, US law requires the provision of detailed written consent prior to each vaccination [4].

Vaccine-related process quality
Adequately trained personnel is of key importance to assure a high quality of vaccine management. This was documented in an intervention study in the Atlanta region: daily temperature monitoring of vaccine storage compartments was 2-3 times more likely if the designated coordinator had a higher level of medical education [13]. A survey of 221 US practices showed that 83% had designated a specific person as responsible for vaccine storage and handling, with a backup in 63% of practices [13]. Another US doi:10.1371/journal.pone.0105119.t003 Wrong temperature in refrigerator 27 16 Wrong vaccination administration 28 16 Staff vaccinated without physician's order 22 13 Wrong patient inoculated 17 10 Reminder send to patient with new family doctor 16 9 A pregnant woman receives rubella inoculation 0 0 *The items offered were based on reports in a German primary care incidents reporting system. survey among 721 primary care offices differentiated between ordering, storing and application of vaccines: in the majority of practices only one person was responsible for ordering vaccines (75%), two or more staff members were responsible for storing (50%) and application of vaccines (77%) [14]. Our survey analyzed responsibilities within the practice team: in more than 70% of practices it is the sole responsibility of the physician in charge to pre-select vaccines, while ordering, storage and stock control are typically delegated to medical assistants. We consider it positive that physicians are involved in vaccine pre-selection, as these decisions may pre-vent programmatic vaccination errors described in the medical literature, such as mixed use of brands for the same vaccination or the use of different vaccines with similar names [15][16][17].

Personnel-related quality
The definition of our quality indicator addressed three typical aspects of personnel-related immunization quality: application of vaccination recommendations, personnel qualification and the physician or a designated person applies the vaccine. Considering the recommendation of catch-up vaccination regimes, studies among pediatricians showed a broad variance between physicians [18][19][20], which was confirmed in our study. Using one clinical vignette we showed discrepancies to current recommendations. These differences may be oblivious due to knowledge deficits (e.g. with regard to immunization recommendations and contraindications), or deliberate due to disagreement with immunization recommendations. Both aspects have been previously associated with the omission of vaccinations, especially in pediatricians [12,19]. Discussed are also increasingly complex immunization schedules as barrier to immunization practices conform to recommendations [20,19].

Storage-related quality
The definition of our quality indicator addressed three typical storage-related aspects: storage device, keeping a storage temperature log and regular storage control. In our study, 79% of practices used a separate refrigerator for vaccines, which was documented in 96% of 695 US primary care practices [14], yet only 59% of 172 Australian [21], and 9 to 22% of 135 Canadian practices [22]. All of the practices surveyed in our study had designated coordinators responsible for storage control, which is higher compared to 83% US private providers who designated a person for the joint tasks vaccine storage and handling [13]. Frequently, vaccine storage conditions are a weak point in the quality chain with the potential to cause disease outbreaks [23][24][25], to reduce the vaccine effectiveness and the tolerability of vaccines [26,27]. A systematic literature review based on 14 studies in developed countries showed that 13.5% (6.4 to 20.7%) of refrigerators had temperatures below the freeze threshold [28]. The difficulty of maintaining refrigerators within the correct temperature range becomes clear considering that even after participating in an intervention study, 50% of the practices fail this criteria [29]. Additionally, the storage equipment used is crucial. An Australian study based on 28 general practitioners showed that the refrigerator type used is associated with maintaining correct temperatures, with purpose built vaccine refrigerators showing better results [30]. Our results are in agreement with these findings: only 92% regularly controlled their storage and only 51% of the practices kept a storage temperature log. The latter result is much lower compared to 73% US primary care practices [14], yet comparable to 53% US private physicians who do not have a log [13]. Thus, at least in our region, future interventions to improve vaccine-related quality should address the issue of a separate refrigerator, a storage temperature log and regular storage control as important content of CME and practice improvement strategies.

Errors and near-errors
In our understanding, the reported average of 3.4 errors and near-errors indicates good error awareness. In accordance with Bundy et al. who analyzed 607 error reports on an US error reporting database (''MEDMARX'') [17] we can confirm high rates for those types of errors reported most frequently in that database: double vaccination (40% vs. 25%) and wrong vaccine (19% vs. 25%). Thus, interventions should focus e.g. on improving vaccination documentation to reduce doubled vaccination (e.g. using electronic health records) and prevent look-alike/sound-alike confusion to reduce using the wrong vaccine.

Limitations, Conclusions and Perspectives
Our cross-sectional physician survey addressed a broad spectrum of issues. Several limitations need to be considered. First, it was performed in a country lacking detailed guidelines on vaccination management and the quality indicators used are based on common reasoning and standards from other countries. Second, we did not compare physicians' answers with actual every-day vaccination management. Third, given the low response rate, a response bias and a bias towards the desired answers cannot be excluded, e.g. regarding time since last CME.
In summary, we demonstrated that about 19% of German primary care practices have good vaccination management, yet broad variance for single aspects. Interventions should focus on storage-related issues. Table S1 Comparison of physician random sample and teaching physicians for physician and practice characteristics.