Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Development of a COVID-19 Vaccination Anxiety Scale to measure COVID-19 vaccine anxiety in Japanese adults

Abstract

Although COVID-19 vaccines are widely available and effective, vaccination anxiety remains a significant public health challenge that is distinct from behavioral vaccine hesitancy. While various scales measure hesitancy, few specifically capture the multifaceted psychological anxiety regarding COVID-19 vaccines, and none have been validated for the Japanese population. To directly quantify this affective component, this study aimed to develop and validate the COVID-19 Vaccination Anxiety Scale for Japanese adults. A two-wave web-based survey was conducted with 500 Japanese adults. Participants completed the newly culturally adapted 25-item scale alongside validation measures assessing fear of COVID-19, vaccination readiness, conspiracist beliefs, and perceived vulnerability to disease. Test–retest reliability was evaluated after a four-week interval. Factor analyses revealed a three-factor structure: Anxiety Related to Vaccine Confidence, Emotional Symptoms of Anxiety, and Anxiety about Infection after Vaccination. The scale showed good internal consistency and high test–retest reliability, and multigroup analyses suggested approximate comparability of the three-factor structure across gender. Construct validity was supported through significant, theoretically aligned correlations with the external criteria. Specifically, vaccination anxiety was strongly associated with lower vaccination readiness, higher fear of infection, and greater conspiratorial beliefs. These findings highlight that vaccination anxiety is a complex emotional response that requires specific measurement. From a public health perspective, the results suggest that addressing affective concerns—rather than solely providing analytic risk data—is crucial for effective communication. The COVID-19 Vaccination Anxiety Scale provides a useful tool for monitoring population-level anxiety and evaluating targeted interventions in Japan. Future studies should examine whether the same structure is replicated in younger and more age-balanced samples and whether the scale can be adapted for use in other settings.

Citation: Fukuda T, Haruyama R, Tanaka Y, Natori S, Koiwa K, Wakashima K (2026) Development of a COVID-19 Vaccination Anxiety Scale to measure COVID-19 vaccine anxiety in Japanese adults. PLoS One 21(7): e0330146. https://doi.org/10.1371/journal.pone.0330146

Editor: Cengiz Erisen, Yeditepe University, TÜRKIYE

Received: August 16, 2025; Accepted: May 29, 2026; Published: July 7, 2026

Copyright: © 2026 Fukuda et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Yes - all data are fully available without restriction; All relevant data are within the paper and the Supporting Information files.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Since 2020, COVID-19 vaccines have been rapidly developed and distributed free of charge in several countries, playing a central role in public health policies. Their effectiveness in preventing severe illness has been confirmed by scientific evidence. For example, a study in the United States reported that the Omicron-variant-specific vaccine (BNT162b2 XBB) had a 62% and 73.3% efficacy in preventing hospitalization and emergency department visits, respectively [1]. Domestic research in Japan has also shown that COVID-19 vaccines (XBB.1.5) can help prevent hospitalization despite their limited effectiveness in preventing infection [2]. Regular vaccination is recommended, however, as its effectiveness may weaken over time [3].

Despite the proven effectiveness and prevalence of vaccines, there remains deep-seated distrust and anxiety regarding their efficacy and safety [4]. While public health discussions often frame this issue primarily in terms of “vaccine hesitancy,” it is crucial to distinguish this behavioral concept from underlying vaccination anxiety. Vaccine hesitancy refers to a broader behavioral tendency to delay or refuse vaccination despite its availability [5], whereas vaccination anxiety refers to the emotional and cognitive distress associated with vaccination [6]. While related, these constructs are not identical. For example, individuals may experience substantial anxiety about vaccination while still choosing to be vaccinated, whereas in other cases, such anxiety may contribute to delay or refusal [7]. This study focuses specifically on vaccination anxiety.

Psychological factors, such as anxiety and distrust, have been identified as key elements associated with lower vaccine acceptance. Lazarus et al. [8] reported that in an international comparative survey of 13,426 people in 19 countries in June 2020, 71.5% of respondents said they would like to be vaccinated if the vaccine was safe and effective. To provide behavioral context, we reviewed later country-level vaccination uptake in the same 19 countries [9]. Across these countries, subsequent vaccination coverage ranged from 33.7% in South Africa and 36.4% in Nigeria to 92.9% in Singapore and 90.1% in China. Notably, some countries with relatively modest early acceptance later achieved high uptake (e.g., Singapore, 67.9% vs. 90.8%), whereas others showed the opposite pattern (e.g., South Africa, 81.6% vs. 35.1%). These descriptive comparisons suggest that early self-reported vaccine acceptance did not map onto later vaccination uptake perfectly, indicating that underlying psychological attitudes are complex and do not fully determine subsequent vaccination behavior.

Although Japan was not included in the Lazarus et al. survey [8], subsequent Japanese data suggest a similar distinction between self-reported attitudes and actual vaccination behavior. According to a domestic survey by Fukunaga [10], prior to April 2021, approximately 30% of respondents reported a lack of trust in the safety and efficacy of the vaccine; importantly, however, about half of this distrustful group still intended to be vaccinated. This suggests a psychological state in which individuals reluctantly choose to receive the vaccine despite harboring significant anxiety. A longitudinal study in Japan found that vaccination intention among the general public increased from 62.1% before vaccine distribution to 72.4% after distribution began [11]. At the population level, vaccine coverage for the primary two-dose regimen later reached 80.4% by December 2022 [12]. Taken together, these findings suggest that concerns or anxiety about vaccination do not necessarily translate into non-vaccination directly. Rather, the relationship between psychological responses and actual vaccine uptake may be shaped by broader social, institutional, and policy contexts. This further supports the importance of examining vaccination anxiety directly, rather than inferring it solely from vaccination behavior or expressed willingness.

Anxiety about vaccination is not unique to COVID-19; it has also been observed in the past with the introduction of influenza and HPV vaccines. According to risk perception theory [13], vaccine-related anxiety can arise when affective, intuitive reactions outweigh analytic, evidence-based assessments of vaccine benefits and adverse-event probabilities at the time of decision making. In such cases, perceived risk may be amplified even when objective risk estimates are low. The intensity and content of such anxiety can vary significantly depending on the social context [14]. The factors specific to COVID-19 vaccine anxiety include, first, the rapid spread and confusion of information in an unprecedented pandemic; second, unfounded anxiety about the new mRNA technology; and third, the inclusion of altruistic motives such as “for the sake of society” and “for the sake of others” in vaccination intentions. These factors are believed to have caused complex and multifaceted psychological reactions. Romate et al. [15] conducted a systematic review and identified fear of infection and severe illness, concerns about vaccine safety and side effects, and anxiety about interactions with preexisting conditions as the main factors associated with COVID-19 vaccine hesitancy. Additionally, trust in vaccines in general, trust in the government and healthcare institutions, beliefs in conspiracy theories, and emotional factors such as health anxiety also significantly influence vaccination anxiety. Cross-national European evidence indicates that conspiracy beliefs are associated not only with lower COVID-19 vaccine uptake but also with weaker support for pandemic-related public health policies [16]. Related work further suggests that COVID-19 conspiracy beliefs are linked to emotionally charged responses such as anger and anxiety [17]. Given the complex interplay among these factors, it is necessary to develop a scale that considers these multifaceted components to accurately assess COVID-19 vaccination anxiety.

Various scales have been developed to capture this multifaceted structure. For example, the Oxford COVID-19 Vaccine Hesitancy Scale developed by Freeman et al. [18] succinctly captures vaccination attitudes and refusal tendencies but does not sufficiently address the content of anxiety. Additionally, its validity has only been verified in relation to the fear of infection, leaving room for further validation. The COVID-19 Vaccine Concerns Scale by Gregory et al. [19] captures concerns specific to COVID-19 but does not address anxiety or fear itself. Therefore, it has a limited ability to capture the diversity of psychological backgrounds. Furthermore, the Multidimensional Vaccine Hesitancy Scale by Balgiu et al. [20] takes a comprehensive approach with eight factors; however, some of these factors are not specific to COVID-19 and instead focus on elements common to general vaccination (cost, inconvenience of vaccination, pain, etc.), limiting its application to COVID-19. Furthermore, there is no standardized scale in Japan, highlighting the need to develop measurement tools tailored to the characteristics of the Japanese population. International surveys have reported comparatively low vaccine confidence in Japan [21], and past vaccine controversies (e.g., the prolonged suspension of proactive HPV vaccine recommendations) [22] may have contributed to persistent safety concerns and distrust. Reviews of COVID-19 vaccine hesitancy in Japan also highlight context-specific themes such as mistrust of institutions and “zero-risk” expectations [23]. Therefore, a validated Japanese-language measure is needed to monitor vaccination-related anxiety and evaluate risk communication and interventions.

To address this gap, this study aimed to develop the COVID-19 Vaccination Anxiety Scale (C19-VAS). We drew upon the scale developed by Al Baseer and Shaheen [6] because it comprehensively and directly captures the emotional distress and multifaceted confidence issues specific to COVID-19 vaccines, overcoming the limitations of previous behavioral or generalized scales. Because the original instrument was designed for university faculty, we revised the item wording to be appropriate and understandable for the general adult population in Japan. We also modified the response format to a five-point Likert scale to improve measurement sensitivity. To examine the construct validity of the C19-VAS, we verified its relationship with the following four existing scales:

The first measure is the Japanese version of the Fear of COVID-19 Scale (FCV-19S-J) [24], which assesses the fear of COVID-19 infection. Individuals with higher levels of fear of the COVID-19 infection are more sensitive to vaccine side effects and efficacy, paradoxically increasing their anxiety about vaccination. Thus, FCV-19S-J and C19-VAS were expected to show a positive correlation (H1).

The second is the Japanese version of the 7C of Vaccination Readiness (7C) [25], which assesses the predictive factors for vaccination intent. As those with higher anxiety about COVID-19 vaccines are likely to have lower readiness for vaccination, it was predicted that the 7C and C19-VAS scores would show a negative correlation (H2).

The third is the Japanese version of the Generic Conspiracist Beliefs Scale (GCBS-J) [26], which assesses general tendencies toward conspiracist beliefs. Individuals with strong conspiratorial thinking tendencies are more likely to harbor distrust regarding the safety of the COVID-19 vaccine and the government’s intentions, leading to heightened vaccination anxiety. Therefore, it was predicted that the GCBS-J and C19-VAS scores would show a positive correlation (H3).

The fourth is the Japanese version of the Perceived Vulnerability to Disease Scale (PVD-J) [27], which assesses awareness of vulnerability to infectious diseases. Individuals with a high perception of vulnerability to infectious diseases are likely to be sensitive to the health effects of vaccines; therefore, it was predicted that the PVD-J and C19-VAS would show a positive correlation (H4).

Materials and methods

Participants and procedure

The survey was conducted between April and May 2025, targeting 500 participants aged 18 and over through the web survey service, Freeasy. The de-identified dataset underlying the findings is provided as supporting information (S1 Data). Of the initial 500 participants, we excluded 3 with incomplete responses, 5 physicians or medical professionals, and 72 who failed the Instructional Manipulation Check (IMC) task [28], leaving 420 participants (208 men and 212 women; mean age = 60.33 years, SD = 10.70) for the primary analysis. Medical professionals were excluded because their occupational exposure to vaccines, vaccine-related information, and vaccination practices differs substantially from that of the general population. Their inclusion might therefore have introduced a qualitatively different response pattern, making it less appropriate to validate the present scale as a general-population measure.

Additionally, to assess test-retest reliability, these participants were surveyed again after a four-week interval. A total of 356 participants who completed both surveys and successfully passed the follow-up IMC task [28] were included in the test-retest analysis. Table 1 presents the participants’ demographic characteristics.

thumbnail
Download:
Table 1. Demographic Characteristics of Participants.

https://doi.org/10.1371/journal.pone.0330146.t001

Ethical consideration

After the survey’s purpose was presented, the form stated that participation was voluntary and anonymous. Personal information would not be disclosed to third parties. Written informed consent was obtained digitally from all participants; only those who agreed to participate by checking an “I agree” box were able to proceed to the questionnaire. The Tohoku University Graduate School of Education’s Ethics Committee approved this study (ID: 24-1-092).

Measures

The same questions were asked in the first and second surveys. Table 2 summarizes the details of the FCV-19S-J, 7C, GCBS-J, and PVD-J (construct measured, reference, number of items, and main dimensions), along with our a priori directional hypotheses regarding their associations with the C19-VAS.

thumbnail
Download:
Table 2. Summary of Measures Used for Construct Validity and A Priori Hypotheses.

https://doi.org/10.1371/journal.pone.0330146.t002

Sociodemographic variables

Respondents were asked to provide information about their age, gender, pregnancy status, occupation, history of COVID-19 infection, presence of chronic diseases, sources of information on COVID-19, presence of family members living in the same household, and number of family members living in the same household.

C19-VAS

The scale developed by Al Baseer and Shaheen [6] was translated into Japanese and slightly modified with permission from the original authors. The original items were independently translated into Japanese by the authors, and content validity and naturalness of expression were rigorously reviewed and refined by an expert panel comprising one faculty member and two graduate students specializing in clinical psychology. Participants were instructed as follows (translated from Japanese): “The following questions ask about your views on COVID-19 vaccination. For each item, please indicate the extent to which you feel it applies to you and select the option that best reflects your response.” While Al Baseer and Shaheen [6] used a 3-point scale, a 4-point to 6-point scale was considered the optimal range [29]. Using a 3-point scale or lower may reduce reliability and validity [30]. Therefore, in this study, participants were asked to respond to each item using a 5-point scale ranging from “1 = does not apply” to “5 = applies.” Higher scores indicated greater anxiety about the COVID-19 vaccination. Additionally, when translating and revising the items, we confirmed the content validity with one faculty member with qualifications in clinical psychology and two graduate students specializing in clinical psychology.

FCV-19S-J

The FCV-19S-J [24] scale consists of 7 items, with responses ranging from “1 = Not at all” to “7 = Strongly agree” on a 7-point scale. In this study, the total score was used for analysis.

7C

The 7C [25] scale consists of 21 items, with 3 items for each of the 7 factors, and responses were acquired through a 7-point scale ranging from “1 = Strongly disagree” to “7 = Strongly agree.” In this study, the total score was used for analysis.

GCBS-J

The GCBS-J [26] scale consists of 15 items across 2 factors, with responses ranging from “1 = Definitely incorrect” to “5 = Definitely correct” on a 5-point scale. In this study, the total score was used for analysis.

PVD-J

The PVD-J [27] scale consists of 15 items across 2 factors, with responses ranging from “1 = Not at all applicable” to “7 = Very applicable” on a 7-point scale. In this study, the total score was used for analysis.

Data analysis

Confirmatory factor analysis (CFA) and multiple-group CFA were performed using IBM SPSS AMOS 29. All other analyses were conducted using the IBM SPSS Statistics version 29.

When translating into Japanese, we shifted from a 3-point scale to a 5-point scale and revised some of the items. Consequently, we decided to conduct an exploratory factor analysis (EFA). The data were randomly sampled into two groups: training and testing. First, an EFA was performed on the training data (N = 210) using maximum likelihood estimation and Promax rotation, excluding unnecessary items. CFA was performed on the test data (N = 210). As an additional analysis, multiple-group CFA was conducted to examine gender invariance. To test goodness of fit, we conducted the following analyses: comparative fit index (CFI), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), and Akaike information criterion (AIC). The cutoff values for acceptable model fit used in this study were as follows: RMSEA < .10 for acceptable fit and <.06 for good fit; CFI > .90 for acceptable fit and >.95 for good fit; and SRMR < .10 for acceptable fit and <.08 for good fit [31]. Error correlation was assumed to be related to the modified index (MI) if the goodness of fit was inadequate. Reliability was calculated for Cronbach’s alpha coefficients (α) and McDonald’s omega coefficients (ω). Correlations between the C19-VAS score and other measures were established by calculating Pearson’s correlation coefficients. The test-retest correlation coefficient was calculated using ICC (intraclass correlation coefficient, two-way random effects, and absolute agreement).

Results

Factor validity

As no items showing ceiling or floor effects were identified, factor analysis was performed on the 30 items of the original C19-VAS. The initial eigenvalues include 14.08, 2.85, 1.75, 1.17, 0.99, in order from the first factor. Based on the scree plot criterion and interpretability of the factors, a three-factor structure was deemed appropriate.

Next, factor analysis was performed, assuming 3 factors; 4 items with factor loadings below 0.40 and 1 item showing double loading were sequentially deleted. The final pattern matrix and inter-factor correlations are shown in Table 3. The cumulative factor contribution rate was 67.59% for the 3 factors across the 25 items. The first factor, “Anxiety Related to Vaccine Confidence,” consists of items such as “I lose confidence in the vaccine’s effectiveness owing to its variety and the different samples tested” and “I am worried about the side effects of the vaccine.” The second factor, “Emotional Symptoms of Anxiety,” comprises items such as “I am overwhelmed by feelings of fear due to excessive thinking about illness and death after vaccination” and “I tremble at the mere thought of vaccinating my family.” The third factor, “Anxiety about Infection after Vaccination,” includes items such as “Even after vaccination, I still dread the thought of being in contact with large numbers of people at work or school” and “I fear attending social gatherings despite being vaccinated.”

thumbnail
Download:
Table 3. Exploratory Factor Analysis of the C19-VAS.

https://doi.org/10.1371/journal.pone.0330146.t003

Finally, a CFA was performed using the three factors obtained through EFA as latent variables. The results of the CFA of the 3-factor model, with paths drawn from each latent variable to the corresponding items, showed that the model fit indices were χ2 = 781.311, p < .001, df = 272, CFI = .894, RMSEA = .095 [90% CI = .087–.102], SRMR = .073, AIC = 887.311, indicating insufficient model fit. MI was used to improve the model fit. The MI between items 6 and 7 (MI = 28.94) was the highest. Therefore, a within-factor error covariance between items 6 and 7 was included, and the model was modified. The results indicated that the modified model was more acceptable (χ2 = 750.866, p < .001, df = 271, CFI = .901, RMSEA = .092 [90% CI = .084–.100], SRMR = .072, AIC = 858.866) and was used as the final model (Fig 1). Descriptive statistics of the C19-VAS and other scales are presented in Table 4.

thumbnail
Download:
Table 4. Descriptive Statistics of the Sample’s Responses to the Study Scales.

https://doi.org/10.1371/journal.pone.0330146.t004

thumbnail
Download:
Fig 1. Final three-factor CFA model of the C19-VAS.

https://doi.org/10.1371/journal.pone.0330146.g001

Multiple-group CFA

To examine whether the same factor patterns apply to the male and female subsamples, we examined the measurement invariance by gender. Specifically, we confirmed the fit for each gender, then compared four models across datasets: a configuration-invariant model with no equivalence constraints on each dataset; a weak measurement invariance model with equivalence constraints only on factor loadings; a strong measurement invariance model with equivalence constraints on both factor loadings and intercepts; and a strict measurement invariance model with equivalence constraints on both factor loadings, intercepts, and error variables.

Regarding the goodness of fit for each gender, male individuals had χ2 = 559.142, p < .001, df = 271, CFI = .891, RMSEA = .104 [90% CI = .091–.116], SRMR = .080, AIC = 667.142, and female individuals had χ2 = 589.683, p < .001, df = 271, CFI = .859, RMSEA = .104 [90% CI = .092–.115], SRMR = 0.085, AIC = 697.683. Although the CFI criterion was not met, the RMSEA was slightly above the criterion value and the SRMR was below the criterion value for both men and women, indicating a marginally acceptable level of fit. We conducted a multi-population simultaneous analysis by gender for the configuration- and measurement-invariant models. The results showed that the fit of the configuration-invariant model was χ2 = 1148.837, p < .001, df = 542, CFI = .877, RMSEA = .073 [90% CI = .067–.079], SRMR = .080, AIC = 1464.837. The fit of the weak measurement invariance model was χ2 = 1186.602, p < .001, df = 563, CFI = .873, RMSEA = .073 [90% CI = .067–.079], SRMR = .076, AIC = 1460.602. The fit of the strong measurement invariance model was χ2 = 1220.149, p < .001, df = 588, CFI = .871, RMSEA = .072 [90% CI = .066–.078], SRMR = .076, AIC = 1444.149. Finally, the fit of the strict measurement invariance model was χ2 = 1260.177, p < .001, df = 613, CFI = .868, RMSEA = .071 [90% CI = .066–.077], SRMR = .079, AIC = 1434.177. Although the CFI value decreased as the number of equivalence constraints increased, the maximum decrease was only.009, and the RMSEA and SRMR values indicated a good fit. Furthermore, when comparing each model, including the AIC values, the strict measurement invariance model showed the best values among the four models, confirming strict measurement invariance (Table 5). Based on the above, as strict measurement invariance was confirmed for gender, the same three-factor structure was applied to construct the scale regardless of gender.

thumbnail
Download:
Table 5. Measurement Invariance of Model among Gender Groups for C19-VAS.

https://doi.org/10.1371/journal.pone.0330146.t005

Examination of gender differences

To examine gender differences in COVID-19 vaccine anxiety, a t-test (two-tailed) was conducted on 420 participants at Time 1. The results showed a significant gender difference (t (418) = 2.34, p = .02, d = .23). This indicates that women had higher total scores on the scale than men.

Reliability

Cronbach’s α and McDonald’s ω were calculated for the subscales of C19-VAS using data from 420 participants at Time 1. The first factor had α = .96 and ω = .96; the second factor had α = .91 and ω = .91; and the third factor had α = .76 and ω = .78, demonstrating sufficient internal consistency. Additionally, the ICC between Time 1 and Time 2 for “Anxiety Related to Vaccine Confidence” was.94 (95% CI [.92–.96]), “Emotional Symptoms of Anxiety” was.91 (95% CI [.89–.93]), and “Anxiety about Infection after Vaccination” was.81 (95% CI [.77–.85]).

Construct validity

To verify the construct validity of the C19-VAS, we performed a correlation analysis with the total scores of the FCV-19S-J, 7C, GCBS-J, and PVD-J at Time 1. When examining the correlation coefficients between the C19-VAS total score and FCV-19S-J, 7C, GCBS-J, and PVD-J, there was a moderate to strong positive correlation with FCV-19S-J (r = .44, p < .001), a strong negative correlation with 7C (r = −.59, p < .001), a strong positive correlation with GCBS-J (r = .55, p < .001), and a small to moderate positive correlation with PVD-J (r = .19, p < .001). As summarized in Table 6, H1–H4 were supported.

thumbnail
Download:
Table 6. Construct Validity: Correlations between the C19-VAS and Validation Measures.

https://doi.org/10.1371/journal.pone.0330146.t006

Discussion

This study aimed to develop the C19-VAS to measure COVID-19 vaccine anxiety in Japanese adults. Factor analysis revealed that the C19-VAS was constructed with 3 factors and 25 items: “Anxiety Related to Vaccine Confidence,” “Emotional Symptoms of Anxiety,” and “Anxiety about Infection after Vaccination.” These three factors differ from those developed by Al Baseer and Shaheen [6]. This is likely owing to the change from a 3-point scale to a 5-point scale and some revisions to the scale items. The scale developed by Al Baseer and Shaheen [6] was designed for university faculty, whereas the C19-VAS was revised for application in the general adult population. Therefore, the factor structure assumed by Al Baseer and Shaheen [6] was not observed.

In the CFA, allowing an error correlation between items 6 and 7 improved model fit to an acceptable, although not ideal, level. Conceptually, both items refer to broad anticipated harm arising from COVID-19 vaccination itself: item 7 concerns the potential negative consequences of vaccination in general, whereas item 6 refers to fear elicited by assuming personal responsibility for such consequences. In this sense, these two items appear to share a residual component related to generalized concern about adverse consequences, beyond the broader latent factor of “Anxiety Related to Vaccine Confidence.” By contrast, most of the other items in this factor refer to more specific sources of reduced confidence, such as side effects, insufficient data, conflicting information, or distrust in the government and in the healthcare system. We, therefore, considered the correlated residual theoretically and conceptually plausible, rather than a purely data-driven adjustment.

Nonetheless, the fit indices should be interpreted with caution. In the single-group CFA, although the final three-factor model improved in the initial solution, the CFI values were not uniformly strong, suggesting some residual item redundancy. For example, items 10 and 19 were semantically similar and showed a particularly high inter-item correlation (r = .83), suggesting that some overlap in item content may still remain. However, rather than relying on a single fit index alone, we retained the three-factor solution based on multiple considerations: the scree plot, interpretability and conceptual distinctiveness of the three domains, the fact that the final 25-item solution still explained a substantial proportion of the total variance, and adequate internal consistency of the resulting factors.

A multiple-group CFA was also conducted to test measurement invariance by gender. Across models, the absolute CFI values were slightly below the conventional.90 criterion, whereas RMSEA and SRMR were within or close to commonly used benchmark values. Therefore, these results should not be interpreted as indicating optimal fit, but rather, as a marginally acceptable fit. Nevertheless, the decrements in fit indices across the configural, metric, scalar, and strict invariance models were small, and strict invariance showed the most favorable AIC among the tested models. According to Chen [32], a ∆CFI of ≤.010 indicates that the invariance assumption is not substantially violated. In our analysis, the total decrement from the configural to the strict invariance model was only.009. Therefore, rather than judging invariance based on the absolute CFI value alone, it was judged based on the overall pattern of evidence, including the relative stability of RMSEA and SRMR and the small change in CFI across increasingly constrained models. Taken together, these findings suggest that the three-factor model is reasonably comparable across both men and women, while also indicating that some refinement of item wording or scale length may further improve model fit in future studies.

Next, the results of the gender difference analysis showed that women had higher COVID-19 vaccination anxiety than men. The results were consistent with those of Sekizawa et al. [33], who showed that women tended to be negative about COVID-19 vaccination. Women experience more side effects from the COVID-19 vaccine owing to hormonal differences between men and women [34], which may have contributed to gender differences in anxiety about COVID-19 vaccination. Additionally, women generally have higher sensitivity to COVID-19-related risks [35], and such personality traits may also influence gender differences in anxiety.

Internal consistency and test-retest reliability were also examined. The α and ω coefficients in C19-VAS returned sufficient values. Test-retest reliability also showed sufficient values for all three factors, confirming test-retest reliability. Subsequently, to verify construct validity, a correlation analysis was conducted between the C19-VAS at Time 1 and the total scores on the FCV-19S-J, 7C, GCBS-J, and PVD-J. The results showed positive correlations with FCV-19S-J, GCBS-J, and PVD-J, and a negative correlation with 7C.

The association with GCBS-J is particularly noteworthy. Similar associations have been reported both beyond the COVID-19 context and across diverse national settings. Health psychology studies have shown that conspiracist beliefs are associated with general anti-vaccination attitudes [36,37] and lower HPV vaccination intention [38]. These findings suggest that a broader conspiracist worldview may predispose individuals to interpret vaccines as threatening, untrustworthy, or harmful. Political science studies have shown comparable associations in cross-national COVID-19 research. For example, lower COVID-19 vaccine acceptance has been associated with lower trust in authorities and scientists and stronger conspiratorial thinking across multiple countries [39,40]. Related comparative research has also shown that conspiracy and misinformation beliefs are linked to emotional states and trust in information sources across countries [41]. In addition, Miller’s work supports the view that specific COVID-19 conspiracy beliefs are embedded in a broader conspiracist worldview shaped by distrust and uncertainty [42,43]. More broadly, Filsinger and Freitag showed that vaccination attitudes can become a source of affective polarization across multiple European countries [44]. Taken together, these findings suggest that the present association between general conspiracist beliefs and COVID-19 vaccination anxiety in Japanese adults is consistent with broader evidence indicating that conspiracist thinking may shape not only vaccine acceptance but also the affective meaning attached to vaccination.

The correlation coefficient with PVD-J was lower than those with other scales, which was likely due to differences in the factors of PVD-J, namely, “perceived infectability” and “germ aversion.” When the correlation between each factor and C19-VAS score was examined, a significant correlation (r = .20, p < .001) was confirmed with the latter but no correlation (r = .09, p = .06) was confirmed with the former. This suggests that perceived infectability is not associated with COVID-19 vaccine anxiety, whereas perceived discomfort in situations where pathogens are likely to be encountered is associated with COVID-19 vaccine anxiety. Duncan et al. [45] distinguished between “perceived infectability,” which is a response based on rational judgment, and “germ aversion,” which is a response based on intuitive judgment. Items such as “I don’t want to wear secondhand clothes because I don’t know who last wore them” reflect “germ aversion” and measure anxious tendencies toward infection risks that lack empirical basis [27]. Regarding the COVID-19 vaccine, providing unfounded conspiracy theories or misinformation increases hesitancy toward vaccination [46]. Many conspiracy theories and rumors exploit intuitive reactions rather than theoretical evidence of adverse events. Therefore, COVID-19 vaccine anxiety and “germ aversion” share a similar cognitive tendency to avoid risks through immediate judgments based on weak evidence, which may explain the observed correlation. Meanwhile, “perceived infectability” is a response based on rational judgment [45]. Therefore, no significant correlation was demonstrated between “perceived infectability” and C19-VAS. From a public health communication perspective, this finding implies that simply providing analytic information about infection risk or vaccine effectiveness may be insufficient for reducing vaccination anxiety. Communication strategies also need to address affective concerns, such as fear, disgust, and anticipated discomfort, in a transparent and reassuring manner. Further investigation is necessary in this regard. Based on these results, the C19-VAS was considered to have sufficient reliability and validity.

This study has some limitations. First, healthcare workers were excluded from the analysis. Because healthcare workers received COVID-19 vaccines earlier and more frequently than the general population, their risk perceptions and psychological responses may differ. Consequently, the factor structure of the C19-VAS, which was developed for the general adult population, might not fully apply to them. Future research should investigate the scale’s validity in healthcare settings or develop a scale tailored to this population. Second, this study did not include questions about vaccination history before taking the COVID-19 vaccine. The presence or absence of vaccination experience may influence attitudes toward the vaccine and anxiety. In the future, it will be necessary to collect detailed information on vaccination history, such as the number of doses administered and the presence or absence of adverse reactions, and examine their association with vaccination anxiety. Third, the analyzed sample was skewed toward older adults, which limits the generalizability of the findings to younger Japanese adults. Previous studies in Japan have suggested that younger adults are more likely to show COVID-19 vaccine hesitancy, whereas older age is associated with greater willingness to vaccinate and more frequent booster uptake [47]. Therefore, this age imbalance may have influenced not only the overall level of reported vaccination anxiety, but also the relative salience of specific psychological factors reflected in the factor structure. Accordingly, the findings should be interpreted primarily as evidence from a relatively older adult sample, and future studies should examine whether the same factor structure and psychometric properties are replicated in younger and more age-balanced populations. Fourth, because the survey was conducted in 2025, the salience of COVID-19 vaccination may have varied across respondents depending on recent experiences, such as infection history or exposure to COVID-19-related information. Although this study assessed current thoughts and feelings about COVID-19 vaccination, rather than a retrospective recall of the early pandemic period, we did not directly model topic salience in the analyses. To address this limitation, future studies should examine how recent exposure, infection history, and related contextual factors may influence C19-VAS responses.

Conclusions

This study clarified the factor structure of the C19-VAS, which demonstrated good internal consistency and test–retest reliability. Although the acute phase of the pandemic has passed, COVID-19 vaccination remains an ongoing public health issue in Japan in 2025, with routine annual vaccination for older adults and high-risk groups, and voluntary vaccination options outside the routine program [48]. Globally, the WHO continues to recommend that countries offer COVID-19 vaccination and notes sustained SARS-CoV-2 evolution, which may necessitate updates to vaccine composition [49].

Importantly, vaccination anxiety is not fully captured by vaccination uptake alone: Affective reactions can coexist with vaccine acceptance and may fluctuate with policy, information environments, and trust. This is consistent with risk-perception frameworks distinguishing “risk as feelings” from analytic risk appraisal [13]. The C19-VAS provides a brief, validated tool to quantify this psychological component, enabling population monitoring, the identification of groups with heightened anxiety, and the evaluation of risk communication strategies and interventions in Japan. While the scale was validated for Japan and the COVID-19 context, it also offers a framework for future work in other settings with appropriate cultural adaptation and psychometric revalidation.

Acknowledgments

We would like to express our sincere gratitude to all the participants who cooperated with this research. We would like to thank Editage for English language editing.

References

  1. 1. Tartof SY, Slezak JM, Frankland TB, Puzniak L, Hong V, Ackerson BK, et al. Estimated effectiveness of the BNT162b2 XBB vaccine against COVID-19. JAMA Intern Med. 2024;184(8):932–40. pmid:38913355
  2. 2. Institute of Tropical Medicine, Nagasaki University. Vaccine effectiveness real-time surveillance for SARS-CoV-2 (VERSUS) study: 11th report. VERSUS Group. 2024. [Cited 2025 July 24]. https://www.tm.nagasaki-u.ac.jp/versus/results/20240524.html
  3. 3. Maeda H, Saito N, Igarashi A, Ishida M, Terada M, Masuda S, et al. Effectiveness of primary series, first, and second booster vaccination of monovalent mRNA COVID-19 vaccines against symptomatic SARS-CoV-2 infections and severe diseases during the SARS-CoV-2 omicron BA.5 epidemic in Japan: Vaccine effectiveness real-time surveillance for SARS-CoV-2 (VERSUS). Expert Rev Vaccines. 2024;23(1):213–25. pmid:38288980
  4. 4. Sallam M. COVID-19 vaccine hesitancy worldwide: A concise systematic review of vaccine acceptance rates. Vaccines (Basel). 2021;9(2):160. pmid:33669441
  5. 5. MacDonald NE, SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: Definition, scope and determinants. Vaccine. 2015;33(34):4161–4. pmid:25896383
  6. 6. Al Baseer NAM, Shaheen HS. A study of some psychological variables as predictors of COVID-19 vaccination anxiety among Faculty Members of Ain Shams University. Sci Rep. 2024;14(1):26615. pmid:39496619
  7. 7. Gilan D, Birkenbach M, Wossidlo M, Sprengholz P, Betsch C, Hahad O, et al. Fear of COVID-19 disease and vaccination as predictors of vaccination status. Sci Rep. 2023;13(1):8865. pmid:37258562
  8. 8. Lazarus JV, Ratzan SC, Palayew A, Gostin LO, Larson HJ, Rabin K, et al. A global survey of potential acceptance of a COVID-19 vaccine. Nat Med. 2021;27(2):225–8. pmid:33082575
  9. 9. Mathieu E, Ritchie H, Rodés-Guirao L, Appel C, Gavrilov D, Giattino C, et al. Coronavirus (COVID-19) Vaccinations. OurWorldinData.org. https://archive.ourworldindata.org/20260305-063307/covid-vaccinations.html. 2020. Accessed 2026 March 5.
  10. 10. Fukunaga H. Social psychology and media coverage of the new corona vaccination: An Internet survey. The NHK Monthly Report on Broadcast Research. 2021;71(7):2–27.
  11. 11. Machida M, Nakamura I, Kojima T, Saito R, Nakaya T, Hanibuchi T, et al. Trends in COVID-19 vaccination intent from pre- to post-COVID-19 vaccine distribution and their associations with the 5C psychological antecedents of vaccination by sex and age in Japan. Hum Vaccin Immunother. 2021;17(11):3954–62. pmid:34723753
  12. 12. Matsumoto N, Mitsuhashi T, Matsuo R, Kadowaki T, Takao S, Yorifuji T. Effectiveness of the Original COVID-19 Vaccine against COVID-19 exacerbations during the omicron wave: A population-based study in Okayama, Japan. JMA J. 2023;6(4):463–9. pmid:37941689
  13. 13. Slovic P, Finucane ML, Peters E, MacGregor DG. Risk as analysis and risk as feelings: Some thoughts about affect, reason, risk, and rationality. Risk Anal. 2004;24(2):311–22. pmid:15078302
  14. 14. Slovic P. Perception of risk. Science. 1987;236(4799):280–5. pmid:3563507
  15. 15. Romate J, Rajkumar E, Gopi A, Abraham J, Rages J, Lakshmi R, et al. What Contributes to COVID-19 Vaccine Hesitancy? A systematic review of the psychological factors associated with COVID-19 vaccine hesitancy. Vaccines (Basel). 2022;10(11):1777. pmid:36366286
  16. 16. Regazzi L, Lontano A, Cadeddu C, Di Padova P, Rosano A. Conspiracy beliefs, COVID-19 vaccine uptake and adherence to public health interventions during the pandemic in Europe. Eur J Public Health. 2023;33(4):717–24. pmid:37290405
  17. 17. Peitz L, Lalot F, Douglas K, Sutton R, Abrams D. COVID-19 conspiracy theories and compliance with governmental restrictions: The mediating roles of anger, anxiety, and hope. Journal of Pacific Rim Psychology. 2021;15.
  18. 18. Freeman D, Loe BS, Chadwick A, Vaccari C, Waite F, Rosebrock L, et al. COVID-19 vaccine hesitancy in the UK: the Oxford coronavirus explanations, attitudes, and narratives survey (Oceans) II. Psychol Med. 2022;52(14):3127–41. pmid:33305716
  19. 19. Gregory ME, MacEwan SR, Powell JR, Volney J, Kurth JD, Kenah E, et al. The COVID-19 vaccine concerns scale: Development and validation of a new measure. Hum Vaccin Immunother. 2022;18(5):2050105. pmid:35380510
  20. 20. Balgiu BA, Sfeatcu R, Țâncu AMC, Imre M, Petre A, Tribus L. The Multidimensional Vaccine Hesitancy Scale: A validation study. Vaccines (Basel). 2022;10(10):1755. pmid:36298620
  21. 21. de Figueiredo A, Simas C, Karafillakis E, Paterson P, Larson HJ. Mapping global trends in vaccine confidence and investigating barriers to vaccine uptake: A large-scale retrospective temporal modelling study. Lancet. 2020;396(10255):898–908. pmid:32919524
  22. 22. Ujiie M. Resumption of active recommendation of the human papillomavirus vaccine in Japan and future challenges for the National Immunization Program. Hum Vaccin Immunother. 2022;18(6):2090777. pmid:35767827
  23. 23. Kuroda M, Ahmed MK, Kuroda K, Lane SD. Understanding COVID-19 vaccine hesitancy among the general population in japan from public health ethical perspectives: Findings from a narrative review. Asian Bioeth Rev. 2024;17(1):141–65. pmid:39896082
  24. 24. Wakashima K, Asai K, Kobayashi D, Koiwa K, Kamoshida S, Sakuraba M. The Japanese version of the Fear of COVID-19 scale: Reliability, validity, and relation to coping behavior. PLoS One. 2020;15(11):e0241958. pmid:33152038
  25. 25. Machida M, Kojima T, Popiel HA, Geiger M, Odagiri Y, Inoue S. Development, validity, and reliability of the Japanese version of the 7C of vaccination readiness scale. Am J Infect Control. 2023;51(4):426–33. pmid:35839960
  26. 26. Majima Y, Nakamura H. Development of the Japanese version of the Generic Conspiracist Beliefs Scale (GCBS-J). Jpn Psychol Res. 2019;62(4):254–67.
  27. 27. Fukukawa Y, Oda R, Usami H, Kawahito J. Development of a Japanese version of the Perceived Vulnerability to Disease Scale. Shinrigaku Kenkyu. 2014;85(2):188–95. pmid:25016839
  28. 28. Masuda S, Sakagami T, Morii M. Comparison among methods for improving response quality of surveys. Jpn J Psychol. 2019;90(5):463–72.
  29. 29. Lee J, Paek I. In search of the optimal number of response categories in a rating scale. J Psychoeduc Assess. 2014;32:663–73.
  30. 30. Lozano LM, García-Cueto E, Muñiz J. Effect of the number of response categories on the reliability and validity of rating scales. Methodology. 2008;4(2):73–9.
  31. 31. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55.
  32. 32. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Model. 2007;14(3):464–504.
  33. 33. Sekizawa Y, Hashimoto S, Denda K, Ochi S, So M. Association between COVID-19 vaccine hesitancy and generalized trust, depression, generalized anxiety, and fear of COVID-19. BMC Public Health. 2022;22(1):126. pmid:35042506
  34. 34. Al-Qazaz HK, Al-Obaidy LM, Attash HM. COVID-19 vaccination, do women suffer from more side effects than men? A retrospective cross-sectional study. Pharm Pract (Granada). 2022;20(2):2678. pmid:35919795
  35. 35. Lewis A, Duch R. Gender differences in perceived risk of COVID-19. Soc Sci Q. 2021;102(5):2124–33. pmid:34908608
  36. 36. Hornsey MJ, Harris EA, Fielding KS. The psychological roots of anti-vaccination attitudes: A 24-nation investigation. Health Psychol. 2018;37(4):307–15. pmid:29389158
  37. 37. Milošević Đorđević J, Mari S, Vdović M, Milošević A. Links between conspiracy beliefs, vaccine knowledge, and trust: Anti-vaccine behavior of Serbian adults. Soc Sci Med. 2021;277:113930. pmid:33873008
  38. 38. Shapiro GK, Holding A, Perez S, Amsel R, Rosberger Z. Validation of the vaccine conspiracy beliefs scale. Papillomavirus Res. 2016;2:167–72. pmid:29074176
  39. 39. Lindholt MF, Jørgensen F, Bor A, Petersen MB. Public acceptance of COVID-19 vaccines: cross-national evidence on levels and individual-level predictors using observational data. BMJ Open. 2021;11(6):e048172. pmid:34130963
  40. 40. Jennings W, Valgarðsson V, McKay L, Stoker G, Mello E, Baniamin HM. Trust and vaccine hesitancy during the COVID-19 pandemic: A cross-national analysis. Vaccine X. 2023;14:100299. pmid:37063307
  41. 41. De Coninck D, Frissen T, Matthijs K, d’Haenens L, Lits G, Champagne-Poirier O, et al. Beliefs in conspiracy theories and misinformation about COVID-19: Comparative perspectives on the role of anxiety, depression and exposure to and trust in information sources. Front Psychol. 2021;12:646394. pmid:33935904
  42. 42. Miller JM. Do COVID-19 conspiracy theory beliefs form a monological belief system?. Can J Polit Sci. 2020;53(9):1–9.
  43. 43. Miller JM. Psychological, political, and situational factors combine to boost COVID-19 conspiracy theory beliefs. Can J Polit Sci. 2020;53(2):327–34.
  44. 44. Filsinger M, Freitag M. Asymmetric affective polarization regarding COVID-19 vaccination in six European countries. Sci Rep. 2024;14(1):15919. pmid:38987619
  45. 45. Duncan LA, Schaller M, Park JH. Perceived vulnerability to disease: Development and validation of a 15-item self-report instrument. Pers Individ Dif. 2009;47:541–6.
  46. 46. Lee SK, Sun J, Jang S, Connelly S. Misinformation of COVID-19 vaccines and vaccine hesitancy. Sci Rep. 2022;12(1):13681. pmid:35953500
  47. 47. Okamoto Y, Hiyama T, Miyake Y, Yoshino A, Miyauchi S, Tanaka J. Attitudes and behavior toward COVID-19 Vaccination in Japanese University Students: A cross-sectional study. Vaccines (Basel). 2022;10(6):863. pmid:35746471
  48. 48. Ministry of Health, Labour and Welfare (Japan). COVID-19 Q&A. https://www.mhlw.go.jp/stf/seisakunitsuite/bunya/vaccine_qa.html. Accessed 2026 March 7.
  49. 49. WHO. Statement on the antigen composition of COVID-19 vaccines. https://www.who.int/news/item/15-05-2025-statement-on-the-antigen-composition-of-covid-19-vaccines. 2025. Accessed 2026 March 7.