1.1. Climate change knowledge: A multidimensional concept

According to Azevedo and Marques [24], climate literacy comprises three key elements: knowledge of climate science, the ability to access and evaluate climate information, and positive attitudes toward adaptation and mitigation strategies. This definition integrates both objective components (knowledge and skills) and subjective elements (attitudes), highlighting climate literacy’s multidimensional nature. As Sato and Park [25] emphasize, “qualifying a person to be climate change literate should require a meticulous assessment of not one but multiple domains of climate change literacy” (pp. 11–12).

Regarding climate knowledge specifically (as distinct from skills or attitudes), existing assessment tools vary considerably in their scope and focus on different dimensions of climate literacy [19]. Some tools emphasize the biophysical processes of climate change [e.g., 26], while others concentrate on specific causes [e.g., food practices, 27] and consequences [e.g., infectious diseases, 28].

In their comprehensive assessment of climate-related knowledge, Tobler et al. [20] identified four key dimensions. The first dimension, physical knowledge, encompasses scientific principles like the greenhouse effect and carbon dioxide’s (CO₂) role. The second dimension focuses on knowledge about climate change and its causes, examining both its existence and human origins. The third dimension addresses knowledge of climate change consequences, including for instance increased extreme weather events and melting polar ice caps. The fourth dimension covers action-related knowledge, which includes strategies and practices for reducing CO₂ emissions. Building on this framework, Taddicken et al. [29] added a fifth dimension—procedural knowledge—which examines how climate knowledge is generated and the scientific uncertainties involved.

As mentioned previously, most existing climate knowledge measurement tools only partially cover these five dimensions [19,25]. Moreover, the action knowledge dimension, when measured, typically focuses on individual behaviors. Many questionnaires, for instance, assess people’s understanding of how personal actions affect CO₂ emissions [e.g., 30–32]. However, solutions to the climate crisis go well beyond individual actions. They include collective-level decisions such as urban planning, public transit implementation, and eco-taxation [33–35]. Despite this, climate knowledge measurement tools rarely assess this broader type of knowledge [19,25].

Moreover, while most assessment tools in the literature focus primarily on global climate change knowledge (e.g., how CO₂ emissions are raising Earth’s temperature), understanding how populations comprehend local climate issues is equally important.

Firstly, different regions of the world experience climate change impacts in distinct ways. In northern regions like Scandinavia, for instance, accelerated melting of glaciers and permafrost contributes to rising sea levels and threatens coastal communities [36]. Meanwhile, Mediterranean countries face increasingly frequent and intense heat waves. In 2024, Greece, Spain, Portugal, France, and Morocco suffered extreme weather events that led to fatalities, widespread wildfires, and public health emergencies [37]. Therefore, measurement tools that focus solely on global climate change consequences cannot assess whether people understand how climate change affects their local area. This limitation makes it impossible to determine if individuals truly comprehend the risks specific to their region.

Secondly, greenhouse gas emissions patterns vary significantly across countries and regions. For example, some European countries—Portugal, France, and the UK—have much lower CO₂ emissions per capita than neighboring countries with comparable living standards, such as Germany, the Netherlands, and Belgium. This difference stems from their energy choices: Portugal, France, and the UK generate most of their electricity from nuclear and renewable sources, while Germany relies on fossil fuels for about half of its electricity production [38]. Significant differences also exist between regions within the same country. In Canada, greenhouse gas emissions in the province of Quebec (9.1 tons of CO₂ equivalent per capita in 2022) are substantially lower than in the province of Saskatchewan (64.4 tons of CO₂ equivalent per capita during the same year). This distinction also stems from their energy source: Quebec relies primarily on hydroelectricity, while Saskatchewan depends mainly on natural gas [39]. As a result, climate knowledge assessment tools that focus solely on global causes or generic solutions cannot effectively measure whether people understand the most relevant actions for reducing emissions or adapting to climate change in their specific region.

1.2. Measuring climate change knowledge

Research on objective climate knowledge assessment features questionnaires of various lengths, from brief scales with fewer than five items [e.g., 40, 41] to extensive ones with over 20 items [e.g., 26, 29]. While comprehensive questionnaires yield rich insights into target audiences, their administration is resource-intensive. Shorter questionnaires present a practical solution but—as noted earlier—often fail to cover all dimensions of climate knowledge. This creates a pressing need for concise yet multidimensional questionnaires that can effectively distinguish between varying levels of climate change knowledge at both global and local scales.

Existing climate knowledge scales show considerable variation in how they phrase and structure items [19,25]. While some questionnaires ask respondents to select correct answers from multiple choices [e.g., 5, 26], others require them to evaluate statements [e.g., 20, 42]. In the latter approach, respondents typically rate their agreement with claims on a scale from strongly agree to strongly disagree [e.g., 29, 41]. This method, however, makes it difficult to separate objective knowledge from attitudes and beliefs [29]. Though some researchers have tried to address this by measuring awareness of expert consensus on climate change [e.g., 41], the challenge of distinguishing between actual knowledge and perceived expert consensus persists.

Moreover, some researchers rely on self-assessment tools to evaluate participants’ climate knowledge [19]. For instance, Pilgreen et al. [43] measured subjective climate knowledge with two self-reported items on a five-point scale ranging from ‘not at all knowledgeable’ to ‘very knowledgeable’ (e.g., ‘How would you rate your knowledge of the issue of climate change?’). While such measures are practical, they also tend to conflate knowledge with confidence or attitudes, underscoring the need for validated tools that more accurately capture objective knowledge.

To this end, researchers often use true/false statements rather than agreement scales to better measure objective knowledge. These questionnaires frequently include an “I don’t know” option to discourage random guessing [e.g., 8, 44]. However, this approach has drawbacks—less confident respondents might select “I don’t know” even when they know the answer, while less motivated participants might choose it as an easy way out [45]. To address this issue, researchers can ask respondents to rate their confidence in each answer, either through a separate question or as part of a Likert scale [e.g., 40, 42].

Regardless of their length, item types, and response options, few climate knowledge questionnaires have undergone rigorous validation [19]. Most are ad hoc instruments with unverified psychometric properties, and publications rarely include validation evidence for these measurement tools. Given the critical nature of climate change, using validated instruments is essential to ensure that climate-related research leads to interventions and policies based on accurate measurements of climate knowledge.

1.3. Aims of the research

There is a need for concise, multidimensional, and reliable tools to assess climate knowledge, both locally—reflecting regional impacts of climate change—and universally, to enable comparisons across populations. Existing research, however, lacks such instruments. This article addresses this gap in two ways. First, we develop and validate a new local instrument using the province of Quebec (Canada) as a case study: the 12-item Multidimensional Assessment of Climate Knowledge specific to Quebec (MACK-12-QC). Second, we explore its broader applicability by proposing an alternative version, the MACK-12, in which Quebec-specific items are replaced with more general ones.

With over 9.1 million inhabitants, the province of Quebec represents more than one-fifth (21.9%) of Canada’s total population [46]. Quebec is the only Canadian province where French is both the official and majority language. About 80% of Quebecers speak French as their first language, while most other Canadians have English as their first official language spoken [47]. This linguistic difference profoundly shapes the province’s culture, education system, and public policies.

Quebec’s distinct character is also evident in public attitudes toward climate change. A 2016 nationwide study showed that among all Canadian provinces, Quebec had the highest percentage of residents who believed climate change is human-caused and showed the strongest support for carbon pricing through emissions trading [48]. More recently, Champagne St-Arnaud et al. [49] found that 85% of Quebecers consider fighting climate change urgent, while only 14% deny its existence—demonstrating high public concern about the climate crisis. However, self-assessed climate literacy levels remain modest. In the same study, 39% of respondents reported being unable to explain “carbon emissions,” while 21% could not explain “adaptation to climate change,” among other concepts. Although these findings suggest limited climate knowledge, no objective assessment of climate literacy has been conducted for this population—making Quebec an ideal case study for developing a valid and reliable tool that measures both global and local climate knowledge.

The article is organized into seven sections. Sections 2–4 describe three studies with large, representative Quebec samples, which were conducted to develop and validate the MACK-12-QC. Building on these results, Section 5 presents complementary analyses that led to a general version of the scale, suitable for comparisons across populations (MACK-12). Section 6 then revisits the study’s objectives, discusses implications for research and practice, and considers its strengths and limitations. Finally, Section 7 concludes the article and outlines directions for future research.

2.1. Materials and methods

2.1.5. Participants and data collection.

The questionnaire was administered through an online panel by Léger, a renowned Canadian market research and analytics company, between July 16, 2024, and July 26, 2024. In the instructions section of the questionnaire (see S2 Text), respondents were asked to answer based solely on their own knowledge, without consulting external sources such as the Internet or other people.

Two thousand respondents (n = 2,000) aged 18 years or older completed the survey. None had participated in the cognitive interviews. Sociodemographic questions—covering age, sex, first language, education, region of residence, and presence of children in the household—enabled the creation of post-stratification weights to ensure a representative sample of Quebec’s adult population. For optional questions used in data weighting, missing values were handled as follows: nine respondents (0.5%) who did not disclose the presence of children in their household were classified as living with children, six respondents (0.3%) who did not provide their education level were classified as having no university degree, and one respondent (0.1%) who did not indicate their first language was classified as a non-native French speaker. Table 1 presents the sample’s sociodemographic characteristics using unweighted values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Sociodemographic characteristics of the sample for each study.

https://doi.org/10.1371/journal.pclm.0000600.t001

2.2. Results

The mean accuracy across all 62 items (using dichotomous scoring) was 60.3% (SD = 17.9%), with individual item accuracy ranging from 9.0% to 89.0%. Participants performed better on true items (M = 71.2%, SD = 21.9%) than false items (M = 50.0%, SD = 19.8%). When asked about the reality of climate change, 92.1% of respondents identified it as true—75.2% selecting “definitely true” and 16.9% “probably true.” Only 4.5% selected “I don’t know,” while 3.5% identified it as false (1.3% “definitely false” and 2.2% “probably false”). To maintain population representativeness, we included all respondents in our analyses, regardless of their beliefs about climate change.

Four items had a correct response rate below 20% (items 2.2, 4.13, 4.14, and 5.4; see S1 Text), while 14 items had a correct response rate above 80% (items 1.8, 1.9, 2.7, 3.2, 3.3, 3.6, 3.13, 4.2, 4.5, 4.12, 5.2, 5.3, 5.8, and 5.10). Additionally, eleven items (1.3, 1.6, 1.10, 2.2, 2.6, 3.4, 3.11, 3.12, 4.13, 4.14, and 5.4) showed a discrimination index below 0.3. Since four of these items were already eliminated based on the accuracy criterion, we retained 37 candidate items for the short questionnaire.

The final scale retained twelve items that assessed diverse climate knowledge subdimensions (see Table 2 for the complete list of items and their psychometric properties). This short questionnaire comprised five true and seven false statements. All items demonstrated adequate corrected item-total correlation with other items in the reduced questionnaire (all r ≥ .219). The final questionnaire yielded a Cronbach’s alpha of .765. We found a strong positive correlation between respondents’ scores on the short scale and their scores on the complete 62-item climate knowledge assessment, r = .900, p < .001.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Items included in the MACK-12-QC, associated item-level statistics, and overall accuracy achieved in each study.

https://doi.org/10.1371/journal.pclm.0000600.t002

Table 3 shows the distribution of responses on the 5-point Likert scale, illustrating respondents’ confidence in their answers. Among the three items answered correctly by less than half of the sample, two exhibited low accuracy mainly because respondents selected ‘I don’t know.’ (items g and l; see Table 3). For these items, when excluding “I don’t know” responses, only 11% of respondents gave incorrect answers. The third low-performing item (item i) showed a different pattern, with a greater proportion of respondents providing incorrect answers. Even after excluding the 24% who reported not knowing the answer, more than one-third of respondents incorrectly endorsed the false statement that offsetting greenhouse gas emissions should be prioritized over reducing emissions at source. Some items elicited both higher confidence and accuracy—for five of the 12 items (a, b, e, j, and k), at least one-third of respondents were certain of their correct true/false answers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Distribution of responses on the MACK-12-QC according to the response scale (dichotomous or Likert) in Study 1.

https://doi.org/10.1371/journal.pclm.0000600.t003

Six respondents (0.3% of the total sample) who did not disclose their education level were excluded from analyses examining climate change knowledge by educational attainment. Mean (± SD) scores on the short questionnaire (out of 60) were 43.1 (± 6.3), 44.1 (± 6.9), and 46.1 (± 6.8) for respondents with primary/secondary, college, and university education, respectively. An ANOVA revealed a small but statistically significant main effect of education level, F(2, 2068) = 34.41, p < .001, η² = .032. Bonferroni-corrected pairwise comparisons showed that questionnaire scores increased significantly with each education level (all p ≤ .021).

2.3. Discussion

Starting with 62 items designed to measure climate knowledge, we selected 12 items to create the short Multidimensional Assessment of Climate Knowledge scale—Quebec version (MACK-12-QC). The scale demonstrates reliability through satisfactory internal consistency. Its validity is supported by two findings: first, the strong correlation between MACK-12-QC scores and scores from the complete item set; second, the positive relationship between MACK-12-QC scores and education levels—a pattern consistent with previous climate knowledge research [e.g., 20, 30].

More than half of respondents correctly answered items about the greenhouse effect, the evolution and causes of climate change, and climate change consequences (dimensions 1, 2, and 3). There was greater variation in response accuracy for items about individual actions’ impact, collective solutions, and climate science accessibility (dimensions 4, 5, and 6). For example, although over 75% of respondents recognized active transportation as a solution to the climate crisis, only slightly more than half knew that transportation is Quebec’s largest source of greenhouse gas emissions. This finding highlights the need to raise public awareness about how goods and passenger transport affect the climate.

While individuals generally understood the limitations of social media as a source of climate change information, they struggled to identify reliable sources. Only 42% correctly identified the Intergovernmental Panel on Climate Change (IPCC) as the most credible source of climate science information. The high number of “I don’t know” responses suggests that this low recognition stems from lack of awareness rather than distrust in the organization.

Fewer than half of respondents recognized that wheat production generates fewer greenhouse gases than beef production of equal weight, aligning with previous studies that show limited public awareness of the environmental impact of food [27,59]. Most respondents also failed to understand that reducing greenhouse gas emissions at the source is more effective than offsetting them after production. This misconception may be reinforced by the proliferation of companies offering carbon credit purchases to consumers, leading some to believe that offsetting emissions is as effective as preventing them.

3.1. Method

3.2. Results and discussion

Table 2 presents item-level statistics for each MACK-12-QC item and mean accuracy across all items. Overall accuracy showed strong consistency between Studies 1 and 2, with only a 0.8% mean difference. Individual item performance was also highly comparable across both studies, with the maximum difference for any single item being 5.7%. This consistency suggests that participants did not look up answers between their first and second questionnaire completions. Though one item slightly exceeded our 80% accuracy threshold, we retained it due to its minimal deviation (1.5% above threshold) and strong discriminating power.

As with Study 1, all 12 items of the short scale showed a high discrimination index (≥ .320) and adequate corrected item-total correlations with other items (all r ≥ .224). Cronbach’s alpha was .782, demonstrating adequate internal consistency [53]. A strong positive correlation emerged between the total scores on the MACK-12-QC from the first and second data collections, r = .814, p < .001, confirming strong test-retest reliability. These results from Study 2 further support the conclusion that the short multidimensional scale is a reliable tool for assessing climate knowledge.

4.1. Method

4.1.2. Measures and analysis.

We analyzed how scores on the MACK-12-QC correlated with measures of climate change denial, environmental concern, perceived urgency of climate action in Quebec, and individual climate-mitigation behaviors. Table 4 lists the items for each domain.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Distribution of climate-related beliefs, attitudes, and behaviors and their correlation with climate knowledge.

https://doi.org/10.1371/journal.pclm.0000600.t004

For items measuring climate change denial, environmental concern, and perceived urgency to act against climate change, response options ranged from 1 (fully agree) to 6 (fully disagree). Responses were reverse coded so that higher scores represented greater agreement. For items about specific climate footprint-reducing actions, response options were: 1) I am currently doing this (3 points), 2) I intend to do so within 1 year (2 points), 3) I have no intention of doing so (1 point), and 4) It is impossible in my situation (1 point). The scores from these 12 action-specific items were combined into a composite score. For all validation items (attitudes, beliefs, and behavior-related variables), participants could select “I don’t know” or “I prefer not to answer.” These responses were excluded from analyses, resulting in varying sample sizes across analyses (see Table 4).

We used Spearman’s rank-order correlation to examine the relationship between MACK-12-QC scores (out of 60) and all previously mentioned variables. Following Study 1’s approach, we conducted a three-level one-way ANOVA to assess whether climate knowledge varied by education level.

4.2. Results

Table 2 presents the statistics for each MACK-12-QC item and the mean accuracy across all items. Item accuracy ranged from 44.7% (SD = 49.7%) to 75.0% (SD = 43.3%). Overall accuracy was consistent with findings from Studies 1 and 2. Participants’ confidence levels in their responses were similar to those in Study 1 (see S1 Table). All items demonstrated strong discrimination (indices ≥ .418) based on overall MACK-12-QC accuracy. Most items showed satisfactory corrected item-total correlation (r ≥ .356), with one exception: the item regarding Quebec’s transportation sector as the largest greenhouse gas emitter (r = .104). The questionnaire achieved a Cronbach’s alpha of .755.

Table 4 presents the distribution of participants’ responses to attitudes, beliefs, and behavior-related variables, along with Spearman’s rank-order correlation coefficients between these variables and MACK-12-QC performance. The correlational analyses revealed several significant relationships: a strong negative correlation between climate change knowledge and denial that climate change is scientifically proven; a small-to-moderate positive association between knowledge and environmental concern; a moderate-to-strong positive relationship between knowledge scores and perceived urgency to address climate change in Quebec; and a moderate positive correlation between climate knowledge and participants’ reported climate-friendly actions, as measured by the composite climate footprint variable.

Mean (± SD) climate knowledge scores (out of 60) were 43.0 (± 6.4), 44.3 (± 6.8), and 45.8 (± 7.0) for respondents with primary/secondary, college, and university education, respectively. The analysis revealed a small effect of education level on climate knowledge, F(2, 2459) = 32.03, p < .001, η² = .025, with scores differing significantly between all education groups (all p < .001). Eight respondents (0.3% of the total sample) were excluded from this analysis because they chose not to disclose their education level.

4.3. Discussion

Psychometric analysis of the MACK-12-QC demonstrated adequate internal consistency, with Cronbach’s alpha values aligning with those from Studies 1 and 2. The positive association between education level and questionnaire performance provided additional evidence of construct validity. Further validation came from the observed relationships between MACK-12-QC scores and participants’ environmental attitudes, beliefs, and behaviors.

Climate change denial had the strongest correlation with climate knowledge scores. As expected from previous research demonstrating a link between climate knowledge and climate change skepticism [20,61], we found a negative relationship. This strong correlation, however, suggests a challenge in distinguishing between knowledge and beliefs. While respondents may acknowledge the scientific consensus on climate change, their lack of trust in this consensus may influence how they answer knowledge-based questions.

The moderate-strong positive relationship between climate knowledge and perceived urgency to act against climate change in Quebec aligns with Shi et al.’s [8] findings, which link understanding of climate change causes to acceptance of climate-friendly policies. Individuals with higher knowledge levels demonstrate greater awareness of both the climate crisis’s severity and the urgent need for mitigation actions.

Our findings revealed a moderate association between climate knowledge and respondents’ efforts to reduce their climate footprint. This aligns with several studies demonstrating links between knowledge and pro-climate behavior or behavioral intentions [5,27,30]. However, social desirability bias in self-reported pro-climate actions might have inflated this relationship [see 62].

Consistent with findings from Bostrom et al. [7] and Tobler et al. [20], concern about environmental problems correlated positively with climate change knowledge—though this correlation was the weakest among all tested variables related to attitudes, beliefs, and behaviors. This suggests that people may be highly concerned about the climate crisis (e.g., due to increasingly frequent extreme weather events worldwide) without necessarily possessing comprehensive knowledge about climate change’s various dimensions. The relationship might have been stronger had our question focused specifically on climate change concerns rather than environmental concerns in general.

Although the questionnaire showed adequate reliability and validity, one item—concerning transportation as Quebec’s largest greenhouse gas emitter—fell below the predetermined item-total correlation threshold (r ≥ .20) in Study 3. Further analysis showed this was the only item where respondents with university or college degrees scored lower (53% and 52%, respectively) than those with primary or secondary education (57%). This unexpected pattern may be explained by workers in the transportation sector (truck drivers, bus drivers, and mechanics) having greater awareness of their industry’s significant role in greenhouse gas emissions. Since these transportation careers typically do not require post-secondary education, this could explain why performance on this item shows only a small correlation with overall questionnaire scores—despite climate knowledge generally increasing with education level. We maintain that this item remains relevant for assessing climate change knowledge in Quebec’s population and should be retained in the questionnaire.

5.1. Method

5.2. Results

Table 5 presents the items included in the MACK-12 along with their psychometric properties. Item f from the Quebec-specific scale (Table 2) was replaced with the statement “By accelerating the melting of glaciers, climate change is reducing freshwater reserves essential for the survival of many populations” (see item f in Table 5). Similarly, Quebec-specific item h (Table 2) was replaced with the statement “To reduce their climate footprint, one must aim to minimize air travel as much as possible, among other things” (see item h in Table 5). With these modifications, the MACK-12 consists of 12 items in total, evenly split between six true and six false statements.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Items included in the MACK-12, associated item-level statistics, and overall accuracy achieved in Studies 1 and 2.

https://doi.org/10.1371/journal.pclm.0000600.t005

All items showed adequate corrected item–total correlations, with values of r ≥ .319 in Study 1 and r ≥ .292 in Study 2. Discrimination indices exceeded .30 in both datasets (see Table 5). Cronbach’s alpha coefficients were .783 in Study 1 and .803 in Study 2, indicating good internal consistency [54]. In Study 1, scores on the MACK-12 were strongly correlated with scores on the full 62-item scale (r = .914, p < .001). ANOVA results (excluding six respondents, 0.3%, who did not report education level) showed mean (± SD) scores of 43.7 (± 6.7), 44.9 (± 7.0), and 47.1 (± 7.0) for participants with primary/secondary, college, and university education, respectively. The analysis revealed a small but statistically significant main effect of education level, F(2, 2068) = 41.81, p < .001, η² = .039. Bonferroni-corrected pairwise comparisons indicated that total scores increased significantly at each successive education level (all p ≤ .007). These findings further support the scale’s construct validity, consistent with prior evidence that climate knowledge correlates positively with educational attainment [25].

Overall accuracy was highly consistent across Studies 1 and 2, with an average difference of only 0.8%. Total MACK-12 scores from the two data collections were strongly correlated (r = .844, p < .001), demonstrating robust test–retest reliability.

5.3. Discussion

These complementary analyses sought to develop a general climate knowledge assessment tool and to evaluate its psychometric properties. The resulting questionnaire, the MACK-12, builds on the original MACK-12-QC but replaces two Quebec-specific items with more general statements, allowing for broader applicability across populations. Overall, the findings indicate that the MACK-12 is a valid and reliable instrument for measuring climate knowledge. Owing to its general scope, the MACK-12 can readily be incorporated into large-scale investigations of the correlates of climate knowledge in diverse adult populations.

These findings offer guidance for adapting the MACK-12 to different regions. While some aspects of climate knowledge are universal (e.g., physical principles, accessibility of climate science), others—such as local consequences and key mitigation actions—are context-dependent. Researchers can replace general items with region-specific ones, focusing primarily on dimension 3 (consequences) and dimension 4 (individual carbon footprint), and potentially dimension 2 (evolution and causes), to better reflect local climate realities and enhance relevance for engagement in climate action.

To ensure that a region-specific adaptation of the MACK-12 is valid and reliable for a given population, researchers should rigorously evaluate its psychometric properties within their target sample. This process can be streamlined by following the procedures outlined in Studies 1–3. It is recommended to include more than 12 items during validation (e.g., four items per dimension instead of two) to allow flexibility in case some adapted items prove unsuitable for the target population. The comprehensive initial item pool (see S1 Text) can serve as a source for additional candidate items. Researchers should also seek expert feedback to verify item accuracy and maintain construct validity [52,53]. Finally, conducting cognitive interviews with a diverse and representative sample of respondents is necessary to establish face validity [52,54].

The resulting version of the scale should be administered to a large, representative sample of the target population, and item-level statistics should be examined. Specifically, the accuracy of each of the 12 selected items should fall within commonly accepted limits (> 20% and < 80%), discrimination indices should be at least .30, and corrected item–total correlations should be ≥ .20 [53–55]. To reduce response bias (e.g., acquiescence bias [63]), items should include both true and false statements. Reliability, particularly internal consistency, can be assessed using Cronbach’s alpha, with values ≥ 0.7 generally considered acceptable [54]. A second data collection approximately two weeks later allows for evaluation of test-retest reliability [53]. Finally, construct validity [53,54] can be further examined by assessing relationships between scale performance and education level [25], as well as other variables known to correlate with climate knowledge, such as attitudes, beliefs and behaviors [8,20,30,61].

6.1. Multidimensional climate knowledge scale

Across all studies, the MACK-12-QC and the MACK-12 demonstrated their effectiveness as tools for measuring climate change knowledge. The scales provided consistent estimates of participants’ knowledge levels in both test-retest measurements and, for the MACK-12-QC, in assessments of two distinct representative population samples. Additionally, the questionnaires showed satisfactory internal consistency throughout all data collections.

While the questionnaires measure six distinct dimensions, each item correlated positively with the combined score of the other 11 items. The scores also showed strong correlation with the initial 62-item questionnaire. In line with previous research on climate knowledge, the short scales demonstrated positive associations with respondents’ education level [e.g., 20, 29]. Further validating the MACK-12-QC questionnaire, results showed expected correlations: negative relationships with climate change denial [e.g., 61] and positive correlations with environmental concern [e.g., 20], perceived urgency to act on climate change [e.g., 8], and engagement in climate-friendly behaviors [e.g., 30]. These consistent relationships confirm that the questionnaire effectively measures overall climate change knowledge. Moreover, based on Studies 1, 2, and 3, Quebec residents demonstrated an intermediate level of climate change knowledge, with a mean accuracy of 62% on the MACK-12-QC. This finding is consistent with climate change knowledge levels reported in other Western countries, including the United States [26], Switzerland [20], and Germany [29].

6.2. Implications for practice and research

To our knowledge, MACK-12-QC is the first validated tool to assess climate change knowledge among residents of the province of Quebec, Canada. In this part of the world where temperatures increase faster than the global average [64] and where key climate-friendly behaviors like public transit and active transportation remain uncommon [60], having an objective measure of citizens’ climate knowledge is crucial for promoting effective mitigation and adaptation practices.

The MACK-12-QC scale can help researchers and decision-makers identify knowledge gaps, misconceptions, and awareness levels among Quebecers. This information can support targeted communication strategies, policy design, and behavior-change campaigns to effectively engage the public in sustainable actions. For instance, more than two-thirds of respondents were not confident in identifying the IPCC as the most reliable source of climate change information. Communication campaigns could therefore aim to raise awareness of this organization’s vital role in advancing scientific knowledge, as well as its main conclusions. Guiding the public toward the most credible source of climate change information, and thus toward the scientific consensus, could help reduce the spread of climate misinformation [65,66].

The MACK-12-QC also revealed that respondents’ most common misconception was about carbon offsetting—specifically, whether compensating for greenhouse gas emissions is preferable to reducing them at the source. This finding aligns with research on compensatory green beliefs, which describes the mistaken idea that climate-friendly actions can offset the carbon emissions from environmentally harmful behaviors [67]. Educational interventions should thus aim to highlight the inaccuracy of this compensatory reasoning, which may help those who wish to minimize their carbon footprint make decisions more aligned with their goals [68].

The literature shows that individuals tend to misperceive the effectiveness of mitigation behaviors [2,32,69]. This aligns with the low accuracy observed for MACK-12-QC items about the carbon footprint of individual actions, including the item identifying the transportation sector as the largest source of greenhouse gas emissions in Quebec. Consistent with several studies showing gaps in public knowledge about the environmental impact of food products [e.g., 27, 59, 70], respondents also showed high uncertainty when comparing greenhouse gas emissions between wheat and beef production. Given the substantial contribution of the transportation and bio-food sectors to Quebec’s overall emissions [71,72], interventions aimed at enhancing public knowledge about the climate impact of transportation and dietary choices would offer a significant opportunity for climate action [69]. More specifically, behavior change campaigns could emphasize the relative impact of everyday actions (e.g., meat consumption, solo driving, recycling) to increase awareness and promote the adoption of the most impactful behaviors. Building on McNeill and Vaughn [73], we underscore the importance of educating the public not only about the physical mechanisms of climate change, but also about the individual and community-based actions needed to address it. Given the importance of personal relevance for learning, such educational interventions should be tailored to individuals’ local contexts and communities [18].

The MACK-12-QC can also serve as a formative tool to highlight and discuss common climate misconceptions with key stakeholders, including decision-makers, educators, and professionals in climate-affected sectors (e.g., agriculture, healthcare, engineering). Future research could adapt and validate the scale for Quebec’s children and adolescents, helping identify knowledge gaps and inform targeted educational initiatives.

Cross-national validation of the questionnaire would allow meaningful comparisons of climate change knowledge across populations and help explain regional differences [see 19]. To this end, we developed the MACK-12, a general climate knowledge assessment tool. Preliminary evidence indicates that it is a reliable instrument for assessing general climate knowledge in a global context.

Both scales allow rapid assessment of climate knowledge gaps and can be applied in interdisciplinary research, including climate communication, education, and policy analysis. They can be incorporated into large-scale observational studies or used in experiments to examine how climate knowledge influences the effectiveness of behavioral interventions, such as eco-labels, on consumer choices.

6.3. Strengths and limitations

This study’s key strength lies in our methodological approach. We built upon previously validated questionnaires [20,29] to identify core climate knowledge domains, while enhancing their scope with new items and an important additional dimension—collective solutions to climate change. Despite their brevity, the MACK-12-QC and the MACK-12 assess six distinct domains of climate change knowledge, making them more comprehensive than most existing tools, which typically focus on general and physical knowledge, the causes, and the consequences of climate change [19]. The scientific accuracy of the included items was validated by a climate change expert, and their clarity was confirmed through pre-testing with laypersons across different age groups and backgrounds. Moreover, unlike most studies on climate change knowledge, which rarely test the reliability and validity of their tools [19], we have demonstrated that both MACK-12-QC and MACK-12 are valid and reliable.

Unlike a simple dichotomous response scale, our Likert scale provided a more nuanced measure of respondents’ confidence in their answers. We minimized guessing by including an “I don’t know” option and maintaining an approximately equal number of true and false statements. Furthermore, using a true/false format, instead of an agree/disagree scale, ensured that we measured knowledge rather than attitudes [see 29]. Nevertheless, a small proportion of respondents denied the reality of climate change, and these participants may have exhibited atypical response patterns. Although we chose to retain them to preserve population representativeness, it is possible that some deliberately avoided the correct answer despite knowing it. This underscores the challenge of measuring climate knowledge independently of individual perceptions or beliefs [see also 25, 41]. A promising avenue for future research would be to explore the relationship between climate change knowledge and denial. Such a study would likely require oversampling deniers to allow meaningful comparisons, given their small representation in the general population.

The use of post-stratification weights allowed us to generate a representative portrait of climate knowledge across Quebec’s population. Our sample accurately reflected the population in terms of age, sex, first language, education, region of residence (including urban vs. rural areas within a region), and presence of children in the household. This constitutes a major strength of our investigation, as most research on climate change knowledge has relied on non-representative samples, limiting the generalizability of their findings [19]. Although the online survey allowed efficient data collection from a large, diverse sample, selection bias is possible, as participants needed literacy and Internet access [74,75]. Convenience sampling may have over-represented individuals interested in climate change [74,75]. The MACK-12-QC was developed specifically for Quebec, reflecting its unique demographics and region-specific climate impacts, and may not generalize to other provinces or countries.

While both MACK-12-QC and MACK-12 are valid measures of climate change knowledge, their compact design means they cannot comprehensively cover all climate knowledge domains (e.g., the full range of climate change consequences). However, this brevity is also a strength, as the scale can be easily integrated into broader studies of climate-related orientations (e.g., attitudes, behaviors) to effectively differentiate between individuals with varying levels of climate knowledge. As highlighted by Lubej et al. [19], there is a gap in the literature on large-scale, cross-cultural, and longitudinal studies assessing climate change knowledge. Given its broad coverage, the MACK-12 is particularly well suited for inclusion in such research. Longitudinal or experimental studies (e.g., involving educational interventions) would further clarify the directionality of observed relationships between climate change knowledge and climate-related attitudes, beliefs, and behaviors. Finally, the questionnaire was reviewed by a single climate science expert, and multiple independent expert reviews would have strengthened the validation of the items.