The heat wave knowledge, awareness, practice and behavior scale: Scale development, validation and reliability

Heat waves are extreme weather and climate events that threaten public health by increasing morbidity and mortality. To reduce the health effects of heat waves, it is necessary to increase the knowledge level of the public, conduct awareness and protection activities and monitor these activity outcomes. The present study aimed to develop and validate a Turkish language scale of heat wave knowledge, awareness, practice and behavior for Turkish nationality. After item generation and creating dimensions, content validity analysis was performed. To evaluate the validity and reliability of the knowledge construct, the difficulty index, discriminant index and Kuder Richardson 20 (KR20) were used. The validity and reliability of the awareness, practice and behavior constructs were evaluated with exploratory and confirmatory factor analyses, and Cronbach’s alpha was used. The 15 items had a good difficulty, discrimination index and KR20 in the knowledge construct. The 14 items were yielded in EFA; 13 items were retained in CFA, and Cronbach’s alpha values of 0.878, 0.768, 0.855, and 0.858 were obtained for total items, practice, awareness and behavior, respectively. Eventually, a Turkish language scale was developed that is reliable and valid for assessing heat wave knowledge, awareness, practice and behavior.


Introduction
Climate change has been described as the greatest global threat of the 21st century endangering people's lives and well-being [1]. The Intergovernmental Panel on Climate Change (IPCC), in the 6th assessment report published in 2021, stated that the average temperature increase will vary between 1.4 and 4.4˚C until 2100 according to different emission scenarios to the best estimates [2]. The Copernicus Climate Change Service released new data showing that the last seven years globally were the seven warmest on record by a clear margin [3]. Climate change has been expected to increase the intensity, frequency, duration and severity of extreme weather events such as heat waves [4]. Heat waves are extreme climatic events in which the weather is extremely hot for a prolonged time period [5]. Although it is generally understood a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 only one scale in Malaysia was developed, which is based on knowledge, risk perception, attitude and practice questionnaires on heat waves [23].
The present study aimed to develop and validate a Turkish language scale for heat wave knowledge, awareness, practice and behavior for Turkish nationality.

Study design and population
This methodological study was conducted between February 2022 and June 2022. The survey data were collected via the LimeSurvey platform. Participants were reached via WhatsApp and Facebook.
The study was conducted in accordance with the Declaration of Helsinki. The eligibility criteria were age �18 years, Turkish citizenship, literacy, and acceptance to participate in the study.
For the sample size, in the guidelines, the general suggestion is a ratio of approximately 5 to 10 subjects per item up to approximately 300 subjects. Because of the knowledge structure included 20 items, and the awareness, behavior and practice structures included 15 items, the sample size was determined as 300 participants. Data collection ended with 308 participants. (Fig 1).

Questionnaire development
Item generation and content validity assessment. A literature review was carried out for item generation and creating constructs for scale by the authors. In this stage, with the literature review, 89 items were created for the 3 constructs of the pool scale (knowledge, awareness, behavior).
The items for the knowledge construct have 3 options (if it was a positive statement, 1: True, 0: False, 0: Not sure; if it was a negative, 1: False, 0: True, 0: Not sure). The scale for awareness and behavior was set on a 5-point Likert scale (if it was a positive statement, 1: strongly disagree 2: disagree, 3: unsure, 4: agree, 5: strongly agree; if it was a negative statement, 1: strongly agree 2: agree, 3: unsure, 4: disagree, 5: strongly disagree).
After the items were determined, the qualitative content validity was made with an expert panel method, and the pool scale was created; also, there was no suggestion to remove it from the item pool.
The invited three experts in the panel method were senior professors (environmental health and public health) or consultants (climate change) with academic titles in their research field. After the qualitative content validity with the panel method, the quantitative content validity evaluation phase has begun.
Quantitative content validity was analyzed with the content validity ratio (CVR) and content validity index (CVI). E-mail or telephone messages were sent to 30 experts for quantitative content validity. A total of 46.7% (n = 14) of the 30 experts whose opinions were consulted responded within 1 week. A reminder e-mail was sent to those who did not respond. In the second week, 6 (37.5%) experts made a return. A total of 20 expert feedback was received. These experts were specialists in medicine (public health, environmental health, emergency services, cardiologist, and family medicine) or environmental sciences (climatologist, environmental engineers). To determine the CVR, the experts were asked their opinions using a threepoint scale (appropriate, appropriate but not necessary, unnecessary) for each item. However, 4 out of 20 experts chose it to be appropriate for similar questions (negatively and positively) but also for almost all (87-89 items). These four expert opinions were excluded. As a result, content validity analyses were conducted with the opinions of 16 experts.

PLOS ONE
The heat wave knowledge, awareness, practice and behavior scale The following formula was applied to determine the CVR.
In the formula, N is the total number of experts, and n e is the number of experts who selected the essential item.
The Lawshe technique was used to evaluate CVR. Based on the Lawshe technique, a minimum CVR of 0.500 and above was considered appropriate for an evaluation with 16 experts (p<0.05) [24].
The content validity index (CVI) was evaluated after CVR. For the CVI, a four-point scale was applied to the participants for each item, considering its simplicity, intelligibility and relevance to the subject. The CVI was calculated using the formula below:

CVI ¼
The number of experts giving a rating of '3' or '4'

Total number of experts
The criterion for content validity of items was set. Items were considered adequate if there was >79% agreement, questionable if there was 70-79% agreement and unacceptable if there was <69% agreement [25].
After the content validity analyses for the pool scale, the initial scale to be applied to the participants was determined.
Construct validity. For the knowledge construct, the difficulty index (p) and discrimination index (DI) were calculated. The total scores of all participants were calculated, and then the upper and lower 27% points of respondents were identified.
The p and DI were calculated as follows: where "N" is the total number of participants in both the high and low groups. "H" and "L" are the number of correct responses in the high and low groups, respectively. There are studies in the literature suggesting values of 30-70 or 20-90 for p [26,27]. Although we initially thought of accepting the range of 30-70, there were items in the 70-80 score range for the risk factors that the researchers considered important. Therefore, we accepted that the 30-80 point range is appropriate. Items with p between 30-80 and DI > 0.24 were considered 'ideal' [26].
Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were applied to demonstrate construct validity for the awareness, practice and behavior constructs. It was targeted to determine items and constructs with using EFA. After EFA, it was tested the confirmation of constructs and items. For EFA and CFA, the study sample was divided into two groups by simple randomization via SPSS. In EFA, the principal component method was used for factor extraction, and it is accepted that eigenvalues are greater than 1. Direct oblimin rotation was used in EFA. Items with factor loading<0.4 or factor loading difference <0.1 with loading to two factors or if only one/two items loading one factor were removed; others were retained. A Kaiser-Meyer-Olkin (KMO) value >0. 6 and Bartlett sphericity test at p < 0.01 indicated that the data met the criteria for factor analysis [28].
The chi-square/df (CMIN/DF), comparative fit index (CFI), goodness of fit (GFI), incremental fit index (IFI), root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) were used in CFA to compute the scale model fitness.

Reliability.
For the knowledge construct, the Kuder-Richardson Formula 20 (KR-20) was used because it was coded as 0-1. For awareness, practice and behavior constructs, Cronbach's alpha was used. A Cronbach's alpha or KR-20 > 0.6 is considered acceptable [29]. Cristobal et al. suggested that corrected item-total correlation values lower than 0.30 are not acceptable [30].

Ethics
This study was approved by the Istanbul University-Cerrahpasa, Non-Invasive Clinical Research Ethics Committee (10.03.2022-334078; 2022/26). When the survey link was clicked, a page was displayed that introduced the study and included the informed consent form. Participants who clicked the "I agree to participate in the study" button were able to access the page containing the survey and scale.

Statistical analysis
The Statistical Package for the Social Sciences version 21.0 for Windows (IBM Corp., Armonk, NY, USA), JASP 0.14.1.0 and Microsoft Office Excel were used for data evaluation and analysis. Categorical variables are presented as frequencies (n) and percentages (%), and numeric variables are presented as the mean ± standard deviation and median (interquartile range (IQR)) values. For two response items, the discrimination index and difficulty index were calculated to evaluate validity, and the Kuder-Richardson 20 value was calculated to evaluate reliability. For 5-point Likert items, exploratory and confirmatory factor analysis and Cronbach's alpha, Spearman's correlation analysis were used to evaluate validity and reliability. Table 1 represents the participants' characteristics. A total of 308 people participated in the study, and most of the participants were female (60.7%). The mean participant's age was 38.5 ±15.2. Most of the participants were 18-35 years old (42.5%), were married (55.7%), had children (52.6%), and had a university degree (80.2%). Most of the participants said, "Income equals expense" (49.2%); 26.9% of the participants said "income less than expenses"; 23.9% of the participants said "income more than expenses". A total of 21.2% of the participants had a chronic disease. A total of 33.7% of the participants had air conditioning in their home. The most common cooling method used was opening windows or balconies (61.1%). The AFA and CFA study groups were similar in terms of basic characteristics (Table 1).

Content validity
The pool scale consisted of 89 items. A total of 44 items had CVR�0.50 and CVI>%79. As a result, in the content validity analyses, 44 items that met the CVR�0.50 and CVI>79% criteria remained on the scale. Of the 44 items, 20 belonged to the knowledge construct, and 24 belonged to the awareness and behavior construct.

Validity and reliability of knowledge construct
Of the 20 items in the knowledge construct, 5 items were removed due to inappropriate values in the difficulty index or discrimination index. (K3, K4, K14, K17, K19). The 20 items and 15 items in the knowledge construct had a good reliability index (pKR20: 0.629 and 0.629, respectively). As a result, the 15-item knowledge construct was evaluated as valid and reliable ( Table 2).

PLOS ONE
The heat wave knowledge, awareness, practice and behavior scale Validity and reliability of awareness, practice and behavior construct Exploratory factor analysis. EFA was applied for the validity of awareness and behavior constructs. The KMO index was 0.881; Bartlett's test of sphericity was found to be significant; and the total variance explained was 62.08% with a 3-factor construct (Table 3).
Although it was designed as an awareness and behavior construct at the beginning of the study, a 3-factor structure was found in the EFA. The items designed as the behavior construct were divided into two different constructs. It was evaluated that some of them predicted behavior and some of them predicted practice; therefore, they were not excluded from the scale and kept in the scale. A total of 10 items were removed due to low factor loading (<0.4), loading two factors with factor loading difference <0.1 or only 1-2 item with loading one factor. The first factor represents "practice" for 41.7% of variance; the second factor "awareness" explained 12.2% of variance, and the third factor "behavior" explained 8.2% of variance (Table 4).
Confirmatory Factor Analysis (CFA). After exploratory factor analysis, confirmatory factor analysis was performed in a different sample. One item (B2) was removed for having a

PLOS ONE
The heat wave knowledge, awareness, practice and behavior scale low standardized regression weight (0.300). After the item was removed, the scale was evaluated as appropriate according to CFA. All items have >0.5 standardize regression weights. The model has good fit indices such as CMIN/DF, CFI, IFI, RMSEA, SRMR and GFI. The CMIN/ DF was 2.081, the CFI was 0.924, the IFI value was 0.925, the RMSEA value was 0.084, the SRMR value was 0.066 and the GFI value was 0.886 in this study. Since the model fit values met the acceptable criteria, the model was considered appropriate. Table 5 and Fig 2 represent the results of confirmatory factor analysis results. Reliability analysis. The subscales and scales have good reliability results. Reliability analysis indicated Cronbach's alpha values of 0.855, 0.768, 0.858 and 0.878 (awareness, practice, behavior and total items, respectively). All items had item-total correlation coefficients greater than 0.50 and corrected item-total correlation coefficients greater than 0.4 (Table 6). Table 7 represents the correlation of the constructs. The correlation coefficients between knowledge, awareness, practice and behavior were statistically significant.

PLOS ONE
The heat wave knowledge, awareness, practice and behavior scale

PLOS ONE
The heat wave knowledge, awareness, practice and behavior scale

Discussion
This study aimed to develop a knowledge, awareness, practice and behavior scale on heat waves that threaten public health and are increasingly common with global climate change. This Turkish language scale on heat waves is the first valid and reliable scale in Türkiye and the second in the world. To our knowledge, only one in Malaysia was developed, which is on a knowledge, risk perception, attitude and practice questionnaire on heat waves. In this study, a valid and reliable scale was developed in the Turkish language for knowledge, awareness, practice and behavior about heat waves.
Many studies emphasize the need to increase knowledge for behavior and practice development [5,19]. After the 2003 heat wave in France, it was found that the devastating health effects predicted in the 2006 heat wave decreased thanks to the increased awareness and policies implemented [31]. Kim et al. found that awareness was an important factor in health-protective behaviors to reduce the risk of heat waves [20]. Lefevre et al. stated that having heard recommendations on heat waves indicated having implemented more heat protection behaviors, and intending to implement more protection behaviors in future hot summers [32]. In our study, we found positively significant correlations between knowledge, awareness, practice and behavior. Many countries have developed national heat health plans and implemented heat wave warning systems. Increased knowledge and awareness are achieved with these implementations and programs. The effectiveness of these implementations and plans in reducing mortality is known [21,33].
In general, it is recommended to begin with a pool of items that is three or four times as large as the final scale [34]. In this study, the 89 generated items were in the pool scale; the 44 items passed the content validity analysis, and the 28 items were retained in the final version of the scale.
The first construct of the scale was a knowledge construct that contains 20 items in the initial scale. There are studies in the literature suggesting values of 30-70 or 20-90 for the difficulty index (p) [26,27]. In this study, items with p between 30 and 80 were considered 'ideal'. Because the items with a p of 70-80 were important to evaluate the knowledge of risky (vulnerable) groups such as children, elderly individuals and pregnant women. According to these criteria, 15 of 20 items were found to be appropriate. The KR-20 value was adequate for 15 item scales (KR-20:0.629). Similar to our results, Arsad et al. found that 16 of 20 items were appropriate in the heat wave knowledge construct [23].
In the EFA, the KMO value was 0.881, and Bartlett's test of sphericity was significant. This indicates that sampling is adequate and that the model is acceptable. Although the awareness and behavior construct was planned, a three-component construct was found. This 3 component explained 62.1% of the total variance. It appears that the behavior construct's items are split into two (behavior and practice). This splitting may be due to the characteristics of the Turkish language and the way of thinking in Turkish. Because "practice" and "behavior" have similar meanings in the Turkish language structure. In EFA, a total of 10 items were removed due to low factor loading (<0.4), loading two factors with factor loading difference <0.1 or one/two items with one factor loading. In addition, in EFA, one item (B2) showed a factor loading of 0.45, while the other 13 items were greater than 0.5. In the CFA, this item is removed because it has a low standardized regression weight (0.300). All other items have >0.5 standardize regression weights. The model has good fit indices such as CMIN/DF, CFI, IFI, RMSEA and GFI. The rule of thumb for (approximately) normed fit indices is that a value greater than 0.9, an RMSEA and SRMR less than 0.08 and CMIN/DF less than 3 indicates an acceptable fit [35]. Some studies suggested that RMSEA values in the range of 0.05 to 0.08 indicate fair fit and 0.08 to 0.10 indicate mediocre fit [36]. The CMIN/DF less than 3 is acceptable, and it is 2.081 in the present study. For CFI and IFI values greater than 0.9 are acceptable. The CFI is 0.924 and the IFI value is 0.925 in this study, which is acceptable since it is greater than 0.9. The SRMR value was 0.066, which is acceptable. The RMSEA value was 0.084 and GFI value was 0.886 in this study, and they were close to recommended values. As a result, the model is good, with 3 components and 13 items.
Cronbach's alpha values of 0.855, 0.768, 0.858 and 0.878 (awareness, practice, behavior and 13 items, respectively) and all item-total correlation coefficients were greater than 0.50, indicating good reliability results in this study. In addition, the corrected item-total correlation coefficients were greater than 0.40, which is acceptable. The Cronbach alpha value, which is often used for internal consistency reliability, is between 0.60 and 0.70 are considered "acceptable in exploratory research," values between 0.70 and 0.90 range from "satisfactory to good" [37]. Arsad et al. showed a cronbach's alpha value varying from 0.78 to 0.84 and an item-total correlation coefficient varying from 0.30 to 0.60 in their scale on risk perception, attitude and practice for heat waves [23].
Lenzholner et al. reported that a general lack of awareness about urban climate adaptations. In addition, it is reported that media campaigns and good practice projects seem most effective in raising awareness [38]. Individual disaster preparedness requires appropriate knowledge and awareness how to behave and to practice for protect oneself [5]. In our study, it was found that knowledge, awareness, behavior and practice showed a statistically significant positive correlation with each other. Li et al. reported that a significant correlation between heat wave knowledge, awareness and practice [19]. Hajat et al provided a list for the evidence of the most commonly provided heat protection advice and advice on optimal clinical and public health practice that is expected to reduce heat-related health problems [39].
Eventually, this scale is a valid and reliable scale for evaluating knowledge, awareness, practice and behavior for heat waves. Although this scale was developed in Turkish, it actually sheds light on an important worldwide problem, and it can be applied in many different countries and cultures by making adaptation studies. As mentioned earlier, another scale developed similarly is in Malaysia [23]. Both two scales have knowledge and practice constructs, and awareness/Risk perception and attitude/behavior construct can be considered similar, as well as their question content.
A few limitations should be considered for this study. First, data were collected via an online survey, and the survey was distributed with WhatsApp and Facebook. Therefore, generalizability is limited. Although the results of this study confirm the psychometric properties of the scale, similar studies in Turkish and other communities are needed for generalizability. Second, all data were collected using self-reported measures. Third, a similar scale could not be applied, and convergent validity was not assessed. Because this was the first study in the Turkish language, and one of the limited studies in the world. This limitation can be actually a strength of the study. This study sheds light on future research.

Conclusion
This study is the first developed scale for heat wave knowledge, awareness, practice and behavior in the Turkish language, and the second scale was developed worldwide (one in the Malay language). The heat wave knowledge, awareness, practice and behavior scale showed good validity and reliability results. The increasing frequency of heat waves is an important public health problem that increases mortality and morbidity. To protect against the negative effects of heat waves, it is necessary to develop different protection and prevention strategies. This newly developed scale can be applied in different cultures and countries by making adaptation researches and used by researchers to measure the level of knowledge, awareness, practice and behavior regarding heat waves. In addition, information acquired with this scale can guide both researchers and policy makers in determining and monitoring the protection and prevention strategies to be developed.