https://orcid.org/0000-0002-5925-2128
Figures
Abstract
Heat waves are extreme weather and climate events that threaten public health by increasing morbidity and mortality. To reduce the health effects of heat waves, it is necessary to increase the knowledge level of the public, conduct awareness and protection activities and monitor these activity outcomes. The present study aimed to develop and validate a Turkish language scale of heat wave knowledge, awareness, practice and behavior for Turkish nationality. After item generation and creating dimensions, content validity analysis was performed. To evaluate the validity and reliability of the knowledge construct, the difficulty index, discriminant index and Kuder Richardson 20 (KR20) were used. The validity and reliability of the awareness, practice and behavior constructs were evaluated with exploratory and confirmatory factor analyses, and Cronbach’s alpha was used. The 15 items had a good difficulty, discrimination index and KR20 in the knowledge construct. The 14 items were yielded in EFA; 13 items were retained in CFA, and Cronbach’s alpha values of 0.878, 0.768, 0.855, and 0.858 were obtained for total items, practice, awareness and behavior, respectively. Eventually, a Turkish language scale was developed that is reliable and valid for assessing heat wave knowledge, awareness, practice and behavior.
Citation: Sayili U, Siddikoglu E, Pirdal BZ, Uygur A, Toplu FS, Can G (2022) The heat wave knowledge, awareness, practice and behavior scale: Scale development, validation and reliability. PLoS ONE 17(12): e0279259. https://doi.org/10.1371/journal.pone.0279259
Editor: Supat Chupradit, Chiang Mai University, THAILAND
Received: September 26, 2022; Accepted: December 3, 2022; Published: December 21, 2022
Copyright: © 2022 Sayili et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Climate change has been described as the greatest global threat of the 21st century endangering people’s lives and well-being [1]. The Intergovernmental Panel on Climate Change (IPCC), in the 6th assessment report published in 2021, stated that the average temperature increase will vary between 1.4 and 4.4°C until 2100 according to different emission scenarios to the best estimates [2]. The Copernicus Climate Change Service released new data showing that the last seven years globally were the seven warmest on record by a clear margin [3]. Climate change has been expected to increase the intensity, frequency, duration and severity of extreme weather events such as heat waves [4]. Heat waves are extreme climatic events in which the weather is extremely hot for a prolonged time period [5]. Although it is generally understood to be a ‘prolonged period of excessive heat’, there is no common definition of a heat wave or temperature thresholds [6].
Heat waves are considered a public health problem, as they are associated with increased hospital emergency admission and heat-related morbidity and mortality [7, 8]. The health effects of heat waves range from sunburn and heat stress to kidney failure, heart attack and even death [6].
Over the past 30 years, studies have reported excess deaths due to heat waves. In Chicago in 1995, more than 700 deaths were reported due to the heat wave [9]. The devastating effects of the heat wave were greatest in 2003, and it is estimated that there were over 70,000 more than expected deaths across Europe, of which 15,000 were in France [10]. In 2010, Russia experienced a 44-day heat wave, also called a "mega heat wave", which is estimated to have caused approximately 11,000 excess deaths [11]. The World Health Organization (WHO) states that between 1998 and 2017, over 166,000 people lost their lives due to heat waves [12]. In 2021, it was reported that there were 434 heat dome deaths compared to 80 community death expectations (a 440% increase) in Vancouver, Canada [13]. A study from China found that 5% excess deaths caused by heat waves between 2006–2011 [14].
The Mediterranean region, including Türkiye, is considered one of the most sensitive regions to climate change, especially because of the increase in temperature, precipitation changes and seasonal temperature variability. Diffenbaugh et al. reported that extreme temperatures are 2 to 5 times more common in the Mediterranean region [15].
There are limited studies evaluating the health effects of heat waves in Türkiye. It is known that hospital emergency service admissions and total deaths increase during heat waves [8]. Between 2013 and 2017, three heat waves in Istanbul caused 419 excess deaths [16]; similarly, in another study, it was reported that 4,281 excess deaths occurred due to heat waves between 2004 and 2017 in Istanbul [17].
According to the health belief model, which evolved gradually in response to very practical public health concerns, an individual’s behavior and practices are related to understanding the severity of the risk and perceiving the benefit of taking action to reduce the threat [18]. The adverse health effects of heat waves can be predicted and prevented by certain public health protection measures, such as to increase the knowledge and awareness of the general population. Individuals reflect this information on their behaviors and practices during heat waves [5].
The public’s knowledge, awareness, behavior and practices toward risks are important factors for reducing the health impact of heat waves [19, 20]. Many conditions, such as age, gender, educational status, socioeconomic status, and chronic disease, can affect the public’s knowledge and practices [19, 21].
In addition, knowledge of heatwave protection and individual preparedness for heatwaves can be enhanced through specific government activities. Various countries have National Heat Health Action Plans (NHHAPs) aimed at improving knowledge and awareness of heat waves among the general population and particularly among vulnerable groups [21]. Some steps are being taken in this regard in Türkiye as well. Although the National Program for Reducing the Adverse Effects of Climate Change on Health was developed in Türkiye in 2015, it is not comprehensive regarding heat waves [22]. In addition, the name of the Ministry of Environment and Urbanization was changed to the Ministry of Environment, Urbanization and Climate Change in 2021. National heat-health plans should be established, the level of public knowledge, awareness, practice and behavior should be determined, and protection and prevention strategies should be acted upon rapidly.
To our knowledge, this study is the first developed scale for heat wave knowledge, awareness, practice and behavior in the Turkish language, and the second scale developed worldwide; only one scale in Malaysia was developed, which is based on knowledge, risk perception, attitude and practice questionnaires on heat waves [23].
The present study aimed to develop and validate a Turkish language scale for heat wave knowledge, awareness, practice and behavior for Turkish nationality.
Method
Study design and population
This methodological study was conducted between February 2022 and June 2022. The survey data were collected via the LimeSurvey platform. Participants were reached via WhatsApp and Facebook.
The study was conducted in accordance with the Declaration of Helsinki. The eligibility criteria were age ≥18 years, Turkish citizenship, literacy, and acceptance to participate in the study.
For the sample size, in the guidelines, the general suggestion is a ratio of approximately 5 to 10 subjects per item up to approximately 300 subjects. Because of the knowledge structure included 20 items, and the awareness, behavior and practice structures included 15 items, the sample size was determined as 300 participants. Data collection ended with 308 participants. (Fig 1).
Questionnaire development
Item generation and content validity assessment.
A literature review was carried out for item generation and creating constructs for scale by the authors. In this stage, with the literature review, 89 items were created for the 3 constructs of the pool scale (knowledge, awareness, behavior).
The items for the knowledge construct have 3 options (if it was a positive statement, 1: True, 0: False, 0: Not sure; if it was a negative, 1: False, 0: True, 0: Not sure). The scale for awareness and behavior was set on a 5-point Likert scale (if it was a positive statement, 1: strongly disagree 2: disagree, 3: unsure, 4: agree, 5: strongly agree; if it was a negative statement, 1: strongly agree 2: agree, 3: unsure, 4: disagree, 5: strongly disagree).
After the items were determined, the qualitative content validity was made with an expert panel method, and the pool scale was created; also, there was no suggestion to remove it from the item pool.
The invited three experts in the panel method were senior professors (environmental health and public health) or consultants (climate change) with academic titles in their research field. After the qualitative content validity with the panel method, the quantitative content validity evaluation phase has begun.
Quantitative content validity was analyzed with the content validity ratio (CVR) and content validity index (CVI). E-mail or telephone messages were sent to 30 experts for quantitative content validity. A total of 46.7% (n = 14) of the 30 experts whose opinions were consulted responded within 1 week. A reminder e-mail was sent to those who did not respond. In the second week, 6 (37.5%) experts made a return. A total of 20 expert feedback was received. These experts were specialists in medicine (public health, environmental health, emergency services, cardiologist, and family medicine) or environmental sciences (climatologist, environmental engineers). To determine the CVR, the experts were asked their opinions using a three-point scale (appropriate, appropriate but not necessary, unnecessary) for each item. However, 4 out of 20 experts chose it to be appropriate for similar questions (negatively and positively) but also for almost all (87–89 items). These four expert opinions were excluded. As a result, content validity analyses were conducted with the opinions of 16 experts.
The following formula was applied to determine the CVR.
In the formula, N is the total number of experts, and ne is the number of experts who selected the essential item.
The Lawshe technique was used to evaluate CVR. Based on the Lawshe technique, a minimum CVR of 0.500 and above was considered appropriate for an evaluation with 16 experts (p<0.05) [24].
The content validity index (CVI) was evaluated after CVR. For the CVI, a four-point scale was applied to the participants for each item, considering its simplicity, intelligibility and relevance to the subject. The CVI was calculated using the formula below:
The criterion for content validity of items was set. Items were considered adequate if there was >79% agreement, questionable if there was 70–79% agreement and unacceptable if there was <69% agreement [25].
After the content validity analyses for the pool scale, the initial scale to be applied to the participants was determined.
Construct validity.
For the knowledge construct, the difficulty index (p) and discrimination index (DI) were calculated. The total scores of all participants were calculated, and then the upper and lower 27% points of respondents were identified.
The p and DI were calculated as follows:
where “N” is the total number of participants in both the high and low groups. “H” and “L” are the number of correct responses in the high and low groups, respectively.
There are studies in the literature suggesting values of 30–70 or 20–90 for p [26, 27]. Although we initially thought of accepting the range of 30–70, there were items in the 70–80 score range for the risk factors that the researchers considered important. Therefore, we accepted that the 30–80 point range is appropriate. Items with p between 30–80 and DI > 0.24 were considered ’ideal’ [26].
Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were applied to demonstrate construct validity for the awareness, practice and behavior constructs. It was targeted to determine items and constructs with using EFA. After EFA, it was tested the confirmation of constructs and items. For EFA and CFA, the study sample was divided into two groups by simple randomization via SPSS. In EFA, the principal component method was used for factor extraction, and it is accepted that eigenvalues are greater than 1. Direct oblimin rotation was used in EFA. Items with factor loading<0.4 or factor loading difference <0.1 with loading to two factors or if only one/two items loading one factor were removed; others were retained. A Kaiser–Meyer–Olkin (KMO) value >0.6 and Bartlett sphericity test at p < 0.01 indicated that the data met the criteria for factor analysis [28].
The chi-square/df (CMIN/DF), comparative fit index (CFI), goodness of fit (GFI), incremental fit index (IFI), root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) were used in CFA to compute the scale model fitness.
Reliability.
For the knowledge construct, the Kuder-Richardson Formula 20 (KR-20) was used because it was coded as 0–1. For awareness, practice and behavior constructs, Cronbach’s alpha was used. A Cronbach’s alpha or KR-20 > 0.6 is considered acceptable [29]. Cristobal et al. suggested that corrected item-total correlation values lower than 0.30 are not acceptable [30].
Ethics
This study was approved by the Istanbul University-Cerrahpasa, Non-Invasive Clinical Research Ethics Committee (10.03.2022–334078; 2022/26). When the survey link was clicked, a page was displayed that introduced the study and included the informed consent form. Participants who clicked the "I agree to participate in the study" button were able to access the page containing the survey and scale.
Statistical analysis
The Statistical Package for the Social Sciences version 21.0 for Windows (IBM Corp., Armonk, NY, USA), JASP 0.14.1.0 and Microsoft Office Excel were used for data evaluation and analysis. Categorical variables are presented as frequencies (n) and percentages (%), and numeric variables are presented as the mean ± standard deviation and median (interquartile range (IQR)) values. For two response items, the discrimination index and difficulty index were calculated to evaluate validity, and the Kuder-Richardson 20 value was calculated to evaluate reliability. For 5-point Likert items, exploratory and confirmatory factor analysis and Cronbach’s alpha, Spearman’s correlation analysis were used to evaluate validity and reliability.
Results
Demographic characteristics
Table 1 represents the participants’ characteristics. A total of 308 people participated in the study, and most of the participants were female (60.7%). The mean participant’s age was 38.5±15.2. Most of the participants were 18–35 years old (42.5%), were married (55.7%), had children (52.6%), and had a university degree (80.2%). Most of the participants said, “Income equals expense” (49.2%); 26.9% of the participants said “income less than expenses”; 23.9% of the participants said “income more than expenses”. A total of 21.2% of the participants had a chronic disease. A total of 33.7% of the participants had air conditioning in their home. The most common cooling method used was opening windows or balconies (61.1%). The AFA and CFA study groups were similar in terms of basic characteristics (Table 1).
Content validity
The pool scale consisted of 89 items. A total of 44 items had CVR≥0.50 and CVI>%79. As a result, in the content validity analyses, 44 items that met the CVR≥0.50 and CVI>79% criteria remained on the scale. Of the 44 items, 20 belonged to the knowledge construct, and 24 belonged to the awareness and behavior construct.
Validity and reliability of knowledge construct
Of the 20 items in the knowledge construct, 5 items were removed due to inappropriate values in the difficulty index or discrimination index. (K3, K4, K14, K17, K19). The 20 items and 15 items in the knowledge construct had a good reliability index (pKR20: 0.629 and 0.629, respectively). As a result, the 15-item knowledge construct was evaluated as valid and reliable (Table 2).
Validity and reliability of awareness, practice and behavior construct
Exploratory factor analysis.
EFA was applied for the validity of awareness and behavior constructs. The KMO index was 0.881; Bartlett’s test of sphericity was found to be significant; and the total variance explained was 62.08% with a 3-factor construct (Table 3).
Although it was designed as an awareness and behavior construct at the beginning of the study, a 3-factor structure was found in the EFA. The items designed as the behavior construct were divided into two different constructs. It was evaluated that some of them predicted behavior and some of them predicted practice; therefore, they were not excluded from the scale and kept in the scale. A total of 10 items were removed due to low factor loading (<0.4), loading two factors with factor loading difference <0.1 or only 1–2 item with loading one factor. The first factor represents “practice” for 41.7% of variance; the second factor “awareness” explained 12.2% of variance, and the third factor “behavior” explained 8.2% of variance (Table 4).
Confirmatory Factor Analysis (CFA).
After exploratory factor analysis, confirmatory factor analysis was performed in a different sample. One item (B2) was removed for having a low standardized regression weight (0.300). After the item was removed, the scale was evaluated as appropriate according to CFA. All items have >0.5 standardize regression weights. The model has good fit indices such as CMIN/DF, CFI, IFI, RMSEA, SRMR and GFI. The CMIN/DF was 2.081, the CFI was 0.924, the IFI value was 0.925, the RMSEA value was 0.084, the SRMR value was 0.066 and the GFI value was 0.886 in this study. Since the model fit values met the acceptable criteria, the model was considered appropriate. Table 5 and Fig 2 represent the results of confirmatory factor analysis results.
Reliability analysis.
The subscales and scales have good reliability results. Reliability analysis indicated Cronbach’s alpha values of 0.855, 0.768, 0.858 and 0.878 (awareness, practice, behavior and total items, respectively). All items had item-total correlation coefficients greater than 0.50 and corrected item-total correlation coefficients greater than 0.4 (Table 6). Table 7 represents the correlation of the constructs. The correlation coefficients between knowledge, awareness, practice and behavior were statistically significant.
Discussion
This study aimed to develop a knowledge, awareness, practice and behavior scale on heat waves that threaten public health and are increasingly common with global climate change.
This Turkish language scale on heat waves is the first valid and reliable scale in Türkiye and the second in the world. To our knowledge, only one in Malaysia was developed, which is on a knowledge, risk perception, attitude and practice questionnaire on heat waves. In this study, a valid and reliable scale was developed in the Turkish language for knowledge, awareness, practice and behavior about heat waves.
Many studies emphasize the need to increase knowledge for behavior and practice development [5, 19]. After the 2003 heat wave in France, it was found that the devastating health effects predicted in the 2006 heat wave decreased thanks to the increased awareness and policies implemented [31]. Kim et al. found that awareness was an important factor in health-protective behaviors to reduce the risk of heat waves [20]. Lefevre et al. stated that having heard recommendations on heat waves indicated having implemented more heat protection behaviors, and intending to implement more protection behaviors in future hot summers [32]. In our study, we found positively significant correlations between knowledge, awareness, practice and behavior. Many countries have developed national heat health plans and implemented heat wave warning systems. Increased knowledge and awareness are achieved with these implementations and programs. The effectiveness of these implementations and plans in reducing mortality is known [21, 33].
In general, it is recommended to begin with a pool of items that is three or four times as large as the final scale [34]. In this study, the 89 generated items were in the pool scale; the 44 items passed the content validity analysis, and the 28 items were retained in the final version of the scale.
The first construct of the scale was a knowledge construct that contains 20 items in the initial scale. There are studies in the literature suggesting values of 30–70 or 20–90 for the difficulty index (p) [26, 27]. In this study, items with p between 30 and 80 were considered ’ideal’. Because the items with a p of 70–80 were important to evaluate the knowledge of risky (vulnerable) groups such as children, elderly individuals and pregnant women. According to these criteria, 15 of 20 items were found to be appropriate. The KR-20 value was adequate for 15 item scales (KR-20:0.629). Similar to our results, Arsad et al. found that 16 of 20 items were appropriate in the heat wave knowledge construct [23].
In the EFA, the KMO value was 0.881, and Bartlett’s test of sphericity was significant. This indicates that sampling is adequate and that the model is acceptable. Although the awareness and behavior construct was planned, a three-component construct was found. This 3 component explained 62.1% of the total variance. It appears that the behavior construct’s items are split into two (behavior and practice). This splitting may be due to the characteristics of the Turkish language and the way of thinking in Turkish. Because “practice” and “behavior” have similar meanings in the Turkish language structure. In EFA, a total of 10 items were removed due to low factor loading (<0.4), loading two factors with factor loading difference <0.1 or one/two items with one factor loading. In addition, in EFA, one item (B2) showed a factor loading of 0.45, while the other 13 items were greater than 0.5. In the CFA, this item is removed because it has a low standardized regression weight (0.300). All other items have >0.5 standardize regression weights. The model has good fit indices such as CMIN/DF, CFI, IFI, RMSEA and GFI. The rule of thumb for (approximately) normed fit indices is that a value greater than 0.9, an RMSEA and SRMR less than 0.08 and CMIN/DF less than 3 indicates an acceptable fit [35]. Some studies suggested that RMSEA values in the range of 0.05 to 0.08 indicate fair fit and 0.08 to 0.10 indicate mediocre fit [36]. The CMIN/DF less than 3 is acceptable, and it is 2.081 in the present study. For CFI and IFI values greater than 0.9 are acceptable. The CFI is 0.924 and the IFI value is 0.925 in this study, which is acceptable since it is greater than 0.9. The SRMR value was 0.066, which is acceptable. The RMSEA value was 0.084 and GFI value was 0.886 in this study, and they were close to recommended values. As a result, the model is good, with 3 components and 13 items.
Cronbach’s alpha values of 0.855, 0.768, 0.858 and 0.878 (awareness, practice, behavior and 13 items, respectively) and all item-total correlation coefficients were greater than 0.50, indicating good reliability results in this study. In addition, the corrected item-total correlation coefficients were greater than 0.40, which is acceptable. The Cronbach alpha value, which is often used for internal consistency reliability, is between 0.60 and 0.70 are considered “acceptable in exploratory research,” values between 0.70 and 0.90 range from “satisfactory to good” [37]. Arsad et al. showed a cronbach’s alpha value varying from 0.78 to 0.84 and an item-total correlation coefficient varying from 0.30 to 0.60 in their scale on risk perception, attitude and practice for heat waves [23].
Lenzholner et al. reported that a general lack of awareness about urban climate adaptations. In addition, it is reported that media campaigns and good practice projects seem most effective in raising awareness [38]. Individual disaster preparedness requires appropriate knowledge and awareness how to behave and to practice for protect oneself [5]. In our study, it was found that knowledge, awareness, behavior and practice showed a statistically significant positive correlation with each other. Li et al. reported that a significant correlation between heat wave knowledge, awareness and practice [19]. Hajat et al provided a list for the evidence of the most commonly provided heat protection advice and advice on optimal clinical and public health practice that is expected to reduce heat-related health problems [39].
Eventually, this scale is a valid and reliable scale for evaluating knowledge, awareness, practice and behavior for heat waves. Although this scale was developed in Turkish, it actually sheds light on an important worldwide problem, and it can be applied in many different countries and cultures by making adaptation studies. As mentioned earlier, another scale developed similarly is in Malaysia [23]. Both two scales have knowledge and practice constructs, and awareness/Risk perception and attitude/behavior construct can be considered similar, as well as their question content.
A few limitations should be considered for this study. First, data were collected via an online survey, and the survey was distributed with WhatsApp and Facebook. Therefore, generalizability is limited. Although the results of this study confirm the psychometric properties of the scale, similar studies in Turkish and other communities are needed for generalizability. Second, all data were collected using self-reported measures. Third, a similar scale could not be applied, and convergent validity was not assessed. Because this was the first study in the Turkish language, and one of the limited studies in the world. This limitation can be actually a strength of the study. This study sheds light on future research.
Conclusion
This study is the first developed scale for heat wave knowledge, awareness, practice and behavior in the Turkish language, and the second scale was developed worldwide (one in the Malay language). The heat wave knowledge, awareness, practice and behavior scale showed good validity and reliability results. The increasing frequency of heat waves is an important public health problem that increases mortality and morbidity. To protect against the negative effects of heat waves, it is necessary to develop different protection and prevention strategies. This newly developed scale can be applied in different cultures and countries by making adaptation researches and used by researchers to measure the level of knowledge, awareness, practice and behavior regarding heat waves. In addition, information acquired with this scale can guide both researchers and policy makers in determining and monitoring the protection and prevention strategies to be developed.
Supporting information
S2 File. The final version of valid and reliable scale.
https://doi.org/10.1371/journal.pone.0279259.s002
(DOCX)
References
- 1. Campbell S, Remenyi TA, White CJ, Johnston FH. Heatwave and health impact research: A global review. Health & place. 2018;53:210–218. pmid:30189362
- 2.
Allan, R. P., Hawkins, E., Bellouin, N. and Collins, B. (2021) IPCC, 2021: Summary for Policymakers. In: Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press.
- 3.
European Commission. Copernicus https://climate.copernicus.eu/copernicus-globally-seven-hottest-years-record-were-last-seven] Accessed 18 July 2022.
- 4. Huang L, Yang Q, Li J, Chen J, He R, Zhang C, et al. Risk perception of heat waves and its spatial variation in Nanjing, China. International journal of biometeorology. 2018;62(5):783–794. pmid:29335771
- 5. van Loenhout JAF, Vanderplanken K, Kashibadze T, Giuashvili N, Gamkrelidze A, Siman-Tov M, et al. Heatwave-protective knowledge and behaviour among urban populations: a multi-country study in Tunisia, Georgia and Israel. BMC public health. 2021;21(1):1–12. pmid:33931063
- 6. Zuo J, Pullen S, Palmer J, Bennetts H, Chileshe N, Ma T. Impacts of heat waves and corresponding measures: a review. Journal of Cleaner Production. 2015;92:1–12.
- 7. Akompab DA, Bi P, Williams S, Grant J, Walker IA, Augoustinos M. Heat waves and climate change: Applying the health belief model to identify predictors of risk perception and adaptive behaviours in Adelaide, Australia. International journal of environmental research and public health. 2013;10(6):2164–2184. pmid:23759952
- 8. Oray NC, Oray D, Aksay E, Atilla R, Bayram B. The impact of a heat wave on mortality in the emergency department. Medicine. 2018;97(52). pmid:30593174
- 9. Semenza JC, Rubin CH, Falter KH, Selanikio JD, Flanders WD, Howe HL, et al. Heat-related deaths during the July 1995 heat wave in Chicago. New England journal of medicine. 1996;335(2):84–90. pmid:8649494
- 10. Robine J-M, Cheung SLK, Le Roy S, Van Oyen H, Griffiths C, Michel J-P, et al. Death toll exceeded 70,000 in Europe during the summer of 2003. Comptes rendus biologies. 2008;331(2):171–178. pmid:18241810
- 11. Shaposhnikov D, Revich B, Bellander T, Bedada GB, Bottai M, Kharkova T, et al. Mortality related to air pollution with the Moscow heat wave and wildfire of 2010. Epidemiology. 2014;25(3):359–364. pmid:24598414
- 12.
WHO. Heatwaves. [https://www.who.int/health-topics/heatwaves#tab=tab_1] Accessed 18 July 2022.
- 13. Henderson SB, McLean KE, Lee MJ, Kosatsky T. Analysis of community deaths during the catastrophic 2021 heat dome: Early evidence to inform the public health response during subsequent events in greater Vancouver, Canada. Environmental Epidemiology. 2022;6(1). pmid:35169667
- 14. Ma W, Zeng W, Zhou M, et al. The short-term effect of heat waves on mortality and its modifiers in China: an analysis from 66 communities. Environ Int. 2015;75:103–109. pmid:25461419
- 15. Diffenbaugh NS, Pal JS, Giorgi F, Gao X. Heat stress intensification in the Mediterranean climate change hotspot. Geophysical Research Letters. 2007;34(11).
- 16. Can G, Şahin Ü, Sayılı U, Dubé M, Kara B, Acar HC, et al. Excess mortality in Istanbul during extreme heat waves between 2013 and 2017. International journal of environmental research and public health. 2019;16(22):4348. pmid:31703402
- 17.
Acar HC. (2022). Heat Wave Impacts on Mortality and Defining Heat Waves in Istanbul. [Thesis on Medicine Speciality]. Istanbul University-Cerrahpasa, Cerrahpasa Faculty of Medicine, İstanbul.
- 18.
Glanz K, Rimer BK, Visnawath K. Health Behavior and Health Education: Theory, Research, and Practice. 4th ed. San Francisco,CA, USA; 2008.
- 19. Li J, Xu X, Ding G, Zhao Y, Zhao R, Xue F, et al. A cross-sectional study of heat wave-related knowledge, attitude, and practice among the public in the Licheng District of Jinan City, China. International journal of environmental research and public health. 2016;13(7):648. pmid:27367715
- 20. Kim M, Kim H, You M. The role of public awareness in health-protective behaviours to reduce heat wave risk. Meteorological Applications. 2014;21(4):867–872.
- 21. van Loenhout JAF, Vanderplanken K, de Almeida MM, Kashibadze T, Giuashvili N, Gamkrelidze A. Heatwave preparedness in urban Georgia: A street survey in three cities. Sustainable Cities and Society. 2021;70:102933.
- 22.
Republic of Turkey, Ministry of Health. İklim Değişikliğinin Sağlık Üzerine Olumsuz Etkilerinin Azaltılması Ulusal Programı ve Eylem Planı. [https://hsgm.saglik.gov.tr/depo/birimler/cevre-sagligi/2-ced/iklim-degisikligi/Iklim_Degisikligi_Eylem_Plani.pdf.] Accessed 18 July 2022
- 23. Arsad FS, Hod R, Ahmad N, Baharom M, Tangang F. The Malay-Version Knowledge, Risk Perception, Attitude and Practice Questionnaire on Heatwaves: Development and Construct Validation. International journal of environmental research and public health. 2022;19(4):2279. pmid:35206467
- 24. Ayre C, Scally AJ. Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Measurement and evaluation in counseling and development. 2014;47(1):79–86.
- 25. Hyrkäs K, Appelqvist-Schmidlechner K, Oksa L. Validating an instrument for clinical supervision using an expert panel. International Journal of nursing studies. 2003;40(6):619–625. pmid:12834927
- 26. Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. JPMA-Journal of the Pakistan Medical Association. 2012;62(2):142–147. pmid:22755376
- 27. Sharma LR. Analysis of difficulty index, discrimination index and distractor efficiency of multiple choice questions of speech sounds of English. International Research Journal of MMC. 2021;2(1):15–28.
- 28. Xiao X, Si Y, Mo Q, Liu Y, Li C, Zhao J, et al. Development and validation of the Chinese version of the Adult Epilepsy Self-Management Scale (C-ESMS) in western China. Epilepsy research. 2018;144:43–48. pmid:29775849
- 29. Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in science education. 2018;48(6):1273–1296.
- 30. Cristobal E, Flavián C and Guinalíu M (2007) Perceived e-service quality (PeSQ). Managing Service Quality: An International Journal 17, 317–340.
- 31. Fouillet A, Rey G, Wagner V, Laaidi K, Empereur-Bissonnet P, Le Tertre A, et al. Has the impact of heat waves on mortality changed in France since the European heat wave of summer 2003? A study of the 2006 heat wave. International journal of epidemiology. 2008;37(2):309–317. pmid:18194962
- 32. Lefevre CE, de Bruin WB, Taylor AL, Dessai S, Kovats S, Fischhoff B. Heat protection behaviors and positive affect about heat during the 2013 heat wave in the United Kingdom. Social Science & Medicine. 2015;128:282–289. pmid:25635375
- 33. Toloo G, FitzGerald G, Aitken P, Verrall K, Tong S. Evaluating the effectiveness of heat warning systems: systematic review of epidemiological evidence. International journal of public health. 2013;58(5):667–681. pmid:23564031
- 34.
DeVellis RF. Scale development: Theory and applications. 4th ed: Sage publications; 2016.
- 35. Sun J. Assessing goodness of fit in confirmatory factor analysis. Measurement and evaluation in counseling and development. 2005;37(4):240–256.
- 36. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychological Methods. 1996;1(2):130–149.
- 37. Hair JF, Risher JJ, Sarstedt M, Ringle CM. When to use and how to report the results of PLS-SEM. European business review. 2019;31(1):2–24.
- 38. Lenzholzer S, Carsjens G-J, Brown RD, et al. Awareness of urban climate adaptation strategies–an international overview. Urban Climate. 2020;34:100705.
- 39. Hajat S, O’Connor M, Kosatsky T. Health effects of hot weather: from awareness of risk factors to effective health protection. The Lancet. 2010;375(9717):856–863. pmid:20153519