A unique double tango: Construct validation and reliability analysis of risk perception, attitude and practice (RPAP) questionnaire on dengue infection

Introduction/background Dengue fever remains a public health threat despite being preventable. A solution to the constant problem of dengue infection will require active intervention and a paradigm shift. Assessing perceived risk and correlating it with the attitude and practice of the community will help in designing appropriate measures. However, possible instruments for these assessments come with limitations. Objective The aim is to develop and validate a new scoring-based questionnaire, using dual statistical approaches to measure risk perception, attitude, and practices (RPAP) related to dengue in the community. Methods The RPAP questionnaire was developed bilingually using the International Society for Pharmacoeconomics and Outcome Research (ISPOR) guidelines. Content analysis was reviewed scrupulously by four expert panels. The initial 35-item scale was tested among 253 Malaysian respondents recruited non-probabilistically via multiple online platforms. Two statistical methods were employed to measure the construct validity: Exploratory Factor Analysis (EFA) as part of the Classical Test Theory (CTT) measurement, while Rasch Measurement Analysis (Rasch) was performed for the Item Response Theory (IRT) measurement. All results were cross-validated with their counterpart to ensure stability. Confirmatory Factor Analysis (CFA) was used to obtain a model fit index. Results 29 questions were retained after the final analysis. Both EFA and Rasch analysis detect multidimensionality. Nine latent factors were extracted from EFA, while only eight factors remained in the final model following CFA: 1) perceived susceptibility; 2) perceived severity; 3) perceived barrier; 4) perceived benefit; 5) cues to action; 6) self-efficacy; 7) attitude; and 8) practice. All items had adequate factor loadings and showed good internal consistency. The final model after CFA achieved a good fit with an RMSEA value of 0.061, SRMR of 0.068, PNFI of 0.649, and GFI of 0.996. Conclusion The RPAP questionnaire contains 29 items and is a reliable and accurate psychometric instrument for measuring the risk perception of dengue fever, attitude, and practice of the community in dengue prevention. The Rasch measurement provides additional rigour to complement the CTT analysis. This RPAP questionnaire is suitable for use in studies related to dengue prevention in the community.


Introduction
Dengue is a vector-borne disease that remains a significant public health problem. The World Health Organisation (WHO) estimated that nearly 390 million dengue infections occur per year with increasing mortality rates worldwide [1]. In Malaysia, the number of reported dengue cases has risen steadily ever since the disease's discovery in 1902 [2,3]. Without effective and safe vaccines, prevention is heavily dependent on vector control targeting the Aedes sp. mosquito.
Entomological accounts of Aedes sp. explain its ability to effectively transmit dengue virus (DENV), from the ability of the egg to survive a lengthy drought [4,5] to the transovarial mode of DENV transmission [6,7]. This vector naturally breeds in human habitat, unlike vectors for other diseases such as malaria. Additionally, a local epidemiological study has elucidated the socio-ecological factors that contribute to ongoing disease transmission. Nonmodifying ecological factors, such as temperature, rainfall, and humidity, have been shown to affect disease transmission [8]. Increasing population and rapid urbanisation have rapidly and dramatically changed the ecological landscape. At the individual level, human activities on daily basis may indirectly create breeding grounds for the Aedes sp. Examples of unintended breeding promotion are improper disposal of solid waste, unplanned landfills, improper storage of water container, or clogged water flow [9,10]. In addition, low levels of participation in community-based dengue prevention programmes, such as Communication for Behavioural Impact (COMBI) [11], also contributed to worsening the problem [3].
Attitudes and practices towards dengue prevention need to be evaluated to ensure the efficacy of health education and information disseminated to the public. Moreover, an accurate measurement of the perceived risk of dengue infection is necessary to correlate with attitude and practice [12]. Risk perception is defined as a multidimensional assessment by an individual of the likelihood of having an unpleasant experience [13]. In order to avoid harm, a person must have a positive attitude and reasonable practices for prevention, as per the Health Belief Model (HBM) theory. This theory proposes six factors that shape a person's behaviour: perceived susceptibility, perceived vulnerability, perceived benefit, perceived barrier to practice, cues to take action, and self-efficacy.
If measuring either risk perception, attitude, or practice is already a challenge, it is even more difficult to measure all three elements together. Administering a valid and reliable psychometric scale is of the utmost necessity. Despite the availability of published local literature regarding dengue knowledge, attitude, and practice (KAP), a slight ambiguity was found during the assessment of the questionnaire or tool used for the measurement in this work. The majority reported the value of internal consistency of their tools, while the validity measures were less likely to be mentioned. For this reason, a sufficient and thoroughly validated questionnaire is necessary for future research.
This study aims to develop and validate a risk perception, attitude, and practice (RPAP questionnaire) for dengue infection that can be used in any community suffering from the disease. This study attempts to further ensure the reliability of the questionnaire through a unique dual approach, featuring both Classical Test Theory (CTT) and Item Response Theory (IRT).

The questionnaire development process
The RPAP questionnaire is a newly developed instrument to measure risk perception, attitude, and community practice related to dengue. It was designed as a self-administered scored survey, and bilingual in both English and Malay. Items were initially developed in English and the translation process followed guidelines by the International Society for Pharmacoeconomics and Outcome Research (ISPOR) [14]. The translation process involved two linguists, two Public Health Physicians, and two academicians, to ensure that the result was accurate, rigorous, and followed good practices in cultural adaption [15]. The steps used to develop the scale were based on DeVellis' protocol [16]: 1. Construct development: The development of the constructs was based on a previous study on a socio-ecological framework of prolonged dengue outbreak in the community. The work is intended to measure three attributes, namely, risk perception, attitude, and practice.

Item pool development:
The items were formulated in reference to the 'Standard Questionnaire On Risk Perception Of An Infectious Disease Outbreak' tool [17] and the application of established human behavioural theory, which is the Health Belief Model (HBM). Two postgraduate DrPH students and two academicians from Universiti Kebangsaan Malaysia (UKM) performed this process systematically. The preliminary set of questionnaires consisted of 43 items constructed during this step.

Pretesting:
The RPAP questionnaire was pretested on ten volunteers to check language understanding among the laymen before conducting the validation study. Some minor adjustments were made, especially regarding the complexity of the constructed sentences.

Study design and settings
The population-based cross-sectional study for validating the RPAP questionnaire was conducted over two weeks in December 2020 during the third wave of the COVID-19 pandemic in Malaysia. Due to the enforcement of a second Movement Control Order (MCO) at the national level, the country on a strict lockdown. The respondents were therefore recruited via an online open-source website, email blasts, and Whatsapp. Each of these modalities linked to a Google form. The choice to use a digital platform was based on two important points; 1) the risk of an in-person survey due to droplet transmission of COVID-19 virus, and 2) the large proportion of the Malaysian population with easy access to digital media [18]. The web-based questionnaire was set to only allow access after the subject had read and responded to the terms and conditions. The common sample size used in validation studies ranges between 5 and 10 respondents per item [19]. The targeted range of respondents in this study was therefore between 175 and 350 samples.

Statistical analysis
The RPAP questionnaire was subjected to double statistical modalities to measure its construct validity. The commonly used method, Classical Test Theory (CTT), was performed using JASP software, while Item Response Theory (IRT) via Rasch Measurement Model Analysis was performed using Bond & Fox Steps software. Both software are free and accessible to the public. Rasch analysis has several advantages in interpreting attitudinal scale [20], as it can simultaneously provide a comprehensive result for latent traits measured [21]. All 35 items were computed in the analysis. For the Exploratory Factor Analysis (EFA) measurement, factor extraction was performed using Principal Component Analysis (PCA), while the promax oblique rotation method was used. The total number of factors retained was determined using the Kaiser criterion with eigenvalues greater than one. Items were suppressed when the factor loading was less than 0.35. The Kaiser-Meyer-Olkin (KMO) index was employed for sample adequacy with Bartlett's test for sphericity valuation. Reliability analysis was performed, and the value of Cronbach's alpha and McDonald's omega were measured.

Background of respondents
Based on Table 1, the mean age of the respondents from the online survey was 33.6 (SD 6.2). The majority were female, ethnically Malay, held a degree in educational achievement, worked as a civil servant, and owned a landed terrace property. More than three-quarters of the respondents had no personal history of dengue infection. However, 51.8% of the respondents had relatives that had at one point contracted the dengue fever. Table 2 presents all the items in the newly developed RPAP questionnaire, with mean marks of the individual questions and the construct.

PLOS ONE
Validity of (RPAP) questionnaire Table 2. Details of the items, constructs and the corresponding questions' mean marks of each question, construct, and overall score (N = 253) based on the initial questionnaire development categories of risk perception, attitude, and practice.

No Code a Construct and Questions b Mean (± SD)
Risk Perception (Total mark: 94) c Objective: To measure the perceived threat of dengue infection.

Classical test theory using exploratory factor analysis (EFA)
The Kaiser-Meyer-Olkin (KMO) index for the RPAP-questionnaire was 0.879, whereas the pvalue for Bartlett's test of sphericity was significant at p < 0.001. The approximated chi-square and degree of freedom of the Bartlett's test were 3816 and 595, respectively. This indicates that the sample size employed in the validation study was adequate to run a factor analysis. Reliability analysis on all 35 items from all constructs yielded an adequate measure of item-total correlation (ITC), with the value of Cronbach's alpha between 0.415-0.908, and the value of McDonald's omega between 0.447-0.912. Two items were rejected via EFA, both from the attitude domain (E5 and E8) due to low factor loading (< 0.35). Some items were reassigned into new latent factors. Nine latent factors were detected from EFA despite the questionnaire's design involving three domains. Domain practice consisted of two factors (Factor 1 and Factor 2), domain attitude consists of two latent factors (Factor 3 and Factor 4), and domain risk perception was comprised of the remaining five latent factors. Table 3 shows the items and their latent factors. The mean score for all the items ranged from 2.18 (item D3) to 7.87 (item D4).

Confirmatory factor analysis (CFA)
The analysis was conducted using the same dataset from the EFA to provide a better visualisation of the model. Although 33 items were included in the measurement study, only 29 items remained: four items (E1, E2, F3, and F11) were removed during analysis. The path analysis, Fig 1, showed that one latent factor (Factor 9) had collapsed, and the items had merged with another construct. Therefore, the final model consisted of eight domains; this new model showed a good model fit. The χ 2 factor model was significant (p < 0.001), and the relative CMIN/DF is 1.96 (< 2.0), which indicates a good fit. According to standard practice, a combination of several fitness indices is more meaningful in interpreting the model fitness. From Table 4, the RMSEA value was 0.061, with an SRMR of 0.068, GFI of 0.996, CFI of 0.86, and PNFI of 0.649; these values achieved the desired target (a good fit). The calculated average

PLOS ONE
Validity of (RPAP) questionnaire

PLOS ONE
Validity of (RPAP) questionnaire

PLOS ONE
Validity of (RPAP) questionnaire variance extracted (AVE), maximum shared variance (MSV), and composite reliability were satisfactory. The standardised factor loading for all items and the correlation coefficient value for each latent factor are shown in Fig 1. Overall reliability analysis for the 29 items produced a Cronbach's alpha value of 0.824 and a McDonald's omega value of 0.841, indicating good consistency (cut off > 0.7).

Item response theory using Rasch measurement analysis
This approach was used to cross-validate the EFA result. As shown in Table 5, the initial 35-item RPAP questionnaire has multidimensionality (value of > 2.0 for the unexplained variance of the contrast). All 35 items' reliability and person reliability were good (> 0.7). A few items were found to be erratic but remained in the final model after a consensus review on the need for these items to provide or contribute to meaningful analysis. An assessment using the Likert scale suggested the merging of scales to achieve a better response (four instead of eight on the Likert scale), as evidenced by low Andrich Threshold logit, obtained 0.29 (1.4 < x < 5.0) and depicted by four-point arrows in the Probability Curve (Fig 2).
The overall percentage of raw variance explained by measures was 64.2% (> 60%). However, the residual contrast analysis showed elements of multidimensionality. In view of this violation of unidirectionality, the items were analysed according to the latent traits obtained from the EFA result. Almost all constructs showed a good and satisfactory result for InFit MNSQ, InFit Z standard, OutFit MNSQ, OutFit Z standard, item reliability, and point measure correlation, as demonstrated in Table 5. Hence, the dual approaches demonstrated similarly good psychometric properties.

Discussion
Researchers are widely adopting questionnaire-based methods in the field of behavioural study due to their practicality, low cost, and convenience for large data collection. The present study has demonstrated that the psychometric performance of RPAP is sufficient across both CTT and IRT analyses, supporting the conclusion that the 29-item scale covering risk perception, attitude, and practice towards dengue infection is reliable and psychometrically valid. A selfreported instrument must be able to function appropriately across heterogeneity of population, be straightforward to be administered and scored, and follow fundamental human behavioural theory. Validation testing using factor analysis (FA) has become more prominent due to the quantitative nature of the measurement that produces objective parameters for comparison [22][23][24].
The two types of FA function differently. EFA is usually used for factor reduction, combining items into their common factors based on the pattern-linearity of the factor loadings [25]. Conversely, CFA harmonises the model obtained from EFA by verifying the discriminant and convergent validity (construct validity) of the extracted factors and their respective items [16]. In this study, the EFA analysis was carried out using the Principal Component Analysis (PCA) as its extraction method; however, this differs with the rotation method. Few studies employing the extraction method have produced valid results that adequately measure the risk perception and KAP domains [22,23]. The oblique (promax) rotation was selected because of the theoretical assumption that the factors may have some correlation with each other rather than being mutually exclusive [26]. Conventional varimax rotation makes no correlated assumption among the factors, and would therefore have been less logical in this situation. Human behaviour (for example, motivation to participate in prevention programme) is very dynamic and shaped by many elements [27,28].

PLOS ONE
Validity of (RPAP) questionnaire The use of the Kaiser criterion to retain several factors led to the detection of nine latent factors extracted in EFA. This multidimensionality of the initial 35 items was similarly detected and proved via Rasch analysis. The Rasch method provided a different dimension reduction analysis by examining the residual components. A higher percentage of the first contrast of the residual (more than 2.0) might signify the existence of another dimension. Hence, this unique combination of measurements provided a good assessment of construct validity and reliability in complementing bidirectional relations [29].
The model derived from CFA was a good fit, as it suggested an acceptable discriminant and convergent validity. The use of multiple fitness indices has been widely adopted by scholars. The model χ 2 result was highly sensitive to a number of observed variables or sample size [30]. The CFA was able to demonstrate good internal consistencies for all measured domains in the RPAP questionnaire, as evidenced by the acceptable range of composite reliability that is comparable with the regularly calculated Cronbach's alpha. These internal consistencies were similar to the items' reliability, as measured by the Rasch analysis, for each domain. Nonetheless, this paper takes a new perspective when reporting other types of reliability alongside Cronbach's alpha, which is used extensively in about 90% of the literature [31]. Although the difference between the results is essentially minuscule, it is worth reporting for both coefficients, as the model used in this work could not be ascertained to have had a tau-equivalent reliability or to be a congeneric model [32][33][34].

Congruency with latent factors
It is worth describing the eight measured latent factors from the initial three domains. Six factors in this questionnaire are thematically congruent with the Health Belief Model (HBM). The HBM is a well-established human behavioural model developed by Godfery Hochbaum, Stephan Kegels, and Irwin Rosenstock during the 1950s. The theory is highly recognised as an effective method of creating behavioural changes. It has been continuously evaluated and rejuvenated over time; nevertheless, its core components are still applicable today [35]. Measuring risk perception is essential to understand the population in the context of dengue infection and transmission before embarking on health education efforts. This research has demonstrated that the components of perceived risk in this study were made of all four HBM attributes, namely, perceived susceptibility, perceived severity, perceived benefit, and perceived barriers; while the attitude components represent the remaining domains, which are cues to action and self-efficacy.
Perceived susceptibility is a subjective perception of the threat of acquiring an illness and may vary between individuals. In a place where dengue is widespread, people tend to be complacent and unable to optimise the prevention measures. This is in line with a study in India, where dengue is endemic and the risk of dengue transmission can be averted by strong preventive measures that are dependent on both perceived susceptibility [36] and a person's knowledge of the disease. Similarly, a strong association between dengue prevention behaviour and perceived high susceptibility has been observed in Malang, Indonesia [37].
Another latent factor is the perceived barrier to dengue infection. This barrier can be defined as the feeling that there are obstacles to performing recommended health behaviours. This inability to take action could stem from intrinsic factors, such as low motivation and the perception of a low risk of infection [38]. External causes such as low community engagement and participation can further contribute to this barrier [39,40]. These problems are not solely experienced in this country; similar issues have been experienced related to dengue elsewhere [41,42]. Perceived barriers should therefore be included in the domain of risk perception.
The latent factor of perceived severity refers to a person's feelings about the seriousness of contracting a disease, or of not seeking treatment. Specific to item D4, the respondents were assessed on treatment-seeking behaviour that indirectly reflects how likely they are to be tested for dengue. Meanwhile, item D3 challenges the idea of death due to dengue. This measure of severity was highlighted in the previous literature, demonstrating that people perceived dengue infection to be more severe than Zika infection, even though both diseases are transmitted by the same vector [43]. They are therefore extra careful to avoid bites from mosquitoes possibly carrying dengue. This component of HBM has a profound effect on the course of other communicable diseases in this country [44]. Perceived benefit was the last latent factor in the risk perception domain; this is the individual's perception of actions intended to reduce the threat of disease. Studies have shown that people are likely to adopt a behaviour change when their actions produce a positive impact [45]. Spending more money, as well as taking part in a dengue campaign (item D9 and D7), would certainly bring health advantages. Hence, the inclusion of these attributes in the RPAP questionnaire adds more value to the risk perception measurement.
The attitude domain is comprised of two latent factors: 1) cues to take action, and 2) selfefficacy. These are also components of the HBM. Cues to take action are motivations needed to trigger behaviours; self-efficacy is defined as the confidence in the ability to successfully perform a behaviour, during the decision-making process of whether to accept a recommended health action. This domain allows us to measure its association with risk perception. A person with a high perceived risk of dengue infection may have a positive attitude towards dengue prevention, as reflected by the HBM and as reported in various literature. For instance, an earlier local study found a positive correlation between high risk of infection and attitudes towards dengue prevention activity [46].
The final domain of the RPAP questionnaire, practice, includes two latent factors that measure good dengue prevention practice: 1) behaviours inside the house, and 2) behaviours outside the home or neighbourhood. Given Aedes sp. mosquitoes' ability to breed easily in a human environment, it is important to ensure that both living spaces and surrounding areas are unlikely to become the breeding source. This work requires total participation by the respondents to keep the place safe. This domain also included several questions specifically about community involvement programmes (i.e., communal work or gotong royong), which have been reported to halt dengue transmission in certain areas [47]. In Malaysia, this community participation has its own platform called COMBI (Communication for Behavioural Impact) that plays a pivotal role in dengue prevention, especially in terms of an eco-bio-social approach [48]. COMBI is an important measure because a sustainable dengue prevention program will depend on the continuous commitment of the population [49].
The newly developed RPAP questionnaire has three main strengths. First, the content of the new instrument is backed by scrupulous research, as evidenced by the comprehensive literature review and structured guidelines, gathering of expert opinions, ample time for cognitive debriefing interviews, and pretesting phase. Secondly, the concurrent use of dual statistical approaches helps to complement the analysis and generate especially powerful results. Finally, the fact that the questionnaire is bilingual helps to reduce possible information bias and comprehension issues among multi-ethnicity Malaysian respondents with different levels of national language proficiency [50,51]. However, the study's greatest limitation is the online recruitment of the sample respondents. The population that was available as samples were confined to groups with a computer or mobile device, internet connection, and the ability to access a web survey. This method excludes socioeconomically disadvantaged populations, likely those experiencing health inequality. In addition, the RPAP questionnaire was not subjected to a concurrent validity test. This lack was due to how few risk perception questionnaires there are currently available that are tailored to the local culture of this analysis; however, other domains are commonly assessed and reported. In that view, this study performed a double advanced statistical analysis to ensure its validity.

Conclusion
The RPAP questionnaire was developed in accordance with the Health Belief Model theory and established infectious disease guidelines; it has been demonstrated to be valid through extensive dual statistical approaches. The RPAP questionnaire is suitable for use in population-based studies and dengue prevention efforts. Ultimately, this questionnaire can quantify individuals' perceived risk of dengue infection, and this data can then used by authorities to strategically develop a sustainable public health intervention programme.