http://orcid.org/0000-0002-3895-530X
Figures
Abstract
Background
To improve individuals’ participation in cardiovascular disease (CVD) screening, it is necessary to understand factors that influence their intention to undergo health checks. This study aimed to develop and validate an instrument that assess determinants that influence individuals’ intention to undergo CVD health checks.
Methods
The concepts and items were developed based on findings from our prior exploratory qualitative study on factors influencing individuals’ intention to undergo CVD health checks. Content validity of the questionnaire was assessed by a panel of six experts and the item-level content validity index (I-CVI) was determined. After pretesting the questionnaire was pilot tested to check reliability of the items. Exploratory factor analysis was used to test for dimensionality using a sample of 240 participants.
Results
The finalized questionnaire consists of 36 items, covering nine concepts. The I-CVI for all items was satisfactory with values ranging from 0.83 to 1.00. The exploratory factor analysis showed that the number of factors extracted was consistent with the theoretical concepts. Correlations values between items ranged from 0.30 to 0.85 and all the factor loadings were more than 0.40, indicating satisfactory structural validity. All concepts showed good internal consistency, Cronbach’s alpha values ranged 0.66–0.85.
Citation: Cheong AT, Chinna K, Khoo EM, Liew SM (2017) Determinants for cardiovascular disease health check questionnaire: A validation study. PLoS ONE 12(11): e0188259. https://doi.org/10.1371/journal.pone.0188259
Editor: Julie Redfern, George Institute for Global Health; Sydney Medical School, University of Sydney, AUSTRALIA
Received: July 30, 2017; Accepted: November 1, 2017; Published: November 16, 2017
Copyright: © 2017 Cheong et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: Financial support for this study was provided by University of Malaya Postgraduate Research Fund (PG049-2014A) via EMK. The funder had no role in study design, analysis and interpretation of this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Cardiovascular disease (CVD) is a major cause of death and health burden globally [1]. Low- and middle-income countries are the most affected [2, 3] where the prevalence and lack of awareness of cardiovascular (CV) risk factors remain high [4]. Thus, health checks are important and necessary for early detection of individuals at high CV risk to allow for timely intervention. However, the uptake rate of CVD health checks remains low with values ranging from 20% to 40% [5, 6].
To improve participation rates of health checks for prevention of CVD, it is crucial to understand factors that influence individuals’ intention to go for such checks. We addressed this by first exploring factors that influenced individuals’ decision-making to undergo health checks for CVD prevention in a previous qualitative study [7]. Following this, a questionnaire was developed to identify the factors that influenced individuals’ intention to undergo health checks for CVD prevention using a quantitative study on a population.
For an instrument to measure determinants for individuals’ intention to undergo CVD health checks, it needs to assess all factors (concepts) that are being measured [8]. A number of questionnaires have been used in previous studies to examine factors influencing individuals’ intention to participate in health checks using the health belief model [9–12] or the theory of planned behaviour [13,14]. However, the items used were not grounded to local context and the concepts measured did not include all determinants found in our qualitative study [7]. Thus, we aimed to develop and validate a questionnaire that assessed all determinants that could influence individuals’ intention to undergo CVD health-checks.
Methods
The development of this questionnaire involved two steps. First the concepts and items for the intended measures were developed. Next the internal validity of the questionnaire was assessed.
Development of concepts and related items
The concepts that represent the determinants of individuals’ intention to undergo CVD health checks were developed based on findings from our previous qualitative study [7]. The qualitative study used a grounded theory approach to develop a conceptual framework to explain individuals’ decision-making process to undergo CVD health checks, taking into account local and cultural context [7]. Participants from three main ethnic groups in Malaysia, Malay, Chinese and Indian were recruited [7]. Nine concepts were identified for the study conceptual framework:
- a. Individuals’ readiness to know the results of CVD health checks
- b. Individuals’ readiness to handle outcomes following CVD health checks
- c. Individuals’ beliefs that the course of CVD can be changed for the better
- d. Perceptions of self being at risk of CVD
- e. Perception of benefits of CVD health checks
- f. Perception of drawbacks of CVD health checks
- g. Preferred method for CVD prevention
- h. External barriers
- i. Influence by significant others
The first two concepts (a & b) measured individuals’ readiness to face outcomes of CVD health checks. The next five concepts (c, d, e, f & g) measured individuals’ perceived relevance of health checks for CVD prevention. The next concept (h) measured external barriers towards CVD health checks while the last concept (i) measured influence by significant others (refer Fig 1). Table 1 describes operational definitions of these concepts.
Items in each concept were statements derived from themes that emerged and wording expressed by participants in the earlier qualitative study [7]. A Likert scale of 1 to 5 was used to indicate participant’s level of agreement with each item; 1 indicated “strongly disagree” and 5 “strongly agree”.
Internal validation of the instrument
The aim of internal validation is to establish if an instrument is valid and reliable to accurately measure what it is supposed to measure [8,15]. In this study, the process of validation consisted of content validation by an expert panel, pre and pilot testing of the questionnaire, assessment of correlation between items, structural validation using exploratory factor analysis and reliability analysis, both internal consistency and test-retest reliability (refer to Fig 2).
Content validation.
The aim of content validation is to examine whether the content of the developed instrument is relevant, important and adequately representing the underlying concept [8,15–17]. In this study content validation was performed by an expert panel that comprised of six content experts: four academic family physicians, one family physician from a public primary care clinic and a psychologist.
The first draft of the questionnaire consisted of 41 items that addressed the underlying concepts. The draft was emailed to members of the panel for their ratings and comments. Each member rated each item according to its relevance to the underlying concept using a 4-point scale: not relevant (score 1) to highly relevant (score 4). For validation purpose, scores of 3 or 4 were taken as indicative of relevance [17]. An item–level content validity index (I-CVI) was the proportion of experts in agreement with relevance of the contents. It was computed as follows:
Items with I-CVI of 0.78 or higher were considered to have good content validity [17,18]. Feedback and comments from the expert panel were used to further refine the items for better clarity and to provide representativeness of the concepts. In cases where items were modified, added or removed, the rating and review process was repeated.
Pretesting of questionnaire.
After content validation process, a face-to-face interview was carried out with participants to pretest the items. Pretesting of an instrument involved testing the questionnaire with participants to ensure it was easily understood and met the purpose of what it was intended to measure [19]. This process aimed to assess readability and clarity of the items in terms of consistency and appropriateness of interpretations from participants [19].
A total of six participants, a male and female each from three main ethnic groups in Malaysia i.e. Malay, Chinese and Indian, were recruited from the community to pretest the questionnaire. Participants from different ethnic groups were recruited to ensure the questionnaire was culturally sensitive and appropriate for use in local community with multi-ethnic background.
Six interview sessions were carried out. During each session, participants first completed the questionnaire on their own. Subsequently, the researcher (ATC) asked participants to report their thoughts on each item. Clarity of the items and ease of completing the questionnaire were assessed. Feedbacks from the participants were used to further improve readability and clarity of the questionnaire.
Pilot test.
A pilot study was conducted among 40 participants at a hypermarket to assess the internal consistencies of the items in the constructs. In the analysis, the correlation values between the items in the respective constructs were checked. In cases where the highest correlation of items with other items in the construct was less than 0.30, the items were modified and a second pilot test was carried out.
Study setting and population.
In this validation study participants were recruited from an urban hypermarket, a very large store selling groceries and household products. The hypermarket was selected purposively as it was surrounded by housing estates with mixed population of varied socioeconomic background. Participants were community dwelling adults who visited the hypermarket, were Malaysians, and aged 30 years or older. The 30 years or older age group was the group recommended for screening of CVD risk factors by the Malaysian Ministry of Health [20]. Those with known history of stroke or coronary heart disease or who could not understand the Malay language (national language of Malaysia) were excluded.
Structural validation (exploratory factor analysis).
Factor analysis is a statistical method used to check dimensionality among items [15]. For a good construct, the items must be adequately inter-correlated to be grouped under a single dimension or factor [8, 15]. In this study, factor analyses were performed on the items in each construct separately. The sample size for the exploratory factor analysis was estimated based on the rule of thumb of five participants for every item [21, 22]. Hence for 36 items in this questionnaire, a minimum sample size of 180 participants was needed.
Before factor analysis, the correlation among the items in the construct was checked. Ideally, the highest correlation of an item with at least one other item in the construct should be between 0.30 and 0.85. Values less than 0.30 indicate non convergence and values more than 0.85 indicate multicollineatity.
In extraction, the principle axis factoring method was used and rotation was performed using the promax oblique rotation method [15, 21]. Sampling adequacy was assessed using Kaiser-Meyer-Olkin (KMO) value, where a KMO value ≥ 0.60 was considered to be acceptable [15]. The Bartlett’s test of sphericity was used to test if the correlation between the items was sizable. In factor analysis, the factor loadings must be > 0.40 [15].
Reliability testing.
Reliability analysis is an assessment of the degree of consistency between multiple items in a construct [15]. In this study, internal consistency of the items was assessed using Cronbach’s alpha, where a value of ≥0.60 is considered reliable and acceptable [8, 15].
The test-retest reliability assesses the stability and reliability of an instrument over time [8, 15].
In this study, both unweighted and weighted kappa (both linear and quadratic weights) were tested to obtain a range of possible results. A kappa coefficient of ≥0.60 is desirable [8, 23].
Data analysis
All statistical analyses were performed using IBM SPSS Statistics software version 22, except for weighted kappa which was performed using a statistical calculator from the Statistical Computation Web Site [23]. Data were cleaned before analysis was conducted. The responses for the items that were worded oppositely were reverse coded. For example, a reverse coding was carried out for the item “I don’t want to think and know about CVD diseases at all”.
Ethical issues
Participation in the study was voluntary. The respondents were explained on the purpose of this study and were assured that the information would be kept strictly confidential and used for the purpose of this study only. Consent was obtained from willing respondents before the questionnaire was administered.
For this study ethical approval was obtained from the Medical Ethics Committee, University of Malaya Medical Centre (20145–274).
Results
Content validation
The first draft had 41 items with seven underlying concepts and one question asking participant’s intention to undergo health checks. Following feedback from the expert panel, six items were removed, and seven items were added, 16 items were refined and rephrased. The six items were removed because they were seen as not reflecting the underlying concept (I-CVI<0.78) or had strong similarities with other items. New items were added to widen the extent of representativeness of the content for the measured concepts. The revised questionnaire with 42 items was then sent to the same expert panel for re-evaluation. The I-CVI values for all items in the revised questionnaire ranged from 0.83 to 1.00.
In the pre-test, generally, the participants found the questionnaire to be easy and readable. Only minor revisions were made substituting more commonly used terms to improve clarity of the items.
Based on the results from the first pilot test, the initial concept of “perceived benefits and drawbacks of health check” were reclassified into two concepts i.e. “perceived benefits of health check” and “perceived drawbacks of health check”; the initial concept of “readiness to face outcome of health check” was also reclassified into two concepts “readiness to know the results of health checks” and “readiness to handle outcome following health check”. Seven items were deleted based on poor correlation (r <0.30) with other items. Two new items were added, and this revised version was sent to expert panel for re-evaluation. The I-CVI was satisfactory with index ranging from 0.83 to 1.00. The second version of the questionnaire consisted of 37 items. It was pilot tested again on another 40 participants. One item was deleted due to poor correlation, and the final version had 36 items with nine underlying concepts. The number of concepts and items in the initial and revised version are shown in S1 Appendix. Table 2 presents the concepts and respective items. The questionnaire was written in Malay (refer to S2 Appendix) and the translated version is shown in Table 2.
Factor analysis and internal consistency
For the final analysis, a total of 605 shoppers were approached and out of this 240 agreed to participate, giving a response rate of 39.7%. The 240 participants were recruited in a similar manner as in the pilot-test for factor analysis.
The median age of the participants was 45.5 years (IQR 17 years), mean age was 46.4 years (SD 11.1 years). The majority of participants were females (60%) and of Malay ethnicity (53.3%). Nearly all participants were aware of heart attack (97.8%) and stroke (98.7%). About one-half (55.0%) of the participants had regular health checks, at least once in two years. Table 3 shows the profiles of the participants.
Factor analyses were performed on the items in each of the nine constructs separately. For each construct, the highest correlation of each item with at least one other item was between 0.30 and 0.85, indicating that all items were correlated adequately (refer S3 Appendix). The Keiser-Meir-Olkin (KMO) values ranged from 0.58 to 0.79. According to Hair et al (2010), a KMO value of > 0.60 is acceptable [15]. The Bartlett test of sphericity was statistically significant for all constructs, indicating that there were sufficient correlation among the items to support the application of factor analysis. The total variance extracted for the constructs ranged from 35.1% to 59.4% and the Cronbach’s alpha values ranged from 0.66 to 0.85. The lowest factor loading value was 0.44.
In test-retest reliability analyses involving 88 participants, the weighted kappa values showed fair to substantial agreement (refer to S4 Appendix).
Discussion
Overall, the internal validity of the questionnaire was found to be acceptable.
This questionnaire has certain similarities and differences from existing questionnaires in medical literature. The concepts measured in this questionnaire; perception of CVD risk, possibility of a change in CVD outcomes and perceived benefits of health checks, are akin to concepts in the Health Belief Model [24]. The pperceived benefit and drawbacks of health checks constructs were similar to concept of attitude towards behaviour in theory of planned behaviour [25]. Significant others such as doctors, friends, spouse or relatives could influence individuals’ intention for health check participation. These social influences resemble the perceived social norms in theory of planned behaviour, which could motivate or demotivate an individual’s behavioural intention. The environmental factors are the external resources, such as financial support from the workplace and availability of nearby facilities or outreach programmes for health checks. Self-efficacy is an individual’s ability to find time, money or transport to undergo health checks [25]. In contrast with other questionnaires, this questionnaire is more comprehensive as it included additional concepts such as readiness to handle outcome and to face results of CVD health checks, which were raised in literature [11, 26–32].
To the best of our knowledge, this is the first questionnaire developed locally that assessed individuals’ intention to undergo CVD health checks. All other questionnaires were developed and used in Western countries such as United Kingdom and Netherlands. However, details of validation process of those questionnaires are lacking. A comparison of the current questionnaire with other similar questionnaires is provided in Table 4 [11, 14, 33].
Recommendations/implications
The questionnaire developed in this study has the potential to help assess an individual’s concern when deciding whether to go for CVD health checks. This information can be used to provide an appropriate advice based on his/her concerns to facilitate their decision-making process to undergo health checks. Tailored messages are perceived to be personally relevant and can increase one’s receptivity to the information presented [34–36]. For example, for opportunistic health check invitations, this questionnaire can be given to potential participants to explore their concerns and intentions about CVD health check, which in turn can be addressed during consultations. This can facilitate shared decision-making between the individual and the health care provider. The questionnaire can also be used by public using web or computer-based application. Based on the information provided by the respondents, tailored messages can be sent to them to facilitate decision-making for CVD health checks.
Strength and limitations
The strength of this questionnaire lies in the fact that the factors and items were grounded on an earlier qualitative study that explored understanding of individuals’ intention to undergo health checks, rather than a hypothesized theory. In this validation study, 240 participants were recruited. The sample size was adequate to address the factor analysis and internal consistency (minimum sample 180 for 36 items). However, due to the limitation of convenience sampling, generalisability of results is cautioned. Nevertheless, it was tested to be valid for use in a multi ethnic society which widens its applicability. More validation studies should be performed in different populations or subgroups.
Conclusion
The questionnaire “Determinants of individuals’ intention to undergo CVD health check” is found to be a valid instrument with good construct and internal consistency. It can be used to determine factors influencing individuals’ intention to undergo CVD health checks so that targeted approach can be taken to improve participation of health checks for prevention of CVD.
Supporting information
S1 Appendix. Comparison of concepts and number of items in the initial and revised version of the questionnaire used for factor analysis.
https://doi.org/10.1371/journal.pone.0188259.s001
(DOCX)
S2 Appendix. The “determinants of intention to undergo CVD health checks” questionnaire.
https://doi.org/10.1371/journal.pone.0188259.s002
(DOCX)
S3 Appendix. The mean, standard deviation of items and correlation matrix of each concept.
https://doi.org/10.1371/journal.pone.0188259.s003
(DOCX)
S4 Appendix. Summary of test-retest reliability for all items in the questionnaire.
https://doi.org/10.1371/journal.pone.0188259.s004
(DOCX)
References
- 1.
World Health Organisation. Projections of mortality and causes of death, 2015 and 2030 [Internet]. WHO. 2015 [cited 2015 Jul 28]. http://www.who.int/healthinfo/global_burden_disease/projections/en/index.html
- 2. Krishnamurthi RV, Feigin VL, Forouzanfar MH, Mensah GA, Connor M, Bennett DA, et al. Global and regional burden of first-ever ischaemic and haemorrhagic stroke during 1990–2010: findings from the Global Burden of Disease Study 2010. Lancet Glob Health. 2013;1:e259–281. pmid:25104492
- 3. Moran AE, Tzong KY, Forouzanfar MH, Rothy GA, Mensah GA, Ezzati M, et al. Variations in ischemic heart disease burden by age, country, and income: the Global Burden of Diseases, Injuries, and Risk Factors 2010 study. Glob Heart. 2014;9:91–9. pmid:24977114
- 4. Mills KT, Bundy JD, Kelly TN, Reed JE, Kearney PM, Reynolds K, et al. Global Disparities of Hypertension Prevalence and Control: A Systematic Analysis of Population-Based Studies From 90 Countries. Circulation. 2016;134:441–50. pmid:27502908
- 5.
Institute for Public Health (IPH). The National Health and Morbidity Survey 2011 (NHMS 2011). Vol. II: Non-Communicable Diseases. Ministry of Health, Malaysia; 2011.
- 6.
The Star online. Half found with health risks [Internet]. 2015 [cited 2015 Jul 13]. http://www.thestar.com.my/News/Nation/2015/02/27/Half-found-with-health-risks-Socso-screening-reveals-worrying-trend-among-employees/
- 7. Cheong AT, Khoo EM, Tong SF, Liew SM. To Check or Not to Check? A Qualitative Study on How the Public Decides on Health Checks for Cardiovascular Disease Prevention. PLoS ONE. 2016;11:e0159438. pmid:27415432
- 8.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 4th ed. Oxford: Oxford University Press, Inc; 2008.
- 9. Norman P. Predicting the uptake of health checks in general practice: invitation methods and patients’ health beliefs. Soc Sci Med. 1993;37:53–9. pmid:8332925
- 10. Norman P. Applying the health belief model to the prediction of attendance at health checks in general practice. Br J Clin Psychol. 1995;34 (Pt 3):461–70.
- 11. Norman P, Fitter M. Predicting attendance at health screening: Organizational factors and patients’ health beliefs. Counselling Psychology Quarterly. 1991;4:143–143.
- 12. Wilson S, Sisk RJ, Baldwin KA. Health beliefs of blue collar workers. Increasing self efficacy and removing barriers. AAOHN J. 1997;45:254–8. pmid:9171531
- 13. Armitage CJ, Norman P, Conner M. Can the Theory of Planned Behaviour mediate the effects of age, gender and multidimensional health locus of control? Br J Health Psychol. 2002;7:299–316. pmid:12614502
- 14. Norman P, Conner M. Predicting Health-Check Attendance Among Prior Attenders and Nonattenders: The Role of Prior Behavior in the Theory of Planned Behavior. Journal of Applied Social Psychology. 1996;26:1010–26.
- 15.
Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. Seventh Edition. United States of America: Pearson Prentice Hall; 2010.
- 16.
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. COSMIN checklist manual [Internet]. 2012 [cited 2015 Dec 11]. http://www.cosmin.nl/COSMIN%20checklist.html
- 17. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29:489–97. pmid:16977646
- 18. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459–67. pmid:17654487
- 19. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003;12:229–38. pmid:12769135
- 20.
Institute for Public Health (IPH). The National Health and Morbidity Survey (NHMS III) 2006. Ministry of Health, Malaysia; 2008.
- 21. Costello AB, Osborne JW. Best practices in exploratory factor analysis: For recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10:1–9.
- 22. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment. 1995;7:286–99.
- 23.
VassarStats. VassarStats: Statistical Computation Web Site. Kappa as a measure of concordance categorical sorting [Internet]. 2016 [cited 2017 Jan 31]. http://vassarstats.net/
- 24. Janz NK, Becker MH. The Health Belief Model: a decade later. Health Educ Q. 1984;11:1–47. pmid:6392204
- 25. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991;50:179–211.
- 26. Simpson WM, Johnston M, McEwan SR. Screening for risk factors for cardiovascular disease: a psychological perspective. Scott Med J. 1997;42:178–81. pmid:9507597
- 27. Shiloh S, Vinter M, Barak M. Correlates of health screening utilization: The roles of health beliefs and self-regulation motivation. Psychology & Health. 1997;12:301–17.
- 28. Pill R, Stott N. Invitation to attend a health check in a general practice setting: the views of a cohort of non-attenders. J R Coll Gen Pract. 1988;38:57–60. pmid:3204566
- 29. Ellis N, Gidlow C, Cowap L, Randall J, Iqbal Z, Kumar J. A qualitative investigation of non-response in NHS health checks. Arch Public Health. 2015;73:14. pmid:25825674
- 30. Jenkinson CE, Asprey A, Clark CE, Richards SH. Patients’ willingness to attend the NHS cardiovascular health checks in primary care: a qualitative interview study. BMC Family Practice. 2015;16:33. pmid:25879731
- 31. Sinclair A, Alexander HA. Using outreach to involve the hard-to-reach in a health check: what difference does it make? Public Health. 2012;126:87–95. pmid:22206983
- 32. Groenenberg I, Crone MR, van Dijk S, Gebhardt WA, Ben Meftah J, Middelkoop BJC, et al. “Check it out!” Decision-making of vulnerable groups about participation in a two-stage cardiometabolic health check: a qualitative study. Patient Educ Couns. 2015;98:234–44. pmid:25457176
- 33. Petter J, Reitsma-van Rooijen MM, Korevaar JC, Nielen MMJ. Willingness to participate in prevention programs for cardiometabolic diseases. BMC Public Health. 2015;15:44. pmid:25637105
- 34. Kreuter MW, Wray RJ. Tailored and targeted health communication: strategies for enhancing information relevance. Am J Health Behav. 2003;27 Suppl 3:S227–232.
- 35. Lustria MLA, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns. 2009;74:156–73.
- 36. Rimer BK, Kreuter MW. Advancing Tailored Health Communication: A Persuasion and Message Effects Perspective. Journal of Communication. 2006;56:S184–201.