The Development and Validation of the Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia

Objectives To develop and validate Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia. Methods The OPAAT was modified from the Malaysian Osteoporosis Knowledge Tool and developed from an exploratory study on patients. Face and content validity was established by an expert panel. The OPAAT consists of 30 items, categorized into three domains. A higher score indicates higher knowledge level. English speaking non-osteoporotic postmenopausal women ≥50 years of age and pharmacists were included in the study. Results A total of 203 patients and 31 pharmacists were recruited. Factor analysis extracted three domains. Flesch reading ease was 59.2. The mean±SD accuracy rate was 0.60±0.22 (range: 0.26-0.94). The Cronbach’s α for each domain ranged from 0.286-0.748. All items were highly correlated (Spearman’s rho: 0.761-0.990, p<0.05), with no significant change in the overall test-retest scores, indicating that OPAAT has achieved stable reliability. Pharmacists had higher knowledge score than patients (80.9±8.7vs63.6±17.4, p<0.001), indicating that the OPAAT was able to discriminate between the knowledge levels of pharmacists and patients. Conclusion The OPAAT was found to be a valid and reliable instrument for assessing patient’s knowledge about osteoporosis and its prevention in Malaysia. The OPAAT can be used to identify individuals in need of osteoporosis educational intervention.

Knowledge of osteoporosis plays an important role in developing attitudes towards the disease which in turn impacts health care behaviors [12]. Patients' health beliefs are defined by attitudes, values and knowledge about health and health services. Although knowledge is not the only component to cause behavioural changes in patients, it is one of the essential components. Therefore patients should be equipped with the knowledge of the various prevention measures available to increase the likelihood of osteoporosis prevention and its fractures. This includes knowledge on physical activity, adequate calcium intake, adequate vitamin D intake, fall prevention and screening of osteoporosis [13].
Primary prevention of osteoporosis is directed at identifying high risk non-osteoporotic individuals, while secondary prevention of osteoporosis refers to the early detection of the disease and prevention of subsequent fragility fracture. Both primary and secondary prevention involve osteoporosis preventing behaviours [14]. Therefore, it is important to educate patients on the importance of screening and prevention, as studies have found that early detection of osteoporosis are the most cost-effective ways to reduce the number of hospital admittance due to osteoporotic fractures [15,16,17,18].
Although there are many methods to increase osteoporosis preventive behaviour such as physician reminders [16] and screening programs [19], patient education has been found to be an effective component in increasing knowledge and frequency of osteoporosis preventive behavior [20,21,22,23]. However, some studies suggest otherwise [24,25]. The differences in these studies' methodologies make it difficult to generalize results, as some studies used qualitative methods [26] whilst others used quantitative methods [23,24,25]. The variations in the results also suggest that knowledge is not the only component that affects behavioural change. Beliefs, attitudes and values may also be a barrier to implementing osteoporosis preventive efforts [12].
In Malaysia, the MOKT [11] and the Malay version of the OKT [8,27] have been validated. However, we wanted to assess the knowledge of osteoporosis and its prevention. Hence, these tools were unsuitable for use in our study as the MOKT assessed knowledge on osteoporosis and its treatment, while the OKT assessed osteoporosis knowledge by asking participants to rate the likelihood of getting osteoporosis based on the type of preventive measure taken [8,11]. Hence, the aim of our study was to develop and validate the English version of the Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia.

Method
This study was divided into 2 phases: the development of the OPAAT, and its validation.

Phase 1: The development of the Osteoporosis Prevention and Awareness Tool (OPAAT)
Despite Malay being the national language of Malaysia, postmenopausal women aged 50 years and above are more fluent in English as schooling was only conducted in the English language then. Hence, the OPAAT was developed in English, based on modifications from the MOKT [11] and findings from a qualitative study which examined the barriers and needs towards an osteoporosis screening and prevention service in Malaysia [28].
We took 10 out of the 50 items from the MOKT, as the other items were related to assessing knowledge on risk factors of osteoporosis, osteoporosis medication or misconceptions about osteoporosis. Six items were rephrased. For item 1, we added the word "fracture" in parenthesis to emphasize that the word "broken bones" means fracture (S1 Table). For item 5, "early on" was removed as patients were unaware that osteoporosis was asymptomatic and the phrase "early on" may confuse them [28]. As for item 13 and 16, we combined the original four questions to develop two questions; as "a loss of height" and "hunchback" were essentially assessing the same thing, and "joint pain" and "swelling of the fingers" were both referring to symptoms of osteoarthritis. Four items from the MOKT were used in its original format.
Results from the qualitative study found that patients, nurses, general practitioners, pharmacists and policy makers lacked knowledge in the following areas: screening and prevention of osteoporosis, and misconceptions of osteoporosis [28]. Therefore 22 new items were added. The final OPAAT consists of 30 items, and was divided into three domains: osteoporosis in general (domain A), consequences of untreated osteoporosis (domain B) and osteoporosis prevention (domain C).
Face and content validity of the OPAAT was established via consultation with an expert panel consisting of four pharmacists with many years of research and clinical experience. Comprehension of the questionnaire was tested on 10 postmenopausal women who understood English. This involved asking the patients for their opinions about the phrasing, format and content of the tool. The patients encountered no difficulty in answering the questionnaire. Hence, no further changes were made.

Phase 2: The validation of the Osteoporosis Prevention and Awareness Tool (OPAAT)
Design. This cross-sectional study was conducted at a primary care clinic of a tertiary hospital from October 2013 to January 2014.
Participants in the patient group. English speaking postmenopausal women aged 50 years and above, who had not been diagnosed with osteoporosis/osteopenia was included (This information was obtained from the patient's medical records). Participants who were feeling too unwell to participate in the study were excluded. The OPAAT was administered to the patient group at baseline and 2 weeks later to assess for reliability.
Participants in the professional group. To assess discriminative validity, pharmacists were recruited from the same tertiary hospital. Pharmacists were expected to have a higher knowledge of osteoporosis than patients. The OPAAT was administered to the pharmacists only once, as we wanted to assess the instrument's ability to discriminate between the knowledge scores of patients and healthcare professionals at baseline.
Sample size for the patient group. Sample size was calculated based on a 5:1 participant ratio for factor analysis [29]. Since the OPAAT had 30 items, the total number of participants needed was 150. Allowing for a 20% loss to follow up, the final number of participants required was 180.
Sample size for the professional group. The total number of pharmacists recruited was based on the number of pharmacists working in the hospital understudy. This group of participants was excluded from factor analysis.
Instruments used-Osteoporosis Prevention and Assessment tool (OPAAT)The OPAAT consist of 30 items with three domains: osteoporosis in general, consequence of untreated osteoporosis and osteoporosis preventive measure. A score of one was given for a correct response and zero for an incorrect or do not know response. The total score was converted into percentage ranging from 0-100. Each domain score was also analyzed.
Procedure. Patients were recruited at the waiting area outside the general practitioner's consultation room as the waiting time to see the general practitioner's appointment ranges from one to two hours. Utilising this period of waiting allowed the research team to collect data without extending the duration of the patient's visit to the hospital.
A 1:2 systematic random sampling method was used to recruit participants, as it was not possible for one researcher to recruit all the eligible participants at the clinic. The medical folders of eligible participants were labelled from 1-40, and a number was randomly drawn from a bag to determine the starting number at the start of each day. This was performed to ensure that sampling was random. Subsequently every 2 nd medical folder was selected for recruitment.
Additionally, 11 participants were also recruited using the "snowballing" method. As the project went on, participants began to refer their friends and family. Although this was a nonrandomized method of recruiting patients, only 11 (7.3%) participants were recruited in this manner.
The study was explained to the participants using an information sheet. Patient's written consent was obtained. Baseline demographic information such as patients' medical history, lifestyle and medication history was collected. Patients answered the questionnaire themselves. For those who experienced some difficulty in reading the questions, the researcher assisted them. The researcher then checked the questionnaire to ensure that all questions were answered. This took approximately 10 minutes. The OPAAT was administered again to the same group of patients after two weeks to assess for reliability. A duration of two weeks was selected for retest, as this time interval is generally accepted to be long enough for participants not to have remembered their original responses, and not long enough for their knowledge of the subject to have changed [30]. Patients were questioned if any significant changes or events occurred within the past two weeks, and all changes were documented.
Pharmacists' baseline information, work experience and education level were also collected using a baseline information form specific for pharmacist. The OPAAT was administered to the pharmacists only once at baseline.
Ethics approval. Written consent was obtained from all participants. This study was approved by the Medical Ethics Committee of the hospital (University Malaya Medical Centre) under study (ref no 920.27).
Data analysis. All data was entered into the IBM SPSS version 20 (IBM Corporation, Armonk, NY, USA). Flesch reading ease was calculated using Microsoft Office Word 2007 (Microsoft Corporation, Redmond, WA, USA). Non-parametric tests were used since data obtained were not normally distributed. A p-value <0.05 was considered as statistically significant.
Factor analysis. The construct validity of OPAAT was examined using exploratory factor analysis (EFA). Traditionally, factor analysis such as EFA and confirmatory factor analysis (CFA) can only be performed when data are of a continuous scale [31,32]. However, Bruin (2006) developed a new algorithm of EFA to account for categorical data. In this study, EFA was performed on three separate domains to explore the appropriateness of factor structure [33]. Factors with eigenvalues greater than one were considered as having significant contribution in explaining the overall model variation and were retained [34,35].
Flesch reading ease. Flesch reading index is a tool used for estimating the reading comprehension level necessary to understand a written document based on the average number of syllables per word and the average number of words per sentence. The Flesch reading ease was calculated using the formula below: Flesch reading ease = 206.835-(1.015x average sentence length)-(84.6 x average number of syllables per word) The Flesch reading score (which range from 0 to 100) indicates the level of difficulty in understanding the document. The lower the score, the greater the difficulty. An average document should have a score of 60-70 [36].
Accuracy rate. The accuracy rate is used to measure the difficulty of a question. It was calculated by the number of correct responses divided by the total number of responses. The higher the accuracy rate, the easier the question was. The optimal level should be 0.5 as a value of higher than 0.75 is deemed to be poor as the question may be too easy. Items with difficulty values between 0.3 and 0.7 are most effective. [37].
Corrected inter-item correlations are the correlations between each item and the total score from the questionnaire. All items should correlate with the total to be considered a reliable scale. A value of less than 0.3 shows a poor correlation and these items should be considered to be excluded. [40].
Test-retest for reliability. For test-retest, categorical data were analysed using the kappa measure of agreement and the Mc Nemar's test. In order to define inter-rater reliability, a kappa measure of agreement was calculated for each item. A kappa value of 0.5 represents moderate agreement, above 0.7 represents good agreement and above 0.8 represents very good agreement [41]. Mc Nemar's test was used to examine the test-retest reliability on the individual items. Continuous data of the individual items and total domain scores were analyzed using the Wilcoxon signed-rank test and Spearman's rho correlation coefficient. According to Cohen 1988, a value of 0.10-0.29 showed a low correlation, 0.30-0.49 moderate correlation and 0.50-1.00 high correlation [42].
Discriminative validity. To assess discriminative validity, the chi square test was used on categorical data of the individual items to detect the difference between the patient group and professional group. The Mann-Whitney U test was used for continuous data of the individual items and total domains score to compare if there was any significant difference between the patient and professional group.
Factors associated with knowledge. Linear multiple regression was used to identify factors associated with knowledge. It used to estimate the linear relationship between a dependent variable (knowledge score) and one or more independent variables (demographic variables).

Results
A total of 253 patients were approached, 19 declined. 234 participants were recruited (patients = 203, hospital pharmacists = 31), [patient response rate = 91.4%, pharmacists response rate = 100.0%]. Patients' demographic data are shown in Table 1. Pharmacists recruited worked in different areas of the pharmacy, with working experience ranging from 1-10 years.

Factor analysis
As shown in Table 2, for domain A, EFA yielded one factor with an eigenvalue of 4.04 which contributed to 81.0% of total variation. Ten items within this domain have factor loadings greater than 0.3 in Table 3, suggesting substantial contribution in explaining the overall variation. In Table 4, for domain B, EFA also produced only one factor with an eigenvalue greater of 1.9, which explained 87.3% of the total variation. All five questions within this domain had factor loadings greater than 0.3 as shown in Table 5. In Table 6, for domain C, EFA generated only one factor with an eigenvalue greater than one (4.4). This factor contributed to 69.4% of total variation. Table 7 showed that the factor loadings of all 12 items within this domain were

Factors associated with knowledge
Knowledge was higher in patients who completed their high school education, and patients who conducted fall prevention activities (R 2 = 0.208, F = 3.949, df = 18, p<0.001). These two factors explained 27.9% of the variances.

Comparison of the Osteoporosis Prevention And Awareness Tool (OPAAT) with other validated instruments
The OPAAT had a similar Flesch reading ease as the MOKT. The Cronbach's α if the OPAAT ranged from 0.27-0.75 which was similar to the MOKT, Osteoporosis and you, OKAT and FOOQ which ranged from 0.60-0.82. This shows that the psychometric properties of the OPAAT were similar to that of other validated instruments for measuring patients' knowledge (Table 11).

Discussion
The OPAAT performed satisfactorily in its psychometric properties and was able to discriminate between knowledge level of patients and pharmacists. This indicates that the English version of OPAAT is suitable to assess knowledge of postmenopausal women about osteoporosis prevention in Malaysia. EFA confirmed that there were three domains (osteoporosis in general, consequences of untreated osteoporosis and osteoporosis prevention) in the OPAAT to assess patient's knowledge on osteoporosis and its prevention. This provides support for the construct validity of our tool. To the best of our knowledge no other osteoporosis knowledge assessment tool has validated the construct of their tool via this method.
Flesch reading ease was at 59.2. This indicates the OPAAT can be understood by patients who have completed primary education. Since all of our participants have completed primary education, they were able to complete the OPAAT without any problems. The mean ± SD accuracy rate was 0.60±0.22 (range:0.26-0.94). Out of the 30 items, four items were considered difficult (accuracy rates <0.3) and five considered easy (accuracy rates >0.7). The optimum difficulty level would be 0.5. This indicates that the OPAAT was moderately easy for the participants to answer.  The construct of the tool was considered to be multi-dimensional and an overall Cronbach's α was unsuitable. We then analyzed the Cronbach's α by domain. All domains demonstrated good and acceptable internal reliability except the domain on the 'consequences of untreated osteoporosis' with a Cronbach α value of 0.286. This could be because there were only 5 items in this domain, and knowing the correct answer for one item may not necessarily mean that they knew the correct answer for the next item. However, increasing the number of items within the domain would have made the questionnaire too lengthy reducing the likelihood of completion. Corrected item-total correlations showed that all items measured the same main component which is satisfaction except items 13/30(43.3%). However all items were retained as removing any of the items did not improve the overall Cronbach's α significantly.
All 30 items performed satisfactorily at test-retest. Kappa measurement of agreement showed that 29/30 items (96.7%) were in very good agreement, and 1/30 items (3.3%) was in good agreement. As for the domains all domains performed satisfactorily except for the domain on "consequences of untreated osteoporosis." Patients may have forgotten the answer they selected at test (as they might have been guessing) as opposed to knowing the right answer. This led to a significant difference in this domain score as it had a small number of items. Although this limits how well this domain can measure the knowledge on the consequences of untreated osteoporosis, the guessing of answer reflects actual practice. Nonetheless, there was no significant difference in the overall scores. This indicates the OPAAT has achieved stable reliability. The domains and items had a high Spearman's rho correlation coefficient ranging from 0.760-0.990. They were all significantly correlated at p<0.05. Therefore, all items were retained.
Although pharmacists were expected to have a higher score than patients for all items, there were three items (items no. 13, 17 and 23) where no significant difference was found. This may be because more than 80.0% of both patients and pharmacists correctly answered items no. 13 and 23, indicating that their knowledge level for these items were high. As for item no. 17 which was pertaining to calcium intake, less than 60.6% of patients and pharmacists answered this item correctly. This concurs with our previous qualitative findings that found that both patients and pharmacists lacked knowledge in this area. [28]. Nonetheless, the overall score and all domain scores of the OPAAT showed a significant difference between the patient and pharmacist group. This indicates that the OPAAT has achieved discriminative validity.
Previous studies have found that the knowledge of osteoporosis in adult women aged 21-90 years in Europe [43,44,45], Canada [9], United States [5,23], Middle East [46], and Australia [6] was low. Conversely, women and men aged 16-79 years in Norway were knowledgeable about osteoporosis [47]. In Asia, the knowledge of osteoporosis ranged from low to moderate for women aged 19-90 in Brunei [48], Singapore [49] and Malaysia [27,50,51]. However, another study in Malaysia found that the knowledge of osteoporosis was moderate in women aged 49-84 [11]. In our study, patients' overall knowledge score was 63.6±17.4, which indicate The Development and Validation of the OPAAT in Malaysia that their knowledge level was moderate. Our results were similar to a previous study conducted in Malaysia which assessed knowledge on osteoporosis and its prevention [11]. This may be because both studies were conducted in the same setting. In addition, participants in both studies were mainly health seeking urban patients. However, we would like to highlight that the cohort of patients used in the Lai et al study was on patients who had osteoporosis, whilst our cohort were patients who were did not have osteoporosis. This shows that there was no difference in knowledge in patients with or without osteoporosis. Another tool, the Osteoporosis Knowledge Questionnaire (OKQ) assessed knowledge on osteoporosis risk factors, diagnosis, prevention and treatment in female population aged 60 and above scored 57.4% [10]. The OKQ score was similar to the OPAAT as they assessed non-osteoporotic postmenopausal population of a similar age group. Additionally, we would like to highlight the lack of knowledge on osteoporosis occurs in women who have not experienced a fracture, as well as those who had a previous fracture [52]. The different tools used to assess knowledge and the different cohorts in which the tool was administered to [11,27,50,51] made comparison between studies difficult. In addition, most studies did not report the use of validated tools to assess knowledge levels [23,24,25,43,44,45,47,48,50,51] Patients' knowledge was lowest on the domain on the 'consequences of untreated osteoporosis.' This concurs with findings from our qualitative research which indicates that there is a need to educate patients in this area [28]. Correspondingly, Osteoporosis and You noted a deficit in knowledge in the area of consequences of untreated osteoporosis [9]. These tools were developed mainly to assess the knowledge of domains of osteoporosis in general and treatment, the OPAAT was developed specifically to evaluate osteoporosis prevention.
In our study, factors with a positive correlation to the knowledge score includes patients with a secondary or higher education level, and patients who conducted fall prevention activities. Similarly, a Greek and Turkish study noted an association with knowledge and level of education [43,44,51]. Additionally, Khan et al's findings concurred with our study as they noted a significant association between knowledge and ethnicity [51]. Conversely, Ailinger et al stated neither education level, age nor the menopause status increase osteoporosis knowledge [5]. Patients who conduct fall preventive measure had more knowledge of osteoporosis. This further justifies the importance of a higher knowledge level about osteoporosis prevention to ensure its implementation. One of the limitations of our study was that convergent validity could not be performed. This was because during the period of our study, no such tool exists. The participants that we recruited also did not represent the ethnic distribution of Malaysia, but it represented the patients who sought treatment in our study site. Nonetheless, a large proportion of our patients had a monthly household income above $1553 (39.9%) which was representative of the married Malaysian household population income [53]. Seventy six percent of our participants were married. [53]. This shows that our participants income were representative of the Malaysian population.
Another limitation of our study was that we used mixed methods of administration. At baseline, majority of participants answered the OPPAT themselves, whilst a minority (2.5%) required assistance. At retest, the OPAAT was administered over the telephone as we wanted to optimize response rates. There is a possibility that participants may answer the items differently due to the mixed modes of administration [54]. However, this effect would be applicable to all participants, hence its effects on the validation process would be negated.

Conclusion
The English version of the OPAAT was found to be a reliable and valid instrument for assessing patient knowledge on osteoporosis and its prevention in Malaysia. OPAAT can subsequently be used to evaluate the effectiveness of the education efforts provided. Future studies, using Bahasa Malaysia and Mandarin versions of the questionnaire are required to assess patient knowledge for Malaysians that are not fluent in English.
Supporting Information S1