https://orcid.org/0000-0002-9568-0226
Figures
Abstract
As health systems transition to ICD-11, it is essential to gauge the readiness and improve existing transition efforts. Assessing the intention to use ICD-11 and factors influencing it is imperative to encourage the use of ICD-11 among the medical record officers (MROs) and assistant medical record officers (AMROs). This study aims to develop and validate a questionnaire on the factors influencing the intention to use ICD-11 among MROs and AMROs in the Ministry of Health, Malaysia. This study comprised a questionnaire development and validation involving 292 participants nationwide from Ministry of Health Malaysia facilities. The questionnaire was developed based on items adapted from the literature. Forward and backward English-Malay translation was done. Then, the questionnaire was examined for content validity, internal consistency reliability, construct validity, face validity, convergent validity, discriminant validity and confirmatory factor analyses. The final version of the questionnaire consists of eleven domains represented by 50 items. The content validity index and modified kappa were excellent for all domains. The Kaiser-Meyer-Olkin sampling adequacy value was appropriate, with a value of 0.790. The questionnaire also demonstrated good internal consistency reliability with Cronbach’s alpha values between 0.850 and 0.992. Confirmatory factor analysis showed a reasonable fit for this eleven-factor model. In conclusion, this questionnaire provides a reliable tool for investigating the intention to use ICD-11 among MROs and AMROs. Positive findings from the psychometric properties support the validity of the questionnaire. This instrument can potentially support personnel in charge of ICD codification, guide the ICD-11 transition at various levels and facilitate research on support dynamics among the MROs and AMROs.
Citation: Ooi ECW, Md Isa Z, Abdul Manaf MR, Ahmad Fuad AS, Ahmad A, Mustapa MN, et al. (2024) Development and validation of the intention to use the ICD-11 questionnaire in the Malaysian medical records context. PLoS ONE 19(9): e0308403. https://doi.org/10.1371/journal.pone.0308403
Editor: Jahanpour Alipour, Zahedan University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN
Received: February 15, 2024; Accepted: July 23, 2024; Published: September 6, 2024
Copyright: © 2024 Ooi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data contains potentially identifying MOH personnel information such as; age, gender, and years of service nationwide. Contact information for data access: Medical Research and Ethics Committee (MREC Secretariat), Ethics and Research Surveillance Sector, Level 2, Blok A1, National Institutes of Health (NIH) No 1 Jalan Setia Murni U13/52, Seksyen U13, Setia Alam, 40170, Shah Alam, Selangor, Malaysia Telephone: +603 – 3362 8398/8399/8404/8408 mrecsec@moh.gov.my.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The International Statistical Classification of Diseases and Related Health Problems (ICD) is a disease classification system that systematically records, interprets, and compares mortality and morbidity statistics collected at different periods and in various nations. ICD allows the codification of the diagnosis of illnesses and other health issues into alphanumeric code, which simplifies data storage, retrieval, and analysis [1]. Essentially, ICD use involves the reporting and codifying of the diagnoses documented by the treating doctors. ICD codification is the clinical coders’ assignment of specific ICD codes to the diagnoses reported by the treating physicians [2]. Current ICD-10 use is a manual process, compared to ICD-11 coding, which is a computerised process to ease the coding experience [3].
In 2019, the 72nd World Health Assembly adopted the ICD-11, which came into effect on Jan 1 2022 [4]. Since then, 64 countries have been in various stages of implementation [5]. For example, pilot implementation activities in Kuwait [6], Iran [7, 8], Rwanda [9] and China [10], whereas information feasibility studies and planning activities have already been done in the USA and Australia respectively [11, 12]. Generally, ICD-11 is a distinct and more efficient system built on formal ontology. It can be used in present IT infrastructures and is adaptable to other classifications and terminology services [13]. The process of using ICD-11 includes the ICD-11 Embedded Classification Tool (ICD-11 ECT). ICD-11 ECT is a search engine developed by the World Health Organization (WHO) to identify suitable ICD-11 codes [3].
From previous experiences, ICD transition has never been a straightforward venture [6, 7, 10]. Up to the latter stages of the change, it is rife with covert difficulties which require adaptations based on local challenges and issues. In other words, there is no one-size-fits-all strategy [7, 14]. The ICD transition’s impact on statistical reporting and reimbursement requires proper planning and firm support from the stakeholders [6]. According to Golpira et al. (2021), conducting feasibility studies in each country is imperative for the ICD-11 transition to succeed [7].
In 2023, the Ministry of Health (MOH), Malaysia has mandated the use of ICD-11 from 2024 onwards at the national level [15–17]. As systems commence transition to ICD-11 in 2024 per the directive, it is critical to learn the factors influencing the intention to use ICD-11 among users in Malaysia as soon as possible [17]. A newly introduced innovation which does not fit the users’ needs will be perceived negatively. Pain points will not be able to be identified and resolved by the policymakers, causing potential users to resist ICD-11 initially. Due to its mandatory implementation, users will be forced to use the system eventually because they do not have the choice [18]. Besides, new technological innovations, like ICD-11, may encounter obstacles due to the technological complexity and the users’ unique characteristics. This is because users transition from manual coding to a computerised process [3]. Previous studies have shown that users of a new technology will reject it if the innovation does not match their attitudes and expectations [19]. Worst, the full potential of ICD-11 may not be realised in Malaysia because users do not intend to use it in the first place. Subsequently, the quality of data collected from healthcare facilities will be negatively impacted. Low-quality data depict inaccurate information on disease trends and the complexity of diseases treated at various levels of facilities in a country [20].
Users have a crucial role in ensuring the successful adoption of ICD-11 since they possess essential knowledge about the factors influencing their adoption decisions [21]. At the Ministry of Health (MOH), Malaysia facilities, ICD usage from codification and reporting is mainly done by the medical record officers (MROs) and the assistant medical record officers (AMROs) [22]. The successful adoption of the ICD-11 depends on the medical records professionals’ intention to use the application [23]. Moreover, measuring the intention to use ICD-11 is more suitable than measuring the usage of the system due to the mandated use of applications like ICD-11 in Malaysia because focusing on the usage will be a hundred per cent [17]. Conversely, intention better reflects the beliefs and motivation of the users towards an innovation like ICD-11 in comparison to innovation usage [24].
With the increasing trend of countries transitioning to ICD-11, including Malaysia, we opine that there is a need for a tool to study and monitor the feasibility of ICD-11 [17]. Several tools have already been used in previous transition-related studies. However, they are mainly focused on assessing the user experience [6], ICD-11 training aspects [25], productivity [8], documentation [26], psychiatry [27], and implementation experience [10]. To the best of our knowledge, no available instruments focus on the intention to use ICD-11 for users in Malaysia. Therefore, an opportunity exists to design and validate an instrument to understand the intention to use ICD-11 and the factors influencing it in the Malaysian context. With that in mind, the current study aims to examine and validate the scale of a model comprising variables influencing the intention to use ICD-11 in the Malaysian setting.
Literature review
Several well-established theories focus on the psychological constructs influencing the intention to use innovation. For example, the Technology Acceptance Model (TAM) [28], Extended Technology Acceptance Model (TAM2) [29], Unified Theory of Acceptance and Use of Technology (UTAUT) [30], Theory of Planned Behaviour (TPB) [31], and Decomposed Theory of Planned Behaviour (DTPB) [32]. These theories have been adjusted and verified in a variety of situations and contexts [33–37].
TPB can better predict and explain behaviours in a compulsory setting in comparison to TAM [18]. At the same time, the extended versions of TAM, like TAM2 and UTAUT, are similar to TPB [33]. However, the decomposed model has superior explanatory and predictive capacity than TPB. This is because, Decomposed Theory of Planned Behaviour (DTPB) offers more profound insights and a more robust picture of the technology adoption phenomenon [18]. For example, DTPB has been widely applied in the domains of agriculture [34], education [35, 36], banking [37, 38] and healthcare technologies [23, 39]. Therefore, in the context of this study, with ICD-11 as the innovation in focus [13], we will apply the DTPB as a framework for questionnaire design and validation. The study hypothesises that the scale based on DTPB will exhibit a satisfactory factor structure.
Decomposed theory of planned behaviour (DTPB)
Based on TPB, the DTPB was introduced in 1995 by Taylor and Todd [32]. According to DTPB, an individual’s attitude (ATT), subjective norm (SN) and perceived behavioural control (PBC) contribute to their behavioural intention (INT) [31, 32]. In the DTPB, attitude is further decomposed into perceived ease of use (PEOU), perceived usefulness (PU) and compatibility (COM), subjective norm into interpersonal (II) and external influences (EI) and perceived behavioural control into facilitating conditions (FC) and self-efficacy (SE) [32].
Intention (INT) is the envisioned outcome that directs a person’s planned actions like using ICD-11. An MRO or AMRO’s intents reveal what could spur their actions in using ICD-11 in a particular way and the related motivations. As a result, intention should be anticipated to impact and influence performance to the degree that the MRO or AMRO possesses behavioural control [40]. Concerning the questionnaire design and DTPB, intention to use ICD-11 is influenced by ATT, SN and PBC [32].
In this study’s context, attitude (ATT) is the degree to which a person has a positive or negative assessment or appraisal of using ICD-11 [41]. In other words, a user’s convictions and characteristics define their attitude. On the use of Electronic Health Records, it has been shown that user’s positive attitude towards a newly introduced technology will improve the chances of the technology being used in the future [21].
Subjective norms (SN) are the social pressures the MROs and AMROs feel. In general, the pressures are from their surroundings to either use ICD-11 or not, as well as the influence that supervisors and colleagues may have on the person’s decision to use ICD-11 [31]. Prior research has demonstrated that respondents are more motivated to carry out the desired behaviour when these social groups significantly impact them [39]. For example, in the use of electronic medical records exchange among physicians, subjective norms are a critical factor influencing doctors’ inclinations to use information systems in their practice [39].
Perceived behavioural control (PBC) is defined as how easy or difficult ICD-11 is viewed to be performed by the MROs and AMROs [31]. PBC considers medical records professionals’ previous experiences and anticipated challenges affecting their confidence in utilising ICD-11 [40]. Innovation adoption among healthcare workers has shown that if the user has the resources and the appropriate guidance, the staff will likely have positive intentions to use the new system [42].
Decomposition of attitude
This study’s three constructs influence attitude: perceived usefulness, perceived ease of use, and compatibility. Perceived usefulness is defined as the degree to which the MROs and AMROs believe that ICD-11 will be able to improve their work performance [43]. Previous studies on innovation acceptance have shown that perceived usefulness predicts attitude toward technology [23, 44]. Specifically in the healthcare context involving healthcare consumers on EHRs, Mathai et al. (2022) have found that users’ attitudes toward a technological system like EHRs are positively impacted by their perceptions that these tools offer them definite benefits [21].
Another significant predictor of attitude toward technological innovation is perceived ease of use. Within the scope of this study, perceived ease of use is defined as the degree of medical records professionals’ belief that ICD-11 is easy to use [32]. Previous studies on using health technology systems have shown an empirical relationship between perceived ease of use and the staff’s attitude. Regardless of the level of education among healthcare professionals, if the technology is not easy to use, users will form an unfavourable attitude towards the technology [23, 32].
The compatibility construct defines the degree to which the MROs and AMROs opine that ICD-11 fulfils the current needs and fits their values [45]. As the agent of change, the MOH must be aware of the users’ needs about ICD-11. With this awareness, MOH can recommend improvements to fulfil the identified needs [46]. This is because adoption happens more quickly when the user believes ICD-11 is compatible with their workflows and values [47].
Decomposition of subjective norm
Two determinants influence the subjective norms in this study: external influence and interpersonal influence [32]. Prior studies have shown that the individual’s social group may strongly influence behaviour, especially when using innovation [48]. Peers, colleagues, and immediate supervisors often form the core of the MROs or AMROs social group, influencing users at the interpersonal level. Important information is communicated frequently in this group of ICD-11 users (50). Studies focusing on the use of electronic brokerages showed that adopters may give more weight to the first-hand accounts from peers or superiors, thereby influencing the subjective norm [49].
Besides interpersonal influence, the user’s external influence affects the subjective norms of the user. This study defines external influences from the WHO, specialists and the MOH [49]. In other words, external influence is not personal or specific to the user. In any new initiatives involving new systems, incentives offered to the users affect the subjective norms of the user in a positive way [39]. In Malaysia, the MOH, with close cooperation from the WHO, has actively organised activities and discussions for regular updates and new information. Besides that, the MOH organised awareness and training sessions to inform and as a platform for the local stakeholders to share their views [50].
Decomposition of perceived behavioural control
Consistent with DTPB and this study, the users’ perceived behavioural control is influenced by the facilitating conditions and their self-efficacy in ICD-11 use [32]. Facilitating conditions among medical records professionals is defined as the extent to which they think the infrastructure and facilities are there to support the use of ICD-11 [30]. For example, ICD-11 ECT supports the ICD-11 code search [3, 13]. Related to the technology used in the healthcare industry, Mathai et al. (2022) found a significant relationship between facilitating conditions and the consumers’ perceived behavioural control [21].
Previous studies have shown that self-efficacy is an additional factor influencing users’ perceived behavioural control [32]. In this study, self-efficacy is an assessment of the MROs and AMROs ability to do what is necessary and to handle potential scenarios related to ICD-11 [51]. Users’ self-efficacy level will influence their choices, readiness, and effort in ensuring the success of ICD-11 implementation. For example, in a study by Hung et al. (2011) involving doctors, self-efficacy directly and significantly impacted the doctors’ perceived behavioural control towards using the Medline system [23].
Materials and methods
Study design
This is a questionnaire development and validation study on the factors influencing the intention to use ICD-11 among MROs and AMROs involved in ICD-11 coding. This study consists of two stages: Questionnaire design phase (Phase I) and psychometric testing phase (Phase II).
Ethics approval
The Research Ethics Committee, Universiti Kebangsaan Malaysia (UKM PPI/111/8/JEP-2023-080) and the Medical Research & Ethics Committee, Ministry of Health (MOH) Malaysia (NMRR ID-23-00756-KIH (IIR) approved this study.
Phase I: Questionnaire design
Item development.
The theoretical model DTPB guided the instrument development. We adapted items from prior studies to the context of this study into 12 parts, where 11 sections consisted of 11 psychological constructs (INT, AT, SN, PBC, PU, PEOU, CO, II, EI, SE, FC). The remaining section consists of questions to capture demographic data. Fifty-eight measurement items (Table 1) were considered for the development of construct measures.
Content validity.
The items then underwent a content validity process by five expert panels to examine the draft questionnaire for relevance, clarity, simplicity, and ambiguity [52]. They comprised three Public Health specialists registered with the National Specialist Register, one Information Technology officer with at least five years of experience and one educator with at least five years of experience in questionnaire development [53].
Several criteria exist for selecting instrument reviewers. Some listed characteristics include documented experience, professional certification, and the ability to present and publish professional papers or initiate research. Given the complexity of the study, experts from specific disciplines can be content reviewers for data collection instruments [54]. To the best of our knowledge, there is no set rule for a minimum number of years of experience to be defined as an expert in content validity exercise. However, based on similar studies conducted in contexts like education [55, 56] and healthcare [53, 57, 58], we have decided on the criteria for experts’ selection to having at least five years of experience in Public Health or questionnaire development or information management. Efforts have been made to ensure that the chosen experts have fulfilled most of the abovementioned criteria.
We then calculated the content validity index (CVI) after the panels scored each scaled item based on relevance, clarity, ambiguity, and simplicity [52]. The panellists also provided opinions or thoughts on the items by completing the questionnaire’s comment section. We improved the draft questionnaire based on the expert panel’s recommendations. Items with I-CVI of ≥0.78 were retained, items with I-CVI of 0.70–0.78 were amended, and items with I-CVI of ≤0.70 were removed [59]. We also computed modified kappa (K), probability of change agreement (Pc), scale-level content validity index based on universal agreement method (S-CVI/UA), and scale-level content validity index based on average techniques (S-CVI/Ave).
Face validity.
We pretested the draft questionnaire on five participants who fulfilled the inclusion criteria and provided their consent. The objective of this face validity was to get input from a convenient sample of participants and to ascertain how well the participants understood the items so that it was free of ambiguities. The participants answered the questionnaire and gave their viewpoints. The questionnaire underwent a minor modification based on the feedback [60].
Translation.
On the language of the questionnaire, we decided to provide a dual language questionnaire rather than one in each language so that respondents could comprehend the questions on a combined level. Even though Malay is the country’s official language, English is a common second language in Malaysia. These two languages are frequently used interchangeably in Malaysia [61]. We used the forward-backward translation method to translate the measurement items from English to Malay. A Malay translator and a Medical Officer fluent in English and Malay forward translated the English version into Malay. Subsequently, two independent translators with full English proficiency translated the Malay version back to English. The study team then harmonised the two translations into a single document.
Phase II: Psychometric testing of questionnaire
Study population.
MROs and AMROs employed by the Ministry of Health, Malaysia (N = 479), were the population of this study. We exclude external personnel from non-MOH facilities sent for attachment at MOH facilities, non-medical records professionals involved in the ICD-11 use process and staff who have just returned from long leaves in the past year [62, 63].
Sampling method, data collection and sample size estimates.
A convenience sampling method was undertaken. Participants were recruited from the ICD-11 training and awareness sessions organised by the MOH. The participants then completed the questionnaire using the provided Google Form link. Personnel who agreed to participate consented online (by checking the "I agree" box) before answering the survey. No information on participants’ identifiers was collected, and participants did not need to log in to their accounts to access the Google Form.
To calculate the appropriate sample size for the survey, we utilised the G*Power 3.1.9 power analysis tool. From the tool, at least 77 responses were needed based on parameters such as effect size of 0.15, α at 0.05 and power at 0.80 [64]. However, evaluating the sample size for EFA, it is advised that samples should be at least 100 [65]. Moreover, a sample size of 160–300 valid observations was recommended for structural equation modelling (SEM), accounting for the target population. For instance, in this study’s context for CFA, a sample size of 200 is deemed large for a population of 400 [66].
Study tool.
The final 50-item questionnaire in both Malay and English consisted of two main sections: ’Intention to use ICD-11’ (6 items), ’Attitude’ (4 items), ’Subjective norm’ (4 items), ’Perceived behavioural control’ (4 items), ’Perceived ease of use’ (6 items), ’Perceived usefulness’ (5 items), ’Compatibility’ (3 items), ’External influence’ (3 items), ’Interpersonal influence’ (4 items), ’Self-efficacy’ (3 items), ’Facilitating conditions’ (8 items). In addition, study participants’ sociodemographic characteristics were collected. A 7-point Likert scale was used to explore all questionnaire items (Strongly agree/ Extremely important; Agree/ Important; Slightly agree/ Slightly important; Neutral; Slightly disagree/ Slightly unimportant; Disagree/ Unimportant; Strongly disagree/ Extremely unimportant). The finalised survey items are shown in the S1 Appendix.
Exploratory factor analysis.
Cronbach’s α coefficient was used for internal consistency reliability assessment, and the proposed domain structure and its substructure were analysed using exploratory factor analysis (EFA). Bartlett’s sphericity test and the Kaiser-Meyer-Olkin (KMO) were used to measure sampling adequacy. The dataset was deemed sufficient for factor analysis if the KMO value exceeds 0.50 and Bartlett’s sphericity test result is p<0.05 [67]. Varimax rotation was utilised for the principal component factor analysis. Ideal values were 0.55 or higher, and items with values lower than 0.55 will be dropped from this study [68]. We used Cronbach’s α coefficient to measure the items’ internal consistency reliability (IC). Items with Cronbach’s α coefficient value of ≥0.70 are deemed satisfactory internal consistency reliability [69].
Confirmatory factor analysis.
This study used confirmatory factor analysis (CFA) to validate the best-fitting factor model among the MRO and AMRO at MOH facilities. CFA was done in first order. A chi-square to the degree of freedom ratio (χ2/df) value of 2.0 with a p-value >0.05, composite fit index (CFI) of 0.95, root mean square error of approximation (RMSEA) of <0.08 were among the criteria used to choose the best-fit model. The determination of the fit indices is based on Hair et al. (2018), that is, using at least one index for absolute fit measure and one incremental fit index [65]. CFA is also utilised to evaluate the constructs’ validity, including the draft questionnaire’s convergent and discriminant validity. Convergent validity is when components that make up a construct converge or share significant variation. The amount that the connected latent construct accounts for of the variation in the corresponding item is shown by the square of the standardised regression weights (SRW). Therefore, to calculate convergent validity, the average variance extract (AVE) formula is as follows:
Factor loadings are represented by λ. A latent construct explains more than 50% of a construct’s variation if each construct’s AVE is more than 0.50. On the other hand, discriminant validity quantifies how well a construct captures a phenomenon not explained by other constructs in the model. Therefore, to assess discriminant validity, we compared whether the AVE of any concept in the model is higher than the square of the correlation between any two constructs [70]. The software used was SPSS Statistics version 27.0 (IBM Corp., Armonk, NY, USA) and Amos 28.0 programs (IBM Corp., Armonk, NY, USA) for data analysis.
Results
Phase I: Questionnaire design
Items of the constructs of decomposed theory of planned behaviour (DTPB).
Translation and adaptation.
Forward translation. Harmonisation of the English to Malay translations from two translators resulted in the final version of the questionnaire. The researchers and the translators discussed the differences until they reached a consensus on the suitability of the words or terms used.
Backward translation. For backward translation, two translators proficient in Malay and English independently translated the completed Malay questionnaire back into English. The researchers and the translators discussed the differences until they reached a consensus on the suitability of the words or terms used. Most items have meanings identical to those of the original English questionnaire.
Content validation.
Five expert panels assessed the draft questionnaire’s content validity for intention, attitude, subjective norms, perceived behavioural control, perceived usefulness, perceived ease of use, self-efficacy, facilitating conditions, compatibility, and interpersonal and external factor domains. For each of the domains, the following indicators of content validity are as follows: (1) Content validity index (I-CVI); (2) Scale-level content validity index based on average methods (S-CVI/AVE); (3) Scale-level content validity index based on universal agreement method (S-CVI/UA); (4) Probability of change agreement (Pc) and; (5) modified kappa (K) based on relevance, clarity, simplicity and ambiguity (Table 2).
No items were deleted based on the expert panel’s comments and the results. However, items II1 and EI1 were deemed to be double-barrelled and restructured. The questionnaire draft, as a result of the comments by the panels, consisted of seven items for the INT domain, four items for the ATT domain, four items for the SN domain, four items for the PBC domain, six items for the PU domain, six items for the PEOU domain, three items for the COM domain, four items for the II domain, four items for the EI domain, eight items for the SE domain and ten items for the FC domain.
Face validity.
Five participants who fulfilled the inclusion criteria completed the questionnaire draft with all items within ten to fifteen minutes. The finding suggests that the participants thought the questionnaire was straightforward and understandable. The 60 items remained with minor adjustments per recommendation from the participants.
Phase II: Psychometric testing
Sociodemographic characteristics of participants.
The population of MROs and AMROs, which shared similar characteristics of the intended sample population for the psychometric assessment, was the subject of this study. A total of 299 participants consented to be involved in the study. One participant did not give consent and seven participants did not complete the questionnaire, leaving 292 valid questionnaires with a response rate of 97.3%. Females comprised the majority (78.8%) and were of Malay ethnicity (81.2%). The respondents’ ages ranged from 23 to 59 (mean 40.2 ± 7.0) years. 49.7% of the respondents had completed at least a high school or diploma. AMROs comprised the majority of respondents (91.1%) with years of experience ranging from 0 to 31 (mean 9.0 ± 6.5) years of experience. The sociodemographic summary of the respondents is tabulated in Table 3.
Construct validity.
For construct validity, 105 Medical Record Officers (MRO) and Assistant Medical Record Officers (AMRO) participated in a cross-sectional study. The participants’ average age was (Mage = 40.1, SD = 7.99). The participants consisted of 102 (97.14%) Malay, with 84 (80.00%) females and 21 (20.00%) males. Using the EFA, we examined the suitable constructs for all the domains. Bartlett’s sphericity test results were less than 0.001, and the KMO showed 0.785 based on the EFA result, suggesting that the data were appropriate for factor analysis. We adopted items with factor loading greater than 0.50.
With eigenvalues greater than 1, 11 components were found to be contributing to 88.30% of the total variance. The first component explained the variance by 32.68%, the second component by 13.14%, the third component by 8.89%, the fourth component by 7.53%, the fifth component by 5.94%, the sixth component by 5.23%, the seventh component by 3.93%, the eighth component by 3.34%, the ninth component by 2.75%, the tenth component by 2.60% and the eleventh component by 2.27% of the variance, respectively.
Following the varimax rotation (Table 6), we excluded FC9-FC10, INT7, and PU1 items. SE1, SE5-SE8 were excluded from the FC, INT, PU and SE domains. There are 51 items in the updated version of the questionnaire. The extracted factors are as in Table 4:
Reliability.
We estimated the internal consistency reliability of each factor and the aggregate value using Cronbach’s alpha. Each factor of the Cronbach’s alpha ranged from 0.850 to 0.992 with an overall Cronbach’s alpha of 0.954. As a result, the internal consistency reliability is satisfactory, confirming that the item correlations on the same components were also satisfactory [72]. The internal consistency of the draft questionnaire is summarised in Table 5.
Confirmatory factor analysis.
For the remaining survey cases (n = 187), a confirmatory factor analysis (CFA) with a maximum likelihood approach was performed, yielding eleven eleven-component structures. The CFA fit indices (CMIN/DF = 1.887, p < 0.001; RMSEA = 0.069; CFI = 0.923) suggest that the measurement model has a reasonably good fit [65]. The factor structure has confirmatory evidence from the indicators mentioned above.
The convergent and discriminant validity of the questionnaire. Tables 6 and 7 summarise the results of the discriminant and convergent validity of the questionnaire. For convergent validity, the AVE of each domain is higher than 0.5; thus, the convergent validity of the questionnaire is achieved. The questionnaire’s discriminant validity is attained, as evidenced by the square of the inter-construct correlations being smaller than the AVE of any of the constructs or domains in the study.
Discussion
To the best of our knowledge, this is one of the early studies detailing the development and validation of a questionnaire to measure the intention of the MROs and AMROs to use ICD-11 in Malaysia. The final questionnaire has 50 items, divided into eleven subscales (FC, INT, PEOU, PU, SN, II, ATT, PBC, SE, COM, and EI). Based on the DTPB theory, we adapted items from the literature. Subsequently, to ensure composite understanding of the questionnaire in Malaysia, we forward and backward translated the questionnaire from English to Malay.
For content and face validity, five expert panels with specialities ranging from Public Health to Questionnaire Development reviewed the items. Ten questions were deemed double-barrelled, and the items were redesigned to ensure each focused on a specific theme. Then, the MROs and AMROs verified the draft items. This approach assured the tool’s content and face validity at the study’s outset, especially regarding language coherence and relevance.
Exploratory and confirmatory factor analysis provided evidence for the construct validity of the questionnaire in evaluating its psychometric qualities. The EFA yielded an eleven-factor model that explained 88.30% of the study’s variance. These factors were then validated using CFA which showed moderately good fit for the instrument. An acceptable fit is defined as having an RMSEA between 0.060 and 0.080; a close fit is described as having a value less than 0.06 [73]. As the incremental fit indicator, the CFI value confirmed the model’s fitness [65]. The questionnaire also has good internal consistency reliability because all eleven subscales have Cronbach alpha values higher than 0.954.
From the eleven-factor model of the questionnaire in CFA, the AVE for each domain of the questionnaire was higher than 0.6, indicating good convergent validity. The square root of AVE of all domains in the questionnaire was higher than the interdomain correlation coefficients, indicating favourable discriminant validity. Given the evidence of the construct validity findings, all the items and domains are valid to measure the factors influencing the intention to use ICD-11 among the MROs and AMROs.
Theoretical and practical implications
This study has theoretical and practical implications. Theoretically, this study has extended the application of the DTPB model involving ICD-11, a new health information system involving medical records professionals in the Malaysian context [13]. The majority of previous studies have mostly measured clinical utility and reliability measures. The practical implication of this study is that the output of this study, that is, the validated questionnaire, will aid in understanding the MROs and AMROs readiness, confidence, attitude, perceptions, and psychological antecedents for the ICD-11 use. Consequently, the information will guide the MOH in creating focused training programs and anticipate potential showstoppers during the implementation of ICD-11 in Malaysia [50, 74]. The implementation initiatives should be more effective if the variables and determinants of ICD-11 acceptance are understood during the early stages of implementation. Other healthcare systems may also utilise this instrument to find gaps or improve current ICD-11 implementation efforts. However, it is to be noted that although this tool may give the MROs and AMROs a stronger voice in ICD-11 implementation initiatives in Malaysia, it cannot, by itself, meet all the users’ needs. Strong communication and coordination across the health disciplines and external stakeholders are essential to ensure users are provided with a clear and supportive environment to ensure the success of ICD-11 implementation in Malaysia.
Strength and limitations
Significantly, from the strength of the questionnaire’s validity, this instrument reduces the gap in the body of literature by providing a linguistically and culturally appropriate tool for evaluating the intention to use ICD-11 among the MROs and AMROs. With the availability of relevant measures, health policymakers and academics can assess the support needs of MROs and AMROs involved in the ICD-11 transition efforts.
Notwithstanding the study’s merits and contributions, it is essential to acknowledge a few limitations. Firstly, selection bias may be introduced by the convenient sampling procedure, which could potentially reduce the findings’ generalizability [75]. The external validity of the questionnaire could be improved by using a more representative and varied sample in future studies. Secondly, the instrument has an unequal number of items across components. The instrument should be as small as possible while maintaining the factor structure and psychometric qualities.
Conclusion
In conclusion, this questionnaire provides a reliable tool for investigating the intention to use ICD-11 among MROs and AMROs. Positive findings from the psychometric properties support the validity of the questionnaire. This instrument can potentially support personnel in charge of ICD codification, guide the ICD-11 transition at various levels and facilitate research on support dynamics among the MROs and AMROs.
Acknowledgments
We would like to thank the Director General of Health Malaysia for permitting us to conduct this study. In addition, we express our sincere gratitude to the translators, experts, and reviewers for their commitment to finalising the draft questionnaire.
References
- 1.
World Health Organization. International Statistical Classification of Diseases and Related Health Problems—Tenth Version. 2nd ed. Vol. 2, World Health Organization. Geneva; 2004. 131 p.
- 2. Alonso V, Santos JV, Pinto M, Ferreira J, Lema I, Lopes F, et al. Problems and Barriers during the Process of Clinical Coding: a Focus Group Study of Coders’ Perceptions. J Med Syst [Internet]. 2020 Mar 1 [cited 2022 Mar 1];44(3):1–8. Available from: https://pubmed.ncbi.nlm.nih.gov/32036459/ pmid:32036459
- 3.
World Health Organization. ICD-11 Reference Guide [Internet]. 2024 [cited 2021 Feb 19]. Available from: https://icd.who.int/icd11refguide/en/index.html#2.01.00Part2ICDmaintenanceandapplication%7Cpart-2-using-icd11%7Cc2
- 4.
World Health Organization. ICD-11 Implementation or Transition Guide [Internet]. 1.05. Vol. version 1. Geneva; 2019 [cited 2022 Jun 10]. 1–29 p. Available from: www.who.int/classifica=ons/network/collabora=ng
- 5.
ICD-11 2023 release is here [Internet]. [cited 2024 Mar 26]. Available from: https://www.who.int/news/item/14-02-2023-icd-11-2023-release-is-here
- 6. Ibrahim Alrashidi M, Al-Salamin M, Kostanjsek N, Jakob R, Azam S, et al. ICD-11 Morbidity Pilot in Kuwait: Methodology and Lessons Learned for Future Implementation. Int J Environ Res Public Health [Internet]. 2022 Mar 5 [cited 2022 Mar 11];19(5):3057. Available from: https://www.mdpi.com/1660-4601/19/5/3057/htm pmid:35270745
- 7. Golpira R, Azadmanjir Z, Zarei J, Hashemi N, Meidani Z, Vahedi A, et al. Evaluation of the implementation of International Classification of Diseases, 11th revision for morbidity coding: Rationale and study protocol. Inform Med Unlocked [Internet]. 2021 Jan 1 [cited 2022 Mar 3];25:100668. Available from:
- 8. Azadmanjir Z, Sheikhtaheri A, Zarei J, Golpira R, Bakhshandeh H, Vahedi A, et al. A study on initial productivity trend in the transition of the ICD-10 to ICD-11 morbidity coding in Iran. Inform Med Unlocked. 2024 Jan 1;44:101440.
- 9. Mugisha M, Byiringiro JB, Uwase M, Abizeyimana T, Ndikubwimana B, Karema N, et al. Integration of International Classification of Diseases Version 11 Application Program Interface (API) in the Rwandan Electronic Medical Records (openMRS): Findings from Two District Hospitals in Rwanda. Stud Health Technol Inform [Internet]. 2020 [cited 2024 Mar 26];272:280–3. Available from: https://ebooks.iospress.nl/doi/10.3233/SHTI200549 pmid:32604656
- 10. Zhang M, Wang Y, Jakob R, Su S, Bai X, Jing X, et al. Methodologies and key considerations for implementing the International Classification of Diseases-11th revision morbidity coding: insights from a national pilot study in China. Journal of the American Medical Informatics Association [Internet]. 2024 Mar 1 [cited 2024 Mar 7]; Available from: pmid:38427850
- 11. Fung KW, Xu J, Bodenreider O. The new International Classification of Diseases 11th edition: a comparative analysis with ICD-10 and. 2022;27(May 2019):738–46.
- 12.
Australian Institute of Health and Welfare. World Health Organization Collaborating Centre for the Family of International Classifications, Australia. Canberra; 2024 Jan.
- 13. Harrison JE, Weber S, Jakob R, Chute CG. ICD-11: an international classification of diseases for the twenty-first century. BMC Medical Informatics and Decision Making 2021 21:6 [Internet]. 2021 Nov 9 [cited 2021 Nov 30];21(6):1–10. Available from: https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-021-01534-6 pmid:34753471
- 14. Krive J, Patel M, Gehm L, Mackey M, Kulstad E, Li JJ, et al. The complexity and challenges of the international classification of diseases, ninth revision, clinical modification to international classification of diseases, 10th revision, clinical modification transition in EDs. American Journal of Emergency Medicine [Internet]. 2015 May 1 [cited 2021 Jun 12];33(5):713–8. Available from: https://pubmed.ncbi.nlm.nih.gov/25863652/ pmid:25863652
- 15.
World Health Organization. WHO Nomenclature Regulations [Internet]. The Twentieth World Health Assembly. Geneva; 1967. p. 3–4. Available from: https://apps.who.int/iris/bitstream/handle/10665/89478/WHA20.18_eng.pdf?sequence=1&isAllowed=y
- 16.
World Health Organization. Eleventh revision of the International Classification of Diseases. Geneva; 2019 Apr.
- 17.
Ministry of Health Malaysia. Pelaksanaan Penggunaan Klasifikasi Penyakit Terbitan WHO, ICD-11 Bagi Pengumpulan dan Pelaporan Data Morbiditi dan Mortaliti di Malaysia Bermula 1 Januari 2024. Putrajaya; 2023.
- 18. Hwang Y, Al-Arabiat M, Shin DH. Understanding technology acceptance in a mandatory environment: A literature review. Information Development. 2016;32(4):1266–83.
- 19. Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: Case study of HealthSpace. BMJ (Online). 2010;341(7782):1091.
- 20. Cahuana-Hurtado L, Gómez-Dantés H, De La Cruz-Góngora V, Chiquete E, Cantú-Brito C. Unveiling the Burden of Miscoding and Misclassification in Stroke Mortality: Analysis of Multiple Cause-of-Death Data in Mexico. Neuroepidemiology [Internet]. 2023 Nov 9 [cited 2024 Jan 10];57(5):284–92. Available from: pmid:37399787
- 21. Mathai N, McGill T, Toohey D. Factors Influencing Consumer Adoption of Electronic Health Records. Journal of Computer Information Systems [Internet]. 2020 Mar 4 [cited 2024 Mar 18];62(2):267–77. Available from: https://www.tandfonline.com/doi/abs/
- 22.
Ministry of Health M. Garis Panduan Pembangunan dan Perkembangan Kerjaya Profesion Pegawai dan Penolong Pegawai Tadbir (Rekod Perubatan). 49 p.
- 23. Hung SY, Ku YC, Chien JC. Understanding physicians’ acceptance of the Medline system for practicing evidence-based medicine: A decomposed TPB model. Int J Med Inform [Internet]. 2012;81(2):130–42. Available from: pmid:22047627
- 24. Chau PYK, Hu PJH. Information technology acceptance by individual professionals: A model comparison approach. Decision Sciences. 2001;32(4):699–719.
- 25. Eastwood CA, Southern DA, Doktorchik C, Khair S, Cullen D, Boxill A, et al. Training and experience of coding with the World Health Organization’s International Classification of Diseases, Eleventh Revision. Health Information Management Journal. 2021;52(2):92–100. pmid:34555947
- 26. Khorrami F, Alipour J, Karami NA, Hayavi-Haghighi MH, Chahooei MK. Quality of documentation of medical records and coding accuracy of ICD-10 versus ICD-11. Journal of Health Administration Fall. 2023;2022(3).
- 27. Nazari A, Huprich SK, Hemmati A, Rezaei F. The Construct Validity of the ICD-11 Severity of Personality Dysfunction Under Scrutiny of Object-Relations Theory. Front Psychiatry [Internet]. 2021 Jul 22 [cited 2024 Mar 30];12:648427. Available from: www.frontiersin.org pmid:34366910
- 28. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–39.
- 29. Venkatesh V, Davis FD. Theoretical extension of the Technology Acceptance Model: Four longitudinal field studies. Manage Sci. 2000;46(2):186–204.
- 30. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS Q. 2003;27(3):425–78.
- 31. Ajzen I. From Intentions to Actions: A Theory of Planned Behavior. Action Control [Internet]. 1985 [cited 2024 Jan 30];11–39. Available from: https://link.springer.com/chapter/10.1007/978-3-642-69746-3_2
- 32. Taylor S, Todd P. Decomposition and crossover effects in the theory of planned behavior: A study of consumer adoption intentions. International Journal of Research in Marketing. 1995;12(2):137–55.
- 33. Benbasat I, Barki H. Quo vadis TAM? G. Balint, Antala B, Carty C, Mabieme JMA, Amar IB, Kaplanova A, editors. J Assoc Inf Syst [Internet]. 2007 [cited 2024 Mar 15];8(4):7. Available from: http://aisel.aisnet.org/jais/vol8/iss4/7
- 34. Omulo G, Daum T, Köller K, Birner R. Unpacking the behavioral intentions of ‘emergent farmers’ towards mechanized conservation agriculture in Zambia. Land use policy. 2024 Jan 1;136:106979.
- 35. He T, Huang Q, Yu X, Li S. Exploring students’ digital informal learning: the roles of digital competence and DTPB factors. Behaviour and Information Technology [Internet]. 2021;40(13):1406–16. Available from:
- 36. Puah S, Bin Mohmad Khalid MIS, Looi CK, Khor ET. Investigating working adults’ intentions to participate in microlearning using the decomposed theory of planned behaviour. British Journal of Educational Technology [Internet]. 2022 Mar 1 [cited 2024 Mar 18];53(2):367–90. Available from: https://onlinelibrary-wiley-com.eresourcesptsl.ukm.remotexs.co/doi/full/10.1111/bjet.13170
- 37. Shih YY, Fang K. The use of a decomposed theory of planned behavior to study Internet banking in Taiwan. Internet Research. 2004;14(3):213–23.
- 38. Irshaidat R, Al Khasawneh MH. Empirical validation of the decomposed theory of planned behaviour model within the mobile banking adoption context. International Journal of Electronic Marketing and Retailing. 2017;8(1):58.
- 39. Hsieh. Physicians’ acceptance of electronic medical records exchange: An extension of the decomposed TPB model with institutional trust and perceived risk. Int J Med Inform [Internet]. 2015 Jan 1 [cited 2022 Apr 19];84(1):1–14. Available from: pmid:25242228
- 40. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991 Dec 1;50(2):179–211.
- 41.
Ajzen I, Crano W. Attitides and Attitude Change. Taylor & Francis Group. 2008. 456 p.
- 42. Chang MY, Kuo FL, Lin TR, Li CC, Lee TY. The Intention and Influence Factors of Nurses’ Participation in Telenursing. Informatics 2021, Vol 8, Page 35 [Internet]. 2021 May 18 [cited 2022 Apr 19];8(2):35. Available from: https://www.mdpi.com/2227-9709/8/2/35/htm
- 43. Davis. User Acceptance of Information Systems: The Technology Acceptance Model (TAM). Information Seeking Behavior and Technology Adoption. 1989;205–19.
- 44. Moores TT. Towards an integrated model of IT acceptance in healthcare. Decis Support Syst [Internet]. 2012;53(3):507–16. Available from:
- 45. Rogers EM. Diffusion of Innovations: Modifications of a Model for Telecommunications. Die Diffusion von Innovationen in der Telekommunikation [Internet]. 1995 [cited 2022 Feb 14];25–38. Available from: https://link.springer.com/chapter/10.1007/978-3-642-79868-9_2
- 46. Alam SS, Ali MY, Jani MFM. An empirical study of factors affecting electronic commerce adoption among SMEs in Malaysia. Journal of Business Economics and Management. 2011;12(2):375–99.
- 47. Vuononvirta T, Timonen M, Keinänen-Kiukaanniemi S, Timonen O, Ylitalo K, Kanste O, et al. The compatibility of telehealth with health-care delivery. [Internet]. 2011 Feb 21 [cited 2022 Sep 30];17(4):190–4. Available from: https://journals.sagepub.com/doi/abs/10.1258/jtt.2010.100502 pmid:21339305
- 48. Iacobucci D, Valente TW. Network Models of the Diffusion of Innovations. J Mark. 1996;60(3):134.
- 49. Bhattacherjee A. Acceptance of E-Commerce Services: The Case of Electronic Brokerages. IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans. 2000;30(4):337–44.
- 50.
Ahmad Fuad A, Ooi ECW, Mimi NM, Azman A, Shahidah AS, Nuraidah MM. ICD-11 Awareness and Training Among MOH Personnel. In: WHO-Family of International Classifications Network Annual Meeting 2023. Bonn, Germany; 2023.
- 51. Bandura A. Self-efficacy mechanism in human agency. American Psychologist. 1982 Feb;37(2):122–47.
- 52. Yaghmaie F. Content validity and its estimation. J Med Educ. 2003;1(5):8–8.
- 53. Suraya Noor Arzahan I, Ismail Z, Munira Yasin S. Content Validity Of A Self-Reported Instrument For Safety And Health (S&H) Culture Practice In Paramedic Training Institute Using A Heterogeneous Expert Panel. Turkish Journal of Computer and Mathematics Education (TURCOMAT) [Internet]. 2021 Apr 16 [cited 2024 Mar 29];12(7):2464–72. Available from: https://www.turcomat.org/index.php/turkbilmat/article/view/3574
- 54. Davis LL. Instrument review: Getting the most from a panel of experts. Applied Nursing Research. 1992 Nov 1;5(4):194–7.
- 55. Akbari R, Yazdanmehr E. A Critical Analysis of the Selection Criteria of Expert Teachers in ELT. 2014;
- 56. Suhaini M, Ahmad A, Bohari NM. Assessments on Vocational Knowledge and Skills: A Content Validity Analysis. European Journal of Educational Research [Internet]. 2021 [cited 2024 Mar 29];10(3):1529–40. Available from: https://www.eu-jer.com/
- 57. Perroca MG. Development and content validity of the new version of a patient classification instrument. Rev Lat Am Enfermagem [Internet]. 2011 [cited 2024 Mar 29];19(1):58–66. Available from: https://www.scielo.br/j/rlae/a/MpMmzXhPLrtb63zpJ8K85mB/?format=html pmid:21412630
- 58. Sampaio FMC, Sequeira C, Lluch Canut T. Content Validity of a Psychotherapeutic Intervention Model in Nursing: A Modified e-Delphi Study. Arch Psychiatr Nurs. 2017 Apr 1;31(2):147–56. pmid:28359426
- 59. Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication. J Caring Sci [Internet]. 2015 Jun 1 [cited 2023 Dec 22];4(2):165. Available from: /pmc/articles/PMC4484991/ pmid:26161370
- 60. Ghani NDH, Abd Rahman MH, Mohamad Fadzil N, Mohammed Z, Mohd Rasdi HF, Shafie NS. Development and validation of parental knowledge, attitude and practice in eye problem among children questionnaire (PEPC-KAPQ). PLoS One [Internet]. 2023 Sep 1 [cited 2023 Nov 5];18(9):e0291062. Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0291062 pmid:37682886
- 61. Kong YC, Danaee M, Kaur R, Thiagarajan M, Zaharah H, Sener M, et al. Development and Validation of a Dual-Language (English and Malay) Needs Assessment Tool for Breast Cancer (NeAT-BC). Diagnostics 2023, Vol 13, Page 241 [Internet]. 2023 Jan 9 [cited 2024 Mar 29];13(2):241. Available from: https://www.mdpi.com/2075-4418/13/2/241/htm pmid:36673050
- 62. Hussein SZ, Khalip N, Hashim R, Harun R, Fazilah NF, Shah NM. Patient Care Delivery: Electronic Nursing Documentation in Malaysia. Makara Journal of Health Research [Internet]. 2021 Aug 30 [cited 2023 Jan 3];25(2):5. Available from: https://scholarhub.ui.ac.id/mjhr/vol25/iss2/5
- 63. Talwar A, Verma M, Sharma T, Talwar A, Verma M, Sharma T. Effectiveness of Capacity Building Program on the Use of Assessment Scales for Critically Ill Patients in Terms of Practice of Nurses. Int J Adv Res (Indore) [Internet]. 2020 [cited 2023 Jan 3];8(3):567–74. Available from: https://www.researchgate.net/publication/340385266
- 64. Franz Faul, Edgar Erdefelder, Lang Albert-Georg Buchner Axel. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. 2007;
- 65. Hair Black WC, Babin BJ, Anderson RE, Black WC, Anderson RE. Multivariate Data Analysis. 2018. 95–120 p.
- 66. Memon MA, Ting H, Cheah JH, Thurasamy R, Chuah F, Cham TH. Sample Size for Survey Research: Review and Recommendations. Journal of Applied Structural Equation Modeling. 2020;
- 67. Fan Y, Zhang S, Li Y, Li Y, Zhang T, Liu W, et al. Development and psychometric testing of the Knowledge, Attitudes and Practices (KAP) questionnaire among student Tuberculosis (TB) Patients (STBP-KAPQ) in China. BMC Infect Dis [Internet]. 2018 May 8 [cited 2023 Dec 23];18(1):1–10. Available from: https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-018-3122-9
- 68.
Hair Black William C., Babin Barry J., Anderson Rolph E. Multivariate Data Analysis. United Kingdom: Pearson; 2013. 1–734 p.
- 69. Musa M, Talip R, Awang Z, Zainudin &. Exploratory Factor Analysis for Technostress Among Primary School Teachers. Malaysian Journal of Social Sciences and Humanities (MJSSH) [Internet]. 2023 Feb 28 [cited 2023 Dec 23];8(2):e002117–e002117. Available from: https://msocialsciences.com/index.php/mjssh/article/view/2117
- 70. Luigi d ell’Olio, Angel Ibeas, Juan de Ona, Rocio de Ona. Public Transportation Quality of Service Factors, Models, and Applications. 2018. 1–231 p.
- 71. Hu PJ, Chau PYK, Liu Sheng OR, Tam KY. Examining the Technology Acceptance Model Using Physician Acceptance of Telemedicine Technology. https://doi.org/101080/07421222199911518247 [Internet]. 2015 Mar 1 [cited 2022 Sep 26];16(2):91–112. Available from: https://www.tandfonline.com/doi/abs/10.1080/07421222.1999.11518247
- 72. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ [Internet]. 2011 Jun 27 [cited 2023 Dec 26];2:53. Available from: /pmc/articles/PMC4205511/ pmid:28029643
- 73. Schreiber JB, Stage FK, King J, Nora A, Barlow EA. Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. J Educ Res [Internet]. 2006 [cited 2024 Jan 8];99(6):323–38. Available from: https://www.tandfonline.com/doi/abs/
- 74.
Ooi ECW, Md Isa Z, Abdul Manaf MR, Ahmad SAF, Mimi NM, Azman A, et al. Planning for ICD-11 Transition in Malaysia. In: WHO-Family of International Classifications Network Annual Meeting 2023. Bonn, Germany; 2023.
- 75. Tavares J, Faria A, Gonçalves D, Mendes D, Silva S, Sousa L. Validation of the Portuguese version of the social isolation scale with a sample of community-dwelling older adults. Int J Nurs Sci. 2023 Apr 1;10(2):151–7. pmid:37128493