Development of the Final Version of the Classification and Assessment of Occupational Dysfunction Scale

Occupational therapy is involved in disability prevention and health enhancement through the prevention of occupational dysfunction. Although many occupational dysfunction scales exist, no standard method is available for the assessment and classification of occupational dysfunction, which may include occupational imbalance, occupational deprivation, occupational alienation, and occupational marginalization. The purpose of this study was to develop the final version of Classification and Assessment of Occupational Dysfunction (CAOD). Our study demonstrated the validity and reliability of CAOD in a group of undergraduate students. The CAOD scale includes 16 items and addresses the following 4 domains: occupational imbalance, occupational deprivation, occupational alienation, and occupational marginalization.


Introduction
Occupational dysfunction is recognized worldwide as a major health-related problem in the field of preventive occupational therapy [1][2][3]. Occupation is considered to be the center of the human experience; it includes things people need to do, want to do, and are expected to do [4,5]. In particular, occupation includes many categories such as leisure, housework, sleep, and personal care [6]. Occupation not only involves work, business, and labor but also includes a wide range of activities such as education, play, activities of daily living, rest, and social participation.
Occupational dysfunction is defined as a negative experience related to engaging in daily activities, and it includes occupational marginalization, occupational imbalance, occupational alienation, and occupational deprivation [7]. Occupational dysfunction may present without obvious medical disease [8]. Moreover, risk of experiencing an occupational dysfunction is not confined to only adult workers but also to people in various developmental stages such as puberty, adolescence, and old age. Occupational marginalization is defined as a person not having the opportunity to engage in desired daily activities [9]. Occupational imbalance is defined as a loss of balance in engaging in daily activities [10]. Occupational alienation is defined as a situation when the inner needs of the individual related to daily activities are not satisfied [11]. Occupational deprivation is defined as a lack of opportunity for daily activities beyond the individual's control [12]. These problems are barriers to social participation and lead to a decrease in health-related quality of life [13]. Preventive occupational therapy requires a valid and reliable assessment tool for occupational dysfunction.
Therefore, we developed a prototype assessment tool called the Classification and Assessment of Occupational Dysfunction (CAOD) [14,15]. The psychometric properties of the prototype were examined in 287 undergraduate students [14]. The rationale for sample choice is that undergraduate students are at high risk of occupational dysfunction because of an irregular lifestyle, poor sleep, dietary abnormalities, academic failure, and human relationship problems [16,17]. Analytical results supported a 4 factor model consisting of 19 items [14]. Overall, the validity and reliability of the prototype were satisfactory [14]. The CAOD prototype was based on the Occupational Based Practice 2.0 (OBP2.0) [14,15]. OBP2.0 was developed by the integration of theoretical study and clinical study in Japan [7]. Since then, various occupational therapists have been involved in its further development [18]. In Japan, OBP2.0 is utilized in a variety of circumstances. For example, OBP2.0 has been applied to diverse patient groups with acute and chronic orthopedic injuries, patients with mental illness, older persons with dementia, children with developmental disorders, and persons with cerebrovascular disorders. OBP2.0 is designed to help the people with occupational dysfunction related to occupational marginalization, occupational imbalance, occupational alienation, and occupational deprivation. OPB2.0 also includes a dissolution approach for belief conflict (DAB) for conflict management [7]. DAB was developed as a comprehensive intervention program for people who are suffering from belief conflicts [7,[18][19][20][21][22]. Therefore, OBP2.0 is a viable solution for occupational dysfunction and clarification of belief conflict [7]. The prototype CAOD based on OBP2.0 is focused on the assessment and classification of occupational dysfunction [14,15].
The factor structure of the prototype CAOD was different from the theoretical expectation in several ways [14]. First, the prototype CAOD suggested occupational functioning as a new factor in addition to occupational marginalization, occupational imbalance, occupational alienation, and occupational deprivation [14]. This is surprising because occupational functioning is the opposite of occupational dysfunction [2,3]. Therefore, occupational functioning was operationally defined as an individual's positive experiences related to engaging in daily activities [14]. Second, occupational alienation and occupational deprivation were integrated as a single factor by exploratory factor analysis (EFA) [14]. For these reasons, we reexamined the psychometric properties and item pools of the prototype CAOD for the development of a final version of CAOD [14].
The purpose of this study was to develop the final version of CAOD and examine its validity and reliability in a group of undergraduate students.

Ethics statement
The research protocol was approved by the Ethics Committee of Kibi International University (No. [11][12][13][14][15][16][17][18][19][20][21][22] and the Research Ethics Committee of Saitama Prefectural University (No. 23068). All participants provided written and verbal informed consent prior to participating. Participation was voluntary, and participants had the right to dropout from the research at any time without giving reason. This study was conducted according to the Declaration of Helsinki.

Development of item pool
To develop the item pool of CAOD, we systematically identified extant items relevant to occupational functioning, occupational marginalization, occupational imbalance, occupational alienation, and occupational deprivation. Using these 5 concepts, we compiled an initial item pool of 40 items consisting of: (1) occupational functioning (8 items), (2) occupational marginalization (8 items), (3) occupational imbalance (8 items), (4) occupational alienation (8 items), and (5) occupational deprivation (8 items). The planned number of items for the final version CAOD was 10-15. Relevant assessments included the work-family conflict scale [23], the sense of isolation scale [24], the scale of the sense of fulfillment of life [25], the Beck depression inventory [26], and many other assessments.
To assess the instrument's content and face validity, a panel of 4 occupational therapy experts (specialists in 2 physical areas and 2 psychosocial areas) reviewed the OBP2.0 item pool. We performed the expert consensus protocol in 3 steps: Step 1, item pool review; Step 2, item adjustment; and Step 3, final review. In Step 1, the item pool was reviewed to assess each question for content and face validity related to the 5 concepts. The content and face validity of 40 items were rated on a 2 point Likert scale, with 1 being "least plausible" and 2 being "most plausible." For inclusion in the item pool of CAOD, an item needed the consent of 75% of the experts in the panel. If the Likert rating was 1, the panelist was asked to suggest additional examples, modifications of wording, or deletions, as appropriate. In Step 2, we revised the language of the collected questions to reflect the results of the Step 1. In Step 3, the panelists reviewed the 40 items again to ensure that the classification of occupational dysfunction adequately represented the 5 domains in the item pool. A total of 40 items of CAOD successfully passed through this process and were sent for field-testing.

Participants
All participants were enrolled in the Kibi International University or the Saitama Prefectural University. A description of the research was given at the end of lectures. A total of 731 participants were contacted. These included 330 undergraduate occupational therapy students, 191 undergraduate physical therapy students, and 210 undergraduate nursing students.

Measures
Participant profile. We obtained relevant demographic data from participants: gender, age, school year, and department.
40 item version of Prototype CAOD. The 40 item version of the Prototype CAOD was used to assess occupational dysfunction types, including occupational marginalization, occupational imbalance, occupational alienation, occupational deprivation, and occupational functioning, based on OBP2.0. The CAOD item design was based on a 7 point Likert scale in which 1, 2, 3, 4, 5, 6, and 7 corresponded to strongly disagree, disagree, slightly disagree, neither agree nor disagree, slightly agree, agree, and strongly agree, respectively.
Japanese version of 36-item short form health survey (SF-36) [29,30]. SF-36 is used to measure health-related quality of life and contains 36 items in 8 dimensions: physical functioning (10 items), role limitations due to physical health problems (4 items), bodily pain (2 items), social functioning (2 items), general mental health (5 items), role limitations due to emotional problems (3 items), energy/vitality (4 items), and general health perceptions (5 items). Moreover, SF-36 has developed algorithms to calculate 3 component summary measures: Physical Component Summary Scale Score (PCS), Mental Component Summary Scale Score (MCS), and Role/Social component summary Scale Score (RCS). This study utilized the 3 component summary measures of SF-36.
Japanese version of Occupational Self-Assessment (OSA) [31]. OSA is a self-reported measure that evaluates a patient's own perceptions of their occupational competence and identity. OSA contains 21 items on 3 scales: performance capacity (11 items), habituation (one's pattern of occupational engagement; 5 items), and volition (one's motivation for participation; 5 items). OSA is measured with a 4 point response (i.e., 4 = I do this extremely well, 3 = I do this well, 2 = I have some difficulties doing this, and 1 = I have a lot of problems doing this).
Sample characteristics. Participant demographics were summarized using descriptive analyses. The normal distribution of all scores of CAOD was examined using the Kolmogorov-Smirnov test (p < 0.05).
Construct validity. The factor structure of the 40 item version of the prototype CAOD was determined by performing exploratory factor analysis (EFA) using maximum likelihood estimation and promax rotation. Item reduction was performed using item floor effects or ceiling effects. Moreover, items not loading on a factor (factor loading of <0.4) or loading on more than 1 factor were eliminated from the scale. Analysis was then performed on the reduced item set. Percentage of variance accounted for by a factor was estimated using eigenvalues.
Structural validity. We performed an EFA followed by a confirmatory factor analysis (CFA) using maximum likelihood estimation. We utilized 2 indices to assess how well the model fits the data. The first index was the comparative fit index (CFI), with critical values of >0.95 [32]. The second index was the root mean square error of approximation (RMSEA). The critical values of RMSEA of 0.08-0.10 show a mediocre fit and those of <0.08 indicate a good fit [33].
Hypothesis testing (convergent and discriminant validity). We performed the square of the correlation between the factors and the average variance extracted (AVE) for CAOD. To analyze discriminant validity, we compared the squared correlation between each pair of constructs against the average of AVE for the factor structure of CAOD. Convergent validity was checked to see whether the square root of each AVE value belonging to each latent construct was > 0.5.
Internal consistency reliability. Internal consistency reliability was evaluated using Cronbach's alpha coefficient.
Concurrent validity. We assessed the concurrent validity of CAOD items by examining the relationships between SOPI and SF-36. We used the Spearman's nonparametric correlation.
Predictive validity. We assessed the predictive validity of CAOD by examining its relationship to OSA. The assessment of the predictive validity of CAOD was performed by comparing the results of CAOD to OSA after a period of 1 week using Spearman's correlation. If we obtain the correlation, there is likely to be occupational dysfunction in the future.
Test-retest reliability. The test-retest reliability of CAOD was evaluated by comparing baseline results to results at 1 week follow up using Spearman's correlation.
Item response. The statistical models used in our analyses are based on the graded item response theory (IRT) using maximum likelihood estimation. IRT estimated item slope parameters and item difficulty parameters in CAOD. Moreover, IRT estimated CAOD's total information curve (TIC) and item information curve (IIC), which indicates the occupational dysfunction level at which a response in a given category or higher becomes probable.
Robustness of CAOD. Robustness of CAOD was tested using a latent class model based multiple-group confirmatory factor analysis. We compare a four model: 1) configural invariance, 2) weak measurement invariance, 3) strong measurement invariance, and 4) structural invariance. To assess the model, three fit indices were used, including the Akaike's Information Criterion (AIC), Bayesian Information Criterion (BIC), and sample-size adjusted BIC [34,35].

Sample Characteristics
In this study, 419 undergraduate students (57.3% response rate) responded, including 201 males, 210 females, and 8 unknowns, with an average age of 19.9 ± 2.09 years. Participants included 107 nursing students, 159 physical therapy students, 151 occupational therapy students, and 2 students from an unknown department. The Kolmogorov-Smirnov test showed that all scores had normal distribution. Table 1 shows the result of EFA for the 40-item prototype CAOD analysis. No floor and ceiling responses were observed. We determined the underlying factor structure of the item set. The 4 factors and 16 items were created from the EFA procedure and includes occupational marginalization (6 items), occupational imbalance (4 items), occupational alienation (3 items), and occupational deprivation (3 items). This factor structure was model fit to OBP2.0.  Table 2 shows the results of hypothesis testing. In brief, CAOD had convergent and discriminant validity.

Hypothesis testing (convergent and discriminant validity)
Internal consistency reliability Table 1 shows the result related to internal consistency. The internal consistency of CAOD (total score and all subscales) was within the acceptable range, between 0.826 and 0.902. Table 3 shows the result related to concurrent validity. The concurrent validity was confirmed by the correlations between the 16 items of CAOD and SOPI and SF-36. CAOD showed a negative correlation to SOPI total score (r = 0.097 to −0.331, p < 0.01). Moreover, CAOD showed a negative correlation to the MCS and RCS components of SF-36 (r = −0.133 to −0.421, p < 0.01). Table 4 shows the result related to predictive validity. The predictive validity was confirmed by the correlations between the 16 items of CAOD and OSA. CAOD showed a negative correlation to OSA (r = −0.116 to −0.425, p < 0.01). Table 4 shows the results related to test-retest reliability. CAOD displayed strong test-retest reliability (r = 0.516 to 0.678, p < 0.01). Table 5 shows the results of item slope parameters (α) and item difficulty parameters (β). Overall, 16 items on CAOD demonstrated satisfactory item response, with item slopes ranging from 1.091 to 1.393. Item difficulty parameters ranged from −CAOD demonstrated satisfa response of CAOD provided the appropriate discrimination index and difficulty index. Figs 2 and 3 present the test response function (TRF) and test information function (TIF) related to CAOD. Overall, CAOD measured an approximately equally wide range of occupational dysfunctions with high precision. Moreover, CAOD was slightly more precise at low levels of occupational dysfunction (i.e., higher theta values).

Psychometric properties of CAOD
We developed and validated the Classification and Assessment of Occupational Dysfunction (CAOD) as a new, self-administered measure for evaluating occupational dysfunction in undergraduate students. Several of our results suggest evidence for the validity and reliability of CAOD (Tables 1 to 5, Figs 1 to 3). To the best of our knowledge, this is the first study on the development of an assessment for classification of occupational dysfunction.
Overall, CAOD had a good model fit. The construct and structural validity of CAOD were assessed by EFA and CFA, and they indicated a good model fit (Fig 1 and Table 1). The      hypothesis testing of this study demonstrated a good value for convergent and discriminant validity of CAOD (Tables 3 and 4). In addition, occupational marginalization may need to be reexamined in the future because we obtained a rather small value. Assessed by Cronbach's alpha coefficient, internal consistency was acceptable (Table 1). In summary, these facts clearly prove that the model of OBP2.0 fits the data. That is, the final version of CAOD was formed by a concept similar to the classification of occupational dysfunction in OBP2.0. Therefore, we think that CAOD is empirically and theoretically well supported. Several results suggest evidence for the concurrent and predictive validity of the measure. First, concurrent validity was assessed by comparison of CAOD, SOPI, and SF-36 (Table 3). A modest negative correlation between CAOD, SOPI, and SF-36 (MCS and RCS) was observed. Occupational dysfunction is identified as a major health-related problem. The results of this  study demonstrate that occupational dysfunction is related to barriers to social participation as well as to decline in health-related quality of life. In addition, it was found that occupational dysfunction and SF-36 PCS (includes physical functioning, body pain, and role physical) correlated poorly. This suggests that CAOD and SF-36 PCS each represent a different aspect of the subjective experience. Second, predictive validity assessed by comparison of CAOD and OSA (Table 4) demonstrates a modest correlation between CAOD and OSA. The results regarding the predictive validity of CAOD showed a moderate negative value for occupational competency and occupational identity. Test-retest reliability of CAOD was very good, with high correlations (Table 4). Although occupational dysfunction may change over time, the reproducibility of CAOD in this study was high.
IRT was used to assess individual item characteristics of CAOD (Table 5). CAOD has high item slope parameters, in the range of 1.091-1.393. Item difficulty parameter scores for CAOD are very wide, ranging from −0.547 to 2.589. Moreover, the results indicate that TRF and TIF of CAOD were sufficiently identified (Figs 2 and 3). The amount of information for CAOD has been sufficiently identified. These results clearly demonstrate strong support for good item response of CAOD.
Results from the latent class model based multiple-group CFA indicate that the CAOD is a structurally valid four factors structure measure of occupational dysfunction in both gender groups and department groups. Overall, CAOD was able to show the robustness of the results.

Practical implications
There are several practical implications for developing CAOD. A valid and reliable assessment is needed for evaluating and intervening with preventive occupational therapy programs based on OBP2.0. CAOD is a screening tool intended to collect broad range information about a person's occupational dysfunction. Moreover, CAOD can be used as a tool to facilitate therapy of occupational dysfunction that includes occupational imbalance, occupational deprivation, occupational alienation, and occupational marginalization. CAOD may assist the patient and the occupational therapist in establishing goals and plans of care for addressing occupational dysfunction. We can prepare CAOD for undergraduate students. Undergraduate students have many lifestyle problems. In the future, we can understand the relationship of occupational dysfunction and mental illness. In addition, we can use CAOD for other participants in preventive occupational therapy.

Limitations
This study design has limitations. First, the survey was conducted with only undergraduate students, raising the question of the generalizability. Moreover, the psychometric properties of CAOD in samples with medical diseases are unknown and should be examined in future studies. Despite these limitations, CAOD is a potentially useful tool for estimating and monitoring the classification of occupational dysfunction.

Conclusion
Overall, the study findings suggest that CAOD is a valid, reliable assessment for assessing occupational dysfunction in undergraduate students. CAOD demonstrates valid psychometric properties for measuring occupational dysfunction, and can be utilized for preventive occupational therapy.