Psychometric evaluation of patient assessment of chronic illness care among Korean cancer survivors

Background The Patient Assessment of Chronic Illness Care (PACIC) was developed in the United States to assess the implementation of the Chronic Care Model (CCM)-based intervention from the patient’s perspective. Although the psychometric properties of the PACIC have been reported in other chronically ill patients, it has not been reported in cancer survivors. Our aim was to evaluate the acceptability, validity, and reliability of a Korean version of the PACIC among cancer survivors (K-PACIC-CS). Methods Among 204 cancer survivors at a university-based hospital in South Korea, we performed psychometric evaluation of the K-PACIC-CS according to acceptability (descriptive statistics, missing values, and floor and ceiling effects), validity (confirmative factor analysis [CFA] and convergent validity), and reliability (internal consistency, i.e., Cronbach’s alpha). Results The item response was high (missing rate = 0.5%). The floor effect was 3.9%– 43.6% and the ceiling effect was 6.9%– 41.2%. The CFA revealed good indices of fit and confirmed the five structures predetermined in the original version of PACIC. The K-PACIC-CS scores had significant positive relationships with cancer survivors’ self-efficacy and health-related quality of life. The total K-PACIC-CS showed excellent internal consistency (Cronbach’s alpha = .94) and those of the subscales were acceptable (Cronbach’s alpha = .76 -.86). Conclusions This study suggests that the K-PACIC-CS is a valid and reliable instrument for measuring implementation of CCM-based chronic care from the survivor’s perspective.


Study design
This study is a secondary analysis using data from a validation study of Cancer Survivors' Self-Efficacy Scale (CSSES) among 204 patients who had completed their primary cancer treatment [35]. Original data were collected after approval from the Institutional Review Board of Inha University Hospital (No. INHAUH-2017-04-025), and data access and analysis were additionally approved by the same institution for this study (No. INHAUH-2020-05-033).

Participants and procedures
We recruited 204 study participants from a university-based hospital in Korea using the convenience sampling approach. Participants were: 1) at least 18 years old; 2) those diagnosed with any type of cancer and who had completed their primary treatment (surgery, chemotherapy, or radiation); and 3) those able to read and write Korean. Subjects with evidence of metastasis and/or recurrence, or those undergoing cancer treatment were excluded.
Data were collected from June to July 2017 at an outpatient clinic at Inha University Hospital. Two trained researchers selected eligible patients by visiting potential enrollees in the outpatient department and checking their electronic medical records. During visits, the researchers explained the study, invited participation, and asked for consent to participate. After accepted participants voluntarily signed an informed consent form, the researchers distributed the questionnaire, which took about 20 minutes to complete. Detailed information about recruitment and procedures are described elsewhere [35].

Sample-size calculation
Based on factorial validity, we chose a sample-size of 200, which is generally acceptable for a confirmatory factor analysis (CFA) [36]. Thus, we used original data (N = 204) [35].

PACIC and translation process
The original version of PACIC [15] was developed for patients with chronic diseases, that is, hypertension, arthritis, depression, diabetes, asthma, and chronic pain conditions. It demonstrated for a good facial, construct, and concurrent validity [15]. Internal consistency, that is, the Cronbach's alpha was.93 for the overall PACIC and the item range was.77 to.90 [15]. The PACIC includes 20 items, comprising five subscales: Patient Activation (items 1-3); Delivery System Design/Decision Support (items 4-6); Goal Setting/Tailoring (items 7-11); Problem Solving/Contextual (items 12-15); and Follow-up/Coordination (items [16][17][18][19][20] [15]. It uses a 5-point Likert scale ranging from 1 ("never") to 5 ("always"), with higher scores indicating better quality of chronic care from the patient's perspective. Each subscale is scored by averaging items completed within the scale, and the overall PACIC score is the average across all 20 items (Table 2).
In Korea, Kim [34] translated the PACIC into Korean (K-PACIC) using the method recommended by the World Health Organization (WHO) for translation and adaptation of instruments to ensure cross-cultural validity. The process involved a forward-backward translation, discussion by a panel of experts in the field, a back-translation, an original author's review, and a reconciliation. The K-PACIC was validated with patients with hypertension and diabetes, and it showed good validity and reliability. Internal consistency as represented by Cronbach's alpha was.93 [34]. We used the K-PACIC in this study with Kim's permission. After reviewing the K-PACIC items, we revised item 19 (from "Told how my visits with other types of doctors, such as an eye doctor or surgeon, helped my treatment" to "Told how my visits with other types of doctors, such as a psychiatrist or rehabilitation physician, helped my treatment"), reflecting the oncology care setting.

Measures
Cancer Survivors' Self-Efficacy Scale. The 11 items of the Cancer Survivors' Self-Efficacy Scale (CSSES) were developed by Foster and colleagues in the United Kingdom (UK) to measure self-efficacy for self-management behaviors in cancer survivors [6]. Each item had 10 response categories with scores ranging from 1 ("not at all") to 10 ("totally confident")-higher scores indicating higher self-efficacy. The Korean version of the CSSES (CSSES-K) has10 items and consists of two subscales, including self-efficacy for managing health problems (five items) and self-efficacy for seeking help and support (five items). It has been validated and has shown good reliability [35]. In the present study, the Cronbach's alpha was.92.
HRQOL. The RAND 36-item Short-Form Health Survey (SF-36) is a reliable and validated instrument for measuring HRQOL [37]. It consists of 36 items, eight health concepts (physical functioning, bodily pain, role limitations due to physical health problems, role limitations due to personal or emotional problems, emotional well-being, social functioning, energy/fatigue, and general health perceptions), and a single item (perceived change in health). Total scores range from 0 to 100, with higher scores indicating better quality of life. In this study, SF-36 was translated according to the guidelines provided by RAND [38]. The Cronbach's alpha was.93.

Statistical analysis
We assessed the psychometric properties of the K-PACIC among cancer survivors (K-PACIC-CS) in terms of acceptability, construct validity, and reliability. We examined acceptability through analysis of data quality using descriptive statistics, including mean and standard deviation, percentage of missing values, and percentage of floor and ceiling effects. The floor and ceiling effects refer to the percentage of respondents using the most extreme (upper or lower) response categories. We considered a frequency less than 30% acceptable [39].
We evaluated the construct validity of the K-PACIC-CS by verifying factorial structure using the CFA and testing convergent validity. We performed the CFA to confirm the factor structure proposed by Glasgow and colleagues [15]. In order to evaluate the model fit, we used the following indices: the comparative fit index (CFI); the normed fit index (NFI); the Tucker Lewis index (TLI); and the root-mean-square error of approximation (RMSEA). If the CFI, NFI, and TLI were greater than.9 and the RMSEA was.05-.08, the results were deemed acceptable [40,41].
In order to test convergent validity, we used Pearson correlation coefficients. We hypothesized that there would be significant positive correlations between the K-PACIC-CS score and the scores of CSSES-K and SF-36. Here, cancer survivors' self-efficacy refers to self-efficacy for self-management behaviors after cancer treatment. According to mapping of the PACIC subscales onto the CCM elements [42], it is conceptually sound to use the PACIC as a measure of self-management support in care quality assessments. Self-efficacy is an important outcome of self-management support intervention [43] and was found to be significantly associated with a patient's perceived self-management support [24,26]. Thus, we expected that the K-PACIC-CS score would be positively correlated with the CSSES-K score.
In order to evaluate reliability, we assessed internal consistency by calculating Cronbach's alpha, considering a value greater than.70 acceptable [44]. In addition, we used descriptive statistics to report participant characteristics and conducted an independent t-test and one-way ANOVA to examine the difference in the K-PACIC-CS scores by participant characteristics. We performed all analyses using the SPSS 25.0 and the AMOS program (SPSS Inc, Chicago, Illinois).

Participant characteristics and the K-PACIC-CS scores
Among 997 potentially eligible participants, 475 did not complete their treatment and 318 refused to participate. Eventually, 204 subjects participated in this study. Table 1 shows participant characteristics and the difference in the K-PACIC-CS scores. The mean age of participants was 54.2 (SD = 11.1) years. Majority of our sample was women (87.7%) and most common types of cancer were breast (75.1%) and gastrointestinal (10.5%) cancers. Almost all (96.6%) had undergone surgery, 66.7% had undergone radiation therapy, and 70.1% had undergone chemotherapy.
Men reported significantly higher K-PACIC-CS scores than women (p = .006); thyroid cancer survivors had the lowest K-PACIC-CS scores (p = .021); participants diagnosed with Stage 3 showed significantly higher K-PACIC-CS scores as compared to those with Stages 0, 1, or 2. Participants whose treatment had been completed less than 12 months ago revealed significantly higher K-PACIC-CS scores than those whose treatment had been completed more than 36 months ago (p = .002) ( Table 1). Table 2 presents descriptive statistics of the K-PACIC-CS. The missing rate was only 0.5%, and only Item 5 had a missing value.

Validity
Factorial validity: CFA. We performed the CFA according to the hypothesized factor structure identified by Glasgow and colleagues [15]. Given the large modification index, we added one correlation of error covariance. Among the four indices for model fit, our data fit well for the CFI, the TLI, and the RMSEA, but not for the NFI (Table 3). Convergent validity. The K-PACIC-CS correlated significantly and positively with both the CSSES-K (r = .25, p < .001) and SF-36 (r = .25, p < .001).

Discussion
The PACIC provides a brief patient assessment of the extent to which chronically ill patients report receiving care that is consistent with the CCM [15]. As we consider the application of CCM to Korean cancer survivors, a psychometric evaluation of the K-PACIC-CS is timely and appropriate. Our analyses demonstrated a good fit of the proposed PACIC factorial structure, reasonable convergent validity, and excellent reliability. To the best of our knowledge, this is the first validation study of PACIC for a cancer population, and our findings may contribute to the implementation of CCM-based cancer survivorship care.

PLOS ONE
The mean scores on the K-PACIC-CS in our sample were somewhat higher than those reported in non-cancer populations in the US [15] and Europe [21,22,24,45] because of the high mean scores of the Patient Activation (mean = 3.82) and Delivery System Design/Decision Support (mean = 3.74) subscales, which may be explained by the nature of the oncology treatment environment. A multidisciplinary team (MDT) approach with team decision-making is currently the standard of cancer treatment [45], so cancer patients seem to participate more frequently in treatment decisions compared to other chronic patient populations.
Regarding floor and ceiling effects, the results vary greatly. Although the PACIC suffers overall from floor effects versus ceiling effects [14,21,24,25,45,46], that was not evident in our study. We found similar results for floor effects (range, 3.9%-43.6%; > 30% for five items) and ceiling effects (range, 6.9%-41.2%; > 30% for five items). Consistent with previous findings [14,21,[45][46][47], we found a floor effect in the Follow-up/Coordination subscale. In particular, Item 17 ("Encouraged to attend program in the community that could help me") had a higher frequency in the lowest response category in several studies (30.7%-92.5%) [14,21,[45][46][47], including ours (40.7%). This suggests that the current cancer care delivery systems in the acute and primary care settings are not well-coordinated.

PLOS ONE
We also observed notable floor effects (38.7%-43.6%) in other items (referral to dietician, consultation, and communication related to consultation), which we did not expect because Korea has adopted the MDT approach [48], which involves consultations and referrals. Current MDT practice, however, may be focused on the initial treatment phase, not the survivorship period. According to Maeng and colleagues [48], the MDT focus in Korea is decisionmaking for further treatment planning and prediction of prognosis. Therefore, we should be careful about interpreting a high floor effect in the Follow-up/Coordination subscale when administering the K-PACIC-CS to long-term cancer survivors.
In addition, Item 9 ("Given a copy of my treatment plan.") showed a high floor effect (31.4%). This contrast with the US value may be related to differences in the US and Korean healthcare systems. In Korea, as opposed to the US, the treatment plan developed by the physician is not shared with the patient, who is not even given a copy of it. Instead, patients are given materials that include information about the next visit and test schedules. Many of our participant responses to Item 9 might have related to those materials rather than the treatment plan. In order to clarify the meaning of Item 9, we recommend using the term '자료' (material) instead of '사본' (copy) when it is translated into Korean.
The K-PACIC-CS had a significant relationship with the CSSES-K and the SF-36 as hypothesized, confirming convergent validity. The correlation coefficient (.25 for each), however, was relatively weak compared to earlier studies [15,47]. Generally, r�.4 is regarded as optimal [50]. Previous researchers have used the Patient Self-Activation Scale [15]and the EUROPEP instrument to assess patient satisfaction with medical care in the primary care setting [47]. Compared to those tools, self-efficacy for self-management and the HRQOL seem to be more comprehensive. This is a limitation of secondary analysis studies; therefore, further investigation is required.
We provided data of factors associated with the K-PACIC-CS, which could verify known groups validity. Due to lack of prior evidence, we could not determine known groups. We suggest cancer stage (early vs advanced) and time since completion of treatment (short-term survivorship phase vs long-term) as a known group. In the current study, survivors with advanced stage cancer showed higher K-PACIC-CS scores than those in early stages. It is possible that

PLOS ONE
patients with advanced cancer are more likely to be involved in decision-making, problemsolving, goal-setting and adjustment, and follow-up. Regarding time since completion of treatment, survivors in the short-term survivorship phase have more frequent visits scheduled than those in long-term survivorship phase, yielding higher K-PACIC-CS scores.

Study limitations
First, our sample consisted mostly of women treated for breast cancer, restricting the generalizability of our findings. Stratified sampling would have been the best choice for this study. Future research using the K-PACIC-CS should report psychometric properties such as Cronbach's alpha. Second, as this was a secondary analysis study, we tested internal consistency with Cronbach's alpha only in terms of reliability; there is a need to also examine reproducibility such as test-retest reliability. Third, we did not evaluate responsiveness, which could contribute to a more psychometrically sound tool for testing the effects of a CCM-based intervention. Lastly, the low missing-data rate of this study (0.5%) suggests the possibility that research assistants might have helped participants respond if they faced difficulties understanding items (a limitation following from secondary analysis).

Conclusions
The psychometric properties of the K-PACIC-CS among cancer survivors are satisfactory and suggest a promising way to evaluate cancer survivors' perspective of a CCM-based survivorship care.