https://orcid.org/0000-0001-8824-8024
Figures
Abstract
Objective
This study examined the reliability and validity of a Shame and Stigma Scale (SSS) and assessed shame and stigma among patients with facial disfigurement from nasopharyngeal carcinoma (NPC).
Methods
Data were collected from 218 patients with NPC through a cross-sectional survey between January 14, 2020, and December 1, 2020. The original SSS is a 20-item scale with four dimensions (i.e., shame with appearance, sense of stigma, regret, and social/speech concern). We used Cronbach’s alpha and McDonald’s omega to assess reliability and exploratory factor analysis (EFA) to assess the factor structure. We also used Pearson correlation analysis to examine the relationship between each item and total score of scale items and convergent validity.
Results
The final 18-item SSS had a Cronbach’s alpha coefficient of .89. The EFA revealed that the SSS has a four-factor structure: sense of stigma, social/speech concern, shame with appearance, and regret. These factors showed satisfactory reliability, with McDonald’s omega coefficients of .87, .77, .86, and .79, respectively. The scale showed significant relationship between each item and total score of scale items with respect to item–total correlations, item–subscale correlations, and item–other-subscale correlations. Convergent validity was supported by the significant positively correlated with the total scores for depression and anxiety.
Citation: Cai Y, Zhang Y, Cao W, Hou F, Xin M, Guo VY, et al. (2022) Preliminary validation of the Chinese version of the Shame and Stigma Scale among patients with facial disfigurement from nasopharyngeal carcinoma. PLoS ONE 17(12): e0279290. https://doi.org/10.1371/journal.pone.0279290
Editor: Meng Li, The University of Texas MD Anderson Cancer Center, UNITED STATES
Received: March 31, 2022; Accepted: December 4, 2022; Published: December 22, 2022
Copyright: © 2022 Cai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because of the privacy implications and some restrictions imposed by the Ethics Committee of Sun Yat-sen University, but is available upon resonable request. Data requests may be sent to Research Data Deposit of Sun Yat-sen University at yjsinfo@mail.sysu.edu.cn.
Funding: This research was funded by the Sun Yat-Sen University Young Teacher Incubation Foundation of China, grant number Ref#20ykpy84. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Several studies have indicated that patients with nasopharyngeal carcinoma (NPC) have more psychiatric symptoms and poorer mental health than the general population [1]. Li found that in a study of 267 patients with NPC, the incidence rate for anxiety and depression were 35.2% and 25.5% respectively [2], which was higher than in the general population (4.4%) [3]. McDowell revealed that in a study of 107 patients with NPC aged 32 to 81, survivors of NPC experience many physical symptoms such as depression (25%), anxiety (37%), and fatigue (28%) [4]. It was reported that over 30% of cancer survivors had been subjected to negative attitudes and stereotypical views toward cancer, and approximately 10% of patients had experienced social discrimination due to cancer [5]. Given the severity of their poor mental health, patients with head and neck cancer (HNC) had higher risk of suicide than in the general population [6].
Shame is an affective state in which a sense of disgrace, dishonor, or humiliation may generate a desire to cover oneself or hide to escape [7]. Stigma is a state of social disapproval that results from a tarnished identity or disfigurement [8]. Previous studies have suggested that shame and stigma are significantly associated with psychological distress [9], poor mental health [10], passive disease coping behaviors, and poor social functioning [10] among patients with lung cancer [11] and breast cancer [9]. One meta-analysis showed that shame and stigma in patients with cancer were associated with poor mental health and other health outcomes such as increased depression (z value, .43, 95% confidence interval (CI), .32 to .54), increased anxiety (z value, .45, 95% CI, .23 to .68), severe impact on body image (z value, .64, 95% CI .29 to .99), and lower medical satisfaction (z value, -.33, 95% CI -.61 to -.06) [12]. Shame and stigma in patients with HNC are related to bodily disfigurement, isolation, and social resistance. Facial disfigurement is considered the single most stressful aspect of HNC [13]. Disfigurement has been associated with greater stigma, defined as shame, social isolation, fear of negative evaluation by others, or being the target of inappropriate behavior [14]. Further studies of cancer-related stigma are urgently needed.
There were some validated assessment tools of stigma and shame for lung cancer [15, 16] and breast cancer [17]. However, due to the complexity and different clinical features of different cancers, the assessment tool should be tailored accordingly. For example, stigma and shame scale for lung cancer has a specific factor to assess patients’ beliefs and stigma regards to smoking behaviors which is often associated with lung cancer [15]. While stigma in patients with breast cancer [18] and HNC patients [19] may be caused by altered body image. The Shame and Stigma Scale (SSS) developed by David W. Kissane is specific to patients with HNC. The item pool was identified from a review of the literature on shame and stigma in patients with bodily disfigurement and patients stigmatized by contagious diseases [19]. The scale has been translated and culturally adapted into a Portuguese version in Brazil and a Chinese version in Taiwan to measure shame and stigma among HNC patients.
There is very limited research on stigma among HNC patients and no researchers have yet investigated stigma associated with NPC. Local research on stigma is in its infancy. The SSS has shown good reliability in measuring the stigma of HNC patients. However, it is unclear whether it can be applied to NPC patients. It is also unclear if the Chinese version of SSS can be used to assess stigma in the Chinese population. This study aimed to validate the SSS in mainland China. We believe the results of this study will help to understand shame and stigma after treatment and to develop interventions that can improve mental health among patients with NPC.
Materials and methods
Study participants and recruitment
Ethical approval was obtained from the Ethics Committee of Sun Yat-sen University (No.2019-145). Data were collected through a cross-sectional survey conducted at Sun Yat-sen University Cancer Center (Guangzhou, China) between January 14, 2020, and December 1, 2020. The inclusion criteria for participants were: 1) aged 18 years or more; 2) documented diagnosis of NPC; and 3) no previous psychiatric disorders and other severe malignancies. The single exclusion criterion was inability to complete the questionnaire independently. Research assistants were recruited and trained to identify eligible patients. The research assistants provided study details to patients in the outpatient clinic and obtained written consent before the conduct of face-to-face interviews. Each interview lasted for 15–25 minutes. The quality of responses to the questionnaire was checked upon completion. We calculated the sample size according to the subject to item ratios of 10:1 [20]. There were 20 items in the original scale, thus the targeted sample size is 200. A total of 218 participants were included in the analysis.
Descriptive statistics
Of the participants (N = 218), the age ranged from 20 to 74 (M = 47.34, SD = 11.39); 69.7% were male; 59.2% were older than 45 years; 56.0% lived in city; 89.9% were married; 55.5% had attended high school or higher; 47.7% had a job; 28.9% had an average monthly household income of RMB 3,001–5,000; 96.8% had medical insurance; 9.2% were diagnosed with NPC more than 5 years previously; and 63.3% had stage III NPC (Table 1).
Measurements
The Chinese version of the SSS.
The scale translation and adaptation procedures were as follows [21]: 1) a research panel of experts in NPC and psychology reviewed and discussed the item pool; 2) two translators translated the scale into Chinese; 3) we invited experts fluent in both Chinese and English to validate the translation and adapt it to the cultural context of China; 4) we did back translation and compare the back-translated versions of the scale with the original; 5) a draft version of the scale was pretested on 19 patients. The original version of the scale measured 4 dimensions, with 8 items (items 1–8) on shame with appearance, 6 items (items 9–14) on social avoidance, 3 items (items 15–17) on a sense of being avoided by others, and 3 items (items 18–20) on general regret at past behaviors. The items were scored from 0 (never) to 4 (always). Higher scores indicated more shame and stigma, which were anticipated in patients with NPC. The Cronbach’s alpha for the overall scale was .93 and the subscale alphas were .92 in shame with appearance, .89 in sense of stigma, .78 in regret, .78 in social/speech concerns respectively in an initial study [19].
The Chinese version of Generalized Anxiety Disorder 7-item scale (GAD-7).
This 7-item self-report scale was developed for screening symptoms of generalized anxiety in the primary care setting [22] and reflects the frequency of symptoms during a 2-week period [23]. Patients with GAD-7 scores of ≥ 10 are classified as at least “moderately ill” [23]. The possible range of scores is from 0 to 21, with a higher score indicating more severe levels of anxiety [23]. The Chinese version of GAD-7 has been shown to have good reliability and validity [24]. The Cronbach’s alpha was .94 in this study.
The Chinese version of Patient Health Questionnaire (PHQ-9).
This 9-item self-report version of the Primary Care Evaluation of Mental Disorders interview utilizes the Diagnostic and Statistical Manual of Mental Disorders fourth edition (DSM-IV) diagnostic criteria for the psychiatric diagnosis of major depression and has been well-validated in primary care and cancer populations [25, 26]. A score of ≥ 10 has good specificity for DSM-IV criteria for major depressive disorder. The possible range of scores is from 0 to 27, with higher scores indicating more severe levels of depression [25]. The PHQ-9 has been widely validated in many Chinese populations with good reliability and validity [27]. The Cronbach’s alpha was .91 in this study.
Statistical analysis
All statistical analyses were conducted using IBM Statistical Package for the Social Sciences (SPSS) version 20.0 (IBM Corp., Armonk, NY, USA). The statistical significance level was a two-tailed value of p < .05. Means and standard deviations (SD) were presented for all items. The floor effect (at the very low end of the scale) and ceiling effect (at the very high end of the scale) were tested to assess if there is limitation in its responsiveness to clinical changes. An item was considered non-responsive if more than 70% of the responses were at the very low or very high end of the scale [28]. We used pearson correlation to determine item–total correlations, item–subscale correlations, and item–other-subscale correlations within the SSS. Cronbach’s alpha coefficients and McDonald’s omega were computed to measure the reliability related to internal consistency.
We used the Kaiser–Meyer–Olkin (KMO) and Bartlett’s sphericity tests to assess sampling adequacy and the appropriateness of the factor analysis. Exploratory factor analysis (EFA) was conducted to measure the factor structure. We used the principal component method for extraction and oblique rotation method. The initial criterion for retaining a factor was an eigenvalue higher than 1.0.
Pearson correlation coefficients were also derived between the SSS (total scores and subscale scores) and external variable (total scores of GAD-7 and PHQ-9) to assess the convergent validity of the SSS.
Results
The mean total score for the SSS was 21.89 (SD = 12.08), with a range of 0 to 69. The percentages of very high scores ranged from .9% to 25.7%, while the percentages of very low scores ranged from 10.1% to 67.9%. There were no ceiling or floor effects. For the individual items, the mean score of item 7 (2.36) was the highest and the mean score of item 3 (.59) was the lowest (Table 2).
Factor structures
Exploratory factor analysis.
In EFA, the eigenvalues of all factors exceeded 1.0. We optimized the scale based on the result of EFA. Items 1 and 4 were deleted as they duplicated the meaning of item 3. This improved the overall value of Cronbach’s alpha from .873 to .886, so items 1 and 4 were excluded before final EFA. The KMO (.902) and Bartlett’s test (χ2 = 2028.174, df = 153, p < .0001) indicated good performance of the factor analysis. Four factors were derived for this scale, leaving 18 items in the final scale.
The four factors of the SSS for NPC were labeled as follows: 1) sense of stigma, 2) social/speech concern, 3) shame with appearance, and 4) regret. The eigenvalues of these factors were 7.50, 1.75, 1.38, 1.12, respectively, and explained 41.7%, 9.7%, 7.7%, 6.2% of the total variance (65.3%), respectively (Table 3).
Item analysis and internal consistency.
The item–total correlation coefficients ranged from .081 to .792 (Table 4). The item–subscale correlation coefficients ranged from .669 to .843 (all p < .01, Table 4). The item–other-subscale correlation coefficients ranged from -.184 to .626 (Table 4). All item–subscale correlation coefficients were higher than the correlation coefficients between the same item and other subscales. The Cronbach’s alpha coefficient was .89 and ranged from .51 to .90 for the four subscales. The McDonald’s omega coefficient was .78 and ranged from .77 to .87 for the four subscales (Table 3).
Convergent validity.
The total scores from the SSS were positively correlated with the total scores for depression (r = .507, p < .01) and anxiety (r = .475, p < .01). Significant positive correlations were also found between scores for three subscales (sense of stigma, shame with appearance, regret) of the SSS and scores for depression (r = .328-.530, p < .01) and anxiety (r = .290-.506, p < .01) (Table 5).
Discussion
This is the first study in mainland China to validate the SSS among a sample of NPC patients. This study adapted the SSS in a Chinese sample with good reliability and structural validity and was tailored to the clinical features of NPC. The scale may have the potential to be widely used to assess stigma and shame among Chinese NPC patients in the future psychosocial researches. The results of this study will help to understand shame and stigma after treatment and to develop interventions that can improve mental health among patients with NPC. The internal consistency measured by Cronbach’s alpha coefficient was .89 and ranged from .51 to .90 for the four subscales. The McDonald’s omega was .78 and ranged from .77 to .87 for the four subscales. The convergent validity of the SSS was supported by its significant correlations with depression and anxiety. We did not see any differences in stigma among NPC patients with different demographic characteristics.
The SSS showed satisfactory internal consistency. The Cronbach’s alpha in our study was .89, which is in the optimal range for internal consistency and consistent with the original scale from Kissane (Cronbach’s alpha = .93) [19]. Kissane’s study was based on a sample of patients with squamous cell carcinoma of the oral cavity, which is different from NPC. In our study, items 1 and 4 were deleted to improve the internal consistency of the scale before factor analysis. Both items had reversed wording and it has been reported that reversed items may weaken scale validity [29]. The remaining items were grouped into four dimensions by factor analysis, including factor 1 (sense of stigma), factor 2 (social/speech concern), factor 3 (shame with appearance), and factor 4 (regret). Based on the result of the EFA, item 7 (“I enjoy going out in public”) was assigned to the social/speech concern subscale in this study rather than to shame with appearance as in the original scale. Considering the results of EFA and the meaning of item 7, it may be more appropriate to classify it into the subscale of social/speech concern. Convergent validity was confirmed by the positive correlations between SSS scores and scores from depression and anxiety scales. This is consistent with findings from previous studies [19]. We found good internal consistency for the SSS. The correlations show that visible disfigurement including scarring, hair loss, sunburn, and other facial disfigurements due to radio-chemotherapy among NPC patients can lead to low self-esteem, loss of self-confidence, regret of treatment, and shame and stigma. These result in negative psychological impacts such as depression, anxiety, and social avoidance [30–33].
We acknowledge several limitations. First, due to the small sample size, we did not conduct confirmatory factor analysis to verify the structure of the SSS and we encourage future studies to apply the scale and verify its factor structure in China. Second, we only recruited participants from one hospital in Guangzhou. The sample may not represent the general population appropriately due to differences in demographic characteristics and clinical severity. And we expect studies would apply the SSS with a more representative sample of Chinese population. Furthermore, this research was primarily focused on the structural validity of the Chinese version of SSS, predictive validity or concurrent validity were not assessed due to lack of gold standard. Finally, the scale lacked sufficient relationship between the total scale and the subscale of social concern. Though its low correlations with total and other subscales (< .3) may suggest potential removal [34], we kept it in the final scale considering that the potential domain-relevant contributions of the items may supersede the absolute correlation values [35]. Given the limitations, the results should be interpreted with caution. Nevertheless, this study carries several implications for further research and current practice.
Conclusions
Our preliminary evidence suggested that the SSS for NPC is valid and reliable, and can provide a reasonable assessment of shame and stigma in NPC patients. However, its application will require additional modifications and validation efforts among more representative samples. Further research is warranted to modify and test the present SSS in a larger and more diverse sample of patients with NPC.
Supporting information
S1 Table. The Chinese version of Shame and Stigma Scale.
https://doi.org/10.1371/journal.pone.0279290.s001
(DOCX)
Acknowledgments
The authors would like to thank all participants who took the time to contribute to this study. The authors would also like to thank doctors in the field for their assistance in the recruitment and screening of participants.
References
- 1. Chen Y, Chen W, Yang Y, Zhao Y, Yang X. Characteristics of symptom distress in Chinese nasopharyngeal carcinoma patients and its relation to mood disturbance: A cross-sectional study. European journal of cancer care. 2019;28(4):e13032. pmid:30859638
- 2. Li R, Su L, Hua Y, Ye JR, Song XR, Tian J, et al. Anxiety and depression status prior to radiotherapy in patients with nasopharyngeal carcinoma and its effect on acute radiation toxicities. European journal of cancer care. 2021;30(6):e13487. pmid:34219307
- 3. WHO. Depression and Other Common Mental Disorders: Global Health Estimates 2017. Available from: https://www.who.int/publications/i/item/depression-global-health-estimates.
- 4. McDowell LJ, Rock K, Xu W, Chan B, Waldron J, Lu L, et al. Long-Term Late Toxicity, Quality of Life, and Emotional Distress in Patients With Nasopharyngeal Carcinoma Treated With Intensity Modulated Radiation Therapy. International journal of radiation oncology, biology, physics. 2018;102(2):340–52. pmid:30191868
- 5. Cho J, Choi EK, Kim SY, Shin DW, Cho BL, Kim CH, et al. Association between cancer stigma and depression among cancer survivors: a nationwide survey in Korea. Psycho-oncology. 2013;22(10):2372–8. pmid:23784964
- 6. Zaorsky NG, Zhang Y, Tuanquin L, Bluethmann SM, Park HS, Chinchilli VM. Suicide among cancer patients. Nature communications. 2019;10(1). pmid:30643135
- 7.
Lewis H. Shame and guilt in neurosis. New York: International University Press; 1971. p 63.
- 8.
Goffman E. Stigma: notes on the management of spoiled identity. New Jersey: Prentice Hall; 1963.
- 9. Warmoth K, Wong CCY, Chen L, Ivy S, Lu Q. The role of acculturation in the relationship between self-stigma and psychological distress among Chinese American breast cancer survivors. Psychology, health & medicine. 2020;25(10):1278–92. pmid:32174167
- 10. Else-Quest NM, LoConte N, Schiller JH, Hyde JS. Perceived stigma, self-blame, and adjustment among lung, breast and prostate cancer patients. Psychology & health. 2009;24(8):949–64. pmid:20205038
- 11. Webb LA, McDonnell KK, Adams SA, Davis RE, Felder TM. Exploring Stigma Among Lung Cancer Survivors: A Scoping Literature Review. Oncology nursing forum. 2019;46(4):402–18. pmid:31225843
- 12. Huang Z, Yu T, Wu S, Hu A. Correlates of stigma for patients with cancer: a systematic review and meta-analysis. Supportive care in cancer: official journal of the Multinational Association of Supportive Care in Cancer. 2021;29(3):1195–203. pmid:32951087
- 13. Katz MR, Irish JC, Devins GM, Rodin GM, Gullane PJ. Reliability and validity of an observer-rated disfigurement scale for head and neck cancer patients. Head & neck. 2000;22(2):132–41. pmid:10679900
- 14. Hagedoorn M, Molleman E. Facial disfigurement in patients with head and neck cancer: the role of social self-efficacy. Health psychology:official journal of the Division of Health Psychology, American Psychological Association. 2006;25(5):643–7. pmid:17014282
- 15. Cataldo JK, Slaughter R Fau—Jahan TM, Jahan Tm Fau—Pongquan VL, Pongquan Vl Fau—Hwang WJ, Hwang WJ. Measuring stigma in people with lung cancer: psychometric testing of the cataldo lung cancer stigma scale. Oncology nursing forum. 2011;38(1):E46–E54. pmid:21186151
- 16. Hamann HA, Shen MJ, Thomas AJ, Craddock Lee SJ, Ostroff JS. Development and Preliminary Psychometric Evaluation of a Patient-Reported Outcome Measure for Lung Cancer Stigma: The Lung Cancer Stigma Inventory (LCSI). Stigma and health. 2018;3(3):195–203. pmid:30393760
- 17. Daryaafzoon M, Amini-Tehrani MA-OX, Zohrevandi Z, Hamzehlouiyan M, Ghotbi A, Zarrabi-Ajami S, et al. Translation and Factor Analysis of the Stigma Scale for Chronic Illnesses 8-Item Version Among Iranian Women With Breast Cancer. Asian Pacific journal of cancer prevention: APJCP. 2020;21(2):449–55. pmid:32102523
- 18. Jin R, Xie T, Zhang L, Gong N, Zhang J. Stigma and its influencing factors among breast cancer survivors in China: A cross-sectional study. European journal of oncology nursing: the official journal of European Oncology Nursing Society. 2021;(52):101972. pmid:33991869
- 19. Kissane DW, Patel SG, Baser RE, Bell R, Farberov M, Ostroff JS, et al. Preliminary evaluation of the reliability and validity of the Shame and Stigma Scale in head and neck cancer. Head & neck. 2013;35(2):172–83. pmid:22368160
- 20. Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation. 2005;10(1):7.
- 21. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value in health: the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2005;8(2):94–104. pmid:15804318
- 22. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Archives of internal medicine. 2006;166(10):1092–7. pmid:16717171
- 23. Mossman SA, Luft MJ, Schroeder HK, Varney ST, Fleck DE, Barzman DH, et al. The Generalized Anxiety Disorder 7-item scale in adolescents with generalized anxiety disorder: Signal detection and validation. Annals of clinical psychiatry: official journal of the American Academy of Clinical Psychiatrists. 2017;29(4):227–34A. PubMed Central PMCID: PMC5765270. pmid:29069107
- 24. Gong Y, Zhou H, Zhang Y, Zhu X, Wang X, Shen B, et al. Validation of the 7-item Generalized Anxiety Disorder scale (GAD-7) as a screening tool for anxiety among pregnant Chinese women. Journal of affective disorders. 2021;(282):98–103. pmid:33401129
- 25. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999;282(18):1737–44. pmid:10568646
- 26. Diez-Quevedo C, Rangil T, Sanchez-Planell L, Kroenke K, Spitzer RL. Validation and utility of the patient health questionnaire in diagnosing mental disorders in 1003 general hospital Spanish inpatients. Psychosomatic medicine. 2001;63(4):679–86. pmid:11485122
- 27. Wang W, Bian Q, Zhao Y, Li X, Wang W, Du J, et al. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. General hospital psychiatry. 2014;36(5):539–44. pmid:25023953
- 28. Cao W, Mo PK, Lau JT. Validation of the Outcome Expectancy Scale for HIV Serostatus Disclosure to Female Sex Partners Among Men Who Have Sex with Men and Women Living with HIV in China. Journal of sex & marital therapy. 2019;45(7):604–17. pmid:30912477
- 29. Chen SL, Tsai JC, Lee WL. Psychometric validation of the Chinese version of the Illness Perception Questionnaire-Revised for patients with hypertension. Journal of advanced nursing. 2008;64(5):524–34. pmid:19146521
- 30. Arunachalam D, Thirumoorthy A, Devi S, Thennarasu . Quality of Life in Cancer Patients with Disfigurement due to Cancer and its Treatments. Indian journal of palliative care. 2011;17(3):184–90. pmid:22346042
- 31. Rumsey N, Clarke A, Musa M. Altered body image: the psychosocial needs of patients. British journal of community nursing. 2002;7(11):563–6. pmid:12447117
- 32. Rumsey N, Clarke A, White P. Exploring the psychosocial concerns of outpatients with disfiguring conditions. Journal of wound care. 2003;12(7):247–52. pmid:12894695
- 33. Williamson D, Gonzalez M, Finlay AY. The effect of hair loss on quality of life. Journal of the European Academy of Dermatology and Venereology: JEADV. 2001;15(2):137–9. pmid:11495520
- 34. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Frontiers in public health. 2018;(6). pmid:29942800
- 35. Cappelleri JC, Jason Lundy J, Hays RD. Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical therapeutics. 2014;36(5):648–62. pmid:24811753