https://orcid.org/0000-0002-4015-0578
Figures
Abstract
Noting concerns about the non-clinical efficacy of the Beck Hopelessness Scale (BHS), specifically the instrument’s ability to discriminate between lower levels of hopelessness, this paper describes the development of the General Hopelessness Scale (GHS) for use with general samples. Following a literature review an item pool assessing the breadth of the hopelessness construct domain was created. This was then placed in survey form and assessed within two independent studies. Study 1 (N = 305, 172 women, 133 men, Mage = 28.68) explored factorial structure, item performance, and convergent validity of the GHS in relation to standardised measures of self-esteem and trait hopelessness. In Study 2 (N = 326, 224 women, 102 men, Mage = 26.52), scrutiny of the GHS occurred using confirmatory factor analysis and invariance tests, alongside item performance and convergent validity analyses relative to measures of affect, optimism, and hope. Factor analysis (using minimum average partial correlations and exploratory factor analysis) within Study 1 revealed the existence of four dimensions (Negative Expectations, Hope, Social Comparison, and Futility), which met Rasch model assumptions (i.e., good item/person fit and item/person reliability). Further psychometric assessment within Study 2 found satisfactory model fit and gender invariance. Convergent validity testing revealed moderate to large associations between the GHS and theoretically relevant variables (self-esteem, trait hopelessness, affect, optimism, and hope) across Study 1 and 2. Further examination of performance (reliability and ceiling and floor effects) within Study 1 and 2 demonstrated that the GHS was a satisfactory measure in non-clinical settings. Additionally, unlike the BHS, the GHS does not assume that administrators are trained professionals capable of advising on appropriate interventions.
Citation: Drinkwater K, Denovan A, Dagnall N, Williams C (2023) The General Hopelessness Scale: Development of a measure of hopelessness for non-clinical samples. PLoS ONE 18(6): e0287016. https://doi.org/10.1371/journal.pone.0287016
Editor: Karl Bang Christensen, University of Copenhagen, DENMARK
Received: May 26, 2022; Accepted: May 30, 2023; Published: June 26, 2023
Copyright: © 2023 Drinkwater et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data files are available from figshare: https://figshare.com/s/cf5ae31bcc0b04d94b91 https://figshare.com/s/91e7372ea710191a9d90.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
This paper validated the General Hopelessness Scale (GHS). This novel, theoretically informed measure was produced to assess hopelessness in general populations. The advancement of the GHS was necessary because the current predominant measure, Beck’s Hopelessness Scale (BHS [1]), originated from work with clinical samples. Accordingly, critics assert that the BHS is not able to effectively distinguish between lower levels of hopelessness [2]. Acknowledging this concern and the fact that non-clinical populations typically possess relatively low levels of hopelessness the authors developed the GHS.
Hopelessness denotes the tendency to overestimate adverse events and underestimate the probability of positive occurrences [3]. This is characterised typically as negative expectation or pessimistic attitude toward self and/or future [4]. Hence, hopelessness expresses as low expectation of goal achievement, reduced belief in probability of success, and feelings of futility [5]. Furthermore, theorists make a distinction between levels of state and trait hopelessness [6]. Trait represents dispositional baseline, whereas state varies as a function of contextual and situation factors (i.e., stressors and mood episode).
Hopelessness is an important construct because it is a principal characteristic of depression and an important etiological and maintaining factor for suicide risk [7–9]. Correspondingly, hopelessness is an integral element of theoretical models of suicide [10]. In this context, the capacity to detect subtle differences at the lower end of the score distribution is important since even moderate levels of hopelessness are indicative of low subjective well-being and psychological difficulties.
Beck Hopelessness Scale (BHS)
The most used measure is BHS [1, 11]. The importance of the instrument is demonstrated by its extensive use in published research and its translation into numerous languages (e.g., Nigerian [12], Urdu [13], Hungarian [14]). Psychometric validation of the BHS identified three factors tapping affect, motivation, and cognition [1]. However, ensuing scrutiny has questioned the validity of this factorial structure [15]. Hence, though several studies have replicated the original model [16–19], others report alternatives solutions comprising one [20–23] to five dimensions [24].
The unidimensional model comprises a single underlying factor with positive and negative item phrasing effects [20–23], whereas two-factor structures reflect negative expectation of the future and perceptions of powerlessness [25–27]. More complex models identify multiple dimensions [24, 28]. Illustratively, Nekanda‐Trepka, Bishop, and Blackburn [28] identified five factors (Motivation and Outcome Expectation, Confidence in the Future, Future Accomplishment, Trust in the Future, and Time Perspective), and Zhang et al. [24] reported a five-factor model for non-suicide attempters (Feelings About the Future, Pessimistic Motivation, Positive Expectation, Negative Expectation, and Future Expectation), and a four-factor model for suicide attempters (Loss of Motivation, Positive Expectation, Negative Expectation, and Future Expectation).
Noting debate about the underlying factorial structure of the BHS, Boduszek and Dhingra [15] tested a range of models using a large student sample (N = 1,733). Analysis revealed that a three-factor solution with method effects provided superior fit. This study was important because it supports the solution proposed by Beck et al. [1] within a non-clinical sample. However, other studies using student samples have failed to reproduce the solution [29, 30]. Moreover Steed [30], following comparisons of BHS with Hope Scale (HS) and Life Orientation Test scores (LOT), concluded that because the HS and LOT were developed to index healthy characteristics and behaviours, they had greater applicability to normal populations than the BHS.
Other authors also note limitations when using the BHS with non-clinical populations. Illustratively, Young et al. [2] observed that the BHS was an inefficient instrument for samples in which the level of hopelessness was expected to be low. Indeed, the BHS works optimally with higher levels of hopelessness, although it can detect changes down to moderately low levels. Acknowledging this, even in clinical samples the BHS may be less sensitive to important changes in depression within the lower range (i.e., patients progressing from improved to totally remitted).
It is important to identify differences within the medium-lower range because scores potentially reveal subsequent psychological states. Thus, the ability to reliably detect differences across the score range, both between groups and within respondents, is crucial. Despite this, recent studies with university-based samples have continued to use the BHS. Adoption of the BHS is typically predicated on the fact that the instrument is generally regarded as the predominant measure of hopelessness [31–34]. Concerning clinical samples, researchers have also noted variations in the predictive validity of the BHS. For example, Niméus, Träskman-Bendz, and Alsén [35] reported that the BHS failed to significantly predict future suicide in hospitalized suicide attempters. Collectively, these findings imply that the BHS is most effective when it is indexing depression ratings, mood disorders and/or personality disorders [35].
Conceptualisation
Noting conceptual concerns about the BHS’s factorial structure and possible non-clinical efficacy, this paper developed an alternative hopelessness measure for use with general, non-clinical populations. That is, for testing within samples where lower levels of hopelessness are anticipated. To ensure that the new instrument appropriately indexed the construct, the authors incorporated core elements from the delineations of Beck and the theory of hopelessness depression [36].
Both approaches represent cognitive-vulnerability-stress models as they centre on the notion that hopelessness develops from negative expectations of future events [37]. In addition, they emphasise the propensity to negative thinking styles, which Beck refers to as dysfunctional attitudes, and Abramson et al. [36] denotes as negative inferences. The interaction between cognitive styles and negative events is theoretically salient through its predictive capacity for depressive symptoms [38].
Furthermore, both explain why ‘some’ individuals develop hopelessness and depression, and others do not. Despite similarities there are important differences between the models. Notably, Beck et al. [1] posits that vulnerability to depression arises from an individual’s universal beliefs and rules for happiness (i.e., those related to perfectionism, performance, and self-worth), whereas Abramson et al. [36] attributed susceptibility to inferences about the cause, consequences, and self-implications of negative life events [39].
Previously, few studies have tried to integrate these accounts. One notable attempt by Pössel and Thomas [40], explored how elements of the cognitive triad assimilated into the model of hopelessness depression. Since mediation occurred only when the whole cognitive triad was included, this was only partially successful. A further attempt by Pössel and Knopf [37] found that inference style was distinct from the depressogenic schemata and cognitive errors element of Beck’s model. Additionally, Pössel and Smith [41] provided support for an integrated model when inferential styles were located between the cognitive errors and cognitive triad. This demonstrated that it was possible to meaningfully combine theoretical accounts.
The present study
To address BHS limitations (i.e., structural variations as a function of sample type and lack of measurement sensitivity within non-clinical samples [16–17, 26]) this paper outlines the development and validation of the General Hopelessness Scale (GHS). The GHS is intended for groups where low scores are anticipated. Although other scales exist, these like the BHS, were designed for use within clinical settings/contexts (i.e., State-Trait Hopelessness Scale [42]; Brief Inventory for Helplessness, Hopelessness and Haplessness [43]; and Hopelessness Scale for Children [44]). Also, they lack the conceptual basis of the BHS. Accordingly, the GHS was developed within a non-clinical sample and scale items derived from core elements of the BHS [1] and Abramsons’s theory [36]. The Hopelessness Depression Symptom Questionnaire (HDSQ, [45]) served also as a reference point for emerging symptoms. The HDSQ is 32-item self-report measure of symptoms that comprise hopelessness depression [36]. The scale serves as a tool for testing the hopelessness theory of depression and acts as a means for assessing individual symptoms in clinical work and research [45].
Acknowledging these theoretically important sources, GHS items index ‘negative perceptions of future events’ (e.g., “If something bad could happen, it probably will”) and the ‘self and one’s own adequacies’, (e.g., “If I don’t get something right after a few tries, I’ll probably never get it right”). Items reference also specific features from cognitive models. Explicitly, ‘attributions for adverse events and consequences’ (e.g., “I often find things to be out of my control”). These reflect the notion that an individual is incapable of taking control of a situation, or that adverse events arise from dispositional factors.
In combination, these components reflect the stability and globality of negative conditions, which Abramson et al. [36] identified as integral to the development of hopelessness. Consistent with this, these items infer or draw normative comparisons with others (e.g., “I doubt I’ll get the things that other people have”). This is reflective of the cognitive distortions of cognitive theory as well as the inferred negative consequences and consensus elements of hopelessness depression theory. Finally, the GHS contains statements with positive valance, which are indicative of a confident outlook (e.g., “I look forward to the future with optimism”). These reflect the hopefulness evident in BHS and benign HDSQ statements.
To test the GHS, the present investigation utilised an established procedure of initially exploring factor structure and item performance (Study 1), before testing the stability of the latent structure with an independent sample (Study 2). Moreover, Study 1 contained the GHS alongside pre-established measures of hopelessness. Study 2 contained measures assessing individuals on adaptive qualities, such as positive affect and hopefulness. This method ensured that the GHS was an accurate and robust index of hopelessness.
Study 1 Materials and methods
Participants
Study 1 comprised 305 participants (172 women, 56%; 133 men, 44%), mean age 28.68 (SD = 10.82), range 18 to 70. These originated mainly from the USA and UK (41% and 25% respectively) (Table 1). Regarding vocational status, 39% of the sample were students, 54% employed and 7% unemployed. Data screening revealed acceptable univariate skewness and kurtosis for study variables (i.e., between -2.0 to +2.0). In addition, non-normality existed, as Mardia’s [46] kurtosis (b2p = 26.19, p < 0.001) and skewness (b1p = 162.19, p < 0.001) inferred significant deviation.
Measures
BHS instrument development.
To create a scale with satisfactory content validity, 31 questions were constructed following examination of extant measures, including the BHS [1] and HDSQ [45]. The semantic content of these scales acted as theoretical reference points to accompany abstract ideas arising from concepts within the cognitive triad theory of depression and the theory of hopelessness depression.
Seventeen items indexed negative expectations of future events (9-items) as well as perceptions of inferiority when comparing the self to others (8-items). Eight items assessed general lack of motivation including unwillingness to proceed (e.g., “I don’t want to do X”) and cognitive dismissal of the outcome (e.g., “There’s no point to doing X”). A further six items referenced hopeful belief and were reverse-keyed. The measure utilised a 7-point Likert scale (1 = Strongly disagree to 7 = Strongly agree).
To ensure item comprehension and face validity the initial iteration of the GHS was reviewed by academics experienced in psychometrics. In addition to the suggestion of minor grammatical and semantic alterations, this process recommended a reworking of an item related to comparing the self to others; the item did not fully reflect other associated statements. Following revisions, the reviewers confirmed that the GHS was suitable for psychometric evaluation. To assess concurrent validity, Study 1 participants completed the Trait element of the State-Trait Hopelessness Scale [42] and the Rosenberg Self-Esteem Scale (RSE) [47].
The Trait element of the State-Trait Hopelessness Scale (THS).
The THS consists of 13 items assessing beliefs and feelings associated with hopelessness, using the phrase ‘typically’ to represent trait hopelessness. Participants answered using a 7-point Likert scale (1 = Strongly agree, 7 = Strongly disagree). Reported reliability is high for the measure (α = .91 [48]). In this study, good alpha and omega reliability existed (α = .93, ω = .93).
The Rosenberg Self-Esteem Scale (RSE).
The RSE is a 10-item measure which uses a 4-point Likert scale (0 = Strongly disagree, 3 = Strongly agree) to assess global self-esteem. The RSE demonstrates excellent internal consistency (α = .92 [47]) with this study showing similarly high reliability (α = .93, ω = .93).
Procedure
Prior to participation, respondents read the study brief. This contained background information about the study and outlined the conditions and requirements of involvement. To participate, respondents provided informed consent and were required to be over 18 years of age. This was the only inclusion criteria. Acknowledgement of consent involved respondents ticking/clicking a box within the online survey hosted by Qualtrics (a web-based online survey tool) indicating that they understood the nature of the study and intended to participate. They were informed that they could cease participation at any point during completion of the measures. Only consenting participants progressed to the online measures. Further instructions asked participants to take their time, complete all questions, and answer openly/honestly. The initial questions requested demographic details (i.e., age, preferred gender, and occupation).
Ethics
The research team gained ethical approval for the project (Developing the General Hopelessness Scale) from the relevant governing/institutional review board (specific details withheld for anonymity).
Analytical strategy
Assessment of the General Hopelessness Scale (GHS) advanced through a series of analytical stages. These involved an initial test of factor structure (MPlus Version 8.6; [49]) via parallel and exploratory factor analysis (EFA), which utilised two criteria for factor extraction: Velicer’s minimum average partial (MAP) test, and an eigenvalue equal to or greater than 1. Velicer’s MAP test computes partial correlations using the covariances among the residuals. Computation of the average squared partial correlation occurs, and the test terminates when this value achieves a minimum result indicating no additional common variance extracted. This is an empirically supported approach for establishing the quantity of factors underlying a measure [50]. Next, Rasch analysis (Winsteps) assessed infit and outfit (MNSQ) for each item, item and person reliability, and item and person separation index. Differential item functioning (DIF) tested response bias among men and women.
Given the presence of non-normality, and ordinal nature of the data, EFA was performed on a polychoric correlation matrix using mean and variance adjusted weighted least squares (WLSMV) estimation. Oblique rotation was applied because correlations among factors were anticipated [51]. Model fit interpretation included chi-square (χ2), Comparative Fit Index (CFI), Root-Mean-Square Error of Approximation (RMSEA), and Standardised Root-Mean-Square Residual (SRMR). RMSEA scrutiny involved reference to its 90% confidence interval (CI). Fit was determined using traditional cut-offs as indicated by the literature [52, 53]. Explicitly, CFI > 0.90, RMSEA < .10, and SRMR < .08 implied satisfactory fit.
Concurrent validity included correlating the GHS with the THS and RSE scales. Cohen’s criteria [54] facilitated interpretation of correlation strength. Specifically, 0.1–0.29 indicated a weak correlation; 0.3–0.49 suggested a moderate relationship; and 0.50 or greater inferred a strong correlation. Cronbach’s alpha and coefficient omega (ω) examined internal consistency of the GHS. Analysis subsequently considered ceiling and floor effects.
Study 1 Results
Exploratory Factor Analysis (EFA).
The MAP test for Study 1 suggested extraction of four factors, which was consistent with the a priori conceptualisation of items into domains of negative expectations, perceptions of inferiority, lack of motivation and an element of hopefulness. The smallest average squared partial correlation was .01, occurring alongside four underlying components. Further assessment revealed satisfactory sampling adequacy; Kaiser-Meyer-Olkin measure (KMO) = .96 and a reasonable item matrix, Bartlett’s Test of Sphericity (p < 0.001).
Consistent with the MAP results, EFA revealed four dimensions possessed eigenvalues > 1. Moreover, the four-factor model revealed satisfactory fit across indices, χ2 (347) = 863.23, p < .001, CFI = .97, RMSEA = .07 (90% CI of .06 to .07), SRMR = .03. Six items loaded below 0.43 [55]. Reanalysis following the removal of these items revealed a more parsimonious solution (i.e., CFI increase of .003, SRMR reduction of .003), χ2 (206) = 569.52, p < .001, CFI = .97, RMSEA = .07 (90% CI of .06 to .07), SRMR = .03. Eight items loaded on Factor 1, six on Factor 2, eight on Factor 3, and three on Factor 4 (Table 2). Factor 1 (labelled as ‘Social Comparison’; Eigenvalue of 12.69) comprised items relating to how someone perceives themselves in an interpersonal spectrum, with hopeless individuals likely perceiving themselves as inferior to others.
Items informing Factor 2 consisted mostly of reversed items and comprised positive valency (named ‘Hope’, Eigenvalue of 2.31). This is like Factor 1 of the Beck Hopelessness Scale, which a user would typically answer as ‘false’ if they were experiencing levels of hopelessness. The third factor (labelled ‘Negative Expectations’, Eigenvalue of 1.32) referred to adverse feelings that individuals have regarding their future. The fourth factor consisted of items that capture an individual’s aversion to initiate or maintain new behaviours (named ‘Futility’, Eigenvalue of 1.01).
All four factors evidenced moderate to large inter-factor correlations. Specifically, Social Comparison demonstrated a large association with Hope (r of -.42), Negative Expectation (r of .57), and Futility (r of .30). Hope evidenced a large negative association with Negative Expectation (r of -.67) and Futility (r of -.42). Negative Expectation also correlated strongly with Futility (r of .37).
Rasch analysis.
Each factor was subjected to Rasch analysis. The properties of most items were satisfactory. However, three items possessed unsatisfactory Infit MNSQ and Outfit MNSQ, and were removed from the scale (see Table 3). Reanalysis indicated that the remaining items possessed satisfactory Infit and Outfit MNSQ between 0.6 and 1.4 [56] inferring a lack of ‘noise’ or randomness within the measure, and no substantial DIF across gender (Table 3). Acceptable item reliability and person reliability (Social Comparison = .92 and .85, Hope = .97 and .91, Negative Expectation = .97 and .89, Futility = .97 and .80) existed for each factor. Moreover, the item separation index and the person separation index (Social Comparison = 3.45 and 2.41, Hope = 6.13 and 3.13, Negative Expectation = 5.89 and 2.83, Futility = 5.27 and 2.06) were all satisfactory, distinguishing high and low ability in the sample [57] and inferring a good spread of items.
Validity and reliability.
Concurrent validity was assessed using the THS and the RSE scale (Table 4). The GHS and its subscales correlated strongly with the THS alongside a large association with the RSE. Using alpha and omega coefficients, reliability for the overall scale was good (α = .94, ω = .95). Social Comparison evidenced good reliability (α = .89, ω = .89). Similarly, Hope was satisfactorily reliable (α = .87, ω = .89), in addition to Negative Expectations (α = .91, ω = .91), and Futility (α = .79, ω = .79).
Ceiling and floor effects.
Examination of GHS score distributions occurred for floor and ceiling effects. Floor effects represent a limitation of a measure whereby the scale cannot determine decreased performance beyond a specific level. Likewise, ceiling effects indicate the opposite extreme [58]. Following the guidelines of Terwee et al. [59], a floor or ceiling effect existed if 15% or more of the participants reported lowest (i.e., 21) or highest (i.e., 147) possible scores. Analysis of the GHS found negligible floor and ceiling effects. Specifically, the lowest score was 21, with 1 (.3%) possessing this. The highest score was 129, also with 1 participant (.3%) indexing this. Negligible floor and ceiling effects also existed for the four factors.
Study 2 Materials and methods
Participants
Study 2 consisted of 326 participants (224 women, 68%; 102 men, 32%), mean age 26.52 (SD = 8.74), range 18 to 71. Participants were typically from the UK (34%). In terms of vocational status, 74% of the sample were students, 24% employed and 2% unemployed (Table 1). Data screening revealed acceptable univariate skewness and kurtosis for study variables, as with Study 1 (i.e., between -2.0 to +2.0). In addition, non-normality existed (kurtosis b2p = 26.96, p < 0.001, skewness b1p = 98.30, p < 0.001).
Measures
Study 2 consisted of a series of antithetical measures. These included the Positive and Negative Affect Scale [60], the Revised Life Orientation Test [61] and the Adult Hope Scale [62]. These measures were chosen for their adversative nature to hopelessness.
The Positive and Negative Affect Scale (PANAS).
PANAS consists of two scales measuring positive and negative affect. Respondents indicate the extent to which they usually experience 20 affects (e.g., ‘jittery’, ‘proud’), using a 5-point Likert scale (1 = very slightly or not at all, 5 = extremely). Scores for each scale are achieved by totalling the positive and negatively valanced words. The present study found good reliability for both scales (PA α = .89, ω = .89; NA α = .88, ω = .88), which is in-line with previous research [60].
The Revised Life Orientation Test (LOT-R).
The LOT-R is a 10-item measure assessing optimism in the form of general expectancy (e.g., ‘In uncertain times, I usually expect the best’) using a 5-point Likert scale (0 = strongly disagree, 4 = strongly agree). The present study found good internal consistency (α = .82, ω = .82).
The Adult Hope Scale (AHS).
The AHS is a 12-item measure of a respondent’s level of hope, which can be divided into two subscales (1) Agency and (2) Pathways. Each item is answered using an 8-point Likert scale ranging from Definitely False to Definitely True. The present study focused on the total scale, which was acceptably reliable (α = .79, ω = .83).
Analytical strategy
CFA (MPlus Version 8.6) examined data-model fit of the superior solution from EFA in an independent sample (Study 2). As with Study 1, due to non-normality and ordinal data, CFA was performed on a polychoric correlation matrix using WLSMV estimation. Satisfactory model fit involved CFI > 0.90, RMSEA < .10, and SRMR < .08.
Measurement invariance was examined by fitting and comparing sequentially nested and increasingly constrained CFA models across gender (men and women). Invariance examined equivalence at the configural (factor structure), metric (factor loadings), and scalar (item intercepts) levels. Invariance was determined by a CFI difference (ΔCFI) ≤ 0.01 and RMSEA difference (ΔRMSEA) ≤ 0.015 [63], or a non-significant change in χ2. Concurrent validity included aligning the GHS with the PANAS, LOT-R and AHS. Cronbach’s alpha and coefficient omega (ω) examined internal consistency prior to consideration of ceiling and floor effects.
Study 2 Results
Confirmatory Factor Analysis (CFA).
A replication of the four-factor model with Study 2 revealed (using CFA) satisfactory fit, χ2 (203) = 778.66, p < .001, CFI = .94, RMSEA = .09 (90% CI of .08 to .10), SRMR = .05. Inspection of standardized parameter estimates indicated that all items loaded above .5 [64] apart from item 9 (loading of .41). The four factors reported large inter-factor correlations with one another (Fig 1).
Note. Ellipses represent latent variables; measured variables are represented by rectangles; error is not shown but was specified for all variables. ** p < .001.
Measurement invariance.
Multi-group analysis comparing men and women supported metric invariance based on the change criteria in fit statistics specified a priori (Table 5): Model 1 vs. Model 2 (ΔCFI = .006, ΔRMSEA = .004). Support also existed for scalar invariance: Model 2 vs. Model 3 (ΔCFI = .004, ΔRMSEA = .014).
Validity and reliability.
Moderate to large correlations occurred between the GHS and its subscales with the PANAS subscales and AHS (Table 6). Large negative correlations existed with the LOT-R measure. The Hope subscale expressed a similarly divergent pattern as witnessed in Study 1, signifying divergent validity. Reliability was good for the overall scale (α = .93, ω = .93), Social Comparison (α = .82, ω = .83), Hope (α = .84, ω = .85), Negative Expectations (α = .89, ω = .89), and Futility (α = .75, ω = .75).
Discussion
Based on Beck’s cognitive theory of depression [65] and Abramson’s theory of hopelessness depression [36], this project developed a novel, psychometrically sound measure of hopelessness for use with general samples. This was necessary because attempts to reproduce the BHS factor structure reported by Beck et al. [1] have produced inconsistent results [e.g., 12, 15, 18, 23, 30]. Additionally, theorists have raised concerns about the ability of the BHS to discriminate between lower levels of hopelessness such as those found in non-clinical samples or improving patients [2, 30]. Indeed, this was only possible when the factor structure was altered to produce a simpler model [11, 17]. Finally, adoption of an integrated approach that combines Beck’s cognitive theory of depression [65] and Abramson’s theory of hopelessness depression [36] opens the possibility for more effective therapeutic interventions in which techniques from different cognitive theories are optimally amalgamated [41]. In this context, the GHS will potentially extend understanding of hopelessness in non-clinical populations and help to facilitate the development of conceptually integrated assessments.
Regarding analysis, EFA identified four factors, which approximated the theoretical underpinnings of items during its construction. Specifically, Negative Expectations, Hope, Social Comparison, and Futility. Rasch analysis and CFA consolidated these factors by informing the removal of underperforming and/or inconsistent items. This, reduced the number of items to 21, strengthened model fit, and improved inter-factor covariance. A concomitant advantage of a shorter emergent measure being enhanced practicality (easier to complete and comprehend). Whilst reliabilities, both for specific factors and overall measure, were good, future research is necessary to establish its efficacy. Nonetheless, the use of a non-clinical sample in the GHS development and the observed concurrent validity with established measures indicated promising scale performance.
Additionally, the study provided information about ceiling and floor effects. These are viewed as problematic when greater than 15% of a sample either produce the highest or lowest available score [59]. Effects of this magnitude did not exist in this study, suggesting that the GHS adequately captured general hopelessness scores. This is a criticism, which scholars have levelled at the BHS [17, 18, 27]. Additional tests of invariance demonstrated satisfactory stability of factor structure, loadings, and intercepts across gender. Thus, analysis specified that the GHS was a psychometrically sound measure of hopelessness.
Conceptually, the synergy of key elements of Beck’s cognitive model of hopelessness [1] and Abramson’s theory of hopelessness depression [36] combined to reflect core characteristics of hopelessness. Specifically, negative, dysfunctional perceptions and a cynical view of the future. Moreover, the GHS assessed these features concomitant with other aspects that contribute towards hopelessness, such as cognitive errors that occur when comparing perceptions of one’s own ability to the seemingly unattainable competence of others. The inclusion of facets of social comparison and futility ensured that the GHS sampled a breadth of construct content.
The inclusion of social comparison stemmed from Abramson’s [36] examples of how hopelessness depression develops. Hence, it encapsulated production of cognitive errors and Beck’s cognitive triad and was commensurate with relationships between social comparison and self-esteem. Particularly, the observation that those who feel comfortable in their perceptions of themselves, in comparison to others, possess positive self-esteem [66]. Accordingly, negative social comparisons can adversely affect self-esteem [67–69] and serve as a precursor of depression, as well as hopelessness. Subsequent studies with the GHS should explore the benefit of this factor.
EFA resulted in the emergence of a nuanced three-item motivation factor “futility”. This reflected aversion to partake in/or maintain behaviours/habits. Explicitly, perceived personal inadequacy and lack/absence of initiative. This interpretation was consistent with the delineation of futility as low self-efficacy and heightened fear of failure (e.g., “I feel that giving up is easier than failing” and “I avoid attempting new things in case I find them difficult”). This emphasis differed from the BHS motivational factor, which focuses on loss of motivation and poor future expectations. Thus, conceptually futility corresponded with negative self-views (cognitive triad) and self-referential style (hopelessness theory). Explicitly, undesirable personal evaluation and internalisation of failure result in lack of motivation and volition [70, 71].
Limitations
Although this study reported that the GHS possessed good psychometric properties it is important to note limitations. One concern was conceptual distinctiveness. This was potentially an issue because of similarities between hopelessness (i.e., the propensity to overestimate adverse events and underestimate the probability of positive occurrences) [3] and locus of control (LOC) (i.e., individual beliefs about control of life events). Accordingly, Rotter [72] defines LOC as the extent to which an individual perceives an outcome as dependent on their own actions or external forces; this is assessed on a continuum from internal to external orientation [73]. Internal LOC views control as personal, whereas external LOC attributes control to external factors. Thus, LOC reflects generalized confidence. Explicitly, experience of the self as either a causal agent (optimism/hope), or a victim of circumstance (pessimism/hopelessness) [74].
In the context of the present paper, Levenson’s [75] reconceptualization of external locus is highly relevant. Levenson divided LOC into control by powerful others, and control by chance or fate. These loci of control represent can be interpreted as fatalism and as contributing to hopelessness [74]. Despite apparent similarities between hopelessness and LOC, it is important to remember that constructs of perceived control (including also self-efficacy [76], learned helplessness [77], and causal attributions [78]) derive from distinct conceptual backgrounds [74]. Correspondingly, studies typically report only moderate positive correlations between hopelessness and LOC [e.g., 79, 80]. Although, the usefulness of cross study comparisons is restricted by investigators’ use of different measurement and samples, it is evident that the two constructs are distinct and reflect different cognitive-perceptual processes.
Explicitly, external LOC denotes a general thinking style that captures elements of hopelessness, which is an extreme reaction to adverse circumstances. Commensurate with this interpretation, Lefcourt [81] drawing on Richter [82], delineates hopelessness as giving up when all avenues of escape appear to be closed and the future holds no hope. Furthermore, consistent with the dimensions identified in this paper, hopelessness embodies pessimism about the future (lack of hope and negative expectations) and resignation to the ineffectiveness of changing the future (futility) [83]. The GHS suggests that these judgments derive from both internalised representations of the self and comparisons with others (social comparison). To extend scholarly understanding and assess the validity of these assumptions future research with the GHS should explore relationships between dimensions of hopelessness and LOC.
A limitation of the present study was the failure to concurrently evaluate the GHS within a clinical sample. Future research should undertake this comparison to ensure that the GHS performs well across different settings. Additionally, assessment of the GHS alongside the BHS is required. This was not possible in the current report due to financial restrictions and sampling constraints (i.e., lack of access to clinical sample). This would establish that the GHS (vs. BHS) is more able to discriminate hopelessness scores in the low to moderate response range. This could also inform the development of more sensitive interventions and recovery monitoring strategies.
Though the present paper demonstrated the validity and internal reliability of the GHS, further research is required to verify the measure’s external reliability (i.e., temporal stability). Hence, subsequent psychometric evaluation should assess the test-retest reliability of the GHS. This is important because external reliability shows that participant scores are consistent and reproduced under comparable conditions on independent occasions [84]. Hence, test-retest reliability is integral to scale performance since it ensures the effective measurement of hopelessness over time.
Moreover, test-retest reliability complements validity by verifying measurement stability. Consequently, in the context of mental health-related outcomes test-retest reliability is necessary to the assessment of treatment and intervention efficacy. A vital consideration when establishing test-retest is the gap between scale completions. The interval needs to be sufficient to demonstrate adequate stability, but not too long for substantial attrition to occur, or allow extrinsic factors to influence scores. Hence with health measures such as the GHS, intervals of 1–2 weeks are typically employed [85]. Although, interval length is often constrained by practicality and subsequently, validation papers often use durations of opportunity [86]. Regardless of period between scale completions, establishing test-retest will further enhance the psychometric properties of the GHS. This is especially necessary since reliability across multiple times, contexts, and users is generally poorly reported within psychological literature [87].
In terms of measurement development, future research could produce an abridged version of the GHS (i.e., 1–3 items per dimension) for use in big panel surveys. Within large test batteries brevity increases end-user accessibility, by reducing survey length, concomitantly facilitating engagement. Additionally, shorter, refined measures reduce the cognitive load placed on respondents and are accordingly less likely to result in fatigue. The development of a short version of the GHS, however, is a long iterative process that should only occur after the measure is thoroughly evaluated and refined. Particularly, it is important to ensure that any emergent scale adequately samples construct breadth.
Acknowledgments
We would like to thank the participants of the study, without whom this study would not have been possible.
References
- 1. Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: The Hopelessness Scale. Journal of Consulting and Clinical Psychology. 1974 Dec;42(6):861–5. pmid:4436473
- 2. Young MA, Halper IS, Clark DC, Scheftner W, Fawcett J. An item-response theory evaluation of the Beck Hopelessness Scale. Cognitive Therapy and Research. 1992 Oct;16:579–87. https://psycnet.apa.org/doi/10.1007/BF01175143
- 3.
Beck AT, Alford BA. Depression: Causes and Treatment. University of Pennsylvania Press; 2014 Apr 4.
- 4. Serafini G, Lamis DA, Aguglia A, Amerio A, Nebbia J, Geoffroy PA, et al. Hopelessness and its correlates with clinical outcomes in an outpatient setting. Journal of Affective Disorders. 2020 Feb 15;263:472–9. pmid:31969280
- 5. Hirsch JK, Visser PL, Chang EC, Jeglic EL. Race and ethnic differences in hope and hopelessness as moderators of the association between depressive symptoms and suicidal behavior. Journal of American College Health. 2012 Feb 1;60(2):115–25. pmid:22316408
- 6. Horwitz AG, Berona J, Czyz EK, Yeguez CE, King CA. Positive and negative expectations of hopelessness as longitudinal predictors of depression, suicidal ideation, and suicidal behavior in high‐risk adolescents. Suicide and Life‐Threatening Behavior. 2017 Apr;47(2):168–76. pmid:27371943
- 7. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer R. Relationship Between Hopelessness and Ultimate Suicide: A Replication With Psychiatric Outpatients. The American Journal of Psychiatry. 1990 Feb;147:190–5. pmid:2278535
- 8. Hirsch JK, Hall BB, Wise HA, Brooks BD, Chang EC, Sirois FM. Negative life events and suicide risk in college students: Conditional indirect effects of hopelessness and self-compassion. Journal of American College Health. 2021 Jul 4;69(5):546–53. pmid:31765290
- 9. Ivanoff A, Jang SJ. The role of hopelessness and social desirability in predicting suicidal behavior: A study of prison inmates. Journal of Consulting and Clinical Psychology. 1991 Jun;59(3):394–9. pmid:2071724
- 10. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE Jr. The interpersonal theory of suicide. Psychological Review. 2010 Apr;117(2):575–600. pmid:20438238
- 11. Kliem S, Lohmann A, Mößle T, Brähler E. Psychometric properties and measurement invariance of the Beck hopelessness scale (BHS): Results from a German representative population sample. BMC Psychiatry. 2018 Dec;18:110. pmid:29699522
- 12. Aloba O, Esan O, Alimi T. Adaptation of the Beck Hopelessness Scale as a suicide risk screening tool among Nigerian patients with schizophrenia. International Journal of Psychiatry in Clinical Practice. 2018 Jan 2;22(1):19–24. pmid:28789569
- 13. Ayub N. Measuring hopelessness and life orientation in Pakistani adolescents. Crisis. 2009 May;30(3):153–60. pmid:19767271
- 14. Perczel Forintos D, Sallai J, Rózsa S. Adaptation of the Beck Hopelessness Scale in Hungary. Psihologijske Teme. 2010 Dec 31;19(2):307–21. https://hrcak.srce.hr/64673
- 15. Boduszek D, Dhingra K. Construct validity of the Beck Hopelessness Scale (BHS) among university students: A multitrait-multimethod approach. Psychological Assessment. 2015 Nov 16;28(10):1325–30. pmid:26569465
- 16. Dyce JA. Factor structure of the Beck Hopelessness Scale. Journal of Clinical Psychology. 1996 Sep;52(5):555–8. https://doi.org/10.1002/(SICI)1097-4679(199609)52:5%3C555::AID-JCLP10%3E3.0.CO;2-D
- 17. Iliceto P, Fino E. Beck Hopelessness Scale (BHS). European Journal of Psychological Assessment. 2015;31(1):31–7.
- 18. Kocalevent RD, Finck C, Pérez-Trujillo M, Sautier L, Zill J, Hinz A. Standardization of the Beck Hopelessness Scale in the general population. Journal of Mental Health (Abingdon, England). 2016 Nov 12;26(6):516–22. pmid:27841054
- 19. Rosenfeld B, Gibson C, Kramer M, Breitbart W. Hopelessness and terminal illness: The construct of hopelessness in patients with advanced AIDS. Palliative & Supportive Care. 2004 Mar;2(1):43–53. pmid:16594234
- 20. Aish AM, Wasserman D. Does Beck’s Hopelessness Scale really measure several components? Psychological Medicine. 2001 Feb;31(2):367–72. pmid:11232923
- 21. Hanna D, White R, Lyons K, McParland MJ, Shannon C, Mulholland C. The structure of the Beck Hopelessness Scale: A confirmatory factor analysis in UK students. Personality and Individual Differences. 2011 Jul 1;51(1):17–22.
- 22. Innamorati M, Lester D, Balsamo M, Erbuto D, Ricci F, Amore M, et al. Factor validity of the Beck Hopelessness Scale in Italian medical patients. Journal of Psychopathology and Behavioral Assessment. 2014 Jun;36:300–7.
- 23. Szabó M, Mészáros V, Sallay J, Ajtay G, Boross V, Udvardy-Mészáros À, et al. The Beck Hopelessness Scale. European Journal of Psychological Assessment. 2016;32(2):111–8.
- 24. Zhang WC, Jia CX, Hu X, Qiu HM, Liu XC. Beck Hopelessness Scale: Psychometric properties among rural Chinese suicide attempters and non-attempters. Death Studies. 2015 Aug 9;39(7):442–6. pmid:25679324
- 25. Neufeld E, O’Rourke N, Donnelly M. Enhanced measurement sensitivity of hopeless ideation among older adults at risk of self-harm: Reliability and validity of Likert-type responses to the Beck Hopelessness Scale. Aging & Mental Health. 2010 Aug 1;14(6):752–6. pmid:20686984
- 26. Nissim R, Flora DB, Cribbie RA, Zimmermann C, Gagliese L, Rodin G. Factor structure of the Beck Hopelessness Scale in individuals with advanced cancer. Psycho‐Oncology: Journal of the Psychological, Social and Behavioral Dimensions of Cancer. 2010 Mar;19(3):255–63. pmid:19274620
- 27. Pompili M, Tatarelli R, Rogers JR, Lester D. The hopelessness scale: a factor analysis. Psychological Reports. 2007 Apr;100(2):375–8. https://doi.org/10.2466%2Fpr0.100.2.375-378 pmid:17564211
- 28. Nekanda‐Trepka CJ, Bishop S, Blackburn IM. Hopelessness and depression. British Journal of Clinical Psychology. 1983 Feb;22(1):49–60. pmid:6831077
- 29. Chang EC, D’Zurilla TJ, Maydeu-Olivares A. Assessing the dimensionality of optimism and pessimism using a multimeasure approach. Cognitive Therapy and Research. 1994 Apr;18:143–60.
- 30. Steed L. Further validity and reliability evidence for Beck Hopelessness Scale scores in a nonclinical sample. Educational and Psychological Measurement. 2001 Apr;61(2):303–16. https://doi.org/10.1177%2F00131640121971121
- 31. Demirtas AS, Yildiz B. Hopelessness and perceived stress: The mediating role of cognitive flexibility and intolerance of uncertainty. Dusunen Adam The Journal of Psychiatry and Neurological Sciences. 2019;32(3):259.
- 32. Lew B, Huen J, Yu P, Yuan L, Wang DF, Ping F, et al. Associations between depression, anxiety, stress, hopelessness, subjective well-being, coping styles and suicide in Chinese university students. PloS one. 2019 Jul 1;14(7):e0217372. pmid:31260454
- 33. Nalipay MJ, Ku L. Indirect effect of hopelessness on depression symptoms through perceived burdensomeness. Psychological Reports. 2019 Oct;122(5):1618–31. https://doi.org/10.1177%2F0033294118789044 pmid:30040031
- 34. Ulas O, Yildirim I. Influence of locus of control, perceived career barriers, negative affect, and hopelessness on career decision-making self-efficacy among Turkish university students. International Journal for Educational and Vocational Guidance. 2019 Apr 1;19:85–109.
- 35. Niméus A, Träskman-Bendz L, Alsén M. Hopelessness and suicidal behavior. Journal of Affective Disorders. 1997 Feb 1;42(2–3):137–44. pmid:9105955
- 36. Abramson LY, Metalsky GI, Alloy LB. Hopelessness depression: A theory-based subtype of depression. Psychological Review. 1989;96:358–72.
- 37. Pössel P, Knopf K. Bridging the gaps: An attempt to integrate three major cognitive depression models. Cognitive Therapy and Research. 2011 Aug;35:342–58.
- 38. Hankin BL, Abramson LY, Miller N, Haeffel GJ. Cognitive vulnerability-stress theories of depression: Examining affective specificity in the prediction of depression versus anxiety in three prospective studies. Cognitive Therapy and Research. 2004 Jun;28:309–45.
- 39. Haeffel GJ, Abramson LY, Voelz ZR, Metalsky GI, Halberstadt L, Dykman BM, et al. Cognitive vulnerability to depression and lifetime history of Axis I psychopathology: A comparison of negative cognitive styles (CSQ) and dysfunctional attitudes (DAS). Journal of Cognitive Psychotherapy. 2003 Jan 1;17(1):3–22.
- 40. Pössel P, Thomas SD. Cognitive triad as mediator in the hopelessness model? A three‐wave longitudinal study. Journal of Clinical Psychology. 2011 Mar;67(3):224–40. pmid:21254051
- 41. Pössel P, Smith E. Integrating Beck’s cognitive theory of depression and the hopelessness model in an adolescent sample. Journal of Abnormal Child Psychology. 2020 Mar;48:435–51. pmid:31832835
- 42. Dunn SL, Olamijulo GB, Fuglseth HL, Holden TP, Swieringa LL, Sit MJ, et al. The State–Trait Hopelessness Scale: Development and testing. Western Journal of Nursing Research. 2014 Apr;36(4):552–70. https://doi.org/10.1177%2F0193945913507634 pmid:24122739
- 43. Lester D. An inventory to measure helplessness, hopelessness, and haplessness. Psychological Reports. 2001 Dec;89(3):495–8. https://doi.org/10.2466%2Fpr0.2001.89.3.495 pmid:11824706
- 44. Kazdin AE, Rodgers A, Colbus D. The Hopelessness Scale for children: Psychometric characteristics and concurrent validity. Journal of Consulting and Clinical Psychology. 1986 Apr;54(2):241–5. pmid:3700812
- 45. Metalsky GI, Joiner TE. The Hopelessness Depression Symptom Questionnaire. Cognitive Therapy and Research. 1997 Jun;21:359–84.
- 46. Mardia KV. Measures of multivariate skewness and kurtosis with applications. Biometrika. 1970 Dec 1;57(3):519–30.
- 47. Rosenberg M. Rosenberg Self-Esteem Scale (RSE). Acceptance and commitment therapy. Measures Package. 1965;61(52):18.
- 48. Burr EM, Rahm-Knigge RL, Conner BT. The differentiating role of state and trait hopelessness in suicidal ideation and suicide attempt. Archives of Suicide Research. 2018 Jul 3;22(3):510–7. pmid:28854122
- 49.
Muthén LK, Muthén BO. MPlus user’s guide. 8th ed. Muthén & Muthén. 2017.
- 50. O’Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers. 2000 Sep;32(3):396–402. pmid:11029811
- 51. Howard MC. A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction. 2016 Jan 2;32(1):51–62. https://doi.org/10.1080/10447318.2015.1087664
- 52.
Kline RB. Principles and practice of structural equation modelling. 4th ed. Guildford Press. 2015.
- 53.
Wang J, Wang X. Structural equation modeling: Applications using MPlus. John Wiley & Sons. 2012.
- 54.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates.
- 55.
Norman GR, Streiner DL. Biostatistics: The bare essentials. St. Louis: Mosby–Year Book.
- 56. Wright BD, Linacre JM, Gusafson JE, Martin-Lof P. Reasonable mean-square fit values. Rasch Measurement Transactions 8: 3. Retrieved. 1994;23(01):2004.
- 57. Souza AC, Alexandre NM, Guirardello ED. Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e servicos de saude. 2017 Jul;26:649–59. pmid:28977189
- 58. Wuang YP, Su CY, Huang MH. Psychometric comparisons of three measures for assessing motor functions in preschoolers with intellectual disabilities. Journal of Intellectual Disability Research. 2012 Jun;56(6):567–78. pmid:21988314
- 59. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology. 2007 Jan 1;60(1):34–42. pmid:17161752
- 60. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988 Jun;54(6):1063–70. pmid:3397865
- 61. Scheier MF, Carver CS, Bridges MW. Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): A reevaluation of the Life Orientation Test. Journal of Personality and Social Psychology. 1994 Dec 1;67(6):1063–78. pmid:7815302
- 62. Snyder CR, Harris C, Anderson JR, Holleran SA, Irving LM, Sigmon ST, et al. The will and the ways: Development and validation of an individual-differences measure of hope. Journal of Personality and Social Psychology. 1991 Apr;60(4):570–85. https://psycnet.apa.org/doi/10.1037//0022-3514.60.4.570 pmid:2037968
- 63. Chen FF, Hayes A, Carver CS, Laurenceau JP, Zhang Z. Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality. 2012 Feb;80(1):219–51. pmid:22092195
- 64.
Hair JF, Black WC, Babin BJ, Anderson RE. & Tatham Ronald L., Multivariate data analysis (6th Eds.). ed: Upper Saddle River, New Jersey, USA: Prentice–Hall, International Inc. 2006.
- 65.
Beck AT, Rush A, Shaw B, & Emery G. Cognitive Therapy of Depression. New York: The Guilford Press; 1979. https://doi.org/10.1046/j.1440-1614.2002.t01-4-01015.x
- 66. Dagnan D, Sandhu S. Social comparison, self‐esteem and depression in people with intellectual disability. Journal of Intellectual Disability Research. 1999 Oct;43(5):372–9. pmid:10546961
- 67.
Ahrens AH, Alloy LB. Social comparison processes in depression. Health, coping, and well-being: Perspectives from social comparison theory. 1997:389–410.
- 68. Erol SA, Ergun AY. Hopelessness and social comparison in Turkish adolescent with visual impairment. Journal of Psychiatric and Mental Health Nursing. 2013 Apr;20(3):222–7. pmid:22070767
- 69. Tylka TL, Sabik NJ. Integrating social comparison theory and self-esteem within objectification theory to predict women’s disordered eating. Sex Roles. 2010 Jul;63:18–31.
- 70. Deci EL. Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of Personality and Social Psychology. 1972 Apr 1;22(1). https://psycnet.apa.org/doi/10.1037/h0032355
- 71.
Sansone C, Harackiewicz JM, editors. Intrinsic and extrinsic motivation: The search for optimal motivation and performance. Elsevier; 2000 Sep 12.
- 72. Rotter J. Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General and Applied. 1966;80:1–28. pmid:5340840
- 73. Tyler N, Heffernan R, Fortune CA. Reorienting locus of control in individuals who have offended through strengths-based interventions: Personal agency and the good lives model. Frontiers in Psychology. 2020 Sep 15;11:553240. pmid:33041920
- 74. Hoffmann H, Kupper Z, Kunz B. Hopelessness and its impact on rehabilitation outcome in schizophrenia–an exploratory study. Schizophrenia Research. 2000 Jun 16;43(2–3):147–58. pmid:10858633
- 75.
Levenson H. Distinctions within the concept of internal-external control: Development of a new scale. In Proceedings of the Annual Convention of the American Psychological Association 1972. American Psychological Association. https://doi.org/10.1037/e465522008-131
- 76. Bandura A. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review. 1977;84:191–215. pmid:847061
- 77.
Seligman ME. Helplessness: On development, depression, and death. WH Freeman; 1992.
- 78. Weiner B. Attribution theory and attributional therapy: Some theoretical observations and suggestions. British Journal of Clinical Psychology. 1988 Feb;27(1):99–104. pmid:3355911
- 79. Asberg K, Renk K. Perceived stress, external locus of control, and social support as predictors of psychological adjustment among female inmates with or without a history of sexual abuse. International Journal of Offender Therapy and Comparative Criminology. 2014 Jan;58(1):59–84. pmid:23070955
- 80. Struijs SY, Lamers F, Spinhoven P, van der Does W, Penninx BW. The predictive specificity of psychological vulnerability markers for the course of affective disorders. Journal of Psychiatric Research. 2018 Aug 1;103:10–17. pmid:29758471
- 81.
Lefcourt HM. Locus of control: Current trends in theory & research. Psychology Press; 2014 Jun 3. https://doi.org/10.4324/9781315798813
- 82. Richter CP. The phenomenon of unexplained sudden death in animals and man. The meaning of death. 1959:302–13.
- 83. Steer RA, Stratford NJ, Beck AT, Brown GK. Factors of the Beck Hopelessness Scale: Fact or artifact? A Journal for Basic Behavioral Research into Personality Dynamics and Clinical Psychology. 1997;11(3):131–44.
- 84. Dagnall N, Denovan A, Drinkwater KG. Longitudinal assessment of the temporal stability and predictive validity of the Revised Paranormal Belief Scale. Frontiers in Psychology. 2023 Jan 27;.13:8713. pmid:36778162
- 85. Polit DF. Getting serious about test–retest reliability: a critique of retest research and some recommendations. Quality of Life Research. 2014 Aug;23:1713–20. pmid:24504622
- 86. Watson D. Stability versus change, dependability versus error: Issues in the assessment of personality over time. Journal of Research in Personality. 2004 Aug 1;38(4):319–50.
- 87. Aldridge VK, Dovey TM, Wade A. Assessing Test-Retest Reliability of Psychological Measures. European Psychologist. 2017 Nov 29;22(4):207–18.