Positive appraisal style

PASTOR theory relies on the concept of appraisal, which is the function of analyzing a stimulus or situation with respect to its meaning for the organism and, on this basis, determining the emotional reaction to the stimulus or situation. Stress reactions result from an appraisal of a stimulus/situation, that is, of a potential stressor, as a threat to one’s goals or needs. Stressor appraisal occurs along the three major threat appraisal dimensions of threat magnitude or cost, threat probability, and coping potential [6]. Appraisal may be implemented via a heterogeneous set of cognitive processes. Some of these processes may be unconscious, non-verbal, and implicit, while others make use of verbal and conscious-explicit mental operations [6]. Appraisal determines the values attributed to a stressor on the aforementioned dimensions. In positive appraisal these values are set to levels that realistically the threat or even slightly underestimate it see Fig 1 for a schematic overview). That is, positive appraisal avoids catastrophizing (magnitude/cost dimension), pessimism (probability dimension), and perceived helplessness (coping dimension). In this sense, positive appraisal is primarily a non-negative way of appraising stressors. At the same time, positive appraisal avoids unrealistically positive (delusional) threat perceptions that might lead to trivialization, blind optimism, or extreme over-confidence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Schematic illustration of the range of appraisals constituting ‘positive appraisal’.

PAS (marked in gray) according to Positive Appraisal Style Theory of Resilience (PASTOR). Adapted from Kalisch and colleagues [8] © Cambridge University Press 2015, reproduced with permission.

https://doi.org/10.1371/journal.pone.0295562.g001

PASTOR assumes that an individual usually appraises similar situations in a similar fashion, and therefore can be characterized by their typical appraisal tendencies (‘appraisal style’) [6]. A negative appraisal style (NAS) consists in a propensity to overestimate the aversive consequences and the probability of challenging situations and to underestimate one’s coping potential. A NAS results in consistent over-reactions to perceived threats, that is, unnecessarily strong, long, or frequent stress reactions. Insofar as stress reactions consume resources (energy, time, cognitive capacity etc.), frequently generating unnecessary reactions can lead to resource depletion and allostatic load effects and eventually increase the likelihood of developing stress-related mental dysfunctions when confronted with substantial adversity [9]. A positive appraisal style (PAS), by contrast, is defined as the absence of such negative biases, but also by the absence of delusional, overly positive appraisal tendencies. As a result, individuals exhibiting PAS will typically generate appropriate stress responses to a given situation as necessary. The mild tendency to under-react will increase the possibilities to replenish and build resources and to learn from exploration and encounters with new situations [8]. Consequently, the likelihood of developing a mental disorder in adverse circumstances is decreased relative to individuals with a negative evaluative style. For this reason, PASTOR claims that PAS is a key resilience factor [8].

PAS is considered a relatively stable, yet malleable individual difference that may change as a function of experience, reflection, or instruction [6,8]. In addition, PAS would theoretically reduce allostatic overload by reducing exaggerated stress responses, which in turn can prevent pathological processes at a very early stage before any specific body or brain system can be lastingly affected. This positions PAS a potential protective factor against many or most potential stress-related dysfunctions.

These theoretical considerations suggest that a PAS questionnaire could be of high benefit for mental health and resilience research. In the present paper, we describe a PAS instrument development for the general population.

Appraisal processes versus appraisal contents

There is an important conceptual distinction in appraisal theory between appraisal processes and appraisal contents. Appraisal processes are the conscious or non-conscious mental operations that generate appraisal outcomes. Appraisal contents, on the other hand, are the mental representations of the meaning of a stimulus or situation for somebody’s needs and goals [8]. Importantly, appraisal contents are the outcome of appraisal processes.

Current paper

This overview motivates the development of new dedicated positive appraisal style self-report instruments, based on the process-content distinction. In the first approach, we develop a self-report questionnaire for the assessment of the frequency of cognitive processes relevant for a positive appraisal style (first approach: Perceived Positive Appraisal Style Scale, process-focused; short: PASS-process). For this purpose, we take items from existing, validated questionnaires that assess cognitive and behavioral strategies to deal with stressful situations, namely brief COPE and CERQ-short. An additional, self-generated subscale included in these analyses covers another important positive cognitive reappraisal tactic: distancing (also known as self-focused reappraisal, detachment, or decentering) [28–32]. We hypothesize that items with similar character on the dimensions of processes vs. contents, cognitive vs. behavioral, and positive vs. negative will cluster together, thus facilitating decisions about the inclusion of single items. We evaluate the resulting factors in terms of interpretability and theoretical compatibility and also apply statistical quality criteria, to eventually reach a solution that is satisfactory on all these levels.

The aim of the second part of this paper is to develop a scale that measures to what extent someone typically produces positive appraisals in stressful situations (Perceived Positive Appraisal Style Scale, content-focused; short: PASS-content). To this end as analyze responses to a new item pool in three independent German samples using factor analysis. Here, we only weakly hypothesize that the items will show a five-factor structure (three separate appraisal dimensions, cross-dimensional appraisals, appraisal background). The final solution considers interpretability, theoretical compatibility, and statistical quality.

The overarching aim of this study is to fill the gap in resilience research by providing two measures of self-perceived positive appraisal style, assessing appraisal processes and appraisal contents, respectively.

Participants

Analyses were based on data from three separate and independent samples of German participants, the Longitudinal Resilience Assessment (LORA; N = 1191) [33], the Gutenberg Brains Study (GBS; N = 3131) [34], and the Mainz Resilience Project (MARP; N = 202) [35–37]. Recruitment lasted from February 2014 to March 2019 for the GBS sample and from 2016 to 2019 for the LORA and MARP studies. All participants took part in a screening interview before inclusion. Participants were mentally healthy at inclusion. Detailed descriptions of sampling and recruitment are published elsewhere. Descriptives of the samples are reported in Table 1. No data on race or ethnicity of participants was collected. All participants in all studies gave written informed consent, and the studies were approved by local ethics committees (Medical Board of Rhineland-Palatinate, Mainz, Germany, and the Ethics Committee of the Department of Medicine at the Goethe University Frankfurt am Main, Germany). Assessments were collected via paper-based or computer-based questionnaires. Authors only had access to pseudonymized data for the purposes of the analyses presented in this paper.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Gender and age distribution of the LORA, MARP, and GBS samples used for the development of the scales.

https://doi.org/10.1371/journal.pone.0295562.t001

Item pool

Data cleaning

Only participants with complete data were considered, see Table 1. The completion rate in MARP was 100%, in LORA 98.6% and in GBS 95.7%. In the GBS sample the questionnaires were part of a later battery, leading to a lower number of participants in the analytical sample.

Item selection

An exploratory factor analysis (EFA) was conducted separately for each approach (process and content focused) in the LORA sample, followed by a confirmatory factor analysis (CFA) in the GBS sample. All analyses were done separately for each sample and on item level. In the EFA, the items of the scales were submitted to Horn’s parallel analyses [39] to identify the appropriate amount of factors. The scales use short Likert scales; therefore, the data was treated as ordinal [40]. Hence, an EFA with ordinary least squares estimation with polychoric correlations was conducted [41]. Oblimin rotation was applied. No continuity correction was applied, as the questionnaires had four or more answer options [42]. EFA results were evaluated based on model fit as well as the criterion to have at least two factors with at least five strongly loading indicators [43].

It is recommended that for a CFA in not-normally distributed data, a robust weighted least square estimator is used. Further, polychoric correlations were used. Model fit of CFA is determined by a variety of indices, including the Tucker-Lewis Index (TLI; ≈ 0.95), Comparative Fit Index (CFI; ≈ 0.95), Root mean squared error (RMSEA; < 0.06), and Standardized Root Mean Squared Residual (SRMR; < 0.08). These indices were preferred over the Chi-squared statistic, as its significance is likely due to the sample size [44]. Additionally, the models were evaluated qualitatively, based on the wording and phrasing of the items, to evaluate the presence of potential method factors.

Validation

The reliability and validity of the final two questionnaires were calculated with the GBS sample. Reliability was computed using McDonald’s omega and Cronbach’s alpha [45–47]. Convergent and discriminant validity were tested within the nomological network by evaluating Pearson’s correlations of the scale score with measures of similar and dissimilar constructs [48]. The nomological network consists of a convergent measure such as optimism, which is a positive appraisal tendency on the probability dimension and known to be positively associated with mental health [49]. As such, optimism can be considered a parallel-level construct and expected to positively correlate with PAS. A further convergent measure is neuroticism, which is a propensity for negative emotionality and emotional dysregulation and a known vulnerability or risk factor [50]. Neuroticism was therefore hypothesized to negatively correlate with PAS. The network further includes a discriminant measure indicated by the concept of impulsive behavior, which is unrelated to positive appraisal and therefore expected to have no relationships with the PAS measure.

In the GBS sample, optimism is measured by the Optimism-Pessimism Short Scale-2 (SOP2) [51]. The scale consists of two items that are rated on a 7-point Likert scale. Neuroticism is measured by neuroticism subscale of the Big Five Inventory (BFI-10) [52], which includes two items rated on 5-point Likert scale. Impulsive behavior was assessed with the i8 scale for impulsive behavior [53]. The scale consists of four subscales, of which Urgency and Spontaneity measure impulsive behavior and Intent and Perseverance capture behavior in the opposite direction. We therefore only used the first two subscales within the nomological network. Furthermore, concurrent validity was assumed to be indicated by a moderate relationship with wellbeing, indicated by WHO-5 [54] as proxy of mental health.

All data, analysis code, and research materials are available by request. Data were analyzed using R version 4.1.2 using the psych [55] and lavaan [56] packages. This study’s design and its analysis were not pre-registered.

Perceived Positive Appraisal Style Scale, process-focused (PASS-process)

Item quality indices calculated in the LORA sample are reported in S2 and S3 Tables. No exclusions were made based on those results.

Confirmatory factor analysis in the GBS sample.

The CFA used a robust weighted least squares estimation with polychoric correlations. The variances of the latent variables of factors 3, 4, and 5 were fixed due to the small numbers of indicators per factor. Factors 1 and 2 were highly correlated in the EFA, thus allowed to be correlated in the CFA model as well. The model fit was moderate to good (TLI = 0.94, NFI = 0.94, RMSEA = 0.094, SRMR = 0.078).

The factor analyses showed a structure that items that belong to the subscales Positive reframing (COPE), Humor (COPE), Positive reappraisal (CERQ), Putting into perspective (CERQ), and Distancing (self-generated) all strongly load on factor 1. Factor 2 was comprised of five items which belong to the subscales of Acceptance in both COPE and CERQ. Factors 3, 4, and 5 were each comprised of items of only one original subscale, respectively: Other-blame (CERQ), Turning to religion (COPE), and Rumination (CERQ), see Table 2. The first and second factors were substantially correlated (r = 0.45, p < 0.000).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results from the CFA for the PASS-process.

Factor loadings by item.

https://doi.org/10.1371/journal.pone.0295562.t002

Perceived positive appraisal style scale, content-focused (PASS-content)

Based on the LORA and MARP samples, eight items were excluded from the pool of 29 items due to unsatisfactory item-total correlation and item difficulty. Item characteristics are reported in S6 Table. Reverse coded items were inverted prior to factor analysis.

Confirmatory factor analysis in the GBS sample.

A confirmatory factor analysis was conducted with one factor consisting of the positively worded items and one putative method factor, see Table 3. A robust weighted least squares estimation with polychoric correlations was used. The resulting model had a good fit (TLI = 0.96, NFI = 0.96, RMSEA = 0.084, SRMR = 0.073).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Factor loadings from the confirmatory factor model.

https://doi.org/10.1371/journal.pone.0295562.t003

Validation

Measurement reliability, as indicated by McDonald’s omega (ω = .88) and Cronbach’s alpha (α = .87, 95% CI [.86, .87]), was very high, as expected given the results of the prior factor analysis. Subsequently, the total item sum score was used. The convergent validity of the new scale was high (optimism: r = .58, 95% CI [.55,.61], neuroticism: r = -.41, 95% CI [-.45,-.37]). The scale also had good discriminant validity: urgency (r = .09, 95% CI [.04,.14]), spontaneity (r = -.06, 95% CI [-.11,-.01]). Concurrent validity was moderate as indicated by the correlation with wellbeing (r = -.46, 95% CI [-.42,.5]). The scale had a substantial correlation with PASS-process (r = .54, 95% CI [.50,.60]).

Process-focused scale

In the first approach on the PASS-process, a structure of five factors was present. The first factor was comprised by items of various clearly positive appraisal-related items. Well correlated with this factor was a second factor comprised of items targeting acceptance tactics. These two factors, which clustered together, appeared to be distinct from the other three factors, each uncorrelated with the first two factors and comprised of items from a single COPE or CERQ subscale only (Other-blame, Turning to religion, and Rumination). Initially, we speculated that these latter strategies could be also related to positive appraisal processes, rather than reflecting appraisal contents, behavioral coping strategies, or negative appraisal processes. While their apparent statistical separation from a cluster of items with unambiguous assignment does not exclude this possibility, it at least indicates that they may also reflect other constructs. We were thus not able to resolve the ambiguity around these items and preferred to not retain them. Retaining the subscales loading on the first two factors resulted in a highly internally consistent instrument, further confirming the observation of statistical clustering of the respective items.

From a theoretical point of view, the close statistical relationship between acceptance-based and non-acceptance-based items in the presence of separability as two factors is interesting. PASTOR employs a broad definition of positive reappraisal in particular as any cognitive change that modifies appraisals towards the better [6,10]. In this classification, trying to accept a difficult situation constitutes a sub-type of reappraisal, or specific reappraisal tactic, in which a situation that is first seen as so negative and annoying that needs to be changed is globally re-evaluated as a situation than can be managed by mentally and behaviorally adjusting to it. That is, an initial disappointing insight into one’s inability to cope actively via situation modification (initial negative appraisal on the coping potential dimension) leads to an awareness of, and switch to, alternative coping possibilities (positive reappraisal on the coping potential dimension). There is an implicit undertone that the situation will be acceptable and not as disastrous as it might appear at first sight (positive reappraisal on the magnitude/cost dimension). PASTOR’s broad classification of reappraisal, extending to acceptance, is grounded in the emotion process model [58]. This model recognizes only two classes of non-behavioral antecedents to emotional responses (namely attention assignment and appraisal) and, consequently, can accommodate only two ways of regulating emotional responses in an antecedent-focused manner (by redirecting attention or by changing the appraisal). Insofar as acceptance does not rely on attention modulation, it must be a way of changing appraisal, or re-appraising. The same classification has been employed in many studies [59–63]. In the clinical literature, however, acceptance is often also juxtaposed to reappraisal [12,64], although it is not stated where in the emotion process chain acceptance-based regulation could be situated.

We interpret our observation of a close statistical link between the tested acceptance and non-acceptance items as supporting the classification of acceptance as a reappraisal tactic, in line with our theoretical considerations. We conjecture that the observed factor separation is due to an easy semantic discriminability of acceptance-based processes from other types of positive appraisal and reappraisal processes. The latter focus not on one’s ability to mentally and behaviorally adjust, but rather shift focus to elements of the situation that are not (as) disastrous. We also note that the brief COPE and CERQ use only two verbs to denominate acceptance (“accept”and “live with“), in contrast to the much more heterogeneous vocabulary used for the other positive (re)appraisal tactics.

Taken together, our results suggest PASS-process measures a single latent construct. From a practical point of view, the high internal consistency of the PASS-process permits sum scoring across all items of the scale. While it is usually not recommended to create sum scores of different factors, this can be done when the separate factor scores are understood to be indicating in the same direction [65]. In the present case, higher scores in each factor can be understood to represent more frequent use of positive appraisal and reappraisal strategies in difficult situations.

Importantly, PASS-process is intended to measure process use (frequency or tendency) [66], not process effectiveness (ability or capacity). Experimental studies on emotion regulation suggest an association between the individual capacity to use reappraisal and the tendency to choose this strategy for emotion regulation [67]. Hence, PASS-process may be found to correlate with positive (re)appraisal effectiveness or capacity measures, such as employed in laboratory tasks [12]. One source of this hypothetical correlation may be a recall bias for pleasant success experiences. Another hypothetical source is a true relationship between frequency of use and effectiveness resulting from learning experiences by which processes that have been successfully used to appraise and reappraise aversive situations will be reinforced and more likely be recruited again in similar situations [68]. The existence of a use-effectiveness relationship, however, has to be tested in future studies.

Content-focused scale

In the second approach on the PASS-content scale, we found a main factor loaded by the positively worded items. The three dimensions magnitude of threat, probability of threat and coping potential, as well as the added elements of cross-dimensional appraisal and appraisal background, were not found in the factor structure. We are hesitant to interpret this as evidence for the absence of the sub-facets of threat appraisal. Experimental studies show that the magnitude and the probability of an aversive outcome, such as a pain stimulus or loss of money, independently shape fear or stress reactions [69–72]. Likewise, experimental manipulations of control perceptions strongly influence aversive responses [73]. It is thus a reasonable possibility that our item pool did not contain enough statements clearly referring to single threat appraisal dimensions. This was likely a consequence of our deliberate effort to generate items that emulate everyday language, where subtle distinctions between aspects of threat may be less present than global, summarizing good-bad distinctions. A tendency for answering along a single good-bad dimension may also have been promoted by our inclusion of negatively framed items, in a sense that the presence of both positive and negative statements may have seduced participants to employ dichotomous thinking. This interpretation is supported by the finding of two anti-correlated factors grouping either positive or negative statements. On this basis, we conjecture that capturing different threat appraisal dimensions in a self-report tool would presumably require a larger and more differentiated item pool that contains no negatively framed items. Insofar as threat reactions are ultimately determined by an integration of appraisals along the various appraisal dimensions [6], such a more fine-grained development approach may, however, be of limited added value.

We interpret the factor comprised of all the negatively worded items as a method factor and excluded all negative items from the final scale. This is justified given its strong inverse correlation to the positive factor, meaning that it did not add further information. Also, in other studies, similar factors based on valence have been found [74–76]. This once more suggests that evaluative language may be strongly organized along a good-bad dimension. In addition, the specific context of filling in a questionnaire, requiring responding in short time to pre-formulated questions and excluding the possibility to formulate and elaborate on own statements, may well penalize more complex, non-dichotomous representations.

Given the instrument’s consistency and its convergent, discriminant, and concurrent validity, the PASS-content sum score can be considered a good measure of one’s perceived tendency to positively appraise difficult situations. As for PASS-process, PASS-content is limited to measuring consciously accessible and verbalizable mental contents, that is, PASS-content does not inform about potential non-conscious appraisals. It therefore also suffers from content under-representation.

Relation of PASS-process and PASS-content

The two questionnaires, PASS-process and PASS-content, measure two different, but related facets of PAS. Frequent use of positive appraisal processes, or strategies, in stressful circumstances (PASS-process) can be assumed to generate more positive stressor appraisals (PASS-content). This interpretation is supported by the high correlation between PASS-process and PASS-content observed in the GBS sample. Nevertheless, the two scales measure distinct constructs, rather than dimensions of the same construct. Firstly, not all use of positive appraisal or reappraisal strategies may lead to positive appraisal contents. This could be due to their ineffectiveness or the possibility that, despite the processes being effective, they do not generate new and consciously available contents measured by the PASS-content. Secondly, contents might be generated by processes not measured by the PASS-process. For example, appraisal contents could be generated in an automatic and unconscious way without the person becoming aware of the appraisal process used to generate them. Accordingly, the only partial correlation of PASS-process and PASS-content suggests that the scales may indeed be employed to assess at least partly separable aspects of the PAS construct.

Limitations and future research

As stated, the two new scales, by being self-report instruments, are limited to consciously accessible and verbally reportable internal information. In addition, self-report instruments are limited by their inherent openness to bias, such as a tendency for socially desirable and internally consistent reporting, recall biases, framing effects, and mood congruency effects. With relevance to the appraisal topic, Koval et al. [77] have recently reported that the use frequency of different emotion regulation strategies, including positive cognitive reappraisal, reported with standard questionnaires is not correlated with their use frequency when measured in the same individuals with ambulatory methods in daily life. This points to the possibility that responding in retrospective self-report is strongly affected by self-models, or self-narratives, rather than veridically reflecting what individuals do or think in concrete situations. This does not exclude that PASS-process and PASS-content transport valuable information about appraisal styles. The appraisal of relevant life situations does not occur at one single moment, but repeatedly and also often retrospectively outside the concrete situation or event (when looking back at a challenging day or time of life, in communication with others, etc.) [78]. That is, appraisal must be understood as an integrating and longer-term process that, not only considers the immediate implications of a situation but also its more lasting consequences, its interactions with contextual factors, and its impact on overarching needs and goals (including a stable and positive identity) [6]. Hence, integrative appraisals are inextricably linked with a person’s self-model, and a person’s self-narrative about how one habitually appraises stressors can probably be understood as being that person’s longer-term integrative appraisal tendency. Appraisal styles may thus be better addressed in a retrospective questionnaire than by an assessment of momentary reactions with ambulatory tools.

Nevertheless, Kalisch et al. [6] have emphasized the importance of developing complementary measurements of positive appraisal that do not require self-report. These would include behavioral tests, task-based functional imaging, physiological, or other bodily measures. A clear advantage of measuring perceived positive appraisal style with PASS-process and PASS-content is their suitability for large samples. Subsequent studies should address the open question which of the different possible routes to measuring appraisal will provide the best predictors for resilience.

A further limitation of the instruments is that both questionnaires use short Likert scales as response options. This has implications both for the assessment of the factor structure, as the data needed to be treated as ordinal [40], and for the variance captured by the scales. While this has been adjusted for in the factor analyses, when it comes to data collection, the scales are likely to only pick up on larger differences. Yet another potential limitation of the instruments is the use of sum scores. Sum scores for instruments built on factor solutions have recently come under criticism, as a sum score is a simpler model solution than the original factor solution [79]. However, it must also be considered that factor loadings are likely to vary from sample to sample and, thus, an equally weighted sum score is a more comparable indicator across studies [65]. Considering the good convergent, discriminant, and concurrent validity reported here for the PASS-process and PASS-content sum scores, the use of sum scores appears appropriate. Additionally, both scales do not include subscales, and comparisons between items or groups of items are not intended (though in theory possible between the five two-item subscales in PASS-process). Rather, the single sum score is meant to indicate a general self-perceived use of positive appraisal processes in the case of the PASS-process and the usual frequency of positive appraisal contents in the case of the PASS-content.

PAS, as measured by a previous version of the PASS-process when it was still under construction, has been shown to be associated with resilience in a cross-sectional sample of 16,000 individuals exposed to the first wave of the COVID-19 pandemic [17]. Future research will have to determine whether using the PAS scales that we introduce here presents any advantage over other instruments that only capture facets of positive appraisal, such as those mentioned in the introduction. That is, whether the PAS scales consistently outperform the related measures in comparative multivariate analyses such as using regularized regression techniques. This can be expected, given that PASS-content in particular was generated with the idea in mind that for those with a strong PAS, negative tendencies on one appraisal dimension can be compensated to some extent by positive tendencies on other dimensions.

The new scales should be employed in large longitudinal studies to investigate the role of PAS in resilience. Specifically, to address whether PAS is protective against specific dysfunctions or general mental health issues, as well as to investigate if it is protective in specific populations or specific kind of adverse circumstance, and less so in others. Future research will also have to determine the test-retest reliability and measurement invariance of the PAS scales and validate the PAS in different translations.

Conclusions

In conclusion, appraisal is considered the key generative factor in stress responses and, consequently, in mental health and resilience [8,63,80]. The two scales presented in this study aim to measure a participant’s perceived general tendency in how they appraise stressors. The PASS-process does so by measuring the processes or strategies someone employs to generate positive appraisals. The PASS-content measures the contents and thoughts that result from such processes. Both scales measure an individual’s self-reported appraisal style. Although they do so by targeting related notions, we note that these are separate constructs. Being self-report instruments, these scales only capture the conscious and verbally accessible dimension of PAS. The interpretation of results should take this limitation into account.