Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Longitudinal prediction of positive and negative mental health in Germany, Russia, and China

  • Jürgen Margraf ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – review & editing

    juergen.margraf@rub.de

    Affiliation Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany

  • Xiao Chi Zhang,

    Roles Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany

  • Kristen L. Lavallee,

    Roles Conceptualization, Formal analysis, Writing – original draft, Writing – review & editing

    Affiliation Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany

  • Silvia Schneider

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – review & editing

    Affiliation Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany

Longitudinal prediction of positive and negative mental health in Germany, Russia, and China

  • Jürgen Margraf, 
  • Xiao Chi Zhang, 
  • Kristen L. Lavallee, 
  • Silvia Schneider
PLOS
x

Abstract

The present study examines a broad range of longitudinal predictors of dimensional positive mental health (PMH) and negative mental health (NMH), using data from the BOOM study. Participants were drawn from university student samples in Germany (1,608), Russia (677) and China (12,057). A structural equation model was conducted with four groups of predictors and PMH/NMH as criteria (outcomes). Five of the six salutogenic predictors were predictive of both positive mental health (positively) and negative mental health, as measured by depression (negatively). Pathogenic predictors anxiety and depression were related to future depression, but not to positive mental health. Stress at baseline was related to both future positive mental health (negatively) and future depression (positively). Being male in this study was associated with increased future depression. Results hold across Germany, Russia, and China. Results support the conceptualization of positive and negative mental health as related, but independent dimensions across three cultures.

Introduction

Historically, mental health has been defined defacto as the absence of psychopathology [13]. However, this view is now recognized increasingly as overly simplistic, with positive mental health being a distinct and important construct for understanding the human condition [1, 2]. With the rise of positive psychology, there is an increased interest in facilitating human flourishing and thriving, not just establishing a neutral state of mind [4]. As awareness of a complete picture of mental health grows [5, 6], the traditional unidimensional model of mental disease is no longer sufficient. As positive mental health (PMH) and mental health problems can present simultaneously, they best viewed as separate interdependent dimensions [1, 2, 7], and should both be taken into account in research on mental health [8]. Still, most studies in clinical psychology and psychiatry continue to exclusively focus on negative aspects of mental health.

In addition to the importance of examining positive mental health, substantial progress in the understanding of the complex causes and correlates of mental health should involve examination of both salutogenic and pathogenic factors [9, 10], in the domains of not only psychological, but also biological and social factors. In the past, true interdisciplinary approaches have been rare exceptions; however the relevance of biological factors has been well established in the past two decades. In addition, social factors, both on the individual level (e.g. socioeconomic status, social mobility, personal ties) and the aggregate level (e.g. community health, neighborhood deprivation, inequality) show close links with physical and mental health [1114]. Some important population parameters have been changing over the last century, with corresponding changes in mental health, including family relationships, social connectedness, expectations for having children, single parenting as risk for poverty and social deprivation, and mass media exposure [15] As a consequence, the settings in which people develop have changed dramatically within only a few decades. Within such work, longitudinal studies represent the gold standard in mental health prediction research (Ghaderi & Scott, 2001), with large-scale such research being ideal but underrealized (e.g. Moffit et al., 2007; Sihvola et al., 2010; Wang, 2004; Beard et al., 2008), especially for positive dimensions of mental health [3].

Pathogenic factors include predispositions (preexisting genetic, somatic, psychological or social characteristics that make the occurrence of a mental health problem possible or more probable), triggering conditions (negative experiences or stressors that trigger the primary occurrence of a mental health problem) [6, 1618], and maintaining factors (dysfunctional reactions, such as avoidance, or cognitive distortions, or ongoing stressors that can prevent resolution of mental health problems and make them chronic [1921]). Examples of salutogenic, protective factors are emotional stability, social support, sustainable relationships, sense of coherence and control, problem solving skills, and social and communication skills. These factors may also be involved in the generation of health, as they enable individuals to realize their potential, to cope with the normal stresses of life, to work productively and fruitfully, and to be able to make a contribution their communities [22]. That is, salutogenic factors may act upon all three classes of pathogenic factors to counteract or balance them [7, 23]. The present study is particularly focused on salutogenic factors and their prediction of positive mental health in contrast to pathological mental states.

Psychological variables

Several psychological variables have particular value as predictors of mental health outcomes in previous research, and are also able to be reliably and validly assessed in the context of a large-scale surveys across cultures [7, 2435]. These include traits such as resilience and personal values, as well as states such as current happiness, life satisfaction, sense of coherence, and the desire to have children [13, 24, 36, 37]. Studies have implicated salutogenic factors, such as sense of coherence (Eriksson & Lindström, 2007), gratitude and agreeableness (Wood et al., 2009), in the development of mental health and life satisfaction (Trumpf et al., 2009; Vriends et al., 2007). Resilience in particular is positively related to positive mental health (Haddadi & Besharat, 2010), and negatively associated with depressive symptoms (Brunwasser, Gillham & Kim, 2009).

Despite the importance of salutogenic factors, most studies still focus on pathogenic predictors and a limited number of specific disorders or negative dimensions, despite high comorbidity (Kessler et al., 2005; WHO, 2004), highlighting the need for more large-scale comprehensive, inclusive research studies. As many people experiencing mental health issues experience more than one type, and as many negative predictors predict more than one disorder type (Beard et al., 2007), it appears important to examine the fundamental processes underlying broad-based positive and negative mental health, and not just one or two narrow subtypes at a time. One recent large-scale longitudinal study of broad based mental disorders found that pathogenic predictors predict incidence and relapse of mental disorder diagnoses, while salutogenic factors predict remission over time [36]. Across the board, there is a need for more studies on salutogenic psychological predictors of positive mental health over time.

Biological variables

There is strong evidence elucidating a bidirectional relationship between mental and physical health, until now primarily focused on health problems and mental disorder (Kolappa et al. 2013). More broadly, it is generally accepted that many physical conditions are associated with serious mental disorders, but the exact nature of these relationships is still unclear and further research beyond the so far dominant cross-sectional designs is called for [38]. Biological variables include physical health [13, 39], body mass index (BMI) [40, 41], sex [42, 43], and age [13, 4446], are important contributors to mental health and can be assessed easily within the framework of a large-scale survey.

Social variables

People with low SES, as measured by income, tend to have lower levels of positive mental health [47, 48]. Further, social support and social rhythm as associated with positive mental health outcomes [13, 18, 24, 35, 36]. The role of perceived social support as a protective factor for mental health problems is well-established [49]. Aside from reducing mental illness incidence, perceived social support also reduces general stress in women and men [50] and has positive effects on somatic health [51]. Social rhythm refers to the regularity with which one engages in social activities throughout the week, and has established links with bipolar disorder, as well as some links with depression and anxiety [52]. Just as daily biological patterns, such as circadian rhythm, temperature fluctuations, and cortisol levels, are integral to good mental health, with disruptions associated with depression [53], so it appears are rhythmic social and behavioral patterns, for example in mealtimes, bedtimes, and patterns of social interaction [35, 54, 55]. While next to no research exists on plans for fertility and mental health, fertility wish may be a potential sign of optimism about the future, and thus a social indicator that may be related to positive mental health outcomes. Indeed, in western societies, those who do have children live longer than those who don’t [56]. Having children can provide older adults with important social support, and plans to have them may indicate a certain level of optimism about the world those children will encounter in the future.

With respect to negative mental health, a meta-analysis of 51 population-based surveys found that adults in the lowest socioeconomic position (SES) have an increased risk of developing depression, with an odds ratio of 1.81 as compared to those in the highest position [57]. Also, recent results from a national survey show that the 12-month prevalence rate for mood disorders is more than twice as high for low (14%) than for high SES (6.3%) [13]. Further, this social class gradation is present across the lifespan [58, 59].

Cross-cultural approach

Despite the increased recognition of the importance of positive factors, so far, most studies into positive factors have been conducted in Western nations, limiting the generalizability of the findings. Depth of inquiry into of the nature of positive protective constructs across cultures is long overdue. Psychological, biological and even sociological theories of mental health typically strive for universal validity, and the claim to be transcultural (i.e., universal). However, social factors and cultural background are widely recognized as a potentially important influences in mental health [1114, 37]. However, we cannot continue to postulate universal validity for psychological theories that have not been tested or may even not be amenable to testing across cultural boundaries. For our theories to be truly transcultural, they must first be studied cross-culturally [60].

Present study

The present study is the primary outcome study in the the “Bochum Optimism and Mental Health (BOOM) Studies” [61], which aim to enhance integrated knowledge of the causes and consequences of positive mental health and mental health problems cross-culturally and over time. The present analysis, specifically, is a large-scale, cross-cultural, multi-national, and longitudinal investigation into primarily salutogenic factors contributing to both positive and negative mental health outcomes, as measured by positive mental health and depression. It is an attempt to contribute to a comprehensive picture of the etiology of mental health. This study examines the following positive constructs thought to be relevant to mental health (positive mental health) and illness (depression, stress, anxiety): resilience, social support, social rhythm, family affluence, physical health and expectations for fertility. Finally, dimensional measures of mental health will be examined, rather than solely categorical measures. Factors will be examined across two time points, and across three countries: Germany, China, and Russia. We limited the number of predictors for methodological reasons [62]. We hypothesized that mental health will be predicted by the balance between positive (protective, salutogenic) and negative (pathogenic) psychological, biological and social factors and that these relationships will have an influence on mental health longitudinally [7, 36].

Method

Procedure

The present study utilizes a subset of data from the Bochum Optimism and Mental Health (BOOM) study, a large-scale, cross-cultural, longitudinal investigation of risk and protective factors in mental health [37, 61]. A comprehensive overview of the full study design, aims, measures, and participants is provided elsewhere [61]. The Ethics Committee of the Faculty of Psychology of the Ruhr University Bochum approved the study. Approval to administer the questionnaires was granted by the Faculty of Psychology at Ruhr University Bochum on May 12, 2011 and renewed on September 2013. The approvals for the German site were communicated to the participating Chinese and Russian Universities who acknowledged these approvals. Data were collected between 2011 and 2016 Participants in the present study were recruited via the internet (German, Russian, and Chinese) and paper mailings (Chinese). The period of time between the first and second time points was about 17 months for each participant. All participants were informed that their answers would be collected pseudonymised, which means that a code is used for anonymisation. This makes it possible to relate individuals‘ answers to one another across different data records, while maintaining the person’s anonymity. Only the project supervisors have access to these codes. Participants provided implicit consent by choosing to continue with the questionnaires after reading the informed consent statement.

German data collection at Ruhr University Bochum was via an online portal, with data collection beginning in 2011. The Ethics Committee of the Faculty of Psychology of Ruhr University Bochum approved the study on May 12, 2011 and renewed on October 2012. The German sample at baseline consisted of 7,890 students from Ruhr University Bochum from 2012 to 2015. In the first follow-up study, 1,608 students participated again. The link was sent to all students enrolled at Ruhr University Bochum in 2012 and sent only to freshmen at from 2013 to 2015. Students were offered the opportunity to take part in a draw for a gift coupon (20 euro) or a tablet computer.

The Russian sample consisted of 3,745 students from Lomonossov University Moscow, University of Voronesh, and University of Orenburg in 2013. In the follow-up study in 2014, 677 students participated again. Participants were recruited via an invitation letter. Data were gathered by online and paper-and-pencil questionnaires administered in a group testing session. Participants received no financial compensation. +

In China, as the data were anonymized from the very beginning of data collection, no statement by an institutional board/ethics committee was required for China. The original Chinese sample at baseline consisted of 13,581 university students from Capital Normal University Beijing, Hebei United University, Shanghai Normal University, Guizhou Finance and Economics University, and Nanjing University with baseline data collected from 2012 to 2013. Of those, 12,744 students participated again in the first follow-up study between 2013 to 2014. Most of them were freshmen at baseline survey.

Participants

University student participants with data at baseline and one-year follow-up were included from three countries: Germany (1,608), Russia (677) and China (12,057). Representativeness of the students samples as compared with the adult residential populations in the three countries was assessed based on the register-assisted census data from 2011 regarding age, gender and education, was ensured via systematized sampling procedures.

Germany.

The German sample consisted of 7,890 (1,608 had data at follow-up) student participants recruited from Ruhr University Bochum. Students were assessed via online survey. German students were recruited by an e-mailed invitation with a link leading to an online questionnaire. The link was sent to all students enrolled at Ruhr University Bochum. They were offered an incentive to take part in a drawing for a gift certificate or a tablet computer.

China.

As the data were anonymized from the very beginning of data collection, no statement by an institutional board/ethics committee was required to collect data in China. The Chinese sample consisted of 13,581 (12,057 had data at both time points) university students from the Capital Normal University Beijing, the Hebei United University, Shanghai Normal University, Guizhou Finance and Economics University, and Nanjing University. Participants, mainly freshmen, were recruited during their first study month via an invitation by mail. The response rate was 94.5%. Data were gathered by an online or a paper-pencil questionnaire administered in a group testing session. Participants received 10 RenMinBi (approximately 1.3 Euros) upon returning the questionnaire.

Russia.

As the data were anonymized from the very beginning of data collection, no statement by an institutional board/ethics committee was required to collect data in Russia. The Russian sample consisted of 4,001 (677 had data at both time points in the present analyses) students from Lomonossov University Moscow, the University of Voronesh, and the University of Orenburg. Participants were recruited via an invitation letter. The response rate was 95.3%. Data was gathered by means of online and paper-pencil questionnaires, administered in a group testing session. Participants received no financial compensation.

Measures

Overview.

As far as possible, established brief standard instruments such as the Depression Anxiety Stress Scales (e.g., DASS 21) were used to measure the constructs of interest. For all questionnaires used in the analysis, validated German versions exist. Russian and Chinese versions of the measures were developed when needed, by using the customary translation-back-translation method as recommended [63]. In cases of discrepancies, this procedure was repeated by the study team until complete agreement was achieved. Measures can be grouped according to the overall design of the research program. Table 2 provides an overview as well as correlations.

General outcomes.

Positive mental health. The 9-item PMH-scale was developed in order to provide a brief, uni-dimensional and person-centered instrument to assess positive mental health [7]. The concept of positive mental health combines mainly emotional, but also psychological and social aspects of well-being into a single general construct [7]. People who are mentally healthy tend to have stable relationships, view their lives as having purpose and direction, experience more positive affect, and are more likely to be self-accepting [64]. Psychometric testing confirmed the scale to be a unidimensional self-report instrument with high internal consistency, good retest-reliability, scalar invariance across samples and over time, good convergent and discriminant validity as well as sensitivity to therapeutic change in a series samples from very different backgrounds [7]. Participants respond to statements such as “I am often carefree and in good spirits, I enjoy my life, I manage well to fulfill my needs, I am in good physical and emotional condition” on a 4-point likert scale ranging from 1 (do not agree) to 4 (agree). Item scores are combined into a sum score with higher scores indicating higher positive mental health. The measurement invariance of positive mental health is established as full strong (Bieda, Hirschfeld, Schoenfeld, Brailovskaia, & Zhang, 2017). Cronbach’s alphas at baseline were .916 (Germany), .891 (China), and .867 (Russia).

Depression, anxiety and stress. Negative mental health was assessed using the widely- used Depression Anxiety Stress Scales (DASS-21) [65]. This short form of the DASS-42 [66] assesses a broad range of psychological distress symptoms. It is composed of three 7-item subscales for depressive, anxiety and stress symptoms over the past week. The subscales may serve as outcome measures and as screening and monitoring instruments [6769]. Items are rated on a 4-point likert scale from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time). Responses can be averaged within subscale or across all three for a total item score. Psychometric properties are well established in both clinical and non-clinical samples [65, 69] and are comparable for the short and long versions [70, 71]. The measurement invariance of these three scales are established as full strong among the German, Russian, and Chinese students samples. In the present study, Cronbach’s alphas at baseline for depression were .886 (Germany), .776 (China), and .794 (Russia). Alphas for anxiety were .774 (Germany), .737 (China), and .775 (Russia). Alphas for stress were .853 (Germany), .771 (China), and .774 (Russia).

Predictors.

Resilience. Psychosocial stress resilience was assessed with an 11-item short version of the Wagnild and Young Resilience Scale (RS-14; 100; RS-11; 102) [72]. Participants responded to items such as “I usually manage one way or another” on a scale ranging from 1 (I disagree) to 7 (I agree). The RS-11 demonstrated good reliability and convergent validity in a German sample [73]. The measurement invariance of resilience is established as partial strong (Bieda, Hirschfeld, Schoenfeld, Brailovskaia, & Zhang, 2017). Cronbach’s alphas at baseline were .891 (Germany), .793 (China), and .776 (Russia).

Social support. Social support was assessed using the 14-item Questionnaire- Social Support measuring perceived and/or anticipated social support (F-SozU K-14) [74]. Participants indicated agreement with statements such as “I experience a lot of understanding and security from others” on a 5-point Likert scale ranging from 1 (not true) to 5 (true). In a German population, this unidimensional measure showed excellent Cronbach`s α and good convergent and discriminant validity [74]. In cross-cultural research, Nover (2012) [75] tested a long version of the Questionnaire- Social Support (F-SozU-22) [76] among pupils from Germany, Luxembourg and Spain, finding partial weak measurement invariance for the three cultural groups. The measurement invariance of the F-SozU K-14 is established as partial strong [77]. Cronbach’s alphas at baseline were .935 (Germany), .948 (China), and .929 (Russia).

Social rhythm. Social rhythm was assessed using the Brief Social Rhythm Scale (BSRS) [35]. This scale consists of ten items, which assess the irregularity with which participants engage in basic daily activities during the workweek and on the weekend. The BSRS assesses waking and bedtimes and breakfast and dinner mealtimes. It also assesses the regularity of time spent with others at work/school and during free time. Participants are asked to rate the general regularity of each activity in their lives in general using a scale ranging from 1 (very regularly) to 6 (very irregularly), with high mean scores indicating high irregularity. This measure can be administered at a single time point, rather than requiring a week of daily data to score. Summary scores are the sum across all 10 items. The BSRS shows a slight positive skewed distribution. It is reliable, distinguishes among categories of mental health and detects relationships with physical and mental health, and is especially useful in large-scale or screening studies, where participant time is limited [35]. In the German representative telephone data, item-total correlations ranged from r = .25 to r = .54. Test-retest-reliability in a subsample study of 1294 people from Germany from the BOOM study who took the measure online or in paper and pencil format at time 1 and time 2 (4 weeks later) was r = .70. Cronbach’s alpha was α = .763 in Germany, .837 in China, and .775 in Russia. The measure was reversed scored in the present study so that high scores equal higher rhythmicity.

Fertility wish. The wish to have children (“Kinderwunsch” in German) was assessed using a single yes-no item asking “Do you want to have a child/children in the future?” Participants who already had a child, were asked “Do you want to have one more child / more children?” and responded indicating “no” or “yes” with how many children they wanted to have. Those answers were recoded into simple “no” or “yes.” In the Chinese and Russian samples, we only used data from people with no existing child. In the German sample, 840 of total 1558 responses to this question were from people who definitively had no existing child, and the rest may or may not have had a child already.

Quality of health. Overall current quality of health was assessed using the EuroQol (EQ-VAS) [7880]. Participants rated current health status on a scale ranging from 0 (worst imaginable health) to 100 (best imaginable health). Validity of EQ-VAS is indicated by convergence with the the five dimensional version of the EuroQol (EuroQol 5D) with WHO-5 and known clinical groups across several countries.

Sociodemographic predictors.

Basic sociodemographic predictors. Sex, age, and relational status were assessed via self-report.

Family affluence. To ensure sufficient comparability across vastly different countries, the Family Affluence Scale (FAS) [81] served as the main cross-cultural measure of socioeconomic circumstances. The FAS is, a four-item measure of family wealth, developed in the WHO Health Behavior in School-aged Children Study. Questions include (either with 2 or 3 response alternatives): “Does your family own a car, van or truck?”, “Do you have your own bed- room for yourself?”, “During the past 12 months, how many times did you travel away on holiday with your family?”, and “How many computers does your family own?”. The FAS total score is calculated by summing up the responses to these items. Convergent validity is established via correlations with the Gross National Product across 35 countries [81]. Cronbach’s alphas at baseline were .315 (Germany), .640 (China), and .379 (Russia).

Statistical analyses

The scales fulfilled the minimum requirement for path comparisons in cross cultural studies: weak invariance. A series of structural equation models were conducted with all the above mentioned predictors at baseline and positive mental health and depression at follow-up as outcomes for each country separately to specify the relations between variables in this study. After that, a multi group analysis was carried out to examine the role of country. The baseline model (M1) with no constrains on the paths will be tested in the first step. Then the same model but with all paths being constrained as the same across the three countries (M2) will be tested. Model fit indices will be examined to further assess the model’s fit. The root mean square of approximation (RMSEA) will be interpreted as follows: values above 0.10 indicate unacceptable fit [82], and values in the range of 0.08 to 0.10 indicate mediocre fit, those between 0.05 and 0.08 indicate fair fit, and those less than 0.05 indicate close fit [83, 84]. The comparative fit index (CFI) [85] with values, which are greater than .90, indicates a good fit. Since equality constraints will mostly lead to decreases in fit indices and the χ2 difference test is highly sensitive in large samples (Oishi, 2007), the rule of ΔCFI not greater than 0.01 [86] is recommended for model comparison. All analyses were calculated with SPSS 23 and R version 3.4.2 with the Package “SEM”.

Missing values were generally between .2 to 2.6%, depending on the measure, with Social Rhythm having 13.3% due to an error omitting 2 questions for some participants (missing at random with no statistical correlation to data). This had no statistical relationship to the results, and therefore participants with questionnaires with missing data relevant to the analyses were deleted from the analyses that involved those questionnaires. Further, as assessment methods had an influence on the data, in all analyses the influence of assessment method was controlled [87]. Internal consistency is computed with Cronbach’s α coefficient. Cronbach’s α > 0.70 indicates acceptable, > 0.80 good, and > 0.90 excellent internal consistency [88]. Data are available in Supporting Information File 1.

Results

Descriptive statistics

Table 1 presents data on participant demographics and descriptive statistics for the predictors and outcomes at baseline.

thumbnail
Table 1. Descriptive statistics of socio-demographic variables and measures.

https://doi.org/10.1371/journal.pone.0234997.t001

Correlations

The correlations among the psychological predictors are shown in Table 2. The salutogenic predictors (positive mental health, life satisfaction, self-efficacy) are generally positively correlated with each other. The baseline negative aspects of mental health, or pathogenic predictors (i.e., stress, anxiety, and depression) are positively correlated with each other as well. The salutogenic predictors generally correlated negatively with the pathogenic predictors. Global assessment of functioning correlated positively with the salutogenic, and negatively with the pathogenic predictors.

thumbnail
Table 2. The correlations among the psychological predictors within country.

https://doi.org/10.1371/journal.pone.0234997.t002

Structural equation model

Table 3 provides the model fit for the unconstrained model (M1) as well as the constrained model (M2). In M1, each country’s regression weights were estimated separately. In M2, the regressions weights were constrained to be equal across countries. Fig 1 displays the constrained model (M2). The same unconstrained model (M1) was tested in each country and model fits well according to the fit indices. M1 was then calculated for all three countries together as a baseline model, also resulting in good fit, with CFI = .916, RMSEA = .087, and SRMR = .072. After the regression coefficients were held equal across three countries (M2), the model still provided an appropriate fit to the data, and ΔCFI did not exceed .01. Thus, the model fit can be considered equal to the M1 [89]. Standardized regression coefficients of the single equations from the unconstrained model (M1) for each country as well as from the constrained Model (M2) are presented in Table 4.

thumbnail
Fig 1. Model with all path coefficients constrained to be equal across China, Germany, and Russia.

https://doi.org/10.1371/journal.pone.0234997.g001

thumbnail
Table 4. Standardized regressions-coefficients from structural equation models.

https://doi.org/10.1371/journal.pone.0234997.t004

The results of the model indicated that nearly all of the six salutogenic predictors were predictive of both positive mental health (positively) and negative mental health, as measured by depression (negatively). The only salutogenic factor not related to future depression was resilience. Pathogenic predictors anxiety and depression were related to future depression, but not to positive mental health. Stress at baseline was related to both future positive mental health (negatively) and future depression (positively). The control variable family affluence was positively related to positive mental health. Being male in this study was associated with increased future depression.

Discussion

To our knowledge, this is a unique longitudinal, prospective study of its size in its examination of the relationship between positive and negative mental health cross-culturally, with such a wide range of psychological salutogenic predictors. The present study extends earlier findings by providing evidence that salutogenic factors not only predict future positive and negative mental health, but may be more likely to predict both positive and negative future mental health than pathogenic variables. The findings speak to the importance of resilience, and extend past findings beyond the typical North American studies, across Euro and Asian cultures, to Germany, Russia, and China. Results indicated that in all three countries, all of the six salutogenic predictors, including somatic health, social support, resilience, positive mental health, social rhythm regularity, and fertility wish, were predictive of both positive mental health (positively) and negative mental health, as measured by depression (negatively). The only salutogenic factor not related to future depression was resilience. Pathogenic predictors anxiety and depression were related to future depression, but, contrary to our predictions, these factors were not important in predicting future positive mental health once baseline salutogenic factors were taken into account. Stress at baseline was indeed related to both future positive mental health (negatively) and future depression (positively). As predicted, and in line with past research, family affluence was positively related to positive mental health [13, 57]. Being male in this study was associated with increased future depression. This result is in contrast to prior research indicating that females typically report greater depression and lower overall internalizing mental health [13, 42]. Notably, many of the effects were small. The large sample size allowed for such a complex model to detect small, but meaningful and significant effects.

The model fit was acceptable when the model was constrained to be equal across cultures, indicating that in general, the variables were predictive in a similar way across China, Germany, and Russia. Interestingly, and unique to this study, the desire to have a child was related to increased well being. It would be interesting to examine this factor in the future as a sign of optimism. Further, it was similarly related to well-being in all three countries. China has had fertility limiting policies in place, but have been relaxing them in recent years, and since before the current data were collected. It is not expected that state attempts to limit fertility would have had a large moderating impact on the influence of fertility desire on well-being, as the relationship from fertility desire to later depression or positive mental health did not display cross-cultural differences (that is, the constrained model was not significantly different from the unconstrained model).

The current findings point to the importance of positive mental health as a dimension in its own right, rather than simply an absence of mental health problems. Positive mental health is indeed a unique predictor not only of future positive mental health, but also of future mental health problems, above and beyond the predictive value of baseline negative mental health. However, it would be an oversimplification to aim toward positive feelings and away from negative feelings without taking context into account. As some authors have recently pointed out, feelings of sadness are healthy and normal in some circumstances, and feelings of positivity can be inappropriate and maladaptive under certain conditions. Life is a balance of positive and negative, both reactions can have their place as appropriate responses to life circumstances, and neither reaction is necessarily pathological [90, 91]. Future researchers are encouraged to examine both positive and negative emotions as not simply ends in themselves but as reactions to a broader context that can themselves be either healthy or unhealthy responses. Positive emotions (i.e., happiness) do not necessarily have to be the ultimate aim. Rather, a focus on euthymia (tranquility or calm), the wisdom that comes from challenge, and mastery of life tasks may be more attainable or even appropriate aims [91].

This study has a number of strengths, including the large sample size, its thorough assessments using standardized instruments and follow-up interview. It also examines relationships among the factors outside of the typical North American sample, extending knowledge and comparing models cross-culturally, across Germany, Russia, and China. Despite its strengths, the present study also has limitations. First, the present study does not compare results to a traditional standard-bearer culture in Western psychological research: the USA. Though comparing results to the US should not be the litmus test of a good study, it would be interesting to see if relationships are the same there, and elsewhere. Second, while the present study was comprehensive, it was not exhaustive. A more exhaustive study would have included more pathogenic variables, such a problematic personality factors. Third, while the model provided a good fit to the data, the effects were small, implying that a more complete picture would need to include additional factors. The pathways to mental health are many and complex. This model provides only some of the whole picture. Fifth, the study used university student samples, which are not representative of the population, and thus generalizability is limited. Sixth, the study has vastly different dropout rates for Chinese (lower dropout) versus German (higher dropout) and Russian (highest dropout rates) participants. In Germany and Russia, the participants were assessed at baseline in various semesters, while in China the participants were primarily assessed at baseline during their first semester, which made them more able to join in the follow-up studies. Further, in Germany the participants were all from the same university, but in Russia they were from different universities under the management of different contact persons (some were no longer in the research area after one year.). Thus, the differences in drop-out rates were likely caused by outside conditions rather than by the participants themselves, and are therefore not thought to have an impact on results. Seventh, the present study indicated higher positive mental health and lower negative mental health for Chinese participants on some measures. Unfortunately, measures of social desireable responding were not included in this study, so we cannot know if those differences were true cultural differences, or may have been due to potential cultural differences in socially desirable responding. Eighth, the present study examined the effects of predictors on later positive and negative mental health, but did not examine the potential reciprocal effects of depression and positive metnal health on some of the predictors. For example, it may be that people with depression are more able to uphold daily social rhythms and routines over time, and that the longitudinal effects are actualy reciprocal rather than unidirectional. Finally, the study lacked clinical sub-samples and the time period studied was relatively short for longitudinal studies, which may partially explain the generally small relationships in the findings.

In sum, positive mental health is predictive of both positive and negative mental health aspects over time, while negative mental health is primariliy predictive of negative mental health, but not positive mental health, with the exception of stress, which predicts both positive and negative mental health over time. These results hold across Germany, Russia, and China. Positive mental health is not simply an absence of mental health problems, but rather is predictive independently. Positive and negative mental health should be viewed as related, but independent dimensions. Positive mental health and salutogenic predictors should be included by default in future studies of the prediction of mental health and disorder, and should be considered more thoroughly as a target for intervention in both alleviating mental disorder and contributing to human flourishing [92].

Acknowledgments

We appreciate the contribution of the participants in this study, as well as Angela Bieda, Julia Brailovskaia, Pia Schönfeld, and Julia Velten for their assistance in data collection and management.

References

  1. 1. Prisciandaro JJ, Roberts JE. A comparison of the predictive abilities of dimensional and categorical models of unipolar depression in the National Comorbidity Survey. Psychological Medicine. 2009;39(7):1087–96. Epub 2008/10/11. pmid:18845012.
  2. 2. Wittchen HU, Hofler M, Gloster AT, Craske MG, Beesdo K. Options and Dilemmas of Dimensional Measures for DSM-5 Which Types of Measures Fare Best in Predicting Course and Outcome? Conceptual Evolution of Dsm-5. 2011:119–43.
  3. 3. Keyes CLM. Mental illness and/or mental health? Investigating the axioms of the complete state model of health. Journal of Consulting and Clinical Psychology. 2005;73(3):539–48. pmid:15982151
  4. 4. Keyes CLM. Promoting and protecting mental health as flourishing: A complementary strategy for improving national mental health. Am Psychol. 2007;62(2).
  5. 5. WHO. Mental health action plan 2013–2020. Geneva: 2013.
  6. 6. WHO. Mental health: a state of well-being 2014 [cited 2016 June 19]. http://www.who.int/features/factfiles/mental_health/en/.
  7. 7. Lukat J, Margraf J, Lutz R, van der Veld WM, Becker ES. Psychometric properties of the Positive Mental Health Scale (PMH-scale). BMC Psychol. 2016;4:8. Epub 2016/02/13. pmid:26865173.
  8. 8. Wood AM, Tarrier N. Positive Clinical Psychology: a new vision and strategy for integrated research and practice. Clinical Psychology Review. 2010;30(7):819–29. Epub 2010/07/27. pmid:20655136.
  9. 9. Margraf J. Hintergründe und Entwicklung. In: Margraf J, Schneider S, editors. Lehrbuch der Verhaltenstherapie Band 1: Diagnostik, Verfahren, Rahmenbedingungen. 3rd ed. Berlin: Springer; 2009.
  10. 10. Margraf J. Was wissen wir wirklich? (What do we really know?). Verhaltenstherapie. 2005;15:136–7.
  11. 11. Bromet E, Andrade LH, Hwang I, Sampson NA, Alonso J, de Girolamo G, et al. Cross-national epidemiology of DSM-IV major depressive episode. BMC Med. 2011;9. Artn 90 pmid:21791035
  12. 12. Pickett KE, James OW, Wilkinson RG. Income inequality and the prevalence of mental illness: a preliminary international analysis. J Epidemiol Community Health. 2006;60(7):646–7. Epub 2006/06/23. pmid:16790839 .
  13. 13. Jacobi F, Hofler M, Siegert J, Mack S, Gerschler A, Scholl L, et al. Twelve-month prevalence, comorbidity and correlates of mental disorders in Germany: the Mental Health Module of the German Health Interview and Examination Survey for Adults (DEGS1-MH). Int J Meth Psych Res. 2014;23(3):304–19. Epub 2014/04/15. pmid:24729411.
  14. 14. Scholten S, Velten J, Margraf J. How are structural and perceived macro-level factors related to mental distress? A preliminary cross-country study. Submitted.
  15. 15. Twenge JM, Gentile B, DeWall CN, Ma D, Lacefield K, Schurtz DR. Birth cohort increases in psychopathology among young Americans, 1938–2007: A cross-temporal meta-analysis of the MMPI. Clinical Psychology Review. 2010;30(2):145–54. Epub 2009/12/01. pmid:19945203.
  16. 16. Margraf J, Ehlers A. Etiological models of panic–medical and biological aspects. In: Baker R, editor. Panic disorder: Research and therapy. Chichester: Wiley; 1989.
  17. 17. Margraf J, Ehlers A. Etiological models of panic–psychophysiological and cognitive aspects. In: Baker R, editor. Panic disorder: Research and therapy. Chichester: Wiley; 1989.
  18. 18. Maercker A, Michael T, Fehm L, Becker ES, Margraf J. Age of traumatisation as a predictor of post-traumatic stress disorder or major depression in young women. Br J Psychiatry. 2004;184:482–7. Epub 2004/06/03. pmid:15172941.
  19. 19. Ehlers A, Margraf J. The psychophysiological model of panic. In: Emmelkamp PM, Everaerd W, Kraaymaat F, van Son M, editors. Fresh perspectives on anxiety disorders. Amsterdam: Swets; 1989.
  20. 20. Hofmann SG. Cognitive factors that maintain social anxiety disorder: a comprehensive model and its treatment implications. Cogn Behav Ther. 2007;36(4):193–209. Epub 2007/12/01. pmid:18049945.
  21. 21. Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behav Res Ther. 1997;35(8):741–56. Epub 1997/08/01. pmid:9256517.
  22. 22. WHO. Promoting mental health. Geneva: 2004.
  23. 23. Margraf J, Siegrist J, Neumer S, editors. Gesundheits- oder Krankheitstheorie? Saluto- versus pathogenetische Ansätze im Gesundheitswesen. Berlin: Springer; 1998.
  24. 24. Michael T, Zetsche U, Margraf J. Epidemiology of anxiety disorders. Psychiatry. 2007;6(4):136–42.
  25. 25. Trumpf J, Becker ES, Vriends N, Meyer AH, Margraf J. Rates and predictors of remission in young women with specific phobia: a prospective community study. Journal of Anxiety Disorders. 2009;23(7):958–64. Epub 2009/07/17. pmid:19604666.
  26. 26. Trumpf J, Margraf J, Vriends N, Meyer AH, Becker ES. Specific phobia predicts psychopathology in young women. Soc Psychiatry Psychiatr Epidemiol. 2010;45(12):1161–6. Epub 2009/11/06. pmid:19888542.
  27. 27. Trumpf J, Margraf J, Vriends N, Meyer AH, Becker ES. Predictors of specific phobia in young women: a prospective community study. Journal of Anxiety Disorders. 2010;24(1):87–93. Epub 2009/10/10. pmid:19815371.
  28. 28. Trumpf J, Vriends N, Meyer AH, Becker ES, Neumer SP, Margraf J. The Dresden Predictor Study of anxiety and depression: objectives, design, and methods. Soc Psychiatry Psychiatr Epidemiol. 2010;45(9):853–64. Epub 2009/09/05. pmid:19730759.
  29. 29. Rudaz M, Craske MG, Becker ES, Ledermann T, Margraf J. Health anxiety and fear of fear in panic disorder and agoraphobia vs. social phobia: a prospective longitudinal study. Depress Anxiety. 2010;27(4):404–11. Epub 2010/02/10. pmid:20143435.
  30. 30. Becker ES, Goodwin R, Holting C, Hoyer J, Margraf J. Content of worry in the community: What do people with generalized anxiety disorder or other disorders worry about? Journal of Nervous and Mental Disease. 2003;191(10):688–91. pmid:14555873
  31. 31. Velten J, Lavallee KL, Scholten S, Meyer AH, Zhang XC, Schneider S, et al. Lifestyle choices and mental health: a representative population survey. BMC Psychol. 2014;2(1):58. Epub 2015/01/30. pmid:25628891.
  32. 32. Vriends N, Becker ES, Meyer A, Margraf J. Incidence of DSM-IV Social Phobia in a Community Sample of Young German Women. German Journal of Psychiatry. 2011;14(2):80–90.
  33. 33. Vriends N, Becker ES, Meyer A, Michael T, Margraf J. Subtypes of social phobia: are they of any use? Journal of Anxiety Disorders. 2007;21(1):59–75. Epub 2006/06/30. pmid:16806802.
  34. 34. Vriends N, Becker ES, Meyer A, Williams SL, Lutz R, Margraf J. Recovery from social phobia in the community and its predictors: data from a longitudinal epidemiological study. Journal of Anxiety Disorders. 2007;21(3):320–37. Epub 2006/08/22. pmid:16919416.
  35. 35. Margraf J, Lavallee K, Zhang XC, Schneider S. Social Rhythm and Mental Health: A Cross-Cultural Comparison. Plos One. 2016;11(3). ARTN e0150312 pmid:26954568
  36. 36. Lukat J, Becker ES, Lavallee KL, van der Veld WM, Margraf J. Predictors of incidence, remission and relapse of Axis I mental disorders in young women: A transdiagnostic approach. Clinical psychology and psychotherapy. 2016. pmid:27256536
  37. 37. Maercker A, Zhang XC, Gao ZH, Kochetkov Y, Lu S, Sang ZQ, et al. Personal value orientations as mediated predictors of mental health: A three-culture study of Chinese, Russian, and German university students. Int J Clin Hlth Psyc. 2015;15(1):8–17. pmid:30487817
  38. 38. Iacovides A, Siamouli M. Comorbid mental and somatic disorders: an epidemiological perspective. Curr Opin Psychiatry. 2008;21(4):417–21. Epub 2008/06/04. pmid:18520749.
  39. 39. Mitchell AJ. Psychiatric aspects of chronic physical disease. Medicine 2012;40(12):662–5.
  40. 40. Becker ES, Margraf J, Turke V, Soeder U, Neumer S. Obesity and mental illness in a representative sample of young women. Int J Obes (Lond). 2001;25:S5–S9. pmid:11466578
  41. 41. Larsson CL, Klock KS, Nordrehaug AA, Haugejorden O, Johansson G. Lifestyle-related characteristics of young low-meat consumers and omnivores in Sweden and Norway. Journal of Adolescent Health. 2002;31:190–8. pmid:12127390
  42. 42. Eaton NR, Keyes KM, Krueger RF, Balsis S, Skodol AE, Markon KE, et al. An invariant dimensional liability model of gender differences in mental disorder prevalence: evidence from a national sample. J Abnorm Psychol. 2012;121(1):282–8. Epub 2011/08/17. pmid:21842958.
  43. 43. Maske UE, Busch MA, Jacobi F, Beesdo-Baum K, Seiffert I, Wittchen HU, et al. Current major depressive syndrome measured with the Patient Health Questionnaire-9 (PHQ-9) and the Composite International Diagnostic Interview (CIDI): results from a cross-sectional population-based study of adults in Germany. BMC Psychiatry. 2015;15. Artn 77 pmid:25884294
  44. 44. Batterham PJ, Christensen H, Mackinnon AJ. Mental health symptoms associatedd wtih morbidity, not mortality, in an elderly community sample. Soc Psychiatry Psychiatr Epidemiol. 2012;47(1):79–85. pmid:21052622
  45. 45. Hoertel N, McMahon K, Olfson M, Wall MM, Rodriguez-Fernandez JM, Lemogne C, et al. A dimensional liability model of age differences in mental disorder prevalence: evidence from a national sample. J Psychiatr Res. 2015;64:107–13. Epub 2015/04/11. pmid:25858414.
  46. 46. Stone AA, Schwartz JE, Broderick JE, Deaton A. A snapshot of the age distribution of psychological well-being in the United States. Proc Natl Acad Sci U S A. 2010;107(22):9985–90. Epub 2010/05/19. pmid:20479218.
  47. 47. Luhmann M, Schimmack U, Eid M. Stability and variability in the relationship between subjective well-being and income. J Res Pers. 2011;45:186–97.
  48. 48. Marmot MG, Wilkinson RG. Social determinants of health. 2 ed. Oxford: Oxford University Press; 2006.
  49. 49. House JS, Umberson D, Landis KR. Structures and Processes of Social Support. Annu Rev Sociol. 1988;14:293–318.
  50. 50. Lindorff M. Who gives what to whom? Managers and the receipt of social support. Int J Psychol. 2000;35(3–4):326–7.
  51. 51. Uchino BN, Cacioppo JT, KiecoltGlaser JK. The relationship between social support and physiological processes: A review with emphasis on underlying mechanisms and implications for health. Psychological Bulletin. 1996;119(3):488–531. pmid:8668748
  52. 52. Grandin LD, Alloy LB, Abramson LY. The social zeitgeber theory, circadian rhythms, and mood disorders: review and evaluation. Clinical Psychology Review. 2006;26(6):679–94. Epub 2006/08/15. pmid:16904251.
  53. 53. Thase ME, Jindal R, Howland RH. Biological aspects of depression. In: Gotlib IH, Hammen CL, editors. Handbook of Depression. New York: Guildford Press; 2002. p. 192–218.
  54. 54. Sylvia LG, Alloy LB, Hafner JA, Gauger MC, Verdon K, Abramson LY. Life events and social rhythms in bipolar spectrum disorders: a prospective study. Behav Ther. 2009;40(2):131–41. Epub 2009/05/13. pmid:19433144.
  55. 55. Ehlers CL, Frank E, Kupfer DJ. Social zeitgebers and biological rhythms. A unified approach to understanding the etiology of depression. Arch Gen Psychiatry. 1988;45(10):948–52. Epub 1988/10/01. pmid:3048226.
  56. 56. Modig K, Talbaeck M, Torssander J, Ahlbom A. Payback time? Influence of having children on mortality in old age. J Epidemiol Community Health. 2017;0:1–7.
  57. 57. Lorant V, Deliege D, Eaton W, Robert A, Philippot P, Ansseau M. Socioeconomic inequalities in depression: A meta-analysis. Am J Epidemiol. 2003;15(2):98–112.
  58. 58. Chang-Quan H, Zheng-Rong W, Yong-Hong L, Yi-Zhou X, Qing-Xiu L. Education and Risk for Late Life Depression: A Meta-Analysis of Published Literature. Int J Psychiat Med. 2010;40(1):109–24. pmid:20565049
  59. 59. Mauz E, Jacobi F. Mental Disorders and Social Inequalities in Three Birth Cohorts. Psychiatr Prax. 2008;35(7):343–52. pmid:18932114
  60. 60. Brink PJ. Transcultural versus cross-cultural. J Transcult Nurs. 1999;10(1):7. Epub 1999/09/07. pmid:10476141.
  61. 61. Margraf J, Schneider S. Bochum Optimism and Mental Health (BOOM) Studies: Protocol for a Multi-National Longitudinal Research Program. [Unpublished Manuscript]. In press 2017.
  62. 62. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9. Epub 1996/12/01. pmid:8970487.
  63. 63. Brislin RW. Back-Translation for Cross-Cultural Research. J Cross Cult Psychol. 1970;1(3):185–216.
  64. 64. Keyes CLM, Shmotkin D, Ryff CD. Optimizing well-being: The empirical encounter of two traditions. J Pers Soc Psychol. 2002;82(6):1007–22. pmid:12051575
  65. 65. Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Brit J Clin Psychol. 2005;44:227–39. pmid:16004657
  66. 66. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995;33(3):335–43. Epub 1995/03/01. pmid:7726811.
  67. 67. Bayram N, Bilgel N. The prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students. Soc Psychiatry Psychiatr Epidemiol. 2008;43(8):667–72. pmid:18398558
  68. 68. Dahm J, Wong DN, Ponsford J. Validity of the Depression Anxiety Stress Scales in assessing depression and anxiety following traumatic brain injury. J Affect Disorders. 2013;151(1):392–6. pmid:23830002
  69. 69. Ng F, Trauer T, Dodd S, Callaly T, Campbell S, Berk M. The validity of the 21-item version of the Depression Anxiety Stress Scales as a routine clinical outcome measure. Acta Neuropsychiatr. 2007;19(5):304–10. pmid:26952943
  70. 70. Lovibond PF, Lovibond SH. The structure of negative emotional states—comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995;33(3):335–43. pmid:7726811
  71. 71. Antony MM, Bieling PJ, Cox BJ, Enns MW, Swinson RP. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychol Assessment. 1998;10(2):176–81.
  72. 72. Wagnild GM, Young HM. Development and psychometric evaluation of the Resilience Scale. J Nurs Meas. 1993;1(2):165–78. Epub 1993/01/01. pmid:7850498.
  73. 73. Schumacher J, Leppert K, Gunzelmann T, Strauss B, Brahler E. The resilience scale—A questionnaire to assess resilience as a personality characteristic. Z Klin Psych Psychia. 2005;53(1):16–39.
  74. 74. Fydrich T, Sommer G, Tydecks S, Brähler E. Fragebogen zur sozialen Unterstützung (F-SozU): Normierung der Kurzform (K-14) (Social Support Questionnaire (F-SozU): Standardization of short form (K-14)). Zeitschrift für Medizinische Psychologie. 2009;18(1):43–8.
  75. 75. Nover T. A cross-cultural comparison of mental health status in adolescents [Doctoral Disseration]2012.
  76. 76. Fydrich T, Sommer G, Menzel U, Höll B. Fragebogen zur sozialen Unterstützung [Questionnaire about social support]. Zeitschrift Für Klinische Psychologie. 1987;16:434–6.
  77. 77. Bieda A, Hirschfeld G, Schoenfeld P, Brailovskaia J, Zhang XC. Universal Happiness? Cross-Cultural Measurement Invariance of Scales Assessing Positive Mental Health. Psychol Assessment. 2017;29(4):408–21. http://doi.org/10.1037/pas0000353.
  78. 78. Group TE. EQ-5D-3L User Guide2013 07/31/2014. http://www.euroqol.org.
  79. 79. Brooks R. EuroQol: The current state of play. Health Policy. 1996;37(53–72). pmid:10158943
  80. 80. Group TE. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. Epub 1990/11/05. pmid:10109801.
  81. 81. Boyce W, Torsheim T, Currie C, Zambon A. The family affluence scale as a measure of national wealth: Validation of an adolescent self-report measure. Soc Indic Res. 2006;78(3):473–87.
  82. 82. MacCallum RC, Wideman KF, Preacher KJ, Hong S. Sample size in factor analysis: The role of model error. Multivariate Behavioral Research. 2001;36(4):611–37. pmid:26822184
  83. 83. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing Structural Equation Models. Beverly Hills, CA: Sage; 1993. p. 136–62.
  84. 84. Steiger JH. Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research. 1990;25(2):173–80. pmid:26794479
  85. 85. Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238–46. pmid:2320703
  86. 86. Vandenberg RJ, Lance CE. A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research. Organ Res Methods. 2000);3(1):4–70.
  87. 87. Zhang X, Kuchinke L, Woud ML, Velten J, Margraf J. Survey Method Matters: Online/Offline Questionnaires and Face-to-Face or Telephone Interviews Differ. Computers in Human Behavior. 2017;71:172–80.
  88. 88. Kline P. The handbook of psychological testing. 2nd ed. London; New York: Routledge; 2000. vii, 744 p. p.
  89. 89. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling. 2008;14:464–504.
  90. 90. Wood AM, Davidson AT. Why the field of moral philosophy must guide any discussion on well‐being. World Psychiatry. 2020;19(1):53–5. pmid:31922676
  91. 91. Linden M. Euthymic suffering and wisdom psychology. World Psychiatry. 2020;19(1):55–6. pmid:31922666
  92. 92. Seligman MEP. Flourish: A Visionary New Understanding of Happiness and Well-Being: Atria Books; 2012.