https://orcid.org/0000-0001-7607-1305
Figures
Abstract
Depression, anxiety and stress symptoms cause substantial psychological and economic burdens around the globe. To mitigate the negative consequences, the negative symptoms should be identified at an early stage. Therefore, the implementation of very brief valid screening tools in mental health prevention programs and in therapeutic settings is advantageous. In two studies on representative German population samples, we developed and validated three ultra-short scales–the “bubbles”–that consist of only one item based on the Depression Anxiety Stress 21 subscales (DASS-21) for the assessment of depression, anxiety and stress symptoms. The results of Study 1 (N = 1,001) and Study 2 (N = 894) revealed that the bubbles are valid instruments that fit the DASS-21 subscales on the factor level. Moreover, the bubbles replicated the association pattern of the DASS-21 subscales with demographic variables, and with variables that belong to the negative and the positive dimension of mental health. Thus, due to their time- and cost-efficiency, the bubbles can be used as brief screening tools in research (e.g., large-scale studies, longitudinal studies, experience sampling paradigms) and in praxis. Their shortness can prevent fatigue, motivation decrease, and participants’ drop-out.
Citation: Brailovskaia J, Schneider S, Margraf J (2024) The “bubbles”-study: Validation of ultra-short scales for the assessment of depression, anxiety and stress symptoms. PLoS ONE 19(3): e0300923. https://doi.org/10.1371/journal.pone.0300923
Editor: Yuwadee Wittayapun, Walailak University, THAILAND
Received: November 9, 2023; Accepted: March 5, 2024; Published: March 20, 2024
Copyright: © 2024 Brailovskaia et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: Julia Brailovskaia received financial support by the Open Access Publication Funds of the Ruhr-Universität Bochum to cover the publication fees. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
One of every eight people in the world suffers from a mental health problem [1]. Mental health problems have a substantial negative effect on all areas of life [2]. They restrict a person’s quality of life and cause high economic burden and substantial financial costs to the community through direct therapy costs and indirect costs caused by, for example, absence from workplace and mortality [3–5]. Depression but also anxiety and stress symptoms belong to the leading causes of disability worldwide [2]. Research from various countries reported a significant increase of the negative symptoms since the global outbreak of the coronavirus disease 2019 (Covid-19) [6–10]. The enhanced level of depression, anxiety, and stress symptoms has been accompanied by a significant increase of suicide-related outcomes (i.e., suicide ideation and suicide attempts) in the general population and in inpatients [11–13].
Against this background, it is of great importance to identify the level of the negative symptoms at an early stage to prevent their further development and maintenance. To achieve this, we need very brief valid screening tools that can be implemented in mental health prevention programs and in therapeutic settings [14]. Currently there is a lack of such very brief validated screening tools. This often results in high drop-out rates especially in time-restricted conditions like large longitudinal panel surveys and in vulnerable populations such as clinical patients with a low attention span who cannot concentrate on long questionnaires [15, 16]. Thus, the development and validations of such brief tools is strongly desirable [17].
In the clinical context, the Beck Depression Inventory (BDI) [18, 19] and the Beck Anxiety Inventory (BAI) [20] are commonly used instruments for the assessment of depressiveness and anxiety. Both instruments consist of, respectively, 21 items which is long for a rapid general screening. Lovibond and Lovibond (21) demonstrated the validity of the Depression Anxiety Stress Scales 21 (DASS-21) that assess symptoms of depression, anxiety and stress over the past week with respectively seven items per subscale. The DASS-21 is closely linked to the Beck’s Inventories [21]. Its psychometric properties and measurement invariance have been shown across various countries [22–26]. Today, the DASS-21 belongs to the internationally well-established and well-validated instruments for the assessed of the negative symptoms in general population and in clinical setting [27–30]. However, 21 items seem still to be not brief enough for a rapid mental health screening. Shorter scales–especially single-items scales–would be of advantage [14, 25, 26, 31].
Previous research reported some shortcomings of single-item scales such as simplifying multidimensional topics and not being able to assess fine-grained differences between individuals [32]. However, they also have remarkable advantageous. The single-item scales are extremally time-saving that prevents fatigue, a decrease of motivation and participants’ drop-out [33]. Therefore, they can be used for brief screenings in clinical settings where patients often display a low attention span. Moreover, large representative studies and longitudinal studies with several measurement time points can benefit from the shortness of such time- and cost-efficient instruments. Available literature described single-items scales that assess constructs such as self-esteem [34], risk-taking [35], narcissism [33], need to belong [36], Fear of Missing Out [37], and happiness [38] to have adequate psychometric properties and to show similar valid results as the long-version measures of the constructs.
Against this background, we aimed to develop and validate ultra-short scales that consist of only one item based on the DASS-21 subscales for the assessment of depression, anxiety and stress symptoms. Therefore, we conducted two studies in population representative samples in Germany as part of a large ongoing project that investigates mental health factors. Notably, the second study aimed mainly to replicate the findings of the first study. The replication is important considering the often described replication crisis in psychology research [39].
In Study 1, we developed three ultra-short scales termed as “depression bubble”, “anxiety bubble”, and “stress bubble” based on expert reviews. Then, we investigated the validity of the bubbles. Therefore, we assessed whether each bubble belongs to the same unidimensional factor as the seven items of the corresponding DASS-21 subscale, we focused on the relationship between the bubbles and the DASS-21 subscales, and we compared their association pattern with other variables that previous research described to be associated with depression, anxiety and stress symptoms.
Considering the association pattern, we focused on available literature. In addition to the association between the bubbles and the symptoms of depression, anxiety and stress as assessed by the DASS-21, we involved demographic variables (age, gender, social status) and the positive mental health (PMH) in the analyses. It is worth noting that previous research has reported inconclusive findings regarding the relationship between negative symptoms and age. Some studies described a positive association [40], other reported a negative link [41–43]. Schönfeld, Brailovskaia [44] found a positive association between age and the negative symptoms in Russia, a negative association in Germany and no significant association in the U.S. Considering gender, higher levels of depression, anxiety and stress symptoms were reported for women than for men [45–49]. Furthermore, the negative symptoms were negatively associated with social status [50–52].
Notably, mental health is not only the absence of psychopathology [2]. Following dual-factor models, mental health consists of two distinct but correlated dimensions: negative and positive [53]. A status of absolute mental health is characterized by a low level of the negative dimension and a high level of the positive dimension [54]. The negative dimension is often operationalized by depression, anxiety and stress symptoms, the positive dimension can be operationalized by positive mental health (PMH) [55, 56]. There is a close positive relationship between the negative symptoms [21, 57]. PMH, i.e., emotional, cognitive and psychological well-being, is negatively associated with the negative symptoms as assessed by the DASS-21 subscales [58–60]. Thus, it was important to assess whether the bubbles are also negatively associated with the positive dimension of mental health operationalized by PMH. Findings on this issue would provide evidence of whether the bubbles show a similar association pattern as the DASS-21, considering the overall mental health construct, including its both dimensions, and therefore contribute to our understanding of the bubbles’ validity (specifically its convergent validity).
Study 2 replicated and extended the findings of Study 1 on the validity of the bubbles. After a further investigation of the question, whether each bubble belongs to the same unidimensional factor as the seven items of the corresponding DASS-21 subscale, we focused again on the associations of the bubbles in comparison to the DASS-21 subscales. Hereby, we extended the examination of the convergent validity of the bubbles by the inclusion of sense of control (negatively coded) that is a further operationalization of the negative dimension of mental health [61] and the inclusion of life satisfaction and social support that are often used in addition to PMH to operationalize the positive dimension of mental health [62]. Previous research has described that a low level of sense of control contributes to higher levels of depression, anxiety and stress symptoms [7, 60, 63]; the negative symptoms were negatively associated with life satisfaction [56, 64, 65], and persons who perceived a high level of social support from their close social network were at a lower risk for the negative symptoms [43, 66]. Thus, the inclusion of the three new variables should extend our understanding of the associations of the bubbles with further representatives of both mental health dimensions and their (convergent) validity. A similar association pattern to that of the DASS-21 could provide further evidence for the suitability of using the bubbles as a replacement for the DASS-21 scales in for example longitudinal research.
Overall, after the development of the bubbles, the findings of both studies should provide initial evidence for their validity.
General methods
For both studies, data were collected in June 2023 by an independent social marketing and research institute (Study 1: Talk Online, www.talkonlinepanel.com/de; Study 2: YouGov, www.yougov.de) via a population-based online-panel survey. Participants were recruited from the German residential population aged 18 years and above. Age (Talk Online: 18 to 29 years, 30 to 39 years, 40 to 49 years, 50 to 59 years, 60 years and older; YouGov: 18 to 24 years, 25 to 34 years, 35 to 44 years, 45 to 54 years, 55 years and older), gender (Talk Online and YouGov: male persons, female persons, gender diverse persons), and region/federal state (Talk Online and YouGov: the 16 German federal states) stratification were implemented to achieve representativeness for the German population. Notably, both institutes have a large number of panel-members from the residential population. The institutes invite members of their panel to participate in the survey as long as the demographics of the present sample correspond to those of the German population in terms of age, gender and region/federal state rates.
Participation was compensated by panel-specific tokens that can be converted in voucher or monetary payment. The requirement for participation was legal age (at least 18 years). There were no specific inclusion or exclusion criteria other than to belong to the residential population aged 18 years and above in Germany. All participants were properly instructed and gave online their informed consent to participate. The survey data were not analyzed at the level of individual persons. Participants were informed that it is not possible to clarify any conspicuous findings on mental health variables, as the data obtained are only evaluated in aggregated and anonymized form (no identification of individual persons is possible). The responsible Ethics Committee approved the current study’s implementation (application number: 20110512). It was pre-registered with AsPredicted.org on May 03, 2023 (Pre-registration Number: #130926). All national regulations and laws regarding human subjects research were followed. The study was conducted in accordance with the Declaration of Helsinki. All data sets used in the present study were complete. Following available literature, the calculation of a confirmatory factor analyses (CFA)–that was a significant part of Study 2 –requires at least a sample size of N = 200 for valid results [67]; considering an exploratory factor analyses (EFA) that was a significant part of Study 1, there are various suggestions on the required sample size that vary between N = 100 and 500 [67]. A priori calculated power analyses (G*Power program, version 3.1) showed that the correlation and regression analyses that were also calculated in the present study required a sample size below N = 200 [68]. However, considering that we aimed to develop new instruments, we decided to collect data of N = 800 to 1000 in each study.
Study 1
Methods
Procedure and participants.
Overall, 1,163 started the survey and 162 (13.9%) dropped out. Thus, the sample of Study 1 included 1,001 participants from Germany. Table 1 shows the demographic variables derived from the present sample. The dataset used in the present study is available in S1 Dataset.
Measures.
Demographics. Participants were asked to indicate their gender, age, marital status, occupation, living environment, and social status.
Depression, anxiety, and stress symptoms. The Depression Anxiety Stress Scales 21 (DASS-21; original version: [21]; German language version: [69]) assessed symptoms of depression, anxiety and stress over the past week with respectively seven items per subscale (e.g., depression subscale: “I felt that life was meaningless”; anxiety subscale: “I felt scared without any good reason”; stress subscale: “I found it hard to wind down”). Previous research described the unidimensional factor structure of the three DASS-21 subscales [21]. The 21 items are rated on a 4-point Likert-type scale (0 = did not apply to me at all, 3 = applies to me very much or most of the time). Higher sum scores indicate higher symptoms. The total sum score of each subscale can range from zero to 21. Current scale reliability was α = .936 for the depression subscale, α = .914 for the anxiety subscale, and α = .917 for the stress subscale.
Bubbles: Depression, anxiety and stress. We developed three bubbles–one per subscale of the DASS-21 –based on the items of the DASS-21 [69]. By the implementation of expert reviews by two psychology trained professionals, who evaluated the appropriateness of context, conciseness and wording, each of the seven items of a subscale was shortened to one word or a brief phrase consisting of several words. For example, the item of the DASS-21 depression subscale “I felt that life was meaningless” was shortened to the word “meaningless”; the item of the DASS-21 anxiety subscale “I felt scared without any good reason” was shortened to the phrase “groundless scared”; and the item of the DASS-21 stress subscale “I found it hard to wind down” was shortened to the word “restless”. Thus, the exact wording of the depression bubble was “sad”, “no initiative”, “joyless”, “depressed”, “no interest”, “worthless”, “meaningless”; the exact wording of the anxiety bubble was “dryness of mouth”, “breathlessness”, “tremble”, “worries”, “panic”, “beating of the heart”, “groundless scared”; and the exact wording of the stress bubble was “restless”, “irritated”, “tense”, “agitated”, “overexcited”, “thin-skinned”, “sensitive” (see Table 2 and Fig 1). The words/phrases that belong to a subscale were included in one visual bubble. Thus, each bubble can be considered as a single-item scale/instrument that includes the content of the seven items from the corresponding DASS-21 subscale in a shortened form. Participants had to rate how much/how often the overall content of each bubble applied to them over the past week (“Please rate how much/how often the overall content of the bubble applied to you over the past week”). Each bubble was rated on a horizontally arranged 4-point Likert-type scale (1 = not at all, 2 = a little bit / sometimes, 3 = substantial / often, 4 = very strong / mostly). The rating scale was presented below each bubble. The higher the rating, the higher the negative symptom represented by the bubble. For example, if a participant rated the depression bubble with “2” on the 4-point Likert-type scale, the person had a score of 2 for symptoms of depression.
Original German language Bubbles: a) Depression Bubble; b) Anxiety Bubble; c) Stress Bubble.
Positive Mental Health (PMH). The unidimensional Positive Mental Health Scale (PMH-Scale; original German language version: [70]) measured the level of PMH. The PMH-Scale is a well-established instrument for the assessment of emotional, cognitive and psychological well-being. It consists of nine items that are rated on a 4-point Likert-type scale (e.g., “I enjoy my life”; 0 = do not agree, 3 = agree). The higher the sum score, the higher the level of PMH. The total sum score can range from zero to 27. Current scale reliability is α = .937.
Statistical analyses.
Statistical analyses were conducted using the Statistical Package for the Social Sciences (SPSS) version 28 [71]. All investigated psychological variables were close to normally distributed (indicated by analyses of skewness, < 2.00, and kurtosis, < 7.00 [72]; see Table 4). We investigated the psychometric properties of the three bubbles. First, we calculated three EFAs to investigate whether each bubble belongs to the same unidimensional factor as the seven items of the corresponding DASS-21 subscale. Notably, available literature emphasized that depression, anxiety, and stress symptoms should be considered as separate entities [73, 74]. Often, a study focuses only one of those constructs [75]. Against this background, we made the decision to calculate an EFA for each construct separately instead of an EFA that includes all three constructs (that are 24 items: 21 DASS items and the three bubbles). Therefore, for each negative symptom, we ran an EFA for the seven items of the DASS-21 subscale with the corresponding bubble using the Maximum Likelihood method (ML; rotation method: promax) [76, 77]. Thus, each of the three EFAs included eight items (i.e., seven DASS-21 items and one bubble). Furthermore, we used the factor loadings of the EFAs to calculate the average variance extracted (AVE) for conclusions on the convergent validity of the bubbles. Convergent validity is established for AVE ≥ .50 [78].
In the next step, we assessed the reliability of the bubbles. As we were not able to investigate their test-retest reliability due to the cross-sectional study design, we calculated the internal consistency (Cronbach’s α) and the composite reliability (CR) [78] of the bubbles in relationship with the DASS-21 subscales to assess their reliability. This step revealed the internal consistency of the DASS-21 subscale items when adding the corresponding bubble. Considering that the bubbles aim to assess the same content as the DASS-21 subscale items, a high internal consistency provides support for the reaching of this aim.
Then, to further assess the validity of the bubbles and their closeness to the original scales, we investigated the correlation pattern of the bubbles with variables that were earlier shown to the linked to depression, anxiety and stress symptoms. Specifically, we assessed their association with demographic variables (age, gender, social status), the DASS-21 subscales and the PMH-Scale by the calculation of Pearson’s zero-order bivariate correlations and Spearman’s rank order correlations [79]. The associations with the mental health variables allowed further conclusions on the convergent validity of the bubbles. Furthermore, we compared the correlations of the bubbles with those of the corresponding DASS-21 subscale to test whether they exhibit a similar correlation pattern. This step revealed further information on the convergent validity of the bubbles. Following Cohen [80], we used the effect size Cohen’s q (small effect: .10 ≤ q < .30; medium effect: .30 ≤ q ≤ .50; large effect: q > .50) for the comparison of the correlations (bubble vs. DASS-21 subscale). To assess the potential predictive power of the bubbles in comparison to the power of the DASS-21 subscales, we calculated linear regression analyses that included, respectively, one of the bubbles as predictor and, respectively, the DASS-21 subscales and the PMH-Scale as outcome. Then, we replicated the regression analyses with the DASS-21 subscales, respectively, as predictor. The cross-sectional study design does not allow true conclusions on causality. However, the findings allowed a further comparison of both measures.
Results
Factor structure of the bubbles in relationship with the DASS-21 subscales: Exploratory Factor Analyses (EFAs).
Depression symptoms. The first EFA that included the DASS-21 depression subscale and the depression bubble (Kaiser-Meyer-Olkin: KMO = .951; Barlett’s test of sphericity: χ2 = 5864.526, df = 28, p < .001) revealed a unidimensional factor structure. The eigenvalue of the factor was 5.204, and it explained 65.1% of the variance which is sufficient for a one-factor structure [81]. Table 3 shows the factor loadings of the 8 items. The AVE was .650 revealing an adequate convergent validity.
Anxiety symptoms. The findings for anxiety symptoms were similar to those of depression symptoms. The EFA that included the DASS-21 anxiety subscale and the anxiety bubble revealed a unidimensional factor structure (KMO = .940; Barlett’s test of sphericity: χ2 = 4710.614, df = 28, p < .001). The eigenvalue of the factor was 4.665 and it explained 58.3% of the variance (see Table 3 for factor loadings). The AVE was .583 revealing an adequate convergent validity.
Stress symptoms. Also, the factor analysis that included the DASS-21 stress subscale and the stress bubble revealed a unidimensional factor structure (KMO = .935; Barlett’s test of sphericity: χ2 = 4710.726, df = 28, p < .001). The eigenvalue of the factor was 4.635 and it explained 57.9% of the variance (see Table 3 for factor loadings). The AVE was .579 revealing an adequate convergent validity.
Reliability of the bubbles in relationship with the DASS-21 subscales.
The overall scale reliability of the seven items of the DASS-21 depression subscale and the depression bubble was α = .935, and their composite reliability was .937; for the seven items of the DASS-21 anxiety subscale and the anxiety bubble, the overall scale reliability was α = .916, and the CR was .917; for the seven items of the DASS-21 stress subscale and the stress bubble, the overall scale reliability was α = .916, and the CR was .916.
Association pattern of the bubbles and convergent validity.
The descriptive statistics (that are means, standard deviations, minimum, maximum) of the bubbles and the other assessed variables are presented in Table 4. Table 5 displays the correlations between the investigated variables. Notably, the bubbles and the corresponding DASS-21 subscales showed very similar correlation pattern.
Correlations with demographic variables. As shown in Table 5, the three bubbles and the DASS-21 subscales were significantly negatively correlated with age. The correlations of the bubbles and the subscales did not show significant differences (bubble vs. DASS-21 subscale: depression: effect size q = .032, anxiety: q = .076, stress: q = .015; all: no effect).
The stress bubble was significantly negatively correlated with gender. The other bubbles and subscales were not significantly correlated with gender (see Table 5). There were no significant differences between the correlations of the bubbles and the subscales (bubble vs. DASS-21 subscale: depression: q = .076, anxiety: q = .0, stress: q = .019; all: no effect).
The three bubbles and the DASS-21 subscales were significantly negatively correlated with social status (see Table 5). The correlations did not significantly differ between the bubbles and the subscales (bubble vs. DASS-21 subscale: depression: q = .023, anxiety: q = .015, stress: q = .021; all: no effect).
Convergent validity. There was a significant positive correlation between depression, anxiety and stress symptoms–assessed by the bubbles and by the DASS-21 subscales. The depression bubble was significantly positively correlated with the anxiety bubble, r = .640, p < .001, and the stress bubble, r = .617, p < .001. The anxiety bubble was significantly positively correlated with the stress bubble, r = .549, p < .001. The positive association between the bubbles corresponds with the close positive relationship between the DASS-21 subscales (see Table 5).
Furthermore, Table 5 shows that the depression bubble was significantly positively correlated with the DASS-21 depression subscale. Both measures were significantly positively correlated with DASS-21 anxiety symptoms (DASS-21 > bubble: q = .524, large effect) and DASS-21 stress symptoms (DASS-21 > bubble: q = .634, large effect) (see Table 5). The anxiety bubble was significantly positively correlated with the DASS-21 anxiety subscale. Both measures were significantly positively correlated with DASS-21 depression symptoms (DASS-21 > bubble: q = .513, large effect) and DASS-21 stress symptoms (DASS-21 > bubble: q = .615, large effect). As shown in Table 5, the stress bubble was significantly positively correlated with the DASS-21 stress subscale. Both measures were significantly positively correlated with DASS-21 depression symptoms (DASS-21 > bubble: q = .742, large effect) and DASS-21 anxiety symptoms (DASS-21 > bubble: q = .749, large effect).
Moreover, the bubbles and the DASS-21 subscales were significantly negatively correlated with the PMH-Scale (see Table 5). The results of the bubbles did not significantly differ from those of the subscales (bubble vs. DASS-21 subscale: depression: q = .079, anxiety: q = .032, stress: q = .091; all: no effect).
Predictive power of the bubbles.
Table 6 shows the results of the regression analyses. All investigated regression models were significant revealing that the bubbles could predict symptoms of depression, anxiety, and stress as well as PMH. Again, the association of the bubbles with the DASS-21 subscales was weaker than the association between the subscales. However, the association between the bubbles and the PMH-Scale did not remarkably differ from the association of the subscales and the PMH-Scale (see Table 6).
Discussion
The main aim of Study 1 was to develop three ultra-short scales, each consisting of one item, based on the subscales of the DASS-21 to assess the level of depression, anxiety, and stress symptoms. Therefore, the content of each DASS-21 item was shortened to single words or brief phrases. The words/phrases that belong to a DASS-21 subscale were included in a visual bubble (see Fig 1). In the next step, we provided initial findings regarding the validity of the three bubbles.
First, we calculated three EFAs, each containing eight items (seven items of a DASS-21 subscale and the corresponding bubble). Each EFA revealed a unidimensional factor structure. This indicated the correspondence of the bubbles and the DASS-21 subscales on the factor level. Specifically, the findings show that the depression bubble belongs to the same factor as the seven DASS-21 depression symptom items; the anxiety bubble belongs to the same factor as the seven DASS-21 anxiety symptom items; and the stress bubble belongs to the same factor as the seven DASS-21 stress symptom items. Furthermore, the AVEs provided first evidence for an adequate convergent validity of the three bubbles. Also, the analysis of the reliability of the bubbles in relationship with the DASS-21 subscales was adequate supporting their correspondence. Notably, the reliability of the DASS-21 subscales with and without the bubbles did not remarkably differ.
The very similar association pattern between both measures and the demographic variables contributed also to the confirmation of the correspondence. The significant negative association of the negative symptoms with age revealed higher levels of depression, anxiety and stress symptoms in younger people. This finding corresponds with previous research from Germany [44] and China [82]. It could at least partly be explained by the fact that the increase of mental health problems since the Covid-19 outbreak has particularly affected younger people [7, 12, 82–86].
In line with earlier research [49], the stress bubble was significantly negatively linked to gender revealing higher stress in female participants. However, the DASS-21 stress subscale and the other instruments were not significantly linked to gender. This finding was unexpected considering previous results on gender and the mental health variables [45, 48]. However, we found the non-significance for both the bubbles and the DASS-21 subscales. Thus, it seemed to be a characteristic of the investigated sample rather than a shortcoming of the bubbles.
All instruments were significantly negatively associated with social status revealing higher depression, anxiety and stress symptoms in people with a lower social status. This result corresponds with available literature [50, 51]. A low social status is often accompanied by low education and the lack of functional strategies to cope with daily hassles [52]. A permanent experience of stressful events in everyday life without the possibility of an adequate coping can negatively impact mental health and result in high levels of depression, anxiety and stress symptoms [87, 88].
Furthermore, the very similar association pattern of the bubbles and the DASS-21 subscales with variables of negative and positive mental health–as shown by the correlation and regression analyses–confirmed their correspondence and provided evidence for the convergent validity of the bubbles. In line with previous results [21, 43, 89], we found a significant positive association between depression, anxiety and stress symptoms–assessed by the bubbles and by the DASS-21 subscales. Notably, the three negative symptoms can enhance the level of each other [43].
PMH that represents the positive dimension of mental is an important protective factor against mental disorders and suicide-related outcomes [90, 91]. Persons with a high level of PMH are characterized by self-efficacy, resilience and the ability to cope with stressful experiences adequately [92]. In line with available literature [93, 94], the bubbles and the DASS-21 subscales were significantly negatively associated with PMH in the current study.
The predictive power of the bubbles and the DASS-21 subscales showed a similar result pattern. Thus, our findings revealed a high correspondence between the bubbles and the DASS-21 subscales and a good convergent validity of the three bubbles. Notably, their association with the DASS-21 subscales was weaker than the relationship between the subscales. However, the association of the bubbles and the DASS-21 subscales with PMH was similarly strong.
To sum up, the results of Study 1 provide initial evidence that the three bubbles are valid instruments for a rapid screening of the level of depression, anxiety and stress symptoms. Their correlations and predictive power are very similar to those of the 21 items of the DASS-21. Against this framework, we can assume that the use of the bubbles could be of great advantage due to their time- and cost-efficiency especially in large-scale representative studies and longitudinal studies.
Study 2
Methods
Procedure and participants.
Overall, 1,066 persons started the survey and 172 (16.1%) dropped out. Thus, the sample of Study 2 consisted of 894 participants from Germany (see Table 1 for demographic variables derived from the present sample). The dataset used in the present study is available in S2 Dataset.
Measures.
Same instruments as in Study 1. DASS-21 for the assessment of depression (current scale reliability of depression subscale: α = .922), anxiety (current scale reliability of anxiety subscale: α = .892) and stress symptoms (current scale reliability of stress subscale: α = .911); depression bubble, anxiety bubble, and stress bubble; PMH-Scale for the assessment of PMH (current scale reliability: α = .927).
Further instruments. Sense of control. Following Niemeyer, Bieda [52], we assessed sense of control by the two items “Do you experience important areas of your life (i.e., work, free-time, family, etc.) to be uncontrollable, meaning that you cannot, or barely can, influence them?” and “Do you experience these important areas of your life as unpredictable or inscrutable?”. The two items are rated on a 5-point Likert-type scale (0 = not at all, 4 = very strong). Higher sum scores indicate lower level of sense of control. The total sum score can range from 0 to 8. Current scale reliability is α = .865.
Life satisfaction. The unidimensional Satisfaction with Life Scale assessed life satisfaction (SWLS; original version: [95]; German version: [96]). This unidimensional instrument includes five items that are rated on a 7-point Likert-type scale (e.g., “In most ways, my life is close to my ideal.”; 1 = strongly disagree, 7 = strongly agree). The higher the sum score, the higher the level of life satisfaction. The total sum score can range from five to 35. Current scale reliability is α = .917.
Perceived social support. We assessed anticipated or perceived support received from the social network by the brief form of the Social Support Questionnaire (F-SozU K-6; original German language version: [66]) that is a unidimensional measure of social support. It consists of six items (e.g., “I experience a lot of understanding and security from others.”) that are rated on a 5-point Likert-type scale (1 = not true, 5 = true). Higher sum scores indicate higher levels of social support. The total sum score can range from 5 to 30. Current scale reliability is α = .901.
Statistical analyses.
Statistical analyses were conducted using SPSS version 28 [71] and the open statistical software jamovi (version 2.3.26.0; www.jamovi.org). All investigated psychological variables were close to normally distributed (indicated by analyses of skewness, < 2.00, and kurtosis, < 7.00 [72]; see Table 4). An EFA and a CFA should not be calculated with the same sample [76, 77]. Therefore, we used the sample of Study 2 to replicate the unidimensional factor structure of the three DASS-21 subscales including the corresponding bubbles by the calculation of three CFAs. Again, we made the decision to calculate a CFA for each construct separately instead of a CFA that includes all three constructs (that are 24 items: 21 DASS items and the three bubbles) based on previous research that emphasized the need to consider depression, anxiety, and stress symptoms as separate entities [73, 74]. Thus, each CFA included eight items (seven items of a DASS-21 subscale and the corresponding bubble). Because of the sample size sensitivity of the chi-square test [97], we took further fit indices into consideration [98]: The comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean residual (SRMR). Considering the cut-off criteria, values > .90 indicate a good fit for the CFI [99], values ≤ .08 indicate a reasonable fit for the RSMEA and values ≤ .05 reveal a good RSMEA fit [100], and values < .08 indicate a good fit for the SRMR [101]. We also examined the factor loadings of the CFAs, with a minimum for factor loading set at .400 [102]. Next, to assess the reliability of the bubbles, we calculated their internal consistency (Cronbach’s α) in relationship with the DASS-21 subscales. This allowed further conclusions on the correspondence between the bubbles and the DASS-21 subscales. Then, we focused again on the associations of the bubbles to replicate the findings of Study 1 and to extend them. We assessed the bubbles’ association with age, gender and social status by Pearson’s zero-order bivariate correlations and Spearman’s rank order correlations [79]. To gain further information about the convergent validity of the bubbles, we assessed their association with the DASS-21 subscales and sense of control (negatively coded), as well as the PMH-Scale, SWLS and F-SozU K-6 by the calculation of Pearson’s zero-order bivariate correlations. We also compared the associations of each bubble with the associations of the corresponding DASS-21 subscale (bubbles vs. DASS-21 subscale, effect size Cohen’s q [80]). Linear regression analyses that included, respectively, one of the bubbles as a predictor and, respectively, a DASS-21 subscale, sense of control, PMH-Scale, SWLS, or F-SozU K-6 as the outcome replicated the potential predictive power of the bubbles. Then, the regression analyses were calculated with the DASS-21 subscales, respectively, as a predictor. This step allowed a further comparison of both measures.
Results
Factor structure of the bubbles in relationship with the DASS-21 subscales: Confirmatory Factor Analyses (CFAs).
Table 7 shows the results of the three CFAs. The CFA that included the depression bubble and the DASS-21 depression subscale in a one-factor structure resulted in a significant chi-square value. The CFI indicated a good fit, the RMSEA indicated a reasonable fit, and the SRMR indicated a good fit. The factor loadings ranged between .594 (depression bubble) and .773 (Item 3 of the DASS-21 depression subscale) (see Table 7).
The CFA that included the anxiety bubble and the DASS-21 anxiety subscale in a one-factor structure resulted in a significant chi-square value. The CFI indicated a good fit, the RMSEA indicated a reasonable fit, and the SRMR indicated a good fit. The factor loadings ranged between .538 (anxiety bubble) and .705 (Item 7 of the DASS-21 anxiety subscale) (see Table 7). The CFA that included the stress bubble and the DASS-21 stress subscale in a one-factor structure resulted in a significant chi-square value. The CFI indicated a good fit, the RMSEA indicated a reasonable fit, and the SRMR indicated a good fit. The factor loadings ranged between .546 (stress bubble) and .785 (Item 5 of the DASS-21 stress subscale) (see Table 7).
Association pattern of the bubbles and convergent validity.
Table 4 shows the descriptive statistics of the bubbles and the other assessed variables (that are means, standard deviations, minimum, maximum).
Correlations with demographic variables. The three bubbles and the DASS-21 subscales were significantly negatively correlated with age and social status (see Table 5). The correlations did not significantly differ between the bubble and the subscale for depression (bubble vs. DASS-21 subscale: age: q = .031, social status: q = .032; both: no effect), anxiety (bubble vs. DASS-21 subscale: age: q = .074, social status: q = .036; both: no effect), and stress (bubble vs. DASS-21 subscale: age: q = .096, social status: q = .019; both: no effect). The stress bubble was significantly negatively correlated with gender. The other measures were not significantly correlated with gender (see Table 5). There were no significant differences between the correlations of the bubbles and the subscales (bubble vs. DASS-21 subscale: depression: q = .031, anxiety: q = .009, stress: q = .029; all: no effect).
Convergent validity. There was a significant positive correlation between depression, anxiety and stress symptoms–assessed by the bubbles and by the DASS-21 subscales. The depression bubble was significantly positively correlated with the anxiety bubble, r = .489, p < .001, and the stress bubble, r = .571, p < .001. The anxiety bubble was significantly positively correlated with the stress bubble, r = .387, p < .001.
Each bubble was significantly positively correlated with the corresponding DASS-21 subscale (see Table 5). Furthermore, both depression measures were significantly positively correlated with DASS-21 anxiety symptoms (DASS-21 > bubble: q = .435, medium effect) and DASS-21 stress symptoms (DASS-21 > bubble: q = .676, large effect). Both anxiety measures were significantly positively correlated with DASS-21 depression symptoms (DASS-21 > bubble: q = .604, large effect) and DASS-21 stress symptoms (DASS-21 > bubble: q = .680, large effect). Both stress measures were significantly positively correlated with DASS-21 depression symptoms (DASS-21 > bubble: q = .578, large effect) and DASS-21 anxiety symptoms (DASS-21 > bubble: q = .412, medium effect) (see Table 5). Furthermore, both depression measures (DASS-21 > bubble: q = .253, small effect), both anxiety measures (DASS-21 > bubble: q = .201, small effect), and both stress measures (DASS-21 > bubble: q = .352, medium effect) were significantly positively correlated with a low level of sense of control (see Table 5).
Considering the variables that represent the positive dimension of mental health, the bubbles and the DASS-21 subscales were significantly negatively correlated with PMH (depression: q = .042; anxiety: q = .001; stress: q = .054, all: no effect) (see Table 5). Moreover, all measures were significantly negatively correlated with life satisfaction (depression: q = .031; anxiety: q = .015; stress: q = .029, all: no effect) and social support (depression: q = .005; anxiety: q = 0; stress: q = .014, all: no effect) (see Table 5).
Predictive power of the bubbles.
As shown in Table 6, all investigated regression models were significant revealing that the bubbles could predict symptoms of depression, anxiety, and stress, sense of control, PMH, life satisfaction, and social support. Again, the association of the bubbles with the negative variables was weaker than the association of the DASS-21 subscales. However, the association between the bubbles and the positive variables did not remarkably differ from the association of the DASS-21 subscales (see Table 6).
Discussion
Study 2 aimed to replicate and to extend the findings of Study 1 on the validity of the three new developed bubbles. The CFAs confirmed the results of the EFAs in Study 1. Thus, the one-factor models fit the data well for depression, anxiety, and stress. This again emphasized the correspondence of the bubbles and the DASS-21 subscale on the factor level. The correlation analyses and the regression analyses replicated the result pattern of Study 1. In line with available literature [44, 51] and the findings of Study 1, the three bubbles and the DASS-21 subscales were significantly negatively associated with age and social status revealing higher depression, anxiety and stress symptoms in younger persons and in persons with a lower social status. Also in line with Study 1, the stress bubble was significantly negatively associated with gender revealing a higher stress level in female participants. Thus, the bubbles and the DASS-21 subscales showed again very similar correlations with demographic variables which emphasizes their correspondence.
In line with earlier research [21] and Study 1, the bubbles and the DASS-21 subscales were significantly positively associated which provided further support for their correspondence. Also, high levels of both depression measures, both anxiety measures and both stress measures were accompanied by a low level of sense of control. This finding corresponds with available literature [60]. A loss of control in important areas of everyday life is a significant characteristic of the negative symptoms [103]. This provides further support of the bubbles’ convergent validity.
As a further replication of Study 1 findings and in line with available literature [60], the bubbles and the DASS-21 subscales were significantly negatively linked to PMH. Notably, life satisfaction and social support are often considered as further factors of the positive dimension of mental health [62]. While they often exhibit a similar association pattern to PMH, it is important to note that the associations are not always consistent. Therefore, it has been recommended to investigate all of them to assess a wide-ranging spectrum of the positive dimension [62, 104, 105]. Against this background, we added the variables life satisfaction and social support in the present investigation of the validity of the bubbles. Notably, the three bubbles and the DASS-21 subscales were significantly negatively linked to life satisfaction and social support. This finding was in line with previous research [43, 56, 65].
Furthermore, in line with Study 1, the predictive power of the bubbles and the DASS-21 subscales showed a very similar pattern. Thus, the results replicated the correspondence of the three bubbles and the DASS-21 scales as well as the bubbles’ convergent validity. Again, their association with the negative variables (i.e., DASS-21 subscales, sense of control) was weaker than the associations of the DASS-21 subscales. However, the association of the bubbles and the subscales with the positive variables was similarly strong.
To sum up, the findings of Study 2 confirmed the evidence gained in Study 1 that the three bubbles are valid time- and cost-efficient instruments for a brief screening of the level of depression, anxiety and stress symptoms.
General discussion
More than 970 million people around the globe suffer from mental health problems [1]. Depression and anxiety disorders belong to the most common [1]. This has a negative impact not only on the affected individuals but also on the worldwide economy [106, 107]. Following the World Health Organization (2), depression and anxiety cost the global economy about one trillion US dollars annually. Depression and anxiety are closely associated with stress symptoms [21, 108]. By an early identification of persons at risk for enhanced depression, anxiety and stress symptoms, we could prevent their further development and maintenance. This would alleviate the suffering of the affected persons and their families, and it could have positive effects on both local and global economies.
To achieve this aim, a brief and valid assessment of the negative symptoms is of great importance. It could help to identify persons at risk and to observe potential changes of the symptoms over time. Against this background, we developed and validated three ultra-short scales for the assessment of depression, anxiety and stress symptoms–the “bubbles” in two population representative studies.
Our results demonstrated that the three bubbles are valid instruments that due to their shortness could be practical and economical tools for an initial very brief screening of the level of the negative symptoms in German language samples. Notably, we replicated the findings of previous research that used longer instruments for the assessment of depression, anxiety and stress symptoms.
The findings of Study 1 showed that the bubbles fit the DASS-21 subscales on the factor level. Furthermore, the bubbles were sensitive enough to replicate the association pattern of the subscales with demographic variables, and with variables that belong to the negative and the positive dimension of mental health.
The findings of Study 2 further confirmed the correspondence between the bubbles and the DASS-21 subscales. Again, the bubbles replicated the association pattern of the subscales for demographics and for further positive and negative variables.
Notably both studies revealed that the associations between the bubbles and the representatives of the negative dimension of mental health were weaker than the associations of the DASS-21 subscales with those variables. However, there was no significant difference considering the associations of the bubbles and the DASS-21 subscales with representatives of the positive dimension of mental health. This finding is in line with earlier research that validated the Single-Item Self-Esteem Scale (SISE) [34] that is an single-item measure of self-esteem. The authors compared SISE with the 10-item Rosenberg Self-Esteem Scale (RSE-Scale) [109, 110]. The relationship between the depression symptoms and self-esteem assessed by the SISE was remarkably weaker than their association with self-esteem measured by the RSE-Scale. However, the SISE self-esteem was strongly associated with subjective happiness and perceived social support [34]. Therefore, it could be that the single-item scales are especially suitable for investigations of associations between the appropriate construct and the positive dimension of mental health. This, however, does not mean that they should not be used in investigations that focus on the negative dimension of mental health.
Thus, similar to other single-item scales (e.g., risk-taking [35], happiness [38]), the bubbles seem to be a valid and economic instrument for a brief screening of the level of depression, anxiety and stress symptoms. Due to their time- and cost-efficiency, the bubbles can be of specific benefit in large-scale surveys, in longitudinal studies in order not to overload participants, to prevent cognitive fatigue and, therefore, attention deficits and enhanced drop-outs, and when a brief screening, for example in mental health prevention programs or assessment centers, is required. Moreover, the bubbles can be of significant benefit in research using the experience sampling paradigm to assess for example the link between the negative symptoms and specific experiences in everyday life. Considering their close correspondence with the DASS-21 subscales, the bubbles could be also used in addition to the subscales for example in experimental research. Specifically, in a study that includes a two-week intervention, the DASS-21 subscales could be used in the baseline measurement and the post-measurement to assess the negative symptoms, and the bubbles could assess the negative symptoms in a brief daily screening during the intervention period. In addition, some research uses a paper-and-pencil format for data collection, followed by a transfer of the data into a digital format. This process typically requires a double-checking and is susceptible to imputation errors. The larger the number of items included in the data collection, the more time is required for the transfer and double-checking and the higher the risk for imputation errors [111]. The shortness of the bubbles can mitigate data processing time and errors in such studies. Moreover, it reduces the amount of paper required, contributing to sustainability and climate protection.
Limitations and future research
The following limitations should be taken into account when interpreting our findings. First, the cross-sectional study design did not allow us to assess the test-retest reliability of the bubbles. The investigation of this form of reliability is important, as single-item scales could be prone to lower reliability than instruments that consist of several items [112, 113].
And we could not assess the time sensitivity of the bubbles–specifically, whether they can measure short- and longer-term changes of depression, anxiety, and stress symptoms. Therefore, we suggest future research to test the bubbles by a longitudinal study design. Second, in order not to overload the participants, we included only a limited number of variables for the examination of the validity of the bubbles and we focused on their convergent validity. Future validation studies of the bubbles should include further measures of depression, anxiety and stress symptoms. Moreover, they should include further negative variables to examine the convergent validity of the bubbles such as insomnia and suicide-related outcomes [114], as well as variables that are not or only very weakly related to the negative symptoms to assess the discriminant validity of the bubbles. Third, future research should examine the validity of the bubbles in clinical patients with different diagnoses. This would provide evidence on the applicability and effectiveness of the bubbles as a measure of negative mental health in various clinical populations. Furthermore, the present work is the initial step for the development and validation of the three bubbles. So far, they are not considered as sensitive instruments for a clinically relevant diagnosis of for example an anxiety disorder. Future studies are required to evaluate their diagnostic accuracy and to specify cut-off points/values for clinically relevant levels of depression, anxiety and stress symptoms. Therefore, investigations in clinical populations would also be of advantage. Hereby, we recommend to involve measures of diagnostic accuracy such as the Area under the Recover Operating Characteristic (AuROC) curve, sensitivity and specificity, positive and negative predictive values (PPV, NPV) [115, 116]. Fourth, we developed the bubbles by transforming the DASS-21 items into short phrases or single words. We cannot exclude the possibility that this transformation potentially impacted the content validity of the constructs we converted into bubbles. Against this background, we recommend future research to assess whether the content of the bubbles can be understood in the same way as that of the DASS-21 subscales. Focus groups that discuss the content of both instruments could be a supportive initiate step in doing so (see also [117]). Furthermore, to test the correspondence of the depression bubble with the DASS-21 depression subscale, of the anxiety bubble with the DASS-21 anxiety subscale, and of the stress bubble with the DASS-21 stress subscale on the factor level, we calculated three EFAs in Study 1 and did not focus on the 21 items of the DASS-21 as an overall concept. Therefore, the analysis of whether the single items of the DASS-21 and the single bubbles have a propensity to be associated with other factors or components of the instrument or not has been neglected. Future research is recommended to test the 21 items of the DASS-21 and the three bubbles in one EFA to further elaborate this issue. Fifth, the validity of the DASS-21 has been shown in various countries [27]. We tested the bubbles in German samples only. Further translations of the bubbles to other languages as German and their validations in other countries are desirable. Sixth, we developed the bubbles only for symptoms of negative mental health. Considering that mental health includes a positive dimension also, the development and validation of further bubbles for variables such as PMH and life satisfaction are desirable.
Conclusion
To sum up, the depression bubble, the anxiety bubble, and the stress bubble revealed to be valid, cost- and time efficient instruments for a brief screening of the level of the negative symptoms. Their use in research and praxis could speed up the working processes and contribute to a rapid identification of persons at risk for enhanced levels of depression, anxiety and stress symptoms as well as related mental health problems. Future studies are required for the validation of the bubbles in clinical context and cross-national.
Supporting information
S1 Dataset. Dataset used for analyses in present study (Study 1).
https://doi.org/10.1371/journal.pone.0300923.s001
(XLSX)
S2 Dataset. Dataset used for analyses in present study (Study 2).
https://doi.org/10.1371/journal.pone.0300923.s002
(XLSX)
References
- 1.
World Health Organization. Mental disorders 2022. Available from: https://www.who.int/news-room/fact-sheets/detail/mental-disorders.
- 2.
World Health Organization. Mental Health 2023. Available from: https://www.who.int/health-topics/mental-health#tab=tab_1.
- 3. Arango C, Díaz-Caneja CM, McGorry PD, Rapoport J, Sommer IE, Vorstman JA, et al. Preventive strategies for mental health. The Lancet Psychiatry. 2018;5(7): 591–604. pmid:29773478
- 4. Liu CH, Zhang E, Wong GTF, Hyun S. Factors associated with depression, anxiety, and PTSD symptomatology during the COVID-19 pandemic: Clinical implications for US young adult mental health. Psychiatry Research. 2020: 113172. pmid:32512357
- 5. Teismann T, Düwel H, Eidt L, Brailovskaia J, Cwik JC. Psychotherapists’ knowledge of guideline recommendations for the treatment of depressed suicidal patients. Frontiers in Psychiatry. 2022. pmid:35492714
- 6. O’Connor RC, Wetherall K, Cleare S, McClelland H, Melson AJ, Niedzwiedz CL, et al. Mental health and well-being during the COVID-19 pandemic: longitudinal analyses of adults in the UK COVID-19 Mental Health & Wellbeing study. The British Journal of Psychiatry. 2021;218: 326–33. pmid:33081860
- 7. Brailovskaia J, Margraf J. Less sense of control, more anxiety, and addictive social media use: Cohort trends in German university freshmen between 2019 and 2021. Current Research in Behavioral Sciences. 2023;4: 100088.
- 8. Alimoradi Z, Ohayon MM, Griffiths MD, Lin C-Y, Pakpour AH. Fear of COVID-19 and its association with mental health-related factors: systematic review and meta-analysis. BJPsych Open. 2022;8(2): 1–26. pmid:35307051
- 9. Alimoradi Z, Gozal D, Tsang HWH, Lin CY, Broström A, Ohayon MM, et al. Gender‐specific estimates of sleep problems during the COVID‐19 pandemic: systematic review and meta‐analysis. Journal of Sleep Research. 2022;31(1): e13432. pmid:34245055
- 10. Alimoradi Z, Broström A, Tsang HWH, Griffiths MD, Haghayegh S, Ohayon MM, et al. Sleep problems during COVID-19 pandemic and its’ association to psychological distress: A systematic review and meta-analysis. EClinicalMedicine. 2021;36: 100916. pmid:34131640
- 11. Zhou S-J, Wang L-L, Qi M, Yang X-J, Gao L, Zhang S-Y, et al. Depression, Anxiety, and Suicidal Ideation in Chinese University Students During the COVID-19 Pandemic. Frontiers in Psychology. 2021;12: 669833. pmid:34421725
- 12. Brailovskaia J, Teismann T, Friedrich S, Schneider S, Margraf J. Suicide ideation during the COVID-19 outbreak in German university students: Comparison with pre-COVID 19 rates. Journal of Affective Disorders Reports. 2021;6: 100228. pmid:34632441
- 13. Berardelli I, Sarubbi S, Rogante E, Cifrodelli M, Erbuto D, Innamorati M, et al. The impact of the COVID-19 pandemic on suicide ideation and suicide attempts in a sample of psychiatric inpatients. Psychiatry research. 2021;303: 114072. pmid:34256349
- 14. Allen MS, Iliescu D, Greiff S. Single item measures in psychological science. Psychologocal Assessment. 2022;38(1): 1–5.
- 15. Diggle P, Farewell D, Henderson R. Analysis of longitudinal data with drop-out: objectives, assumptions and a proposal. Journal of the Royal Statistical Society Series C: Applied Statistics. 2007;56(5): 499–550.
- 16. Sharma T, Guski LS, Freund N, Meng DM, Gøtzsche PC. Drop-out rates in placebo-controlled trials of antidepressant drugs: A systematic review and meta-analysis based on clinical study reports. International Journal of Risk & Safety in Medicine. 2019;30(4): 217–32. pmid:31561390
- 17. Kogar H. Development of a Short Form: Methods, Examinations, and Recommendations. Journal of Measurement and Evaluation in Education and Psychology. 2020;11(3): 302–10.
- 18.
Hautzinger M, Keller F, Kühner C. Beck Depressions-Inventar (BDI-II; Revision; 2. Aufl.). Göttingen: PsychCorp; 2009.
- 19.
Beck AT, Steer RA. Manual for the revised Beck Depression Inventory. San Antonio, Tex: The Psychological Corporation; 1987.
- 20.
Beck AT, Steer RA. Manual for the Beck Anxiety Inventory. San Antonio, Tex: The Psychological Corporation; 1990.
- 21. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy. 1995;33(3): 335–43. pmid:7726811.
- 22. Scholten S, Velten J, Bieda A, Zhang XC, Margraf J. Testing measurement invariance of the Depression, Anxiety, and Stress Scales (DASS-21) across four countries. Psychological Assessment. 2017;29(11): 1376–90. pmid:28125249
- 23. Norton PJ. Depression Anxiety and Stress Scales (DASS-21): psychometric analysis across four racial groups. Anxiety, Stress, & Coping. 2007;20(3): 253–65. pmid:17999228.
- 24. Oei TPS, Sawang S, Goh YW, Mukhtar F. Using the depression anxiety stress scale 21 (DASS-21) across cultures. International Journal of Psychology. 2013;48(6): 1018–29. pmid:23425257
- 25. Cao C-H, Liao X-L, Jiang X-Y, Li X-D, Chen IH, Lin C-Y. Psychometric evaluation of the depression, anxiety, and stress scale-21 (DASS-21) among Chinese primary and middle school teachers. BMC psychology. 2023;11(1): 209–24. pmid:37452365
- 26. Chen IH, Chen C-Y, Liao X-l, Chen X-m, Zheng X, Tsai Y-C, et al. Psychometric properties of the Depression, Anxiety, and Stress Scale (DASS-21) among different Chinese populations: a cross-sectional and longitudinal analysis. Acta Psychologica. 2023;240: 104042. pmid:37783184
- 27. Zanon C, Brenner RE, Baptista MN, Vogel DL, Rubin M, Al-Darmaki FR, et al. Examining the dimensionality, reliability, and invariance of the Depression, Anxiety, and Stress Scale–21 (DASS-21) across eight countries. Assessment. 2021;28(6): 1531–44. pmid:31916468
- 28. Lee D. The convergent, discriminant, and nomological validity of the Depression Anxiety Stress Scales-21 (DASS-21). Journal of Affective Disorders. 2019;259: 136–42. pmid:31445339
- 29. Yohannes AM, Dryden S, Hanania NA. Validity and responsiveness of the Depression Anxiety Stress Scales-21 (DASS-21) in COPD. Chest. 2019;155(6): 1166–77. pmid:30594559
- 30. Cao C-h, Liao X-l, Gamble JH, Li L-l, Jiang X-Y, Li X-D, et al. Evaluating the psychometric properties of the Chinese Depression Anxiety Stress Scale for Youth (DASS-Y) and DASS-21. Child and Adolescent Psychiatry and Mental Health. 2023;17(1): 106–21. pmid:37679819
- 31. Kyriazos TA, Stalikas A, Prassa K, Yotsidi V. Can the Depression Anxiety Stress Scales Short be shorter? Factor structure and measurement invariance of DASS-21 and DASS-9 in a Greek, non-clinical sample. Psychology. 2018;9(5): 1095–127.
- 32.
Nunnally JC, Bernstein IH. Psychometric theory. New York: MC-Graw-Hill; 1994.
- 33. Konrath S, Meier BP, Bushman BJ. Development and validation of the single item narcissism scale (SINS). PLOS one. 2014;9(8): e103469. pmid:25093508
- 34. Brailovskaia J, Margraf J. How to measure self-esteem with one item? Validation of the German single-item self-esteem scale (G-SISE). Current Psychology. 2020;39(6): 2192–202.
- 35. Szrek H, Chao L-W, Ramlagan S, Peltzer K. Predicting (un) healthy behavior: A comparison of risk-taking propensity measures. Judgment and Decision Making. 2012;7(6): 716–27. pmid:24307919
- 36. Nichols AL, Webster GD. The single-item need to belong scale. Personality and Individual Differences. 2013;55(2): 189–92.
- 37. Riordan BC, Cody L, Flett JAM, Conner TS, Hunter J, Scarf D. The development of a single item FoMO (fear of missing out) scale. Current Psychology. 2018;39: 1215–20.
- 38. Abdel-Khalek AM. Measuring happiness with a single-item scale. Social Behavior and Personality: an International Journal. 2006;34(2): 139–50.
- 39. Maxwell SE, Lau MY, Howard GS. Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist. 2015;70(6): 487–98. pmid:26348332
- 40. Henderson AS, Jorm AF, Korten AE, Jacomb P, Christensen H, Rodgers B. Symptoms of depression and anxiety during adult life: evidence for a decline in prevalence with age. Psychological Medicine. 1998;28(6): 1321–8. pmid:9854273
- 41. Christensen H, Jorm AF, Mackinnon AJ, Korten AE, Jacomb PA, Henderson AS, et al. Age differences in depression and anxiety symptoms: a structural equation modelling analysis of data from a general population sample. Psychological Medicine. 1999;29(2): 325–39. pmid:10218924
- 42. Gambin M, Sękowski M, Woźniak-Prus M, Wnuk A, Oleksy T, Cudo A, et al. Generalized anxiety and depressive symptoms in various age groups during the COVID-19 lockdown in Poland. Specific predictors and differences in symptoms severity. Comprehensive Psychiatry. 2021;105: 152222. pmid:33388494
- 43. Brailovskaia J, Schönfeld P, Zhang XC, Bieda A, Kochetkov Y, Margraf J. A Cross-Cultural Study in Germany, Russia, and China: Are Resilient and Social Supported Students Protected Against Depression, Anxiety, and Stress? Psychological Reports. 2018;121(2): 265–81. pmid:28836915
- 44. Schönfeld P, Brailovskaia J, Margraf J. Positive and negative mental health across the lifespan: A cross-cultural comparison. International Journal of Clinical and Health Psychology. 2017;17(3): 197–206. pmid:30487895
- 45.
World Health Organization. Depression 2023. Available from: https://www.who.int/news-room/fact-sheets/detail/depression.
- 46. Karger A. Gender differences in depression. Bundesgesundheitsblatt-Gesundheitsforschung-Gesundheitsschutz. 2014;57: 1092–8.
- 47. Girgus JS, Yang K. Gender and depression. Current Opinion in Psychology. 2015;4: 53–60.
- 48. Vanderminden J, Esala JJ. Beyond symptoms: Race and gender predict anxiety disorder diagnosis. Society and Mental Health. 2019;9(1): 111–25.
- 49. Gyllensten K, Palmer S. The role of gender in workplace stress: A critical literature review. Health Education Journal. 2005;64(3): 271–88.
- 50. Goyal D, Gay C, Lee KA. How much does low socioeconomic status increase the risk of prenatal and postpartum depressive symptoms in first-time mothers? Women’s Health Issues. 2010;20(2): 96–104. pmid:20133153
- 51. Wetherall K, Robb KA, O’Connor RC. Social rank theory of depression: A systematic review of self-perceptions of social rank and their relationship with depressive symptoms and suicide risk. Journal of affective disorders. 2019;246: 300–19. pmid:30594043
- 52. Niemeyer H, Bieda A, Michalak J, Schneider S, Margraf J. Education and mental health: Do psychosocial resources matter? SSM-Population Health. 2019;7: 100392. pmid:30989104
- 53. Keyes CL, Shmotkin D, Ryff CD. Optimizing well-being: the empirical encounter of two traditions. Journal of Personality and Social Psychology. 2002;82(6): 1007–22. pmid:12051575
- 54. Antaramian SP, Huebner ES, Hills KJ, Valois RF. A dual‐factor model of mental health: Toward a more comprehensive understanding of youth functioning. American Journal of Orthopsychiatry. 2010;80(4): 462–72. pmid:20950287
- 55. Schönfeld P, Brailovskaia J, Bieda A, Zhang XC, Margraf J. The effects of daily stress on positive and negative mental health: Mediation through self-efficacy. International Journal of Clinical and Health Psychology. 2016;16(1): 1–10. Epub 2016/01/01. pmid:30487845.
- 56. Margraf J, Lavallee K, Zhang X, Schneider S. Social Rhythm and Mental Health: A Cross-Cultural Comparison. PLoS One. 2016;11(3): e0150312. pmid:26954568.
- 57. Schönfeld P, Brailovskaia J, Zhang XC, Margraf J. Self-Efficacy as a Mechanism Linking Daily Stress to Mental Health in Students: A Three-Wave Cross-Lagged Study. Psychological Reports. 2018;122(6): 2074–95. pmid:30235979
- 58. Teismann T, Forkmann T, Brailovskaia J, Siegmann P, Glaesmer H, Margraf J. Positive mental health moderates the association between depression and suicide ideation: a longitudinal study. International Journal of Clinical and Health Psychology. 2018;18: 1–7. pmid:30487904
- 59. Teismann T, Brailovskaia J, Totzeck C, Wannemüller A, Margraf J. Predictors of remission from panic disorder, agoraphobia and specific phobia in outpatients receiving exposure therapy: The importance of positive mental health. Behaviour Research and Therapy. 2018;108: 40–4. pmid:29981937
- 60. Precht L-M, Margraf J, Stirnberg J, Brailovskaia J. It’s all about control: Sense of control mediates the relationship between physical activity and mental health during the COVID-19 pandemic in Germany. Current Psychology. 2021;42: 8531–9. pmid:34690477
- 61. Southwick FS, Southwick SM. The loss of a sense of control as a major contributor to physician burnout: a neuropsychiatric pathway to prevention and recovery. JAMA Psychiatry. 2018;75(7): 665–6. pmid:29799941
- 62. Bieda A, Hirschfeld G, Schönfeld P, Brailovskaia J, Zhang XC, Margraf J. Universal Happiness? Cross-Cultural Measurement Invariance of Scales Assessing Positive Mental Health. Psychological Assessment. 2017;29(4): 408–21. pmid:27322203.
- 63. Keeton CP, Perry-Jenkins M, Sayer AG. Sense of control predicts depressive and anxious symptoms across the transition to parenthood. Journal of Family Psychology. 2008;22(2): 212–21. pmid:18410208
- 64. Mahmoud JSR, Staten RT, Hall LA, Lennie TA. The relationship among young adult college students’ depression, anxiety, stress, demographics, life satisfaction, and coping styles. Issues in Mental Health Nursing. 2012;33(3): 149–56. pmid:22364426
- 65. Weinstein L, Laverghetta A. College student stress and satisfaction with life. College Student Journal. 2009;43(4).
- 66. Lin M, Hirschfeld G, Margraf J. Brief form of the Perceived Social Support Questionnaire (F-SozU K-6): Validation, norms, and cross-cultural measurement invariance in the USA, Germany, Russia, and China. Psychological Assessment. 2019;31(5): 609–21. pmid:30589275
- 67. Kyriazos TA. Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. 2018;9(08): 2207–30.
- 68. Mayr S, Erdfelder E, Buchner A, Faul F. A short tutorial of GPower. Tutorials in Quantitative Methods for Psychology. 2007;3(2): 51–9.
- 69. Nilges P, Essau C. Die Depressions-Angst-Stress-Skalen. Der Schmerz. 2015;29(6): 649–57. pmid:26205682
- 70. Lukat J, Margraf J, Lutz R, van der Veld WM, Becker ES. Psychometric properties of the positive mental health scale (PMH-scale). BMC Psychology. 2016;4(1): 8. pmid:26865173
- 71.
IBM Corp. IBM SPSS Statistics for Windows, Version 28.0. Armonk, NY: IBM Corp; 2021.
- 72.
Hair J, Black WC, Babin BJ, Anderson RE. Multivariate data analysis (7th ed.). Upper Saddle River, New Jersey: Pearson Educational International; 2010.
- 73. Rahimnia F, Mazidi A, Mohammadzadeh Z. Emotional mediators of psychological capital on well-being: The role of stress, anxiety, and depression. Management Science Letters. 2013;3(3): 913–26.
- 74. Phillips AC, Carroll D, Der G. Negative life events and symptoms of depression and anxiety: stress causation and/or stress generation. Anxiety, Stress, & Coping. 2015;28(4): 357–71. pmid:25572915
- 75. Siegmann P, Teismann T, Fritsch N, Forkmann T, Glaesmer H, Zhang XC, et al. Resilience to suicide ideation: A cross‐cultural test of the buffering hypothesis. Clinical Psychology & Psychotherapy. 2018;25(1): e1–e9. pmid:28853242
- 76.
Field A. Discovering statistics using IBM SPSS statistics. 4th, editor. London: Sage Publications Ltd; 2013.
- 77. Schmitt TA. Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment. 2011;29(4): 304–21.
- 78. Shrestha N. Factor analysis as a tool for survey analysis. American Journal of Applied Mathematics and Statistics. 2021;9(1): 4–11.
- 79.
Bühner M. Einführung in die Test-und Fragebogenkonstruktion. München: Pearson Deutschland GmbH; 2011.
- 80.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.
- 81.
Kline R. Exploratory and confirmatory factor analysis. In: Petscher Y, Schatschneider C, Compton DL, editors. Applied quantitative analysis in the social sciences. New York, NY: Routledge; 2013. p. 171–207
- 82. Chen IH, Chen C-Y, Pakpour AH, Griffiths MD, Lin C-Y, Li X-D, et al. Problematic internet-related behaviors mediate the associations between levels of internet engagement and distress among schoolchildren during COVID-19 lockdown: A longitudinal structural equation modeling study. Journal of Behavioral Addictions. 2021;10(1): 135–48. pmid:33570506
- 83. Hawes MT, Szenczy AK, Klein DN, Hajcak G, Nelson BD. Increases in depression and anxiety symptoms in adolescents and young adults during the COVID-19 pandemic. Psychological Medicine. 2021;52(14): 3222–30. pmid:33436120
- 84. Fung XCC, Siu AMH, Potenza MN, O’Brien KS, Latner JD, Chen C-Y, et al. Problematic use of internet-related activities and perceived weight stigma in schoolchildren: a longitudinal study across different epidemic periods of COVID-19 in China. Frontiers in Psychiatry. 2021;12: 675839. pmid:34108898
- 85. Chen C-Y, Chen IH, Hou W-L, Potenza MN, O’Brien KS, Lin C-Y, et al. The relationship between children’s problematic internet-related behaviors and psychological distress during the onset of the COVID-19 pandemic: a longitudinal study. Journal of Addiction Medicine. 2022;16(2): e73–e80. pmid:33770003
- 86. Chen C-Y, Chen IH, Pakpour AH, Lin C-Y, Griffiths MD. Internet-related behaviors and psychological distress among schoolchildren during the COVID-19 school hiatus. Cyberpsychology, Behavior, and Social Networking. 2021;24(10): 654–63. pmid:33877905
- 87. Frison E, Eggermont S. The impact of daily stress on adolescents’ depressed mood: The role of social support seeking through Facebook. Computers in Human Behavior. 2015;44: 315–25.
- 88. Brailovskaia J, Schillack H, Margraf J. Tell me why are you using social media (SM)! Relationship between reasons for use of SM, SM flow, daily stress, depression, anxiety, and addictive SM use–An exploratory investigation of young adults in Germany. Computers in Human Behavior. 2020;113: 106511.
- 89. Truskauskaite-Kuneviciene I, Kazlauskas E, Ostreikaite-Jurevice R, Brailovskaia J, Margraf J. Positive mental health and adjustment following life-stressors among young adults. Current Psychology. 2020;41: 1951–6.
- 90. Carrozzino D, Christensen KS, Mansueto G, Brailovskaia J, Margraf J, Cosci F. A clinimetric analysis of the Euthymia, Resilience, and Positive Mental Health Scales. Journal of Affective Disorders. 2021;294: 71–6. pmid:34274790
- 91. Teismann T, Naghavi A, Asgari Z, Grundein L, Brailovskaia J. Posttraumatic stress disorder symptoms and suicidal ideation/behavior: The moderating ef-fects of positive mental health and social support among Iranian and German students. Current Psychology. 2022;42: 20757–64.
- 92. Brailovskaia J, Teismann T, Margraf J. Positive mental health mediates the relationship between Facebook Addiction Disorder and suicide-related outcomes: A longitudinal approach. Cyberpsychology, Behavior, and Social Networking. 2020;23(5): 346–50. pmid:32216638
- 93. Trompetter HR, Lamers SMA, Westerhof GJ, Fledderus M, Bohlmeijer ET. Both positive mental health and psychopathology should be monitored in psychotherapy: confirmation for the dual-factor model in acceptance and commitment therapy. Behaviour Research and Therapy. 2017;91: 58–63. pmid:28160719
- 94. Capaldi CA, Liu L, Dopko RL. Positive mental health and perceived change in mental health among adults in Canada during the second wave of the COVID-19 pandemic. Health Promotion & Chronic Disease Prevention in Canada: Research, Policy & Practice. 2021;41(11): 359–77.
- 95. Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. Journal of Personality Assessment. 1985;49(1): 71–5. pmid:16367493
- 96. Glaesmer H, Grande G, Braehler E, Roth M. The German version of the satisfaction with life scale (SWLS): Psychometric properties, validity, and population-based norms. European Journal of Psychological Assessment. 2011;27: 127–32.
- 97.
Oishi S. The application of structural equation modeling and item response theory to cross-cultural positive psychology research. In: Ong AD, van Dulmen MHM, editors. Oxford handbook of methods in positive psychology. New York, NY: Oxford University Press; 2007. p. 126–38
- 98. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online. 2003;8(2): 23–74.
- 99. Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2): 238–46. pmid:2320703
- 100. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychological Methods. 1996;1(2): 130–49.
- 101.
Byrne BM. Structural equation modeling with EQS and EQS/Windows. Thousand Oaks, CA: Sage Publications; 1994.
- 102. Ford JK, MacCallum RC, Tait M. The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology. 1986;39(2): 291–314.
- 103.
Skaff MM. Sense of control and health. In: Adldwin CM, Park CL, Spiro A, editors. Handbook of health psychology and aging. New York, NY: Guilford Press; 2007. p. 186–209
- 104. Bieda A, Hirschfeld G, Schönfeld P, Brailovskaia J, Lin M, Margraf J. Happiness, life satisfaction and positive mental health: Investigating reciprocal effects over four years in a Chinese student sample. Journal of Research in Personality. 2019;78: 198–209.
- 105. Hu S, Cai D, Zhang XC, Margraf J. Relationship between social support and positive mental health: A three-wave longitudinal study on college students. Current Psychology. 2020: 1–10.
- 106. Scholten S, Lavallee K, Velten J, Zhang XC, Margraf J. The Brief Daily Stressors Screening Tool: An introduction and evaluation. Stress and Health. 2020;36(5): 686–92. pmid:32449236
- 107.
Fernando S. Mental health worldwide: Culture, globalization and development. New York, NY: Springer; 2014.
- 108. Pawlaczyk M, Siembida J, Balaj K, Rajewska-Rager A. The assessment of stress level, anxiety, depressive symptoms, and defense mechanisms among Polish and English medical students. Annals of General Psychiatry. 2020;19(1): 29. pmid:32391067
- 109. Rosenberg M. Rosenberg self-esteem scale (RSE). Acceptance and commitment therapy Measures package. 1965;61: 52.
- 110. von Collani G, Herzberg PY. Eine revidierte Fassung der deutschsprachigen Skala zum Selbstwertgefühl von Rosenberg. Zeitschrift für Differentielle und Diagnostische Psychologie. 2003;24(1): 3–7.
- 111. Bergkvist L, Rossiter JR. The predictive validity of multiple-item versus single-item measures of the same constructs. Journal of Marketing Research. 2007;44(2): 175–84.
- 112. Milton K, Bull FC, Bauman A. Reliability and validity testing of a single-item physical activity measure. British Journal of Sports Medicine. 2011;45(3): 203–8. pmid:20484314
- 113. Mohajan HK. Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University Economic Series. 2017;17(4): 59–82.
- 114. Buysse DJ, Angst J, Gamma A, Ajdacic V, Eich D, Rössler W. Prevalence, course, and comorbidity of insomnia and depression in young adults. Sleep. 2008;31(4): 473–80. pmid:18457234
- 115. Florkowski CM. Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. The Clinical Biochemist Reviews. 2008;29(Suppl 1): S83–S7. pmid:18852864
- 116. Šimundić A-M. Measures of diagnostic accuracy: basic definitions. The Journal of the International Federation of Clinical Chemistry and Laboratory Medicine. 2009;19(4): 203–11. pmid:27683318
- 117. Brailovskaia J, Lin M, Scholten S, Zhu M, Fu Y, Shao M, et al. A Qualitative Cross-Cultural Comparison of Well-Being Constructs: the Meaning of Happiness, Life Satisfaction, and Social Support for German and Chinese Students. Journal of Happiness Studies. 2022;23(4): 1379–402.