https://orcid.org/0000-0001-9930-057X
Figures
Abstract
A recent meta-analysis, of 38 studies with data from 43 independent samples (total N = 24,668), claimed evidence for positive reciprocal prospective effects, and hence for both top-down and bottom-up processes, between general and domain-specific self-esteem. However, the meta-analytic cross-lagged effects were estimated while adjusting for a prior measurement of the outcome variable and it is known that such adjusted cross-lagged effects may be spurious due to correlations with residuals and regression to the mean. In the present reanalyses, we found all of the prospective effects to be spurious. Consequently, claims about increasing prospective effects and top-down and bottom-up processes between general and domain-specific self-esteem can be questioned. It is important for researchers to be aware of the limitations of cross-lagged panel analyses, and of analyses of correlational data in general, in order not to overinterpret findings.
Citation: Sorjonen K, Melin B (2024) Spurious prospective effects between general and domain-specific self-esteem: A reanalysis of a meta-analysis of longitudinal studies. PLoS ONE 19(2): e0298158. https://doi.org/10.1371/journal.pone.0298158
Editor: Amitav Banerjee, Dr DY Patil Medical College, Hospital and Research Center, INDIA
Received: May 5, 2023; Accepted: January 15, 2024; Published: February 13, 2024
Copyright: © 2024 Sorjonen, Melin. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data underlying the results presented in the study are available at the Open Science Framework at https://osf.io/qu2hd/.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Researchers have suggested that low self-esteem may increase the risk for various negative outcomes, including poor physical and mental health [1,2], worse work experiences and economic prospects [2,3], low quality of social relations [4], and criminal behavior [2]. Consequently, it may be important to grasp determinants of self-esteem. Although self-esteem is often regarded as a general construct, it has been suggested that people’s self-esteem (or the related “self-concept”) can vary between more specific domains, e.g. academic ability and physical appearance [5,6]. A top-down model claims influence from general to domain-specific self-esteem [7,8] while a bottom-up model suggests effects in the opposite direction [5,9,10].
Dapp et al. [11] estimated meta-analytic prospective effects of initial general self-esteem on subsequent domain-specific self-esteem while adjusting for a prior measurement of the same domain-specific self-esteem, and vice versa. Dapp et al. did this for eight different domain-specific self-esteems, namely academic abilities, physical appearance, athletic abilities, math abilities, morality, romantic relationships, social acceptance, and verbal abilities. With a few exceptions, both the prospective adjusted effect of general self-esteem on subsequent domain-specific self-esteem and the prospective adjusted effect of domain-specific self-esteem on subsequent general self-esteem were positive and statistically significant. Dapp et al. concluded that their findings provided robust evidence for reciprocal effects, and consequently for both top-down and bottom-up processes, between general and domain-specific self-esteem.
However, it is known that cross-lagged effects while adjusting for a prior measurement of the outcome may be spurious due to correlations with residuals and regression to the mean [12–15]. As an example, let us assume that cabdrivers tend to drive more than gardeners. Consequently, if a cabdriver and a gardener has driven equally much a particular week, we may suspect that the cabdriver has driven less than usually, i.e. experienced a negative residual, or that the gardener has driven more than usually, i.e. experienced a positive residual. However, as residuals tend to regress toward a mean value of zero between measurements, we should expect a more positive change in driving to the subsequent week for the cabdriver compared with the gardener. On group level, we should expect a positive effect of a dichotomous “cabdriver vs. gardener” variable on subsequent driving while adjusting for initial driving even if no group-level change has taken place. As regression to the mean is independent of the direction of time, if the effect is spurious, we should also expect a positive effect of “cabdriver vs. gardener” on initial driving while adjusting for subsequent driving. The impact of correlations with residuals and regression to the mean on adjusted cross-lagged effects is demonstrated in a simulation included in one of our recent publications [16].
The objective of the present reanalyses was to evaluate if the meta-analytic findings by Dapp et al. [11] really provided evidence for reciprocal prospective effects, and consequently for top-down and bottom-up processes, between general and domain-specific self-esteem or if the findings may have been spurious due to correlations with residuals and regression to the mean. As mentioned above, research has indicated potential negative consequences of low self-esteem and clinical practitioners and decision-makers may think that increased self-esteem could be one way to avoid or alleviate those negative consequences. Therefore, it is important to scrutinize research in the area, and to identify effects that could be spurious, in order to help avoid investments that may not be the best use of limited resources.
Method
We refer to Dapp et al. [11] for more comprehensive information on selection of studies, descriptive data, etc. In short, Dapp et al. extracted six zero-order correlations between general and eight types of domain-specific self-esteem measured at two occasions from 38 studies with data from 43 independent samples (total N = 24,668, mean age at time 1 = 12.6 years (SD = 3.5 years), mean proportion of female participants = 53% (SD = 22%)). The domain-specific self-esteems were academic abilities, physical appearance, athletic abilities, math abilities, morality, romantic relationships, social acceptance, and verbal abilities. Dapp et al. estimated meta-analytic cross-lagged effects between general and domain-specific self-esteem while adjusting for a prior measurement of the outcome with meta-analytic structural equation modeling (MASEM).
Using zero-order correlations provided by Dapp et al. (at the Open Science Framework at https://osf.io/beu29/, from where we retrieved data 2023-04-21) and Eq 1 [17], we estimated meta-analytic effects of initial global self-esteem on subsequent domain-specific self-esteem while adjusting for initial domain-specific self-esteem, and vice versa. Here, both a hypothesis of true prospective effects and a hypothesis of spurious prospective effects due to correlations with residuals and regression to the mean predicted positive effects (Table 1, rows 1 and 4).
Additionally, we used Eq 1 to estimate meta-analytic effects of initial general self-esteem on initial domain-specific self-esteem while adjusting for subsequent domain-specific self-esteem, and vice versa. Here, a hypothesis of true increasing prospective effects predicted negative effects, which would mean that among individuals with the same subsequent domain-specific self-esteem, those with high initial general self-esteem had had lower initial domain-specific self-esteem and had, consequently, experienced a larger increase in domain-specific self-esteem between measurements compared with those with the same subsequent domain-specific self-esteem but with lower initial general self-esteem. Similarly, a negative effect would mean that those with high initial domain-specific self-esteem had experienced a larger subsequent increase in general self-esteem compared with those with the same subsequent general self-esteem but with lower initial domain-specific self-esteem. Contrarily, as regression to the mean is independent of the direction of time, a hypothesis of spuriousness predicted these effects to be positive (Table 1, rows 2 and 5).
Moreover, we used Eq 2 [18] to estimate meta-analytic effects of initial general self-esteem on the subsequent domain-specific self-esteem–initial domain-specific self-esteem difference, and vice versa. Here, a hypothesis of true increasing prospective effects predicted positive effects. Contrarily, a hypothesis of spuriousness predicted these effects to be either close to zero (if concurrent, rX1,Y1, and cross-lagged, rX1,Y2, correlations were approximately equally strong) or negative (if concurrent correlations were stronger than cross-lagged correlations) (Table 1, rows 3 and 6).
We conducted a multilevel random effects meta-analysis for each of the six effects in Table 1 for each of the eight domain-specific self-esteems, i.e. 48 meta-analyses in total. Different effects from the same study were aggregated with a multilevel approach. Then, a random meta-analytic effect, with a 95% confidence interval, was estimated across the independent effect sizes. Analyses were conducted on Fisher’s z-transformed standardized regression effects, but these were inverted back to non-transformed effects for presentations. Analyses were conducted with R 4.1.3 statistical software [19] employing the metafor package [20]. Data, a list of studies included in the meta-analyses, forest-plots, and an analysis script are available at the Open Science Framework at https://osf.io/qu2hd/.
Results
Meta-analytic findings are presented in Table 2. Most estimated effects exhibited statistically significant, although not necessarily high, heterogeneity, as estimated by Cochran’s Q and I2, which estimates percentage of variation across effects attributable to heterogeneity rather than random variance. With two exceptions, the effects of initial general self-esteem on subsequent domain-specific self-esteem while adjusting for initial domain-specific self-esteem (β(g1,s2.s1)), and vice versa (β(s1,g2.g1)), were positive and statistically significant. This suggested a larger subsequent increase in domain-specific self-esteem, for example social acceptance, for those with high, compared with low, initial general self-esteem (Fig 1A), and vice versa (Fig 1D).
Predicted social (A-C) and general (D-F) self-esteem at two occasions, separately for those with high (Z = 1), average, and low (Z = -1) general (A-C) and social (D-F) self-esteem at T1. Separately for situations when conditioning on average level of the outcome at T1 (A and D), average level of the outcome at T2 (B and E), and when not conditioning on the outcome (C and F).
However, all effects of initial general self-esteem on initial domain-specific self-esteem while adjusting for subsequent domain-specific self-esteem (β(g1,s1.s2)), and vice versa (β(s1,g1.g2)), were positive and statistically significant (Table 2). This suggested, contrarily, a larger subsequent decrease in domain-specific self-esteem for those with high, compared with low, initial general self-esteem (Fig 1B), and vice versa (Fig 1E).
Furthermore, with one exception, the effects of initial general self-esteem on the subsequent domain-specific self-esteem–initial domain-specific self-esteem difference (β(g1,s2-s1)), and vice versa (β(s1,g2-g1)), were negative and statistically significant (Table 2). This suggested, again, that those with high initial general self-esteem had experienced a larger subsequent decrease in domain-specific self-esteem compared with those with low initial general self-esteem (Fig 1C), and vice versa (Fig 1F). In summary, the findings agreed with a hypothesis of spurious prospective effects due to correlations with residuals and regression to the mean rather than with a hypothesis of true increasing prospective effects (compare effects in Table 2 with predictions in Table 1).
Discussion
The present study set out to evaluate if the meta-analytic findings by Dapp et al. [11] really provided evidence for reciprocal prospective effects, and consequently for top-down and bottom-up processes, between general and domain-specific self-esteem. The present findings suggested that findings by Dapp et al. probably were spurious due to correlations with residuals and regression to the mean. Consequently, the conclusion by Dapp et al. can be challenged.
In the present reanalyses we found, with a few exceptions, positive meta-analytic reciprocal prospective effects between general and domain-specific self-esteem. This could indicate, as suggested by Dapp et al. [11], a reinforcing loop of self-esteem including both bottom-up and top-down processes. However, we also found positive effects of initial general self-esteem on initial domain-specific self-esteem while adjusting for subsequent domain-specific self-esteem, and vice versa. Moreover, we found negative effects of initial general self-esteem on the subsequent domain-specific self-esteem–initial domain-specific self-esteem difference, and vice versa. These latter findings indicated, contrarily, degrading influence between general and domain-specific self-esteem. These contradictory findings of simultaneous reinforcing and degrading effects suggested that the prospective effects between general and domain-specific self-esteem were spurious, probably due to correlations with residuals and regression to the mean.
As an example, picture two individuals, A and B, with the same initial social self-esteem but who differ in initial general self-esteem, with A having a higher score. Due to the positive association between general and social self-esteem (r = 0.45 according to the meta-analytic estimation by Dapp et al. [11]) in combination with assumed less than perfect reliability in measurements, we should assume a higher true social self-esteem and, consequently, a more negative residual in the initial measurement of social self-esteem for A compared with B. Furthermore, as residuals tend to regress toward a mean value of zero between measurements, we should expect a more positive, but spurious, subsequent change in social self-esteem for A compared with B.
This study is part of a series where we have reanalyzed meta-analyses using cross-lagged panel analyses. All claimed prospective effects in Table 3 were found to be spurious. An overall message in these challenging studies is that cross-lagged effects while adjusting for a prior measurement of the outcome often do not prove anything over and above a cross-sectional correlation combined with less than perfect reliability in measurements. And the cross-sectional correlation could be spurious due to confounding by a third variable, common method bias, etc. The limitation is not alleviated by meta-analytic aggregation of several cross-lagged effects. It is important for researchers to be aware of this limitation in order not to overinterpret findings, something that appears to have happened to Dapp et al. [11]. The continued output of studies using cross-lagged panel analyses suggests that knowledge of these limitations, although far from new, is lacking in the research community. Hence, continued communication of these limitations is warranted. We recommend researchers to analyze models with effects both forward and backward in time and without adjustment for the outcome variable, as we have done in the present study, in order to identify possibly spurious prospective effects.
Limitations
The present study shared some of the limitations of the reanalyzed meta-analysis by Dapp et al. [11]. For example, 42 of 43 included samples were Western, from the United States, Canada, Europe, Australia, or New Zealand. A single sample was from Asia (China) and no samples were from Africa or South America. Hence, it remains an open question if the present main finding, i.e. that prospective effects between global and domain-specific self-esteem appear to be spurious due to correlations with residuals and regression to the mean, generalize to a broader cultural context.
The measurements of global and domain-specific self-esteem in the included studies may not always have been optimal. Moreover, in the present reanalyses we did not consider possible moderating influences of age and sex composition of the sample, time lag between measurements, used instruments, etc. However, it is important to bear in mind that such characteristics were constant across the analyzed models and cannot, consequently, explain why some models suggested increasing and others decreasing prospective effects between global and domain-specific self-esteem.
Extended statistical models, e.g. random-intercept cross-lagged panel models [34,35] and latent growth curve models [36,37], have been proposed as improvements of the traditional two-wave cross-lagged panel model. These extended models are presumably better at differentiating within- from between-person processes of change and, consequently, allow stronger inference about causal effects. Conflation of within- and between-person processes of change has been discussed in relation to, for example, self-esteem [38], academic self-concept [39], and self-efficacy [40]. We cannot rule out that analyses with extended and improved models would withstand scrutiny and indicate increasing effects between general and domain-specific self-esteem. Therefore, the present findings should not be seen to rule out the possible existence of such effects once and for all. Rather, and more specifically, the present study challenges conclusions by Dapp et al. [11] based on findings from meta-analytic two-wave cross-lagged panel models. It should be noted that we could not conduct analyses with extended models as they require data from more than two waves of measurement, which is what we, via Dapp et al., had at our disposal. It should also be noted that analyses of three or more waves of measurement with random-intercept cross-lagged panel models would probably not give definite answers to whether general and domain-specific self-esteem have true reciprocal prospective effects on each other or not. This because the method can be susceptible to spurious findings due to correlations with residuals and regression to the mean in a similar way as traditional two-wave cross-lagged panel models [41]. This agrees with a general conclusion that it is very difficult, maybe even impossible, to prove causal effects in correlational, i.e. non-experimental, data.
To reiterate, the objective of the present study was not to evaluate if true prospective effects between general and domain-specific self-esteem actually exist. Rather, the more limited objective was to evaluate if the data analyzed by Dapp et al. [11] allowed their conclusion that such true prospective effects exist. For this objective, our reanalyses of the data with two waves of measurement used by Dapp et al. [11], which did not allow analyses with RI-CLPM, were sufficient. Our reanalyses permitted us to conclude that the prospective effects presented by Dapp et al. [11] were spurious and, consequently, that their claims about increasing prospective effects between general and domain-specific self-esteem were not supported by their own meta-analytic data.
Conclusions
The present reanalysis of a meta-analysis by Dapp et al. [11] found prospective effects between global and domain-specific self-esteem to be spurious, probably due to correlations with residuals and regression to the mean. Hence, reciprocal increasing effects between global and domain-specific self-esteem, claimed by Dapp et al., as well as hypotheses of top-down and bottom-up processes in self-esteem, can be called into question. It is important for researchers to be aware of the limitations of adjusted cross-lagged effects, and of analyses of correlational data in general, in order not to overinterpret findings.
References
- 1. Sowislo JF, Orth U. Does low self-esteem predict depression and anxiety? A meta-analysis of longitudinal studies. Psychological Bulletin. 2013;139: 213–240. pmid:22730921
- 2. Trzesniewski KH, Donnellan MB, Moffitt TE, Robins RW, Poulton R, Caspi A. Low self-esteem during adolescence predicts poor health, criminal behavior, and limited economic prospects during adulthood. Developmental Psychology. 2006;42: 381–390. pmid:16569175
- 3. Krauss S, Orth U. Work experiences and self-esteem development: A meta-analysis of longitudinal studies. Eur J Pers. 2022;36: 849–869.
- 4. Harris MA, Orth U. The link between self-esteem and social relationships: A meta-analysis of longitudinal studies. Journal of Personality and Social Psychology. 2020;119: 1459–1477. pmid:31556680
- 5. Shavelson RJ, Hubner JJ, Stanton GC. Self-concept: Validation of construct interpretations. Review of Educational Research. 1976;46: 407–441.
- 6. Rentzsch K, Schröder–Abé M. Stability and change in domain–specific self–esteem and global self–esteem. Eur J Pers. 2018;32: 353–370.
- 7. Rentzsch K, Schröder-Abé M. Top down or bottom up? Evidence from the longitudinal development of global and domain-specific self-esteem in adulthood. Journal of Personality and Social Psychology. 2022;122: 714–730. pmid:34542309
- 8. Brown JD, Dutton KA, Cook KE. From the top down: Self-esteem and self-evaluation. Cognition and Emotion. 2001;15: 615–631.
- 9.
Harter S. The development of self-esteem. Self-esteem issues and answers: A sourcebook of current perspectives. Psychology Press; 2006. pp. 144–150.
- 10. Harter S, Whitesell NR, Junkin LJ. Similarities and differences in domain-specific and global self-evaluations of learning-disabled, behaviorally disordered, and normally achieving adolescents. American Educational Research Journal. 1998;35: 653–680.
- 11. Dapp LC, Krauss S, Orth U. Testing the bottom-up and top-down models of self-esteem: A meta-analysis of longitudinal studies. Journal of Personality and Social Psychology. 2023;124: 1111–1131. pmid:36326678
- 12. Castro-Schilo L, Grimm KJ. Using residualized change versus difference scores for longitudinal research. Journal of Social and Personal Relationships. 2018;35: 32–58.
- 13. Eriksson K, Häggström O. Lord’s paradox in a continuous setting and a regression artifact in numerical cognition research. PLoS ONE. 2014;9: e95949. pmid:24752244
- 14. Glymour MM, Weuve J, Berkman LF, Kawachi I, Robins JM. When is baseline adjustment useful in analyses of change? An example with education and cognitive change. American Journal of Epidemiology. 2005;162: 267–278. pmid:15987729
- 15. Sorjonen K, Melin B, Ingre M. Predicting the effect of a predictor when controlling for baseline. Educational and Psychological Measurement. 2019;79: 688–698. pmid:32655179
- 16. Sorjonen K, Nilsonne G, Melin B. Distorted meta-analytic findings on peer influence: A reanalysis. Heliyon. 2023;9: e21458. pmid:37954301
- 17.
Cohen J, Cohen P, West SG, Aiken LS. Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah: Lawrence Erlbaum Associates; 2003.
- 18.
Guilford JP. Fundamental statistics in psychology and education. 4th edition. New York: McGraw-Hill; 1965.
- 19.
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. 2023.
- 20. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Soft. 2010;36.
- 21. Gedikli C, Miraglia M, Connolly S, Bryan M, Watson D. The relationship between unemployment and wellbeing: an updated meta-analysis of longitudinal evidence. European Journal of Work and Organizational Psychology. 2022; 1–17.
- 22.
Sorjonen K, Melin B. Spurious prospective associations between unemployment and wellbeing: Reanalysis of a meta-analytic cross-lagged panel analysis. PsyArXiv; 2023 Mar.
- 23. Giletta M, Choukas-Bradley S, Maes M, Linthicum KP, Card NA, Prinstein MJ. A meta-analysis of longitudinal peer influence effects in childhood and adolescence. Psychological Bulletin. 2021;147: 719–747. pmid:34855429
- 24. Sorjonen K, Ingre M, Melin B, Nilsonne G. Unmasking artifactual links: A reanalysis reveals No direct causal relationship between self-esteem and quality of social relations. Heliyon. 2023;9: e20397. pmid:37767502
- 25. Sorjonen K, Melin M, Melin B. Spurious prospective effects between work experiences and self-esteem: A reanalysis of a meta-analysis of longitudinal studies. Eur J Pers. 2023;
- 26. Smith MM, Sherry SB, Ray C, Hewitt PL, Flett GL. Is perfectionism a vulnerability factor for depressive symptoms, a complication of depressive symptoms, or both? A meta-analytic test of 67 longitudinal studies. Clinical Psychology Review. 2021;84: 101982. pmid:33556805
- 27.
Sorjonen K, Melin B. Spurious prospective effect of perfectionism on depression: Reanalysis of a meta-analytic cross-lagged panel analysis. PsyArXiv; 2023 Apr.
- 28. Talsma K, Schüz B, Schwarzer R, Norris K. I believe, therefore I achieve (and vice versa): A meta-analytic cross-lagged panel analysis of self-efficacy and academic performance. Learning and Individual Differences. 2018;61: 136–150.
- 29.
Sorjonen K, Melin B. Spurious prospective effect of self-efficacy on academic performance: Reanalysis of a meta-analytic cross-lagged panel analysis. PsyArXiv; 2023 Mar.
- 30. Wang Y, Chung MC, Wang N, Yu X, Kenardy J. Social support and posttraumatic stress disorder: A meta-analysis of longitudinal studies. Clinical Psychology Review. 2021;85: 101998. pmid:33714168
- 31. Sorjonen K, Melin B. Prospective associations between social support and posttraumatic stress disorder may be spurious: A re-meta-analysis of longitudinal studies and a simulation. Journal of Affective Disorders. 2023;340: 174–180. pmid:37557992
- 32. Wu H, Guo Y, Yang Y, Zhao L, Guo C. A meta-analysis of the longitudinal relationship between academic self-concept and academic achievement. Educ Psychol Rev. 2021;33: 1749–1778.
- 33.
Sorjonen K, Ingre M, Melin B, Nilsonne G. Questioning the reciprocal effects model of academic self-concept and achievement: A reanalysis of a meta-analysis of longitudinal studies and a simulation. PsyArXiv; 2022 Oct.
- 34. Hamaker EL, Kuiper RM, Grasman RPPP. A critique of the cross-lagged panel model. Psychological Methods. 2015;20: 102–116. pmid:25822208
- 35. Mulder JD, Hamaker EL. Three extensions of the random intercept cross-lagged panel model. Structural Equation Modeling: A Multidisciplinary Journal. 2021;28: 638–648.
- 36.
Grimm KJ, Ram N, Estabrook R. Growth modeling: Structural equation and multilevel modeling approaches. New York, NY: Guilford Press; 2017.
- 37.
Bollen KA, Curran PJ. Latent curve models: A structural equation perspective. Hoboken, N.J: Wiley-Interscience; 2006.
- 38. Zuffianò A, Sette S, Manfredi L, Gregori F, Lopéz-Pérez B, Polias S, et al. The relation between self-esteem and regulatory emotional self-efficacy in daily life: A study among university students. Identity. 2023;23: 36–49.
- 39. Núñez-Regueiro F, Juhel J, Bressoux P, Nurra C. Identifying reciprocities in school motivation research: A review of issues and solutions associated with cross-lagged effects models. Journal of Educational Psychology. 2022;114: 945–965.
- 40. Joshanloo M. Self-esteem predicts positive affect directly and self-efficacy indirectly: a 10-year longitudinal study. Cognition and Emotion. 2022;36: 1211–1217. pmid:35786410
- 41. Sorjonen K, Nilsonne G, Melin B, Ingre M. Uncertain inference in random intercept cross-lagged panel models: An example involving need for cognition and anxiety and depression symptoms. Personality and Individual Differences. 2023;201: 111925.