Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Symptom Checklist-90-Revised: A structural examination in relation to family functioning

Abstract

The accurate assessment of psychopathological behaviours of adolescents and young adults is imperative. Symptom Checklist-90-Revised (SCL-90-R) is one of the most comprehensive and widely used scales addressing this purpose internationally. Interestingly, associations between the different SCL-90 symptoms and family functioning have been highlighted. Nevertheless, the scale’s factorial structure has often been challenged. To contribute in this area, this study scrutinizes the psychopathological dimensions of the Symptom Checklist-90-Revised (SCL-90-R) in a large cohort of high school students (Mean age = 16.16; SD = .911) from Greece. It addresses this aim by: a) using first order and bi-factor confirmatory factor analysis, and exploratory structural equation models and; b) investigating the factors’ associations with family functioning. A total of 2090 public Greek High School students completed the SCL-90-R and the Family Adaptability and Cohesion Scale IV (FACES-IV) covering family functioning, satisfaction and communication. Six different solutions, yielded by separate permutations of CFA, ESEM, and bifactor models, were evaluated. Based on global fit, the clarity, reliabilities and the family functioning links of the dimensions in the models, the ESEM oblique model with the theorized nine factors emerged as the optimum. This model had adequate fit, and symptom dimensions were well defined. Also six of the nine factors demonstrated external associations with family functioning, satisfaction and communication. The clinical assessment benefits of these results are discussed.

Citation: Gomez R, Stavropoulos V, Zarate D, Palikara O (2021) Symptom Checklist-90-Revised: A structural examination in relation to family functioning. PLoS ONE 16(3): e0247902. https://doi.org/10.1371/journal.pone.0247902

Editor: Luca Cerniglia, International Telematic University Uninettuno, ITALY

Received: July 28, 2020; Accepted: February 16, 2021; Published: March 12, 2021

Copyright: © 2021 Gomez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper.

Funding: VS has been co-financed by the European Union (European Social Fund – ESF) and Greek national funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF) – Research Funding Program: Heracleitus II. Investing in knowledge society through the European Social Fund.

Competing interests: The second author, Dr Vasileios Stavropoulos, is an associate editor for PLOS One. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

The Symptom Checklist-90-Revised [1,2] is a widely used self-report questionnaire for measuring a range of psychological and psychiatric symptoms. It involves nine primary symptom dimensions, which entail somatization (SOM, for distress related to one’s body/physiological experiences), obsessive-compulsive (O-C, for intrusive thoughts and compulsive actions), interpersonal sensitivity (IS describing self-perceived inadequacy/inferiority in relationships with others), depression (DEP, for low mood and decreased sense of meaning), anxiety (ANX, for anxious symptoms and experienced tensions), hostility (HOS, for aggressiveness towards others), phobic anxiety (PANX, for fears related to specific stimuli), paranoid ideation (PI, for projections to others and persecutory cognitions), and psychoticism (PSY, for psychotic and schizophrenic behaviours; [1,2]). The instrument aims to contribute to accurately assessing one’s subjective (e.g. not clinically obserbable) experience of psychopathology, without requiring the involvement of mental health professionals (i.e. saving resources; [3]). Derogatis, Lipman, Rickels, Uhlenhuth and Covi (1974) [4] first introduced the Hopkins Symptom Checklist (HSCL) to self-assess the subjective experience of SOM, O-C, IS, DEP, and ANX symptoms. The five HSCL symptom categories were expanded with the addition of HOS, PANX, PI, and PSY by Derogatis, Lipman and Covi (1973) [5], resulting to the Symptom Check List 90 items (SCL-90). In 1994, Derogatis revised the SCL-90 to amend/replace psychometrically underperforming ANX and O-C items. Furthermore, three global distress measures and seven general items were added, informing the current version of the SCL-90-R [3].

The scale has been extensively used internationally, as well as in Greece, where the sample of the current study comes from, to assess diverse clinical, forensic and community populations across various age ranges [3,6]. Indicatively, the SCL-90-R has been successfully employed for the psychopathological examination of: a) female adolescents presenting with binge eating disorder, anorexia and bulimia nervosa and their parents [7]; b) early adolescents, who had lost a parent within their first years of life, and their surviving parents [7]; c) adult eating disorder (ED) diagnosed clients attending outpatient treatment programs [8]; d) adult substance use disorder patients [9,10]; e) impulse control disorder patients [9] and; f) adult incarcerated offenders [11]. In that context, the SCL-90-R subscales present to be of particular relevance regarding the psychopathological symptoms of adolescents and young adults [12]. Preti and colleagues (2019) [12] argue that the majority of mental healths symptoms first present during these years. Additionally, prefatory behaviours of major and life-long disorders, such as schizofreneia and bipolar disorder, although maybe not at a diagnosable level, tend to emerge the same period [13]. Developmental challenges, such as the intergration of one’s identity and life-orientation, their transition from more limited child responsibilities to adult independence and likely contradicting influences/messages received by peers, family and social institutions (e.g. church) have been implicated with higher psychological vulnerability for adolescents and emergent adults [14]. Thus, timely comprehensive assessment, such as that offered via the SCL-90-R, could be crucial for the containement/prevention of future psychopathological escalations in these populations.

SCL-90-R and family functioning

Interestingly, the presence of psychopathology in adolescent and early adult populations has been disctinctively associated with one’s family functioning [7,15–23]. Literature has hypothesized that such associations may be explained bi-directionally [24]. Specifically, dysfunctional family relationships may precipitate and perpeatuate reactive psychopathological manifestations [24]. For instance, chaotic family boundaries and emotional distance between family members may invite (and/or accommodate pre-existing) depressive or anxious behaviours [25]. On the other hand, evidence also suggests that the presence of impactful psychopathological presentations by an off-spring, may also trigger (and/or exacerbate pre-existing) family dysfunctions [26]. Likewise, aggression and hostility by a young adult may compromise the level of communication within their family or the level of satisfaction experienced within one’s family [27].

To depict such associations, Tafà and colleagues (2017) [7] studied a sample of female adolescents presenting with ED and their parents. They used the SCL-90-R to measure their psychopathology and the Family Adaptability and Cohesion Evaluation Scale IV edition (FACES-IV) for their perceived family functioning [28]. The latter assumes that family functioning can be described by two main dimensions, flexibility and cohesion. Flexibility refers to the level of change in roles and rules (e.g. leadership), which is necessary to enable families to grow (e.g. progressively offsprings acquire more responsibilities). Although a moderate/balanced level of flexibility is deemed necessary for a family to achieve changes without compromising its stability, too low levels may reflect rigidity. This does not enable the family as a whole and/or the family members independently to evolve. Similalry, excessive flexibility induces chaos, which undermines a family’s sense of stability. In that line, cohesion refers to the level of emotional bonding experienced between family members. A family with a balanced sense of cohesion allows its members to be connected/ interdependent without violating their independency. Nonethelss, too high cohesion may eventuate a state of enmeshment, which deprives members from their independence. Similarly, too low levels of cohesion may invite a state of disengagement between members, which reduces their sense of connection. Enmeshment and disengagement can also compromise family functioning hosting forms of psychopathology, especially when combined with either rigidity or chaos [28]. Olson and colleagues (2006) [28] additionally suggest that balanced flexibility and cohesion are correlated with better communication between family members and a sense of satisfaction regarding one’s family. Indeed, the work of Tafà and colleagues (2017) [7] confirmed that ED adolescents and their parents have distinctive psychopathological profiles, whilts their family functioning patterns may predict different forms of comorbid psychopathology. For instance, anorexic adolescents presented more anxious, depressed, hostile and obsessive-compulsive compared to their bulimic and binge-disorder counterparts. Interestingly, when considering one’s family functioning, higher levels of hostility linked with higher levels of family rigidity among anorexic adolescents. Finally, higher reported levels of somatization tied with higher levels of family enmeshment among bulimic adolescents.

Despite these, there is a dearth of evidence comprehensively examining a variety of different symptoms (such as those described in SCL-90-R) and different family functioning aspects using a large cohort of young people [29]. Such findings would be important for guiding the implementation of evidence-based prevention and intervention programs. This could be achieved by selectively targeting either different types of families or specific family functioning aspects such as flexibility (i.e. family boundaries’ tightness) and cohesion (i.e. emotional distancing between family members) to address certain types of symptoms [30]. This study aims to answer such questions by examining the associations of the different dimensions of psychopathological symptoms of the SCL-90-R with aspects of family functioning in a large and normative cohort of Greek adolescents and young adults.

SCL-90-R structure

For such research to be effectively implemented, the structure of the SCL-90-R and its included sub-dimensions need to be first established. Nevertheless, these have been challenged via the use of different analytical methods [31–35]. First, exploratory procedures such as principal component analysis (PCA) and exploratory factor analysis (EFA) have been used to examine the SCL-90-R symptoms dimensions [12,31,32,36]. Such methods allow for cross-loadings that may compromise the clarity of the factors when inter-item associations occur [37]. Indeed, the initial SCL-90 PCA study by Derogatis and Cleary (1977) [38] showed that the PSY factor was not well defined, whilst ANX and PANX items overlapped. Cyr and colleagues (1985) [36] also supported that the nine dimensional structure of the SCL-90-R is lacking stability due to multiple cross-loadings. Subsequent PCA and EFA studies have generally supported these findings since then [39–42].

Confirmatory procedures.

These highlight the importance of implementation of confirmatory factor analysis (CFA) to study the SCL-90-R structure [31,33–35]. In CFA items link exclusively to their designated dimensions, and cross-loadings are not allowed [43,44]. Interestingly, CFA studies of the SCL-90-R have supported at least five different possible structures: a) uni-dimensional (i.e. all items loading on a single general psychopathology factor); b) nine first-order factors (i.e. nine factors directly related to the items without any overarching/inclusive dimension); c) the nine-dimensional oblique (i.e. assuming some correlation between the nine dimensions); d) the higher order dimensional structure (the nine primary dimensions linking on one higher order general dimension) and; e) the bi-factor structure (all items linking with one general dimension and concurrently with their own nine specific factors [9,12,31–36].

Among these models, a number of past CFA literature has shown better global fit for the bi-factor model [9,12]. As bi-factor CFA models are more flexible in accommodating different response patterns, they tend by definition to fit better [45]. Nonetheless, this requires further investigation for several compelling reasons. Firstly, the bi-factor model has not always been supported. For example, Urbán et al. (2016) [32] favoured the bi-factor structure in only one of the two samples they examined. Secondly, the findings for bi-factor SCL-90 solutions have shown numerous cross-loadings [9], as well as negative loadings on designated dimensions [31]. Thirdly, although research has generally reported substantial reliability for the general factor [9,12,32,35], the specific factors have often not shown acceptable reliability [12,32,35].

In this context, CFA limitations have also been illustrated [46,47]. Specifically, the zero CFA cross-loadings may compromise the accurate interpretation of items in multidimensional measures (as is the case with the SCL-90-R). This is due to items often simultaneously providing information (at least to some degree) for dimensions other to their primary designated one [47]. This limitation refers also to the bi-factor CFA model, where the different dimensions are not inter-correlated (orthogonal model). Thus, the overall/general dimension captures all the shared information (i.e. variances) across the different items, whilst the sub-dimensions capture an amount of information which is distinct [47].

Exploratory equation modelling procedures.

Relatedly, to date there is no robustly accepted factor model for SCL-90-R. Clearly more studies are needed in this respect [31,32]. However, more PCA, EFA, CFA, and Bifactor CFA (BCFA) studies are unlike to be fruitful. Alternatively, the application of proven superior modelling procedures that can counterbalance their limitations, such as exploratory structural equation modeling (ESEM) and bi-factor exploratory structural equation modeling (BESEM) could provide more reliable findings. This possibility is also reinforced by coceptual and theoretical arguments assuming that: a) psyhopathological symptoms are interrelated and; b) behavioural presentations of different diagnostic entities, such as those inquired in SCL-90, may be shared (i.e. sleep disturbanses common between anxiety and depression, [48]).

Asparouhov and Muthén (2009) [49] have advocated the implementation of exploratory structural equation modeling (ESEM) with targeted rotation, as being especially useful for assessing the structure of multi-dimensional scales. ESEM combines the positives of the EFA (i.e. enabling cross-loadings) and CFA (i.e. being conceptually driven, and allowing the examination of a pre-defined structure) approaches. Subsequently, in an ESEM structure, items link to their designated specific dimensions, as well as all other dimensions (at rates approximating zero). Studies have evidenced that this allows ESEM to be superior to the simple EFA and CFA approaches when testing factor structures [37,46]. Interestingly, the ESEM analysis with targeted rotation can be upgraded to a bi-factor ESEM (BESEM) analysis. This uniquely concentrates the positives of the CFA and EFA approaches, and also bi-factor modelling [37,47]. This is crucial for demistyfying the complex associations between the distinct and the overarching dimensions of psychopathological symptoms [48]. In the present study, such complex structures and associations are examined based on SCL-90-R ratings. Not surprisingly, these have been revealed to be multi-dimensional, with various cross-loadings, and entailing both general and specific psyhcopathological dimensions [12,32,35]. Despite these, to date, neither the ESEM nor BESEM approach have been employed for studying the factor structure of the SCL-90-R. It is additionally noted that even if the ESEM model or the BESEM model shows good global fit; it is still necessary for the dimensions in the model to be clearly defined [47].

Current study

Taking these into consideration, the primary goal of the present research was to assess the dimensionality of the SCL-90-R using CFA, bi-factor CFA, ESEM and BESEM approaches. A large and normative cohort of adolescents and young adults from Greece was examined. Several different alternative structures were comparatively studied. These entailed: a) the CFA uni-dimensional; b) the first order CFA nine-dimensional oblique model (with primary dimensions for SOM, OC, IS, DEP, ANX, HOST, PANX, PI, and PSY); c) the higher order CFA model (with the nine primary dimensions linking on a single overarching psychopathological dimension); d) the BCFA structure (with one general dimension and specific sub-dimensions for SOM, OC, IS, DEP, ANX, HOST, PANX, PI, and PSY); e) the ESEM model (with dimensions for SOM, OC, IS, DEP, ANX, HOST, PANX, PI, and PSY); and f) the bi-factor ESEM (BESEM) structure (with one general dimension and specific dimensions for SOM, OC, IS, DEP, ANX, HOST, PANX, PI, and PSY). Model-based reliabilities (omega; [50]) for the various dimensions were also calculated in relation to the best fitting structure. In addition, external associations of the different psychopathological dimensions were examined in relation to family functioning aspects, as assessed with the FACES-IV scale, in line with Tafà and colleagues work (2017) [7].

Research methods

Subjects and procedure

The sample involved 2090 students of public Greek high schools in the broader Athens metropolitan area. The maximum estimated sampling error for this number of respondents is minus/plus 2.14% (95% confidence level; Z = 1.96)1. This satisfies the 4% requirement proposed (Hill, 1998) [51]. In addition, a conducted G-Power evaluation for a-priori estimations, linear multiple regression R2 deviation from 0, an effect size f2 = .15, an error probability of α = .05, power (1-β error probability) of .95, a non-centrality parameter λ = 63.60, a critical F of 1.30, an actual power of .951 and a number of 90 predictors, suggested that a minimum sample of 429 would be required [52]. This condition is also safely addressed here. Last, the current sample size is well above the rule of thump of minimum 20 participants per item considering EFA/CFA procedures (i.e. 20 x 90 = 1800 here; [53]). Participation, provided parents’ consent, exceeded 95% of approached population. Their average age was M = 16.16 (SD = 0.91). All students attending a public Greek high school were eligible to participate in the study. After-hours, vocational, public high schools were not exempted by the selection process. The presence/ absence of psychopathological symptoms was not assessed prior the study. Thus, students who likely presented with psychopathology were also not exempted. The mean (SD) age for males (49.7%) and females (50.3%) was 16.22 years (0.96) and 16.11 years (0.86), respectively. Although females and males differ significantly on age, t = 2.37, p < 0.05, this difference was of negligible effect size (Cohen’s d = .12).

Upon written approval of the Greek Ministry of Education, teacher and written parents’ consent (for the minors included in the sample) were secured (i.e. the parents/guardians of minors addressing the study sent letters of participation acceptance-or the opposite-to the school principal, who communicated the final participants’ list to the research team). Minors themselves had to finally orally aggree to participate. To encourage participation students were explained that if they did not partcipate in the study, they would need to attend the subjects taught duirng the time of the data collection. Data collection took place during class hours during the 2010–2012 school years. It is noted that access to students of public Greek high schools for research purposes, necessitates an initial permission from the research ethics committee of the pedagogical institute of the ministry of education (protocol number for the current data collection is 101359Γ2). A team of 13 specially trained researchers collected data in participants’ classrooms at either the beginning or end of school day. Participation was anonymous and students were not penalized if decided to discontinue.

Measures

Family Adaptability and Cohesion Evaluation Scales IV (FACES IV).

Family relationships health/ well-being, satisfaction, and communication were measured using the Greek version [54] of the FACES IV [28]. The scale reflects the Circumplex Model of Marital and Family Systems [55]. It entails 42 items measuring family functioning in terms of two aspects: a) family cohesion and; b) family flexibility. In the Circumplex Model, cohesion and flexibility are viewed as curvi-linear, and thus very low and very high scores for both these aspects are deemed as maladaptive. In contrast, intermediate scores are considered balanced and adaptive [56]. Corresponding to this, FACES IV has six scales. They include two balanced sub-scales (Balanced Cohesion and Balanced Flexibility), two subs-cales measuring maladaptively low and high cohesion (Disengaged vs Enmeshed), and two sub-scales measuring maladaptively low and high flexibility (Rigid vs Chaotic). For all FACES IV items, responses ranged between 1 = strongly disagree to 5 = strongly agree. Eelevated reported numbers on the two balanced sub-scales are refelective of more adaptive family relationships. Counterintuitively poorer family relationships are reflected by elevated scores in the four maladaptive sub-scales for cohesion and flexibility. For this study, the raw scores from each subscale informed the analyses.

For more robust and holistic evaluation, cohesion ratio, flexibility ratio, and total circumplex ratio scores are recommened to be calculated [28]. These more accurately reflect the relative amount of balance/unbalance for cohesion, flexibility, and the overall (circumplex) family functioning, respectively. Their values fluctuate from 1 to 10, with one indicating equal amount of balanced and unbalanced levels. Higher rates of suggest healthier family functioning [56].

The FACES IV Package also included the Family Communication Scale (FCS, [57]) and the Family Satisfaction Scale (FS, [58]). The FCS and the FSS (10 items each) assess levels of communication and satisfaction within the family, with responses from 1 =“strongly disagree” to 5 = “strongly agree” and higher scores indicating more positive performance across both aspects respectively. The FCS and FSS were additionally completed by the current sample. Like the FACES IV, the FCS and FSS are self-report measures that may be addressed by any family member above 12 years of age. For the current study, these measures were completed by the participants. The internal consistency (Cronbach α) for the 2 adaptive and the 4 maladaptive main FACES IV sub-scales in the current sample fluctuated from .60 to .71, while for the FCS and FSS these equaled 0.91.

The Symptom Checklist-90-R (SCL-90-R; [2]).

Given the Greek speaking population, the Greek adaptation of the SCL-90-R was used in the study [59]. Further to the SCL-90-R information described in the introductory part, it is highlighted that different symptom dimensions are hypothesized to include unequal number of items (i.e. SOM, 12; O-C, 10; IS, 9; DEP, 13; ANX, 10; HOS, 6; PANX, 7; PI, 6; PSY 10). An item example is “Having thoughts that are not your own” (PSY subscale). Distinct items reflect different symptoms and are scored across 5-points (0 = “not at all” and 4 = “very much”), with more elevated numbers being indicative of more intense symptoms. For this study, the item scores were used as observable indicators in the CFA/ESEM analyses. The internal consistency (Cronbach’s α) values for the subscales in this data were SOM = .85; O-C = .79, IS = .81, DEP = .86, ANX = .85, HOS = .84, PANX = .73, PI = .74, and PSY = .78 and .97 for the general psychopathology factor.

Statistical analysis

The Mplus software 7.3 [60] was employed to address all calculations. Given the argument made by Urbán et al. (2014) [31] that analytic methods for the SCL-90-R need to consider the nature of the ratings in this measure, the robust maximum likelihood estimator (MLR) was used. This estimator adjusts for violations of normality and is favoured for items with five or more answering points [60–62] such as those of SCL-90-R.

To examine which of the six different CFA (i.e. unidimensional; nine-dimensional oblique; higher order), BCFA, ESEM or the BESEM models was most applicable, a sequence of four successive steps was adopted. These entailed assessment of: a) global model fit criteria; b) dimension clarity; c) dimension reliability and; d) external associations envisaged with family functioning (as assessed with FACES-IV) based on the literature [15–22].

Therefore, we initially inspected the global fitness of each of the six models separately. Then we compared them to define those with the optimum fit. To assess the 6 models fit independently the Hu and Bentler (1999) [63] recommendations were followed. They proposed that RMSEA scores of .06, or lower reflect sufficient fit, .07 to .0158 moderate, .08 to .10 marginal, and >.10 poor fit. Regarding the CFI and TLI, rates of .95 and higher reflect sufficient fit, between .90 and .95 average, and less than .90 poor fit [63]. Next, comparative model evaluation for the nested models involved the estimation of the chi square difference (Δχ2, [37]). Provided the susceptibility for inflation that applies to χ2 and Δχ2 estimates, the current study additionally considered RMSEA and CFI discrepancies between models [60]. ΔCFI ≤ .010 and/or ΔRMSEA ≤ .015 were viewed as reflecting significant differences [64,65]. Adopting previous suggestions (e.g., [66,67]), we relied more on the differences in CFI values, given that these are less compromised by model complexity, sample size, and overall fit measurements.

Second, the two best fitting structures (as concluded in the first step) were evaluated. Their different dimension loadings/cross-loadings were assessed to confirm the quality/clarity of the dimensions. For this purpose, a rate of ≥.30 was adopted to indicate salience [68]. The model with the optimum fit based on the two initial steps was then chosen.

Third, we tested the omega coefficient (ω) reliabilities of the dimensions included in the optimum fit model [69]. The ω coefficient is understood as a saturation test that accounts for covariance among items in measuring scale reliability [50]. Bi-factor models employ omega hierarchical (ωh) for general factors and omega subscale (ωs) for specific factors [50,69]. The ω coefficient rates vary from 0 to 1 reflecting higher reliability as values increase [70]. The ratio of ωh to ω (called relative omega) indicates the proportion of variance in the overall score that is associated to the general dimension. Cut-off criteria for ωh, as determined by Reise, Bonifay, and Haviland (2013) [71], suggest a minimum value of .50 for acceptable scale reliability and a value of .75 for a more meaningful scale reliability. Furthermore, we used the standards recommended by Smits et al. (2014) [35] to classify all the ω values: substantial ≥: 30, moderate 20 to <: 30; and low <: 20.

Fourth, we also tested the external associations of the factors in the optimum fit structure with the FACES IV scores. Following suggestions outlined in Park et al. (2018) [72], FACES IV, FCS and FSS scale scores were regressed on to the SCL-90-R dimensions of the optimum model. Gender was included as a covariate in the regressions to control for confounding effects.

Results

SCL-90-R models fit

The global fit indices for the sequence of SLC-90-R alternative structures assessed are presented in Table 1. Following Hu and Bentler’s (1999) [63] proposals, for all the CFA models (e.g. unidimensional; nine-dimensional oblique; higher order and bi-factor), the RMSEA suggested good fit, whilst the CFI and TLI poor fit. For both the ESEM and BESEM structures, the RMSEA advocated good fit, while the CFI and TLI adequate fit. Furthermore, both these models showed better fit than all the other models, when the ΔCFI values (> .01) between all structure pairs were considered. Between the ESEM and the BESEM models, the AIC and BIC values were lower for the BESEM model. Thus, this was proposed as a better fitting structure than the ESEM one. However, there was no significant difference between these models in terms of ΔCFI and ΔRMSEA. Thus, it can be taken that when all the global fit values for these two models are considered together, the support for the ESEM and BESEM are somewhat comparable. Therefore, we then examined the clarity of the dimensions in both these models.

thumbnail
Download:
Table 1. Fit of all the models tested in the study.

https://doi.org/10.1371/journal.pone.0247902.t001

Dimensions’ clarity/factor loadings in the two best fitting models.

Tables 2 and 3 presents the loadings for the dimensions of the ESEM and the BESEM structures. Table 4 provides an overview of the number of focused and non focused salient factor loadings in the ESEM and BESEM structures. As evident, most of the focused items associated significantly with their respective dimensions for the ESEM, with the exception of PI. Also the numbers of salient cross-loadings for all the different dimensions were low, ranging from 0 to 3, and there was no negative loading for any symptom. In contrast, relatively fewer items loaded on their BESEM specific dimensions. The majority of the symptoms for seven BESEM dimensions (OC, IS, DE, AN, PA PI, and PY) did not load saliently on their designated dimension. In addition, more salient loadings were present in the ESEM structure vs the BESEM. Although there were many significant cross-loadings in both these structures (ESEM & BESEM), these were relatively more significant in the ESEM. However, considering both models, only few cross-loadings were salient (ranging from 0 to 3 in the ESEM model, and 0 to 1 in the BESEM model). Also, while there were many significant negative cross-loadings across both structures, there were relatively more negative cross-loadings in the BESEM model. Additionally, the intercorrelations of all the SCL-90-R factors in the ESEM model ranged from .10 to .45. Conclusively, the ESEM dimensions were better defined than those of the BESEM. We therefore progressed in the examination of the two final steps of our procedure focusing on this structure.

thumbnail
Download:
Table 2. Standardized factor loadings in the SCL-90-R ESEM model.

https://doi.org/10.1371/journal.pone.0247902.t002

thumbnail
Download:
Table 3. Standardized factor loadings in the SCL-90-R BESEM model.

https://doi.org/10.1371/journal.pone.0247902.t003

thumbnail
Download:
Table 4. Summary of targeted and non-targeted factor loadings in the ESEM and BESEM models.

https://doi.org/10.1371/journal.pone.0247902.t004

Dimensions’ Ω reliabilities

In order to compute the omega coefficient reliabilities for the factors in the SCL-90-R ESEM, we used the targeted/focused factor loadings in this model. As shown (Table 5), for the ESEM model, the ω values for all factors, except PI, were above .20. ranging from .31 to .80. Furthermore, all ω except those for PI, PSY and ANX were above the adequacy threshold of .50 [71]. The SCL-90-R factor reliabilities were additionally computed using the factor loadings of the designated symptoms in the CFA 9-factor oblique model. As shown in Table 5, the ω value values (based on the CFA model) for all factors were high, ranging from .74 to .94.

thumbnail
Download:
Table 5. Inter-correlations and reliabilities of the factors in the SCL-90-R ESEM model.

https://doi.org/10.1371/journal.pone.0247902.t005

Psychopathological dimensions & family functioning aspects

Table 6 provides the standardized regression coefficients for the associations of the three main FACES IV subscales (i.e. cohesion ratio; flexibility ratio; family ratio), family satisfaction, and family communication (whilst accounting for gender), and the nine dimensions in the SCL-90-R ESEM optimum fit model. As evident for the ESEM structure (see Table 6), SOM associated negatively to family communication. DEP and PSY were also reversely associated to flexibility ratio, family satisfaction and family communication. Interestingly, ANX positively linked to flexibility ratio, family ratio, family satisfaction, and family communication. HOS was negatively associated to family cohesion ratio, family ratio, family satisfaction and family communication. Finally, O-C, PANX, and PI did not significantly tie with any of the family aspects examined. Conclusively, there were signicant external links between family functioning aspects and SOM, IS, DEP, ANX, HOS, and PSY, but not for O-C, PANX, and PI.

thumbnail
Download:
Table 6. Standardized path coefficients for the predictions of the FACES IV ratio scores for cohesion, flexibility and total scale, family communication and family satisfaction by the SCL-90-R subscale factor scores in the ESEM model.

https://doi.org/10.1371/journal.pone.0247902.t006

Optimum structure selection

We selected the ESEM model with nine primary factors as our preferred model for the following reasons: (1) this structure had better global fit than all the CFA models; (2) although the ESEM and BESEM had comparable global fit indices; the ESEM patterns of loadings/cross loadings revealed better clarity. Our selection was also reinforced by the nine primary ESEM dimensions showing acceptable reliabilities. Finally, signicant associations between family functioning aspects and the SOM, IS, DEP, ANX, HOS, and PSY dimensions of the deemed as optimum model emerged.

Discussion

The present study’s aim was twofold: a) to shed light in the controversies considering the structural dimensions of psychopathological symptoms, as assessed with the SCL-90 R and; b) to examine potential associations of specific family functioning aspects with the SCL-90 distinct psychopathological dimensions revealed. To address these aims, a large cohort of 2090 adolescents and young adults from Greece was examined. Considering the first aim, six different structural SCL-90-R models, reflecting different diagnostic conceptions were tested. First, the unidimensional CFA structure was assessed. This aligns with a transdiagnostic perspective, assuming that a general psychopathological tendency explains the different presentations [73]. Therefore, symptom specific approaches may be less effective [48]. Second, the nine-dimensional CFA structure was examined. This contents that symptom specific dimensions should be emphasized; while it contradicts the notion of a general psychopathological propensity [31,32]. Third, a higher order CFA dimensional structure with one overarching and nine direct symptom dimensions was assessed. This assumes that although different symptom dimensions may occur, to some extent these share the common influence of a general psychopathological tendency [9]. Fourth, the BCFA structure assuming one general dimension and nine symptom specific dimensions was evaluated. This structure assumes that although specific symptom dimensions apply, all (and not just part) of their common variations are attributed to a general psychopathology dimension (which therefore needs to be targeted, [47]). Fifth, the nine-dimensional ESEM was tested. This advocates the existence of distinct psychopathological dimensions, which may somehow correlate in the absence of a general psychopathological dimension [46]. Finally, the BESEM structure was evaluated. This assumes one general psychopathology dimension and nine presentation specific dimensions. These may also correlate with each other independent of the general tendency that to some extent implicates with all of them [46,47]. Adopting the theoretical structure of the SCL-90-R, the nine group/specific dimensions across all the solutions examined were SOM, O-C, IS, DEP, ANX, HOS, PANX, PI, and PSY. To conclude the best fitting structure, we appreciated four sequential steps: a) global fit; b) dimension clarity; c) dimension reliability and; d) the associations between the SCL-90 dimensions revealed were explored in relation to family functioning aspects. The fourth step of this process concurrently addressed our second aim.

SCL-90-R dimensional structure

Considering global fitness, for all four CFA structures (unidimensional CFA, nine-dimensional CFA, higher order dimensional, and nine-dimensional BCFA), the RMSEA rates suggested good fit. In contrast, the CFI and TLI advocated poor fitness. Thus, when more than one index is considered, these structures present to have poor fit. Therefore, it is evident that, due to confirmatory procedures not allowing cross-loadings, the absence of inter-relationships between symptom specific dimensions, not accounting to a general psychopathology dimension should be excluded [43,44]. Regarding the nine-dimensional ESEM, and nine-dimensional BESEM structures tested, the RMSEA revealed good fit, and the CFI and TLI sufficient fit. Thus, both these structures were superior to the previous four (ΔCFI > .01 in all instances of pair-model comparisons). As in ESEM structures items may link to their designated specific dimensions, as well as all other dimensions [37,46], this finding confirms the occurrence of interassociations between the symptom specific dimensions, independent of the applicability of a general dimension. Furthermore, there was no significant discrepancy between the ESEM and BESEM structures in regards to ΔCFI and ΔRMSEA. Nonetheless, additional assessment of both these structures considering their dimension clarity via the use of dimensional loadings was revealing. This evaluation step suggested that the pattern of dimension loadings was clearer in the ESEM structure than the BESEM one. Finally, the ESEM structure demonstrated overall sufficient reliabilities. Given Morin and colleague’s (2015) [47] recommendation that the acceptance of a bi-factor model requires adequate reliability rates for the dimensions it proposes, the ESEM structure was chosen as the preferred structure for the SCL-90-R, at least when considering a large sample from Greece. This means that although distinct symptom dimensions may occur, in the absence of a general psychopathological tendency, these are interassociated and not completely independent to each other. This finding aligns with the numerous cross-loadings suggested in Arrindell and colleagues’ findings (2017) [9]. Furthermore, it reinforces theoretical arguments assuming that: a) psyhopathological symptoms are interrelated and; b) behavioural presentations of different diagnostic entities, such as those inquired in SCL-90, may be shared [48]. Therefore, it is likely that symptom specific interventions may also have indirect effects on non-targeted symptoms.

Our results expand the extant knowledge considering the SCL-90 dimensional structure. In contrast with previous research that has employed exclusively EFA (e,g., [38–42] and CFA (e.g., [9,12,31–35]), our work additionally assessed the dimensional structure of SCL-90-R via the employement of ESEM and BESEM structures. Thus, the results for the ESEM and BESEM for the SCL-90-R are unique. Due to this, we were able to consider alternative conceptualizations of psychopathological symptoms, that had not been previously evaluated. In that line, the theorized nine-factor oblique model for the SCL-90-R was expanded. This was achieved via the acknowledgment of interassociations between the different dimensions, irrespective of an inclusive/ general psychopathological dimension [2].

In the light of these innovations, our results appear comparative with past literature. Considering at least one global fit index suggesting sufficient fitness, past evidence has also proposed sufficient fitness for the uni-dimensional structure [12,32,33]. In addition, past CFA results have similalry revealed at least adequate fit for the nine-dimensional oblique strcuture, the second order dimensional structure and the bi-dimensional CFA structure. Nonetheless, although previous studies have favoured better global fit for the CFA bi-dimensional structure (over the CFA one-factor, nine-factor oblique and higher order factor models), this was not compared with alternative ESEM structures [9,12].

Psychopathological dimensions and family functioning

Our results regarding the links of family effects with psychopathological presentations (as measured with the SCL-90-R) are aligning with past evidence involving young individuals [7,15–22,74]. Nevertheless, the present work methodologically expands past literature in at least two ways: a) the examination of a large and normative cohort of Greek high school students and; b) the concurrent assessment of all the nine symptom specific dimensions of the SCL-90-R and the five major family aspects, as reflected by FACES-IV. Interestingly, results highlighted that SOM associated negatively to family communication. This corresponds with evidence suggesting that unexpressed feelings and thoughts could invite psycho-somatization [16]. DEP and PSY were similalry negatively associated to family flexibility, satisfaction and communication. This result also corresponds with literature suggesting that DEP and PSY may either be exacerbated by adverse external experiences, such as those related to family functioning; or negatively impact the level of the family relationships of the carrier [25,75]. In contrast, ANX positively tied to family flexibility, satisfaction, communication and the overall family functioning (as reflected by the family ratio). This finding also entertains past literature (e.g., [76]). It may be therefore assumed, that anxiety, when not comorbid with more compromising symptoms (i.e. PSY & PI), constitutes a presentation of better-adapted individuals, likely less impacted by adverse family conditions [77]. In that line, HOS was negatively linked to family cohesion, satisfaction, communication and the overall family functioning. This also corresponds with past evidence proposing aggression as an externalized repercussion and/or a cause of poorer family conditions [7,27,78]. Nonetheless, O-C, PANX, and PI did not present significant associations with family functioning aspects in contrast with past evidence for relevant clinical populations [79,80]. This may be attributed to the nature of the sample, which (being normative) could soften links more prominent in clinical populations [81]. Overall, there were ties between specific different family functioning aspects and SOM, IS, DEP, ANX, HOS, and PSY.

Implications & conclusion

Our results have significant theoretical/conceptual, assessment/diagnostic and prevention/intervention implications. Regarding the theoretical value of the findings, the support of the ESEM with the nine distinct synmptom dimensions as the optimum SCL-90 structure is deemed important. It implies that although distinct symptom dimensions occur, these are interassociated, while a general psychopathology factor presents not applicable. Considering the use of the SCL-90-R in assessment/ diagnosis, results do not favor the use of the general factor score. The authors of the SCL-90-R [2] proposed that this scale should be scored for the nine symptom scales (dimensions) and also the total score, as derived by the sum of all items. Furthermore, they suggested that these scores can be applied concurrently. For this suggestion to be valid, it would be required that a bi-dimensional structure should be appreciated as the optimum [71]. Two bi-dimensional SCL-90-R structures were tested (i.e. the BCFA and the BESEM) and neither was concluded as presenting the best fit. This implication is further reinforced by the rejection of the unidimensional structure. As the ESEM structure with the nine obliquely related dimensions was the preferred one, the ideal SCL-90-R scores employed in psychological assessment are those referring to the distinct symptom dimensions.

The study results also illustrate insights for family based prevention and treatment of psychopathological manifestations. In particular, specific suggestions apply given the revealed associations between SOM, DEP, ANX, HOS, and PSY with family functioning aspects. Specifically, family communication strategies should be emphasized by parents for the prevention and treatment of SOM symptoms. Similarly, increasing family flexibility, satisfaction and communication should be considered in young individuals experiencing DEP. Considering HOS and aggression, prevention and intervention protocols aiming to boost family cohesion, satisfaction and communication may also need to be emphasized. Last, family flexibility/boundaries should also be targeted for individuals presenting with PSY symptoms.

Such inittiatives might be of a particular importance when SCL-R-90 assessment is employed among adolescents and early/young adults. The majority of mental health symptoms, as well as prodromal manifestations of life-long disorders (e.g. psychosis) first present during these years (13–25, [12,13]. Thus, addressing family functioning in ways that do not exacerbate risk and promote healthy behaviours could prove crucial for young individuals’ concurrent and prospective adaptation [14].

Limitations & further research

The major strengths of this project entail: a) the employment of a large sample; b) the application (unlike previous studies) of innovative and advanced ESEM and BESEM approaches [46] to test the structure of the SCL-90-R and; c) the comprehensive examination of all the different symptom dimensions of the SCL-90 with family functioning aspects as assessed with the FACES-IV. Despite these positives, several limitations need to be taken into account when evaluating our findings. First, it is possible that factors such as gender and ethnicity could influence ratings of SCL-90-R items. The failure to control for these effects in this study could have confounded the results. Second, as ethics approval for this study did not permit collection of information about individuals prior to inviting them to participate, there is no information about those who knew of the study but did not respond to the invitation to take part, and therefore how this affected the results. Third, as this study examined a community sample, it is uncertain if the findings are applicable to clinically diagnosed adolescents and young adults. Fourth, as the SCL-90-R and the other measures used in the study are self-report questionnaires, it is possible the ratings may have been influenced by the method used to collect them, thereby subjecting them to confounding by common method variance effects. Fifth, indirect external validities of the dimensions were not examined. Future research should include a wider range of relevant external factors, such as clinical disorders, to provide more accurate relevant testing of the external validities of the ESEM model dimensions. Notwithstanding all these limitations, our findings do support the use of the SCL-90-R in clinical practice and research. Last, they do invite for more studies in this area, that may account for the restrictions highlighted in the present work.

References

  1. 1. Derogatis L.R. (1994). SCL-90-R: Administration, scoring and procedures manual (3rd ed.). Minneapolis, MN: NCS Pearson.
    • 2. Derogatis L.R., & Savitz K.L. (2000). The SCL-90-R, Brief Symptom Inventory, and Matching Clinical Rating Scales. In Maruish M.E. (Ed.), The use of psychological testing for treatment planning and outcomes assessment (p. 679–724). New York, NY: Routledge.
      • 3. Vaurio R. (2011) Symptom Checklist-90-Revised. In: Kreutzer J.S., DeLuca J., Caplan B. (eds) Encyclopedia of Clinical Neuropsychology. Springer, New York, NY. https://doi.org/10.1007/978-0-387-79948-3_2012.
        • 4. Derogatis L.R., Lipman R.S., Rickels K., Uhlenhuth E.H., & Covi L. (1974). The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory. Behavioural Science, 19(1), 1–15. pmid:4808738
        • 5. Derogatis L.R., Lipman R.S., & Covi L. (1973). SCL-90: an outpatient psychiatric rating scale-preliminary report. Psychopharmacology Bulletin, 9(1), 13/28. pmid:4682398
        • 6. Stavropoulos V., Moore K. A., Lazaratou H., Dikeos D., & Gomez R. (2017). A multilevel longitudinal study of obsessive compulsive symptoms in adolescence: male gender and emotional stability as protective factors. Annals of general psychiatry, 16(1), 42. pmid:29201132
        • 7. Tafà M., Cimino S., Ballarotto G., Bracaglia F., Bottone C., & Cerniglia L. (2017). Female adolescents with eating disorders, parental psychopathological risk and family functioning. Journal of Child and Family Studies, 26(1), 28–39. http://doi.org/10.1007/s10826-016-0531-5.
        • 8. Solmi M., Collantoni E., Meneguzzo P., Degortes D., Tenconi E., & Favaro A. (2018). Network analysis of specific psychopathology and psychiatric symptoms in patients with eating disorders. International Journal of Eating Disorders, 51(7), 680–692. pmid:29846016
        • 9. Arrindell W.A., Urbán R., Carrozzino D., Bech P., Demetrovics Z., & Roozen H.G. (2017). SCL-90-R emotional distress ratings in substance use and impulse control disorders: One-factor, oblique first-order, higher-order, and bi-factor models compared. Psychiatry Research, 255, 173–185. pmid:28558358
        • 10. Maremmani A. G., Maiello M., Carbone M. G., Pallucchini A., Brizzi F., Belcari I., et al. (2018). Towards a psychopathology specific to substance use disorder: should emotional responses to life events be included?. Comprehensive psychiatry, 80, 132–139. pmid:29091779
        • 11. Baier A., Fritsch R., Ignatyev Y., Priebe S., & Mundt A. P. (2016). The course of major depression during imprisonment–A one year cohort study. Journal of affective disorders, 189, 207–213. pmid:26451505
        • 12. Preti A., Carta M.G., & Petretto D.R. (2019). Factor structure models of the SCL-90-R: Replicability across community samples of adolescents. Psychiatry Research, 272, 491–498. pmid:30611969
        • 13. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub.
          • 14. Masten A. S., & Kalstabakken A. W. (2018). Developmental perspectives on psychopathology in children and adolescents. In Butcher J. N. & Kendall P. C. (Eds.), APA handbooks in psychology®. APA handbook of psychopathology: Child and adolescent psychopathology (p. 15–36). American Psychological Association. https://doi.org/10.1037/0000065-002.
            • 15. Chapman L.K., & Woodruff-Borden J. (2009). The impact of family functioning on anxiety symptoms in African American and European American young adults. Personality and Individual Differences, 47(6), 583–589. http://doi.org/10.1016/j.paid.2009.05.012.
            • 16. Ferro M.A., & Boyle M.H. (2014). The Impact of Chronic Physical Illness, Maternal Depressive Symptoms, Family Functioning, and Self-esteem on Symptoms of Anxiety and Depression in Children. Journal of Abnormal Child Psychology, 43(1), 177–187. https://doi.org/10.1007/s10802-014-9893-6.
            • 17. Guberman C., & Manassis K. (2011). Symptomatology and Family Functioning in Children and Adolescents with Comorbid Anxiety and Depression. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 20(3), 186–195. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3143694/. pmid:21804847
            • 18. Hughes E.K., & Gullone E. (2008). Internalizing symptoms and disorders in families of adolescents: A review of family systems literature. Clinical Psychology Review, 28(1), 92–117. pmid:17509739
            • 19. Queen A.H., Stewart L.M., Ehrenreich-May J., & Pincus D.B. (2012). Mothers’ and Fathers’ Ratings of Family Relationship Quality: Associations with Preadolescent and Adolescent Anxiety and Depressive Symptoms in a Clinical Sample. Child Psychiatry & Human Development, 44(3), 351–360. https://doi.org/10.1007/s10578-012-0329-7.
            • 20. Rapee R. (1997). Potential role of childrearing practices in the development of anxiety and depression. Clinical Psychology Review, 17(1), 47–67. pmid:9125367
            • 21. Santesteban-Echarri O., MacQueen G., Goldstein B.I., Wang J.L., Kennedy S.H., Bray S., et al. (2018). Family functioning in youth at-risk for serious mental illness. Comprehensive Psychiatry. pmid:30193153
            • 22. Tamplin A., & Gooyer I.M. (2001). Family functioning in adolescents at high and low risk for major depressive disorder. European Child & Adolescent Psychiatry, 10(3), 170–179. http://doi.org/10.1007/s007870170023.
            • 23. Wang Y., Tian L., Guo L., & Huebner E.S. (2020). Family dysfunction and Adolescents’ anxiety and depression: A multiple mediation model. Journal of Applied Developmental Psychology, 66, 101090. http://doi.org/10.1016/j.appdev.2019.101090.
            • 24. Dollberg D. G., & Keren M. (2020). Factors contributing to continuity and discontinuity in child psychopathology from infancy to childhood: An explorative study. Clinical Child Psychology and Psychiatry, 25(4), 891–908. pmid:32508128
            • 25. Jacobvitz D., Hazen N., Curran M., & Hitchens K. (2004). Observations of early triadic family interactions: Boundary disturbances in the family predict symptoms of depression, anxiety, and attention-deficit/hyperactivity disorder in middle childhood. Development and psychopathology, 16(3), 577–592. pmid:15605626
            • 26. Si Y. (2020). The sources of parenting stress in Chinese families of children with attention-deficit/hyperactivity disorder: An overview and future directions. Movement and Nutrition in Health and Disease, 4. http://doi.org/10.5283/mnhd.21.
            • 27. Rutter M. (1994). Family discord and conduct disorder: Cause, consequence, or correlate?. Journal of family psychology, 8(2), 170. https://doi.org/10.1037/0893-3200.8.2.170.
            • 28. Olson D.H, Gorall D.M, & Tiesel J.W. (2006). FACES-IV package: Administration. Minneapolis, MN: Life Innovations.
              • 29. Jozefiak T., & Wallander J. L. (2016). Perceived family functioning, adolescent psychopathology and quality of life in the general population: a 6-month follow-up study. Quality of Life Research, 25(4), 959–967. pmid:26410101
              • 30. Fritz J., de Graaff A. M., Caisley H., Van Harmelen A. L., & Wilkinson P. O. (2018). A systematic review of amenable resilience factors that moderate and/or mediate the relationship between childhood adversity and mental health in young people. Frontiers in psychiatry, 9, 230. pmid:29971021
              • 31. Urbán R., Kun B., Farkas J., Paksi B., Kökönyei G., Zsolt U., et al. (2014). Bifactor structural model of symptom checklists: SCL-90-R and Brief Symptom Inventory (BSI) in a non-clinical community sample. Psychiatry Research, 216(1), 146–154. pmid:24524946
              • 32. Urbán R., Arrindell W.A., Demetrovics Z., Unoka Z., & Timman R. (2016). Cross-cultural confirmation of bi-factor models of a symptom distress measure: Symptom Checklist-90-Revised in clinical samples. Psychiatry Research, 239, 265–274. pmid:27039011
              • 33. Rytilä-Manninen M., Frojd S., Haravuori H., Lindberg N., Marttunen M., Kettunen K., et al. (2016). Psychometric properties of the Symptom Checklist-90 in adolescent psychiatric inpatients and age- and gender-matched community youth. Child and Adolescent Psychiatry and Mental Health, 10(1). pmid:27429645
              • 34. Sereda Y., & Dembitskyi S. (2016). Validity assessment of the symptom checklist SCL-90-R and shortened versions for the general population in Ukraine. BMC Psychiatry, 16(1). pmid:27561313
              • 35. Smits I.A.M., Timmerman M.E., Barelds D.P.H., & Meijer R.R. (2014). The Dutch Symptom Checklist-90-Revised: Is the Use of the Subscales Justified? European Journal of Psychological Assessment, 31, 263–271. http://doi.org/10.1027/1015-5759/a000233.
              • 36. Cyr J.J., McKenna-Foley J.M., & Peacock E. (1985). Factor Structure of the SCL-90-R: Is There One? Journal of Personality Assessment, 49(6), 571–578. pmid:4093836
              • 37. Marsh H.W., Morin A.J.S., Parker P.D., & Kaur G. (2014). Exploratory Structural Equation Modeling: An Integration of the Best Features of Exploratory and Confirmatory Factor Analysis. Annual Review of Clinical Psychology, 10(1), 85–110. pmid:24313568
              • 38. Derogatis L.R., & Cleary P.A. (1977). Factorial invariance across gender for the primary symptom dimensions of the SCL-90. British Journal of Social and Clinical Psychology, 16, 347–356. pmid:588890
              • 39. Bonynge E.R. (1993). Unidimensionality of SCL-90-R scales in adult and adolescent crisis samples. Journal of Clinical Psychology, 49(2), 212–215. pmid:8486803
              • 40. Carpenter K.M., & Hittner J.B. (1995). Dimensional characteristics of the SCL-90-R: Evaluation of gender differences in dually diagnosed inpatients. Journal of Clinical Psychology, 51(3), 383–390. pmid:7560139
              • 41. Shutty M.S., DeGood D.E., & Schwartz D.P. (1986). Psychological dimensions of distress in chronic pain patients: A factor analytic study of Symptom Checklist-90 responses. Journal of Consulting and Clinical Psychology, 54(6), 836–842. pmid:3794030
              • 42. Vassend A., & Skrondal A. (1999). The problem of structural indeterminacy in multidimensional symptom report instruments. The case of SCL-90-R. Behaviour Research and Therapy, 37(7), 685–701. pmid:10402693
              • 43. Jöreskog K.G. (1969). A General Approach to Confirmatory Factor Analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343.
              • 44. Morin A.J.S., Tran A., & Caci H. (2013). Factorial Validity of the ADHD Adult Symptom Rating Scale in a French Community Sample. Journal of Attention Disorders, 20(6), 530–541. pmid:23729493
              • 45. Bonifay W., Lane S. P., & Reise S.P. (2016). Three Concerns With Applying a Bifactor Model as a Structure of Psychopathology. Clinical Psychological Science, 5(1), 184–186. http://doi.org/10.1177/2167702616657069.
              • 46. Marsh H.W., Muthén B., Asparouhov T., Lüdtke O., Robitzch A., Morin A.J.S., et al. (2009). Exploratory Structural Equation Modelling, Integrating CFA and EFA: Application to Students’ Evaluations of University Teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439–476. https://doi.org/10.1080/10705510903008220.
              • 47. Morin A.J.S., Arens A.K., & Marsh H.W. (2015). A Bifactor Exploratory Structural Equation Modeling Framework for the Identification of Distinct Sources of Construct-Relevant Psychometric Multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal, 23(1), 116–139. https://doi.org/10.1080/10705511.2014.961800.
              • 48. Gomez R., Stavropoulos V., Vance A., & Griffiths M. D. (2019). Re-evaluation of the latent structure of common childhood disorders: Is there a general psychopathology factor (p-factor)?. International Journal of Mental Health and Addiction, 17(2), 258–278. https://doi.org/10.1007/s11469-018-0017-3.
              • 49. Asparouhov T., & Muthén B. (2009). Exploratory Structural Equation Modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397–438. http://doi.org/10.1080/10705510903008204.
              • 50. McDonald R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: L. Erlbaum Associates.
                • 51. Hill R. (1998). What sample size is “enough” in internet survey research. Interpersonal Computing and Technology: An electronic journal for the 21st century, 6(3–4), 1–12.
                • 52. Faul F., Erdfelder E., Lang A. G., & Buchner A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175–191. pmid:17695343
                • 53. Myers N. G., Suyeon A., & Ahn Y. S. (2011). Sample size and power estimates for a confirmatory factor analytic model in exercise and sport: A Monte Carlo approach. Research Quarterly for Exercise and Sport, 82(3), 412–423. pmid:21957699
                • 54. Koutra K., Triliva S., Roumeliotaki T., Lionis C., & Vgontzas A. N. (2012). Cross-Cultural Adaptation and Validation of the Greek Version of the Family Adaptability and Cohesion Evaluation Scales IV Package (FACES IV Package). Journal of Family Issues, 34(12), 1647–1672. https://doi.org/10.1177/0192513x12462818.
                • 55. Olson D.H., Sprenkle D.H., & Russell C.S. (1979). Circumplex Model of Marital and Family Systems: I. Cohesion and Adaptability Dimensions, Family Types, and Clinical Applications. Family Process, 18(1), 3–28. pmid:437067
                • 56. Olson D. (2011). FACES IV and the Circumplex Model: Validation Study. Journal of Marital and Family Therapy, 37(1), 64–80. pmid:21198689
                • 57. Olson D. H., & Barnes H. (2004). Family communication. Minneapolis: Life Innovations.
                • 58. Olson D. H. (1995). Family Satisfaction Scale. Minneapolis, MN: Life Innovations
                  • 59. Donias S., Karastergiou A., & Manos N. (1991). Standardization of the symptom checklist 90 rating scale in a Greek population. Psychiatriki, 2, 42–48.
                  • 60. Muthen L. K., & Muthen B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.
                    • 61. DiStefano C. (2002). The Impact of Categorization With Confirmatory Factor Analysis. Structural Equation Modeling: A Multidisciplinary Journal, 9(3), 327–346. https://doi.org/10.1207/s15328007sem0903_2.
                    • 62. Lubke G.H., & Muthén B.O. (2004). Applying Multigroup Confirmatory Factor Models for Continuous Outcomes to Likert Scale Data Complicates Meaningful Group Comparisons. Structural Equation Modeling: A Multidisciplinary Journal, 11(4), 514–534. http://doi.org/10.1207/s15328007sem1104_2.
                    • 63. Hu L., & Bentler P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.
                    • 64. Chen F. F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. http://doi.org/10.1080/10705510701301834.
                    • 65. Cheung G. W., & Rensvold R. B. (2002). Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. http://doi.org/10.1207/s15328007sem0902_5.
                    • 66. Guo B., Aveyard P., Fielding A., & Sutton S. (2009). The Factor Structure and Factorial Invariance for the Decisional Balance Scale for Adolescent Smoking. International Journal of Behavioral Medicine, 16(2), 158–163. pmid:19225896
                    • 67. Guo B., Kaylor-Hughes C., Garland A., Nixon N., Sweeney T., Simpson S., et al. (2017). Factor structure and longitudinal measurement invariance of PHQ-9 for specialist mental health care patients with persistent major depressive disorder: Exploratory Structural Equation Modelling. Journal of Affective Disorders, 219, 1–8. pmid:28501679
                    • 68. Hair J.F. Jr., Black W.C., Babin B.J., & Anderson R.E. (2010). Multivariate Data Analysis (7th ed.). Essex: UK, Pearson.
                      • 69. Zinbarg R. E., Revelle W., Yovel I., & Li W. (2005). Cronbach’s α, Revelle’s β, and Mcdonald’s ωh: their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. http://doi.org/10.1007/s11336-003-0974-7.
                      • 70. Brunner M., Nagy G., & Wilhelm O. (2012). A Tutorial on Hierarchically Structured Constructs. Journal of Personality, 80(4), 796–846. pmid:22091867
                      • 71. Reise S.P., Bonifay W.E., & Haviland M.G. (2013). Scoring and Modeling Psychological Measures in the Presence of Multidimensionality. Journal of Personality Assessment, 95(2), 129–140. pmid:23030794
                      • 72. Park J.L., Silveira M., Elliott M., Savalei V., & Johnston C. (2018). Confirmatory Factor Analysis of the Structure of Adult ADHD Symptoms. Journal of Psychopathology and Behavioral Assessment. https://doi.org/10.1007/s10862-018-9698-y.
                      • 73. Stanton K., McDonnell C. G., Hayden E. P., & Watson D. (2020). Transdiagnostic approaches to psychopathology measurement: Recommendations for measure selection, data analysis, and participant recruitment. Journal of Abnormal Psychology, 129(1), 21. pmid:31868384
                      • 74. Tafà M., Cerniglia L., Cimino S., Ballarotto G., Marzilli E., & Tambelli R. (2018). Predictive values of early parental loss and psychopathological risk for physical problems in early adolescents. Frontiers in Psychology, 9, 922. pmid:29928249
                      • 75. Anagnostopoulou N., Kyriakopoulos M., & Alba A. (2019). Psychological interventions in psychosis in children and adolescents: a systematic review. European child & adolescent psychiatry, 28(6), 735–746. pmid:29728871
                      • 76. Peleg‐Popko O., & Dar R. (2001). Marital quality, family patterns, and children’s fears and social anxiety. Contemporary Family Therapy, 23(4), 465–487. https://doi.org/10.1023/a:1013057129790.
                      • 77. Nakajima M., Takano K., & Tanno Y. (2017). Adaptive functions of self-focused attention: Insight and depressive and anxiety symptoms. Psychiatry research, 249, 275–280. pmid:28135598
                      • 78. Kazdin A. E., & Wassell G. (2000). Therapeutic changes in children, parents, and families resulting from treatment of children with conduct problems. Journal of the American Academy of Child & Adolescent Psychiatry, 39(4), 414–420. http://doi.org/ 10.1097/00004583-200004000-00009.
                      • 79. Albert U., Bogetto F., Maina G., Saracco P., Brunatto C., & Mataix-Cols D. (2010). Family accommodation in obsessive–compulsive disorder: Relation to symptom dimensions, clinical and family characteristics. Psychiatry Research, 179(2), 204–211. pmid:20483467
                      • 80. Riggio H. R., & Kwong W. Y. (2011). Paranoid thinking, quality of relationships with parents, and social outcomes among young adults. Journal of Family Issues, 32(8), 1030–1049.
                      • 81. Chung T., Martin C. S., Armstrong T. D., & Labouvie E. W. (2002). Prevalence of DSM-IV alcohol diagnoses and symptoms in adolescent community and clinical samples. Journal of the American Academy of Child & Adolescent Psychiatry, 41(5), 546–554. pmid:12014787