https://orcid.org/0009-0004-0063-8917
Figures
Abstract
Categorical diagnostic systems for psychopathology, such as the DSM and ICD, have long been criticized for their limited validity and reliability. Dimensional models, like the Hierarchical Taxonomy of Psychopathology (HiTOP), offer an alternative by focusing on transdiagnostic dimensions that better capture the complexity of mental health disorders. While HiTOP’s internalizing spectrum has been studied extensively in adults, its applicability and structure in children and adolescents remain less clear. Further, understanding sociopsychological indicators associated with internalizing dimensions in this age group could improve developmental psychopathological interventions. We analyzed data from 4,142 participants aged 5–21 (65.7% male; mean age = 10.46) from the Healthy Brain Network. Using exploratory and confirmatory factor analyses, we tested the internalizing structure proposed by HiTOP, with an additional focus on invariance across sex, age, and diagnostic groups. The hierarchical structure was tested through hierarchical CFA and the extended Bass-Ackward method. Structural equation modeling (SEM) examined latent factor relationships, and sociopsychological variables associated with the factors. A four-factor structure was identified: Distress, Nervousness, Social Fears, and Obsessions and Compulsions (OC). The model demonstrated partial invariance and strong fit indices. Sociopsychological variables, i.e., predictors and a quality of life indicator of the factors, included parental attitudes, discipline, bullying, and daily functioning. DSM categories and CBCL scores mapped well onto the latent factors. These findings suggest the potential clinical utility of a dimensional model for internalizing disorders in youth. Future studies should further examine the role of sociodemographic factors on dimensional constructs, explore predictive developmental trajectories longitudinally, and verify the structure of all HiTOP spectra across age groups to advance dimensional models in pediatric psychopathology research, as well as their implementation in clinical practice.
Citation: Engler IM, Langer N (2025) Toward dimensional psychiatry in youth: A data-driven analysis of transdiagnostic internalizing symptoms in childhood and adolescence. PLOS Ment Health 2(12): e0000509. https://doi.org/10.1371/journal.pmen.0000509
Editor: Juan Felipe Cardona, Universidad del Valle, COLOMBIA
Received: May 11, 2025; Accepted: November 17, 2025; Published: December 17, 2025
Copyright: © 2025 Engler, Langer. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data and code utilized for all analyses can be found at https://osf.io/br8wd/files/osfstorage.
Funding: This work was supported by the Swiss National Science Foundation (SNSF) Grant No. 10001C_220048, awarded to NL (https://www.snf.ch/en). This grant includes the salary of multiple collaborators, including IE. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
The prevalence of mental health disorders is increasing at a concerning rate globally, reaching unprecedented levels and causing a significant social, physical health, and economic burden [1,2]. A significant challenge in current mental health research is the reliance on group-average case-control studies and traditional diagnostic frameworks as those exemplified by classification systems of psychopathology [3,4]. Systems such as the International Classification of Diseases (ICD) and the Diagnostic and Statistical Manual of Mental Disorders (DSM) define, structure, and categorize mental health disorders based on symptom-based diagnostic criteria [5,6]. These frameworks facilitate communication among clinicians, patients, and researchers, while supporting the diagnostic processes and treatment selection [7,8]. However, numerous limitations of conventional nosologies have been highlighted, including the high comorbidity between many mental disorders [9,10], significant heterogeneity within syndromes [11–13], and the low to moderate reliability of diagnoses, such as low inter-rater and test-retest reliability [4,14,15]. Furthermore, many undiagnosed or subclinical cases remain unaddressed due to the polythetic and dichotomous nature of these categorical classification systems [16,17]. Importantly, emerging research from genetics, systems neuroscience, and behavioral science has increasingly challenged the validity of traditional diagnostic labels [18–20].
In response to these issues, alternative nosological systems have emerged, e.g., clinical staging models [21], network models [22], and the Research Domain of Criteria (RDoC) [23], among others. One prominent example is the Hierarchical Taxonomy of Psychopathology (HiTOP) Consortium’s project, which aims to overcome the shortcomings of traditional classification systems by introducing a hierarchical, dimensional, and empirically driven model that maps a wide range of interconnected psychopathologies [3]. HiTOP organizes psychopathological traits and symptoms into hierarchically ordered transdiagnostic dimensions. At the base of the hierarchy lie lower-order traits and symptoms, which are grouped by their co-occurrence into syndromes. These syndromes are further clustered into subfactors, which are then nested within broader transdiagnostic spectra. Five spectra have been differentiated: somatoform, internalizing, thought disorders, detachment, and externalizing. At the top of the model, a general factor of psychopathology, the p-factor, presides over all other dimensions, which “represents features common across all of psychopathology”, as well as a vulnerability underlying all psychopathologies [3,24]. However, the existence and validity of the p-factor have been questioned, as they depend on the chosen statistical method and the structural conceptualization of a dimensional model [25,26]. Nonetheless, this hierarchical and dimensional structure allows HiTOP to capture the complexity and interconnectivity of mental health conditions more accurately than traditional diagnostic systems [27,28].
In the HiTOP model, the conceptual basis of the dimensions is a separation between inherently internalizing and inherently externalizing disorders. This principle follows a long-standing finding from childhood studies by Thomas Achenbach, whereby “externalizing” refers to outwardly expressed symptoms, and “internalizing” to inwardly directed ones [29]. Likewise, HiTOP’s externalizing spectrum represents a range of outward maladaptive behaviors, whereas the internalizing spectrum represents an inward manifestation of emotional dysfunction [30]. It encompasses four subfactors: Distress, Fear, Sexual Problems, and Eating Pathologies. The internalizing domain is particularly important due to its wide scope, its association to impaired functioning (e.g., executive functioning or disability level), the high degree of comorbidity among its related disorders, and its raising prevalence [31–33].
Albeit the basic dichotomy between an internalizing and an externalizing spectrum initially emerged from childhood studies, most research on HiTOP has concentrated on the structure of psychopathology in adulthood. As a result, the applicability of HiTOP to early childhood and adolescence remains largely unexamined, despite these being critical developmental periods marked by substantial neural reorganization, and during which many mental health disorders first emerge [34,35]. A recent exploratory study has begun to address this gap by investigating the validity of the broader HiTOP structure in healthy children and adolescents, as well as in a clinical referral sample. Here, only a small proportion of their sample was younger than 11 years of age, an age-group that is often under-examined in internalizing investigations, and the model was tested for the complete group, without examining model aptitude for subgroups (e.g., sex, age) within the sample. The authors identified four domains that are closely related to HiTOP but did not fully replicate its spectral structure [36]. Consequently, the robustness of detailed symptom-level models within clinical pediatric populations and across different measurement methods remains mostly uncertain.
The present study aims to address current shortcomings and challenges in the field of youth mental health research through the analysis of a large clinical pediatric sample (N = 4,142). With this purpose, the study investigates internalizing symptoms and traits to identify and validate HiTOP’s transdiagnostic structure of internalizing psychopathology, while also evaluating the model’s suitability across the full age range from childhood to late adolescence. Additionally, we investigate the impact of internalizing symptoms on daily functioning, as defined by the WHO, and identify potential predictive risk factors. Overall, this study aims to inform the adaptation of the HiTOP model for younger populations with internalizing mental health problems and establish its viability for future clinical implementation. This may ultimately contribute to more targeted mental health interventions and improved outcomes in younger generations.
2. Methods
Ethics statement
The analyses included data from 4,830 participants from the Healthy Brain Network (HBN) [37]. The HBN is an ongoing large-scale multimodal pediatric data acquisition project conducted by the Child Mind Institute in the New York area (USA). The Healthy Brain Network (HBN) study was approved by the Chesapeake Institutional Review Board (https://www.chesapeakeirb.com/), and the approval explicitly includes permission for secondary data analyses. Data access requires completion of the HBN Data Usage Agreement (DUA), which we obtained prior to conducting the present analyses. Prior to conducting the research, written informed consent was obtained from participants ages 18 or older. For participants younger than 18, written consent was obtained from their legal guardians and written assent obtained from the participant. The current study involved secondary analysis of de-identified data, accessed on February 26, 2024, and did not involve direct interaction with human participants.
Sample
Sample characteristics, such as age, sex, race, number of diagnoses and CBCL T-scores, are displayed in Table 1.
Measures
The 6,126 available behavioral and demographic variables of the HBN were first screened for thematic and analytical aptitude: variables on physical, bodily and cognitive measures, protocol items, age-specific questionnaires, demographic information (including basic demographics and risk factors), and non-internalizing symptoms and traits, according to their source questionnaires, were first excluded. After, variables with more than 30% of missing data, retrospective and self-report variables were discarded, in addition to variables with too little variance (e.g., high single-category endorsement, i.e., variables that had 85% of answers in the same answer category) and that have similar wording (e.g., variables “Cries often” and “Often cries”) to reduce redundancy. Additionally, variables with low sampling adequacy (i.e., their Kaiser-Meyer-Olkin value was < 0.80) [38], and with factor loadings ≤0.20 were removed (see Fig 1). This selection resulted in 78 parent-reported variables with acceptable distributions and missingness (≤22% per variable, mean missingness ≈11.6%) and thematic coherence to the subfactors of the internalizing spectrum as presented in the HiTOP model. The remaining missing data, which was not fully missing at random (i.e., some participants did not receive certain questionnaires if the study clinicians did not deem them necessary) was imputed as described below (see Data Preprocessing). The 78 variables stemmed from six different questionnaires (Child Behavior Checklist, Extended Strengths and Weaknesses Assessment of Normal Behaviors, Mood and Feelings Questionnaire, Screen for Anxiety Related Disorders, Strengths and Difficulties Questionnaire, HBN’s Symptom Checker; see Table 2) were selected for factor analysis (FA), as well as nine additional variables representing possible sociopsychological predictors of internalizing psychopathology for structural equation modeling (SEM). A sum score variable, reflecting daily functioning, was selected as an indicator variable for SEM from the WHO Disability Assessment Schedule (WHODAS), which we refer to as a “quality of life (QoL) indicator” [39]. The variable contents, scale and source questionnaires are listed in Table 2. For further analyses beyond FA and SEM, the youth’s age, sex, race, their pubertal timing, and the diagnostic categories of their DSM-based diagnoses, as well as the Child Behavior Checklist’s (CBCL) sum scores were considered.
All available HBN variables were screened following data- and content-driven approaches. Variables were sequentially excluded based on different criteria, following broader (e.g., “non-internalizing variables”) to more precise (e.g., “KMO ≤ 0.8) exclusion criteria. The final selection included 78 variables, each one assigned to the four HiTOP subfactors, as well as an additional set of predefined demographic and clinical variables utilized in further analyses, but not in the factor analysis. FA = Factor Analysis.
Data preprocessing
All analyses were performed in R (version 4.3.1). First, missing data were imputed using a multivariate imputation algorithm (“mice” package in R). For ordinally scaled variables with few categories (i.e., two to six answer categories), classification and regression trees were used as the imputation method, for variables with many categories (i.e., upwards of 8 answer categories) random forests was utilized, and for variables with two categories, logistic regression [40]. We validated the imputation method through a simulation procedure. For the simulation, we randomly removed a percentage of data from the complete dataset, which corresponded to the mean percentage of missing data per observation in the full dataset (~11.6%). These missing values were imputed with the methods described above, and the distributions between imputed and real data compared. The distributional overlap was satisfactory. The pertinent figures can be consulted in the “Imputation” section of the supplementary materials, along with further details on the procedure (Fig A in S1 Text). This approach ensured that the retained variables were psychometrically adequate and analytically robust. After imputation, the diagnostic category variables were dummy-coded. Followingly, a z-standardization was performed on all numerical variables except age. Then, the data were randomly stratified into two halves while controlling for age and sex, resulting in two comparable datasets for EFA and CFA (Dataset 1 (DS1) = 2,045 participants; Dataset 2 (DS2) = 2,097 participants).
Exploratory factor analysis (EFA)
For the EFA, dataset 1 (DS1) was utilized. The suitability of the data for factor analysis was assessed through the calculation of the Kaiser-Meyer-Olkin (KMO) test [38]. Parallel analysis was applied to assess the number of factors to extract [41]. Oblique rotation was selected as the matrix rotation method, as is apt for correlated factors [42]. The model was calculated using the Mean and Variance Adjusted Weighted Least Squares (WLSMV) estimator. The selected fit indices and values to evaluate acceptable to good model performance were RMSEA < 0.08/ < 0.06, SRMR < 0.08, and CFI and TLI > 0.90/ > 0. 95 [43,44]. All reported fit indices, unless otherwise noted, are scaled to keep them comparable between models. χ2 is reported but not considered as a main model evaluation criterion because of its sensitivity and therefore unreliableness with large sample sizes. Since fit indices alone may be insufficient to appraise model appropriateness in certain cases, the factor loading structure was analyzed for cohesion, as well as consistency with previous theories and findings on HiTOP’s internalizing spectrum [45,46]. The internal consistency of the extracted latent factors was determined with Cronbach’s α [47]. Finally, for model building, only the primary loadings, i.e., the highest variable-to-factor loadings, were considered for factor allocation. Approximately 35% of the variables had a second loading with λ > 0.2, which could have been added as a cross-loading. Nonetheless, to simplify the interpretability of the initial latent factors, and because patterns of co-occurrence could be implemented in later CFA steps, no cross-loadings were added.
Confirmatory factor analysis (CFA)
The goal of the CFA was to verify the factor structure using dataset 2 (DS2) through an iterative process that tested various model variants, and incorporated modification indices. To ensure the aptitude of changes to the CFA model, ANOVAs were performed, and fit indices were evaluated. To further argue for the strong relationships between internalizing psychopathologies, we compared the resulting CFA model with correlated factors to a model with equal structure, but with an added constraint of uncorrelated factors.
Bifactor and hierarchical modeling
The factor model was extended by exploring possible hierarchical and bifactor CFA structures. Additionally, the so-called “bass-ackward” method was implemented, as a growing body of literature argues for its relevance and accuracy in hierarchical analyses [48,49]. This method applies Principal Component Analysis (PCA) to find hierarchical components and removing redundant ones, i.e., components with high correlation (Pearson’s r ≥ 0.9) and congruence (Tucker’s φ ≥ 0.9) across all levels of the hierarchy [36,48]. Considering the results from parallel analysis and the EFA and CFA, we also adopted a four-component solution for the bass-ackward model. Following the recommendations by Forbes, the resulting model was fit as a hierarchical CFA and fit indices to the other models were compared [48]. The bass-ackward method was implemented utilizing the complete sample size to ensure sufficient power for analysis.
Multigroup analysis
To assess the robustness of the factor structure, three separate multigroup analyses (MGA) were conducted, comparing model invariance for male (N = 2,635) and female (N = 1,507) participants, for prepubescent (N = 2,560) and postpubescent (N = 1,582) participants, and for participants with an internalizing diagnosis (N = 844), with an externalizing diagnosis (N = 2,744), and with no given clinical diagnosis (N = 554). Configural, metric and scalar invariance were considered as adequate indicators of measurement equivalence between subgroups [42,50]. Differences in |ΔRMSEA > 0.010| indicated measurement inequivalence [51].
For the age-based MGA, the mean age of puberty onset in the children and adolescents was used, resulting in a split of the sample at 10.84 years of age. This parent-reported puberty onset variable was taken from HBN’s pre-interview questionnaire. For the diagnosis-based comparison, all participants with no diagnosable disorder, or with missing diagnoses due to missing data or incomplete clinical evaluations were grouped together into the “No Clinical Diagnosis” category, as they did not receive a formal clinical diagnosis. The internalizing group included the DSM categories of anxiety disorders, bipolar disorders, depressive disorders, feeding and eating disorders, gender dysphoric disorders, obsessive compulsive disorders, and traumatic stress disorders. In turn, the externalizing group was comprised of disruptive and impulse control disorders, elimination disorders, neurocognitive disorders, neurodevelopmental disorders, personality disorders, schizophrenic disorders and substance addiction disorders. The complete dataset (N = 4,142) was utilized for all MGAs to ensure large-enough subsample sizes. A fourth MGA combining the subsampling of sex and age groups, i.e., creating the subgroups of, e.g., postpubescent males or prepubescent females, was performed and is only reported in the supplementary materials (Table E in S1 Text).
Structural equation modeling (SEM)
The purpose of SEM was expanding the model by assessing the relationship between the latent variables, as well as subsequently testing predictors of the latent variables. SEM was conducted with the full dataset of 4,142 participants. The structural model was built by testing different latent factor regressions, in part guided by existing literature on the causal processes between disorders. Further, the factor covariances from the measurement model (CFA) aided in the identification of potential regression paths to include in the model. This initiated an iterative process of model testing. In each structural model, the factor regressions were inspected. High regression scores, their statistical significance, the model’s fit indices (CFI, TLI, RMSEA and SRMR), as well as ANOVAs between the models, if nested, attested the strength and validity of a relationship between latent factors. This way, relevant latent factor regressions to keep in the model were selected.
Predictor and QoL indicator analysis
Beyond the latent factor relationships, nine sociopsychological predictors were added to the model. Their selection was guided by their availability in the HBN dataset and their relevance as identified in prior literature (e.g., parental behaviors and rearing practices, or peer victimization) [52–54]. The predictors encompassed constructs such as parents having a negative attitude towards their child, parental life dissatisfaction, peer bullying, mother involvement, inconsistent discipline, poor monitoring or corporal punishment by the parents, as well as a child or adolescent’s age and sex. The variable content, scale and source questionnaires are listed in the supplementary materials (Table A in S1 Text). Piece-wise SEM (pwSEM) [55] was executed to analyze the effect of the predictors on the latent factors. Fit measures, effect sizes and the individual predictors’ R2 were compared between the models. In addition to exclusion of badly performing predictors through pwSEM, lasso regression was implemented to identify the best-performing predictors. Variable selection thus combined automated and criteria-based steps: poorly performing predictors were sequentially removed through the iterative SEM process, i.e., based on p < 0.001, β > 0.1, and lack of model fit improvement. Only variables that survived lasso regularization and met the significance and effect size thresholds were retained in the final model.
Finally, the WHO Disability Assessment Schedule’s (WHODAS) sum score, which represents overall daily functioning in children and adolescents, was included as a QoL indicator from the four latent factors. This measure captures domains such as autonomy and self-care (e.g., being able to take care of their own hygiene), mobility (e.g., being able to walk a mile) and socio-emotional health and participation in society (e.g., controlling adverse feelings). After incorporating the sum score in the model, non-significant effects of the latent factors on daily functioning were removed again, retaining only those with a significance level of p < 0.001 and an effect size of β > 0.1.
3. Results
Sample characteristics and aptitude of data
Sample characteristics are shown in Table 1. Age, sex, and race distributions remained similar between the complete sample and the two stratified halves (DS1 and DS2). Approximately 80% of the sample had at least one diagnosis, 18.55% of which were internalizing psychopathologies. No statistically significant differences in age or sex were identified between the full and split datasets. Both halves showed comparable aptitude for factor analysis, with mean KMOs of 0.94.
Exploratory factor analysis
Parallel analysis in DS1 produced a four-factor solution, explaining 47.6% of the total variance (CFI = 0.92, TLI = 0.91, RMSEA = 0.041; Table 3). Alternative factor extraction methods produced differing factor numbers. Only the four-factor solution is reported here, as it proved the most robust and statistically viable in subsequent analyses (i.e., did not have convergence issues). Details on alternative solutions and full loading matrices are provided in the supplementary materials (Table B, C in S1 Text).
The factors were labeled Distress, Nervousness, Social Fears, and OC (Obsessions and Compulsions). Distress comprised 27 variables (Cronbach’s α = 0.91), with strongest loadings for sadness, pessimism, self-hatred, apathy and worthlessness. Nervousness included 23 variables (α = 0.91) such as excessive worrying, fearfulness, generalized anxiety, anxiety attacks and separation fears. Social Fears encompassed 13 variables (α = 0.89), including enjoys social activities, relaxed in social situations, handles anxiety in social situations, preference for solitude and shyness. OC summed 15 variables (α = 0.74) reflecting obsessions, animosity, compulsions, and whining and crying. This model is hereupon referred to as model “EFA”. Fig 2 visualizes the loading structure; full details are provided in the supplementary (Fig B in S1 Text; Table C in S1 Text).
A. Heatmap of the latent factor loadings across all variables for Distress, Nervousness, Social Fears, and OC in descending order of loading size. The highest loading per variable is marked with an asterisk (*). B. Bar plot of the first and second highest loadings per variable, ordered by descending loading size. The primary, i.e., highest loading, is indicated by the color of the bars, the second loading by the dashed, dotted or straight lines, e.g., the primary loading for “Enjoys little” is on Distress, whereas the second-highest loading is on Social Fears. Only second loadings with λ ≥ 0.2 are present. The lower the primary loading, the more second loadings with λ ≥ 0.2. The variable contents can be seen on the y-axis, and additional information consulted in Table A in S1 Text. OC = Obsessions and Compulsions; w/o = without.
Confirmatory factor analysis
The four-factor model was successfully validated in a separate dataset (DS2), with all loadings remaining significant (p-value < 0.001) and acceptable fit indices (model “CFA i”; cf. Table 3). Model fit improved after adding theoretically justified residual covariances (model “CFA ii”), while constraining factor correlations (model “CFA iii”) significantly worsened fit. Followingly, model CFA ii was retained for subsequent analyses. ANOVAs between the CFA models confirmed the choice for this model. All latent factor covariances of model CFA ii had a p-value < 0.001. The highest standardized covariances were φ(Distress, OC) = 0.70, φ(Distress, Nervousness) = 0.67, and φ(Nervousness, OC) = 0.64, whereas the lowest one was φ(Social Fears, OC) = 0.37.
Bifactor and hierarchical modeling
Alternative CFA model structures, such as CFA bifactor, hierarchical and second-order hierarchical models, were computed (Table 4). The hierarchical and second order models showed similar fit to model CFA ii. Although the bifactor model achieved marginally better fit (ΔRMSEA = 0.002 relative to CFA ii), the improvement was not considered substantial. Given the risk of overfitting, the more parsimonious correlated-factor model was preferred. This is further elucidated in the discussion section. A complementarily (extended) bass-ackward analysis largely reproduced the same four-factor structure of model CFA ii with minor variable reassigments (< 5%), but the character of the factors remained unchanged (Table D in S1 Text). This model identified an overarching factor at the top linking Distress and Fear. Fear separated into Nervousness and Social Fears, whereas Distress remained intact, but was related to a fourth factor, OC, at the lowest level of the hierarchy (see Fig 3). However, higher-order models based on this structure failed to meet recommended fit thresholds (RMSEA < 0.05; see Table 4) and were not pursued further.
This structure was modeled implementing the Extended Bass-Ackward Method on the complete dataset. This model includes the thresholds of Pearson correlation r ≥ 0.9 and Tucker’s congruence coefficient φ ≥ 0.9 to ensure the removal of redundant components. The correlations indicate the relatedness of the individual participant’s component scores, whereas the congruence assesses the similarity of the variable loadings for all components. Based on the set correlation and congruence thresholds for the components across levels, the extended Bass-Ackward method recommended the removal of four components, indicated by empty labels in the figure. abs(weight) = absolute regression weights, i.e., loadings across levels.
Multigroup analysis
Measurement invariance across sex, age, and diagnostic groups was tested using multigroup CFA based on model CFA ii. Fit indices were similar across groups (female > male, postpubescent > prepubescent), with ΔCFI and ΔRMSEA values below 0.01, indicating configural, metric and scalar invariance (Table 3). Model fit was descriptively best in participants without clinical diagnoses, which must be interpreted with caution due to sample size variation and method effects. Implications of all MGAs can be found in the Discussion section. Configural, metric and scalar fit of all three MGAs are displayed in Table E in S1 Text.
Structural equation modeling
A structural model including all latent factor regressions was estimated (e.g., Distress being predicted (~) by Nervousness, Social Fears and OC). Due to model complexity and bidirectional effects, the model could not converge. Thus, effects were first examined iteratively and then integrated into a final model (SEM i), which achieved good fit (Tables 3, 5). All regression scores for the tested structural models are listed in supplementary Table F in S1 Text.
Predictor and QoL indicator analysis
Subsequent predictor analyses via piecewise SEM and lasso regression identified key predictors (e.g., sex, parental negative attitude, peer bullying, parental life dissatisfaction, inconsistent discipline; Model “SEM i”). Not all variables remained significant or with β ≥ 0.1 (cf. Table G, H in S1 Text), and as such were removed from the model once again. A retested model (Model “SEM ii”) reached acceptable fit indices, and was supplemented with a QoL indicator from the WHODAS, representing daily functioning. This last model (Model “SEM iii”; Fig 4) established significant influences of Distress, Social Fears and OC, but not Nervousness, unto daily functioning, and abided by Hu and Bentler’s proposed fit index thresholds for acceptable to good model fit. The variance explained by the predictors for Distress, Nervousness, Social Fears and OC were 21%, 13%, 7%, and 20%, respectively.
Model SEM iii with latent factors, regression effects between latent factors in purple, predictors in green and the QoL indicator in blue. All effect sizes are standardized and have a p-value < 0.001. D = Distress; N = Nervousness; OC = Obsessions and Compulsions; SF = Social Fears.
Clinical plausibility
To assess clinical validity, latent factors scores were compared across diagnostic groups, i.e., the type of diagnosis and the diagnostic categories of the participant’s diagnoses were compared with their predicted latent factor scores (Fig 5). As expected, the internalizing group exhibited significantly higher scores for all four factors compared to the two other groups (Fig 5A). Within diagnostic categories, Distress was most elevated in the depressive disorders, Nervousness in anxiety disorders, and OC in obsessive-compulsive-related and depressive disorders. Depressive and anxiety disorders had the highest predicted Social Fears latent factor score. In general, categories of internalizing disorders mapped higher unto the latent factor scores than those of externalizing disorders (Fig 5B).
A Mapping of the latent factors with the type of diagnosis. Internalizing disorders had the highest score on all latent factors, followed by externalizing disorders, and no clinical diagnosis. These differences were all statistically significant. B Mapping of the latent factors with diagnostic categories of DSM. The category of “Others” included diagnostic categories with fewer than 10 observations. Categories included are Bipolar and Related Disorders (n = 7), Feeding and Eating Disorders (n = 3), Gender Dysphoria (n = 2), Neurocognitive Disorders (n = 4), Schizophrenia Spectrum and other Psychotic Disorders (n = 5) and Substance and Addictive Disorders (n = 3). The low sample sizes may lead to the latent factor score of these not to be representative. Yellow = Externalizing disorders; Orange = Internalizing disorders; Blue = No clinical diagnosis.
Adapted CBCL sum scores (excluding overlapping CBCL variable utilized in the factor analysis were correlated with the latent factor scores (Fig 6). Nervousness showed the strongest association with the CBCL Anxiety-Depression scale, whereas Social Fears and Distress demonstrated the strongest relationship with the Withdrawal-Depression scale. No CBCL sum score directly reflects obsessive-compulsive or sexual symptomatology, thus, the OC factor was not strongly associated with any dimensional sum score. Overall, externalizing dimensions (e.g., Attention Problems and Rule-Breaking Behavior) corelated less strongly with the latent factors, except for Aggressive Behavior. Implications of these results are reviewed in the Discussion section.
Mapping of latent factor scores with adapted CBCL sum scores. Here, the CBCL sum score of “Other Problems” refers to symptom and trait domains such as nail biting, being cruel to animals, increased sleep, and enuresis, among others.
4. Discussion
This study aimed to replicate HiTOP’s structure of transdiagnostic internalizing psychopathology for childhood and adolescence. An alternative, four factor structure was discovered, which was invariant to sex and age. The resulting dimensional model was compared with traditional DSM diagnoses and the CBCL diagnostic tool, further supporting the plausibility and coherence of the identified latent factors.
To gain a more comprehensive understanding of internalizing pathologies predictors for and one QoL indicator of the factors were examined. Key predictors included experiences of inconsistent discipline, peer bullying, and parental life dissatisfaction, as well as parents having a dysfunctional or negative attitude towards their children. Importantly, individuals with higher scores on the OC, Social Fears and Distress factors exhibited greater impairments in daily functioning.
Based on the variables associated with the four latent factors, the labels of “Distress”, “Nervousness”, “Social Fears” and “OC” (Obsessions and Compulsions) were assigned. Please note that in the following section, “latent factors” (or “factors”) are to be understood as the four factors resulting from the factor analysis in this work, whereas “subfactors” refer to those ascertained by the HiTOP consortium in their hierarchical model. The Distress and Nervousness latent factors closely resembled HiTOP’s Distress and Fear subfactors, respectively [3]. For example, both the Distress latent factor and subfactor include constructs such as dysphoria, anhedonia, suicidality, and irritability, and both Nervousness and Fear are associated with specific fears, phobias, and anxiety. The overlap between our Distress latent factor and HiTOP’s Distress subfactor was considerable, and we thus opted to adopt the HiTOP label. Contrarily, since HiTOP’s Fear includes a broader range of anxiety-related components than our Nervousness factor, these labels were not aligned. Further, unlike the HiTOP model, where Fear also includes traits of social and performance anxiety, our factor model found these to be distinct from all other anxious and nervousness symptoms, forming a cohesive latent factor labeled Social Fears. Components of obsessions, checking, and rituals are distributed between Distress and Fear in the HiTOP, whereas in our model, they were combined within the OC factor. This factor included further behavioral disinhibitions such as the touching of one’s own genitals, binge-eating, and excessive screaming, whining, and crying. Further nuances emerged with food-related variables: variables related to eating less than usual or weight worries were allocated to Distress, whereas variables on food disinhibitions and eating more or uncontrollably were allocated within the OC factor. This indicates potentially separate domains of eating pathologies that may evolve differently across development.
The content of the four factors also closely aligns with the diagnostic criteria for Major Depressive Disorder, General Anxiety Disorder (GAD), Social Anxiety Disorder (SAD), and Obsessive-Compulsive Disorder (OCD) in the DSM-5. For example, GAD in children and adolescents includes worries about the safety of parents and guardians, and physical manifestations of anxiety such as head or stomach aches; features that are also encompassed within the Nervousness factor. Nonetheless, the latent factors extend beyond the dichotomous, categorical boundaries of the DSM-5 diagnoses. In fact, our analyses revealed covariances between the latent factors, which mirror well-documented patterns of comorbidity in the literature. For example, the strong association between mood and anxiety disorders, as reflected in our Distress and Nervousness factors, is consistent with findings of high pediatric comorbidity [56]. The progression from depressive to anxiety disorders, and vice versa, has been observed [57]. Similarly, anxiety disorders are often highly comorbid with each other: In a sample of Norwegian youth with SAD, 58.5% were found to meet the criteria for GAD, and 25.7% that for specific phobias [58]. These results provide an explanatory approach for the strong association between Nervousness and Social Fears. In addition, both depressive and anxiety disorders have been linked to OCD [59–61]. According to the DSM-5 diagnostic criteria obsessive-compulsive symptoms often “cause marked anxiety or distress”, thus explaining the distress-related symptoms (e.g., crying, whining and screaming) identified within the OC factor. This connection underscores the overlap between the OC factor and the broader Distress and Nervousness factors, further highlighting the interconnected nature of these psychopathologies8. Tentatively, the addition of cross-loadings would have further modeled these patterns of comorbidity, albeit we opted to exclude them from our model to facilitate the interpretation of the latent factors, and because comorbidity was further integrated into the model through factor covariances and regressions.
In sum, our factors showed strong symptomatic coherence, high internal consistency, and many parallels to previous findings on co-occurrence and comorbidity. One further strength of such a model is the ability to characterize each individual on a dimensional level for each of the latent factors. This specifies a child or adolescent’s symptom and trait manifestation within the internalizing domain, highlighting areas of gravity, as well as potential treatment targets. The model achieved satisfactory fit, and moreover, was found to be very robust, as its structure was confirmed through the analysis of a separate dataset with CFA. Despite the model’s robustness, we deemed it important to inspect its validity for different subgroups within the sample – a valuable analytical step mostly overlooked in past literature. Various multigroup analyses showed invariance to sex and age, providing evidence that there were no sample effects, i.e., our sample including more male and prepubescent participants did not skew the factor model. The model also seems invariant to type of diagnosis, with it even being a good fit for participants whose primary diagnosis is externalizing. One possible interpretation is that the DSM diagnostic categories do not adequately capture the symptomatology and thus purvey a diagnosis whose diagnostic criteria are not fully or correctly met. Alternatively, it could be that too many children and adolescents receive externalizing diagnoses because those symptoms are more salient to parents and peers, although they also present clinically relevant internalizing symptoms and traits. This highlights the issue of the high rate of comorbidity across internalizing and externalizing disorders once again. Nonetheless, the model provided evidence of invariance and thus seems broadly generalizable to the subgroups that were inspected.
Generalizability is only one aspect of clinical plausibility. To further support the model’s plausibility, we mapped the DSM’s diagnostic categories onto the latent factor scores, revealing a strong overall alignment between the categories and the factors. For example, depressive disorders, anxiety disorders and obsessive-compulsive disorders mapped the highest on Distress, Nervousness and OC, respectively. The diagnostic categories of externalizing psychopathologies generally mapped the lowest on all four latent factors. Additionally, individuals with an internalizing pathology had consistently stronger scores on all four factors than those with an externalizing diagnosis and those with no clinical diagnosis at all. Taken together, these results highlight a high consistency between the proposed latent factors and existing clinical tools, further substantiating the model’s clinical plausibility.
Our findings have practical implications for screening and intervention. The identified dimensional structure can inform the development of new screening tools that capture broad internalizing dimensions. Such tools could serve both clinical and non-clinical settings: For example, in schools they could facilitate early identification of individuals at elevated risk across domains. In clinical settings, dimensional scores could complement categorical classifications by providing a profile of severity and comorbity. Further, in intervention planning, a dimensional model could help tailor treatments to shared underlying processes, as well as to the individual. In this way, our study provides an empirical foundation for the eventual development of age-appropriate screening and diagnostic tools with clear translational value.
Finally, the exploration of possible correlates of the four latent factors yielded several notable findings. Among the predictors, only a few of the selected variables survived all levels of analysis and most of their relevance concentrated on Distress. This observation raises the possibility that Distress mediates the influence of these predictors on other latent factors, given its substantial covariances with and regression effects on Nervousness and OC. The retained predictors in the model align with previous research. For example, peer bullying (or “victimization”) has consistently been linked to depression and to internalizing psychopathology altogether [62]. Similarly, parental discipline (or “dysfunctional parenting”) has shown strong associations with both depression and OCD [63–65]. Furthermore, impairments in various aspects of daily functioning have been reported in children with depressive, social anxiety and obsessive-compulsive symptoms, agreeing with our findings of the impact of Distress, Social Fears and OC on daily functioning [66–69].
Limitations and future research
As with any factor analysis, the results of this work are influenced by the included variables, i.e., their content and informants. Regarding informants, the present study included variables based on parental reports, which might introduce a systematic bias [70,71]. Externalizing symptoms are often more salient and therefore more readily reported by parents, whereas internalizing symptoms typically require greater sensitivity and may be underestimated. The under-reporting of internalizing symptoms should be considered when interpreting our findings. Nonetheless, the imbalance in parents-reports between externalizing and internalizing domains is less of a problem here, since we only included internalizing symptoms. Considering variable content, studies examining the internalizing spectrum in adults have identified varying factorial structures depending on the included constructs (for an overview, see [30]). Our selection of pertinent variables to include in the factor analysis followed a data-driven and a content-driven approach. The former is purely objective, since it removes variables statistically unsuitable for analysis, e.g., the child-reported variables had mostly >80% missing data and were thus deemed non-imputable. The latter subsumes more subjective decisions, as the exclusion or inclusion of a variable was ultimately (hu)man-made but we tried to make this process as literature driven as possible: We followed the indications of the literature on each variable’s questionnaire concerning the constructs that the variable is intended to measure, and matched these with the constructs included in HiTOP’s internalizing spectrum. For example, if a questionnaire’s documentation explicitly stated the intent of measuring externalizing symptoms, we discarded it.
Adding to the limitation specified at the very beginning of this section, due to variable selection and availability, our model does not entail all domains relevant to internalizing psychopathology. Specifically, trauma and stressor-related disorders were excluded due to insufficient variable representation or excessive missing data within our sample. On top of that, alternative model conceptualizations to ours may provide an equally valid or even better representation of pediatric internalizing psychopathology. To address this, we explored three alternative factor model structures commonly proposed in the literature: the bifactor and hierarchical models, as well as a modified hierarchical model with Distress positioned hierarchically over OC. Beyond factor analysis, we additionally implemented the extended bass-ackward hierarchical approach. While the alternative CFA models demonstrated adequate fit, it is difficult to determine whether they are as equally suitable as our selected model. For example, the additional variance captured by a bifactor may not be theoretically meaningful (i.e., is residual covariance) and “only a statistical summary of the covariance among psychopathological symptoms” [72]. In other words, the general factor in bifactor models has a propensity to reflect residual covariance among items, that albeit statistically strong is theoretically ambiguous. As a result, it may represent a statistical artifact rather than a meaningful dimension. Although model fit was an important consideration for our model selection, our evaluation also took theoretical interpretability and empirical support into account, since solely relying on model fit has been deemed inadequate for highly complex models in the past [73,74]. Ultimately, our model constitutes a robust solution congruent with literature and is in line with the principle of parsimony. Additionally, the results from the bass-ackward analysis largely confirmed our variable-to-factor structure, presenting an internalizing psychopathological structure of Distress, Nervousness, Social Fears and OC, as well. The modeled hierarchical structure was cohesive and congruent with previous literature, e.g., the internalizing dimension being largely characterized by Distress and Fear is consistent with findings on the internalizing spectrum [75,76]. This approach also reflected our expectation that OC might partially load onto the Distress factor. Despite theoretical coherence, the hierarchical models derived from the bass-ackward analysis did not yield satisfactory fit indices, limiting the strength and interpretability of conclusions regarding the higher-order structure. In addition, computational limitations in subsequent MGAs and SEM prevented us from implementing the full three-tier hierarchical model. Nonetheless, we considered it essential to include an alternative, well-established hierarchical modeling approach in our analyses. Replicating findings across different analytical frameworks fosters comparability with previous work and highlights the importance of testing dimensional structures across complementary analytic strategies. Future research should continue to investigate the hierarchical organization of internalizing symptoms in children and adolescents to clarify the developmental validity of dimensional models like HiTOP.
The results of the multigroup analyses provided a basis to assume model invariance to the subgroups we tested for. Nonetheless, there were considerable differences in sample size between the different subgroups, which may lead to somewhat unstable fit indices as well as model non-convergence. To remediate this issue, we performed sample-size matched MGAs, which delivered similar results to the standard ones. We thus cautiously concluded that our results seem plausible, and our model is invariant, but further research must confirm them. For the diagnostic group comparison, the superior fit of the group with no clinical diagnosis may have arisen from the inclusion of the categories of participant’s that had no diagnosis due to an incomplete evaluation or due to missing data, which, in reality, could possibly receive a diagnosis if the assessments were to be completed. Nonetheless, it could also be a methods effect due to the group’s small sample size.
A further limitation is the labeling of our factors. We adopted HiTOP’s Distress label for one of our latent factors due to high overlap of its associated psychopathological constructs. Our other factors did not directly align with HiTOP’s, as the model does, for example, not include a factor explicitly covering obsessions, compulsions, behavioral disinhibitions and sexual fixations such as our OC factor. To this end, we adopted our own labels for three of the latent factors, suggesting that the structure of psychopathology in childhood may differ to that in adulthood.
Finally, one must consider the chosen predictors. We found parental life dissatisfaction and peer bullying, among other predictors, to be greatly associated with internalizing psychopathologies. Yet, many more sociopsychological variables not considered in this study due to lack of data availability surely influence internalizing psychopathology and should be examined in future longitudinal investigations. Nonetheless, the effects found show great potential for therapeutic targets and interventions, as well as early identification of children and adolescents at risk of developing internalizing psychopathologies.
Further recommendations for the future are the analysis of HiTOP model for other age groups, such as for elders, as well as for the whole dimensional psychopathological spectrum, instead of only focusing on the internalizing domain. This way, one could effectively test the hierarchical and transdiagnostic dimensional structure at different age ranges. In addition, variables from dimensional behavioral questionnaires may be more apt for this kind of investigation. The inclusion of non-behavioral variables, such as neuroanatomical or functional data, could reinforce the veracity of the model. For this, big data approaches should continue to be implemented to ensure well-powered analyses.
5. Conclusion
The findings of this study argue for an alternative understanding of dimensional internalizing psychopathologies in children and adolescents by establishing a robust four-factor structure. These factors not only align with DSM diagnostic categories, but also offer deeper insights into symptom co-occurrence patterns, exemplified by the distinct composition of the OC factor. Importantly, the model showed invariance across sex and age, suggesting its generalizability. Behavioral correlates, such as inconsistent discipline and peer bullying, were identified as predictors, particularly within the Distress factor, reinforcing findings from prior research and identifying actionable targets for intervention. The findings underscore the clinical plausibility of the proposed model. Taken together, the findings extend the understanding of pediatric psychopathology while addressing the complexity of the internalizing symptom and trait domain and add nuance to the dimensional conceptualization of psychopathology at the symptom and factor level. This work contributes to the growing body of literature advocating for transdiagnostic and dimensional approaches to psychopathology, specifically focusing on childhood and adolescence, which will hopefully lead to improved interventions and mental health outcomes in youth.
Supporting information
S1 Text.
Table A. Study variables. Overview of variables, scale, and source questionnaires for all variables utilized in FA, MGA, and SEM. Table B. Fit Indices of Three, Four and Eight Factor Solutions. Fit indices of models with an alternative number of factors, i.e., the 3- and 8-factor solutions, are presented here. Table C. Factor Loading Structure of the Four Factor Model. Raw factor loadings of all variables to the factors in the four factor solution of the EFA, in addition to all variable communalities. Table D. Variables with Different Factor Allocation in CFA and Bass-Ackward Analysis. Differences between the CFA and Bass-Ackward method in regards to variable-to-factor allocation. Table E. Additional Fit Indices for MGAs. Reporting of additional fit indices of the MGAs reported in the main manuscript, i.e., strict invariance indices, and fit indices of an additional MGA looking at sex and age combined subgroups (e.g., “male-prepubescent” or “female-postpubescent”). Table F. Regression Effects in SEM. Standardized regression scores between latent factors for all structural models in SEM. Table G. Standardized Regression Coefficients in Piece-Wise SEM of Predictor (Indicator) Analysis. Standardized regression coefficients of the predictor selection process utilizing piece-wise SEM. Table H. Lasso Regression of Predictor (Indicator) Analysis. Standardized regression coefficients of the predictor selection process utilizing lasso regression. Fig A. Imputation of Simulated and Real Missing Values. Density and distribution curves of imputed simulated missing values, imputed real missing values and datasets with missing values and imputed values. Fig B. Path Diagram of the Four Latent Factor Solution. Path diagram with variable-to-factor overview corresponding to the factor loading structure of the four factor solution.
https://doi.org/10.1371/journal.pmen.0000509.s001
(DOCX)
S1 Checklist. Questionnaire references (Table 2).
https://doi.org/10.1371/journal.pmen.0000509.s002
(DOCX)
References
- 1. Arias D, Saxena S, Verguet S. Quantifying the global burden of mental disorders and their economic value. EClinicalMedicine. 2022;54:101675. pmid:36193171
- 2. Wu Y, Wang L, Tao M, Cao H, Yuan H, Ye M, et al. Changing trends in the global burden of mental disorders from 1990 to 2019 and predicted levels in 25 years. Epidemiol Psychiatr Sci. 2023;32:e63. pmid:37933540
- 3. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM, et al. The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. J Abnorm Psychol. 2017;126(4):454–77. pmid:28333488
- 4. Markon KE, Chmielewski M, Miller CJ. The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychol Bull. 2011;137(5):856–79. pmid:21574681
- 5. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed.; 2013.
- 6.
World Health Organization. International Statistical Classification of Diseases and Related Health Problems. 11th ed.; 2019. Available from: https://icd.who.int/
- 7. First MB, Westen D. Classification for clinical practice: how to make ICD and DSM better able to serve clinicians. Int Rev Psychiatry. 2007;19(5):473–81. pmid:17896228
- 8. Reed GM, First MB, Kogan CS, Hyman SE, Gureje O, Gaebel W, et al. Innovations and changes in the ICD-11 classification of mental, behavioural and neurodevelopmental disorders. World Psychiatry. 2019;18(1):3–19. pmid:30600616
- 9. Caron C, Rutter M. Comorbidity in child psychopathology: concepts, issues and research strategies. J Child Psychol Psychiatry. 1991;32(7):1063–80. pmid:1787137
- 10. Clark LA, Cuthbert B, Lewis-Fernández R, Narrow WE, Reed GM. Three Approaches to Understanding and Classifying Mental Disorder: ICD-11, DSM-5, and the National Institute of Mental Health’s Research Domain Criteria (RDoC). Psychol Sci Public Interest. 2017;18(2):72–145. pmid:29211974
- 11. Fried EI, Nesse RM. Depression is not a consistent syndrome: An investigation of unique symptom patterns in the STAR*D study. J Affect Disord. 2015;172:96–102. pmid:25451401
- 12. Newson JJ, Pastukh V, Thiagarajan TC. Poor Separation of Clinical Symptom Profiles by DSM-5 Disorder Criteria. Front Psychiatry. 2021;12:775762. pmid:34916976
- 13. Olbert CM, Gala GJ, Tupler LA. Quantifying heterogeneity attributable to polythetic diagnostic criteria: theoretical framework and empirical application. J Abnorm Psychol. 2014;123(2):452–62. pmid:24886017
- 14. Aboraya A. The Reliability of Psychiatric Diagnoses: Point-Our psychiatric Diagnoses are Still Unreliable. Psychiatry (Edgmont). 2007;4(1):22–5. pmid:20805925
- 15. Regier DA, Narrow WE, Clarke DE, Kraemer HC, Kuramoto SJ, Kuhl EA, et al. DSM-5 field trials in the United States and Canada, Part II: test-retest reliability of selected categorical diagnoses. Am J Psychiatry. 2013;170(1):59–70. pmid:23111466
- 16. Krueger RF, Bezdjian S. Enhancing research and treatment of mental disorders with dimensional concepts: toward DSM-V and ICD-11. World Psychiatry. 2009;8(1):3–6. pmid:19293948
- 17. Okasha A. Would the use of dimensions instead of categories remove problems related to subthreshold disorders?. Eur Arch Psychiatry Clin Neurosci. 2009;259 Suppl 2:S129-33. pmid:19876670
- 18. Smoller JW, Andreassen OA, Edenberg HJ, Faraone SV, Glatt SJ, Kendler KS. Psychiatric genetics and the structure of psychopathology. Mol Psychiatry. 2019;24(3):409–20. pmid:29317742
- 19. Brainstorm Consortium, Anttila V, Bulik-Sullivan B, Finucane HK, Walters RK, Bras J, et al. Analysis of shared heritability in common disorders of the brain. Science. 2018;360(6395):eaap8757. pmid:29930110
- 20. Xia CH, Ma Z, Ciric R, Gu S, Betzel RF, Kaczkurkin AN, et al. Linked dimensions of psychopathology and connectivity in functional brain networks. Nat Commun. 2018;9(1):3003. pmid:30068943
- 21. Scott J, Leboyer M, Hickie I, Berk M, Kapczinski F, Frank E, et al. Clinical staging in psychiatry: a cross-cutting model of diagnosis with heuristic and practical value. Br J Psychiatry. 2013;202(4):243–5. pmid:23549937
- 22. Borsboom D, Cramer AOJ. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol. 2013;9:91–121. pmid:23537483
- 23. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–51. pmid:20595427
- 24. Allegrini AG, Cheesman R, Rimfeld K, Selzam S, Pingault J-B, Eley TC, et al. The p factor: genetic analyses support a general dimension of psychopathology in childhood and adolescence. J Child Psychol Psychiatry. 2020;61(1):30–9. pmid:31541466
- 25. Caspi A, Houts RM, Fisher HL, Danese A, Moffitt TE. The general factor of psychopathology (p): Choosing among competing models and interpreting p. Clin Psychol Sci. 2024;12(1):53–82. pmid:38236494
- 26. van Bork R, Epskamp S, Rhemtulla M, Borsboom D, van der Maas HLJ. What is the p-factor of psychopathology? Some risks of general factor modeling. Theory & Psychology. 2017;27(6):759–73.
- 27. Haslam N, McGrath MJ, Viechtbauer W, Kuppens P. Dimensions over categories: a meta-analysis of taxometric research. Psychol Med. 2020;50(9):1418–32. pmid:32493520
- 28. Krueger RF, Kotov R, Watson D, Forbes MK, Eaton NR, Ruggero CJ, et al. Progress in achieving quantitative classification of psychopathology. World Psychiatry. 2018;17(3):282–93. pmid:30229571
- 29. Achenbach TM. The classification of children’s psychiatric symptoms: a factor-analytic study. Psychol Monogr. 1966;80(7):1–37. pmid:5968338
- 30. Watson D, Levin-Aspenson HF, Waszczuk MA, Conway CC, Dalgleish T, Dretsch MN, et al. Validity and utility of Hierarchical Taxonomy of Psychopathology (HiTOP): III. Emotional dysfunction superspectrum. World Psychiatry. 2022;21(1):26–54. pmid:35015357
- 31. Askari MS, Rutherford CG, Mauro PM, Kreski NT, Keyes KM. Structure and trends of externalizing and internalizing psychiatric symptoms and gender differences among adolescents in the US from 1991 to 2018. Soc Psychiatry Psychiatr Epidemiol. 2022;57(4):737–48. pmid:34773140
- 32. Tanner J, Zeffiro T, Wyss D, Perron N, Rufer M, Mueller-Pfeiffer C. Psychiatric Symptom Profiles Predict Functional Impairment. Front Psychiatry. 2019;10:37. pmid:30853916
- 33. Vedechkina M, Bennett M, Holmes J. Dimensions of internalizing symptoms are stable across early adolescence and predicted by executive functions: Longitudinal findings from the Adolescent Brain and Cognitive Development (ABCD) study. Dev Psychopathol. 2023;:1–10. pmid:37272416
- 34. Galván A. Adolescence, brain maturation and mental health. Nat Neurosci. 2017;20(4):503–4. pmid:28352110
- 35. Vasileva M, Graf RK, Reinelt T, Petermann U, Petermann F. Research review: A meta-analysis of the international prevalence and comorbidity of mental disorders in children between 1 and 7 years. J Child Psychol Psychiatry. 2021;62(4):372–81. pmid:32433792
- 36. Forbes MK, Watts AL, Twose M, Barrett A, Hudson JL, Lyneham HJ, et al. A Hierarchical Model of the Symptom-Level Structure of Psychopathology in Youth. Clin Psychol Sci. 2025;13(2):207–21. pmid:40330987
- 37. Alexander LM, Escalera J, Ai L, Andreotti C, Febre K, Mangone A, et al. An open resource for transdiagnostic research in pediatric mental health and learning disorders. Sci Data. 2017;4:170181. pmid:29257126
- 38. Kaiser HF. An Index of Factorial Simplicity. Psychometrika. 1974;39(1):31–6.
- 39. Ustün TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, et al. Developing the World Health Organization Disability Assessment Schedule 2.0. Bull World Health Organ. 2010;88(11):815–23. pmid:21076562
- 40. Buuren S van, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations inR. J Stat Soft. 2011;45(3).
- 41. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179–85. pmid:14306381
- 42.
Brown TA. Confirmatory factor analysis for applied research. 2nd ed. New York: Guilford Press; 2015.
- 43. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55.
- 44. Marsh HW, Hau K-T, Wen Z. In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler’s (1999) Findings. Structural Equation Modeling: A Multidisciplinary Journal. 2004;11(3):320–41.
- 45. Forbes MK, Greene AL, Levin-Aspenson HF, Watts AL, Hallquist M, Lahey BB, et al. Three recommendations based on a comparison of the reliability and validity of the predominant models used in research on the empirical structure of psychopathology. J Abnorm Psychol. 2021;130(3):297–317. pmid:33539117
- 46. Greene AL, Eaton NR, Li K, Forbes MK, Krueger RF, Markon KE, et al. Are fit indices used to test psychopathology structure biased? A simulation study. J Abnorm Psychol. 2019;128(7):740–64. pmid:31318246
- 47. Cronbach LJ. Coefficient Alpha and the Internal Structure of Tests. Psychometrika. 1951;16(3):297–334.
- 48. Forbes MK. Improving hierarchical models of individual differences: An extension of Goldberg’s bass-ackward method. Psychol Methods. 2024;29(6):1062–73. pmid:36780281
- 49. Goldberg LR. Doing it all Bass-Ackwards: The development of hierarchical factor structures from the top down. Journal of Research in Personality. 2006;40(4):347–58.
- 50. Putnick DL, Bornstein MH. Measurement Invariance Conventions and Reporting: The State of the Art and Future Directions for Psychological Research. Dev Rev. 2016;41:71–90. pmid:27942093
- 51. Chen FF. Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal. 2007;14(3):464–504.
- 52. Hazzard VM, Mason TB, Smith KE, Schaefer LM, Anderson LM, Dodd DR, et al. Identifying transdiagnostically relevant risk and protective factors for internalizing psychopathology: An umbrella review of longitudinal meta-analyses. J Psychiatr Res. 2023;158:231–44. pmid:36603318
- 53. Lin J, Guo W. The Research on Risk Factors for Adolescents’ Mental Health. Behav Sci (Basel). 2024;14(4):263. pmid:38667059
- 54. Spiroiu F. A Critical Evaluation of Risk and Protective Factors Influencing the Development and Maintenance of Internalizing Disorders in Adolescents. Int J Psychiatr Res. 2018;1(1):1–15.
- 55. Lefcheck JS. piecewiseSEM: Piecewise structural equation modeling in R for ecology, evolution, and systematics. Methods Ecol Evol. 2016;7(5):573–9.
- 56. Cummings CM, Caporino NE, Kendall PC. Comorbidity of anxiety and depression in children and adolescents: 20 years after. Psychol Bull. 2014;140(3):816–45. pmid:24219155
- 57. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry. 2003;60(8):837–44. pmid:12912767
- 58. Jystad I, Bjerkeset O, Haugan T, Sund ER, Vaag J. Sociodemographic Correlates and Mental Health Comorbidities in Adolescents With Social Anxiety: The Young-HUNT3 Study, Norway. Front Psychol. 2021;12:663161. pmid:33935922
- 59. Langley AK, Lewin AB, Bergman RL, Lee JC, Piacentini J. Correlates of comorbid anxiety and externalizing disorders in childhood obsessive compulsive disorder. Eur Child Adolesc Psychiatry. 2010;19(8):637–45. pmid:20349255
- 60. Osland S, Arnold PD, Pringsheim T. The prevalence of diagnosed obsessive compulsive disorder and associated comorbidities: A population-based Canadian study. Psychiatry Res. 2018;268:137–42. pmid:30025284
- 61. Peris TS, Bergman RL, Asarnow JR, Langley A, McCracken JT, Piacentini J. Clinical and cognitive correlates of depressive symptoms among youth with obsessive compulsive disorder. J Clin Child Adolesc Psychol. 2010;39(5):616–26. pmid:20706915
- 62. Forbes MK, Magson NR, Rapee RM. Evidence that Different Types of Peer Victimization have Equivalent Associations with Transdiagnostic Psychopathology in Adolescence. J Youth Adolesc. 2020;49(3):590–604. pmid:32026235
- 63. Alloy LB, Abramson LY, Smith JM, Gibb BE, Neeren AM. Role of parenting and maltreatment histories in unipolar and bipolar mood disorders: mediation by cognitive vulnerability to depression. Clin Child Fam Psychol Rev. 2006;9(1):23–64. pmid:16718583
- 64. Lennertz L, Grabe HJ, Ruhrmann S, Rampacher F, Vogeley A, Schulze-Rauschenbach S, et al. Perceived parental rearing in subjects with obsessive-compulsive disorder and their siblings. Acta Psychiatr Scand. 2010;121(4):280–8. pmid:19694627
- 65. Yap MBH, Jorm AF. Parental factors associated with childhood anxiety, depression, and internalizing problems: a systematic review and meta-analysis. J Affect Disord. 2015;175:424–40. pmid:25679197
- 66. Dickson SJ, Oar EL, Kangas M, Johnco CJ, Lavell CH, Seaton AH, et al. A Systematic Review and Meta-Analysis of Impairment and Quality of Life in Children and Adolescents with Anxiety Disorders. Clin Child Fam Psychol Rev. 2024;27(2):342–56. pmid:38782783
- 67. du Plessis LJ, Lochner C, Louw D, Hendricks G, Bowles S, Fischer M, et al. A comprehensive view of functional impairment in children and adolescents with obsessive-compulsive disorder adds value. Early Interv Psychiatry. 2022;16(9):994–1001. pmid:34743402
- 68. Hoff AL, Kendall PC, Langley A, Ginsburg G, Keeton C, Compton S, et al. Developmental Differences in Functioning in Youth With Social Phobia. J Clin Child Adolesc Psychol. 2017;46(5):686–94. pmid:26630122
- 69. Usami M, Iwadare Y, Watanabe K, Ushijima H, Kodaira M, Okada T, et al. A case-control study of the difficulties in daily functioning experienced by children with depressive disorder. J Affect Disord. 2015;179:167–74. pmid:25863914
- 70. Collishaw S, Goodman R, Ford T, Rabe-Hesketh S, Pickles A. How far are associations between child, family and community factors and child psychopathology informant-specific and informant-general?. J Child Psychol Psychiatry. 2009;50(5):571–80. pmid:19207620
- 71. Robinson M, Doherty DA, Cannon J, Hickey M, Rosenthal SL, Marino JL, et al. Comparing adolescent and parent reports of externalizing problems: A longitudinal population-based study. Br J Dev Psychol. 2019;37(2):247–68. pmid:30394545
- 72. Murray AL, Eisner M, Ribeaud D. The Development of the General Factor of Psychopathology “p Factor” Through Childhood and Adolescence. J Abnorm Child Psychol. 2016;44(8):1573–86. pmid:26846993
- 73. Falk CF, Muthukrishna M. Parsimony in model selection: Tools for assessing fit propensity. Psychol Methods. 2023;28(1):123–36. pmid:34647757
- 74. Marsh HW, Balla J. Goodness of fit in confirmatory factor analysis: The effects of sample size and model parsimony. Qual Quant. 1994;28(2):185–217.
- 75. Dornbach-Bender A, Ruggero CJ, Waszczuk MA, Gamez W, Watson D, Kotov R. Mapping emotional disorders at the finest level: Convergent validity and joint structure based on alternative measures. Compr Psychiatry. 2017;79:31–9. pmid:28754505
- 76. Forbes MK, Sunderland M, Rapee RM, Batterham PJ, Calear AL, Carragher N, et al. A detailed hierarchical model of psychopathology: From individual symptoms up to the general factor of psychopathology. Clin Psychol Sci. 2021;9(2):139–68. pmid:33758691