The construct of self-regulation

SR refers to the ability to actively guide oneself toward a desired emotional, behavioral, or cognitive state (e.g., [4]). It involves adapting internal states (i.e., emotions and cognitions) and behaviors to align with contextual demands [5]. Context, in this regard, encompasses situational factors, social expectations, and personal goals. SR is a complex, multidimensional construct that depends on dynamic interactions between traits (e.g., temperament, impulsivity) and states (e.g., hormonal and physiological conditions) and involves various subskills, such as delay of gratification, executive functions (EFs) and self-control (see [5] for a glossary of related terminology). Although SR and self-control are sometimes used interchangeably, they are more accurately understood as distinct but related concepts. Self-control specifically addresses behavioral regulation in the face of conflicts between competing goals, such as resisting short-term rewards to prioritize long-term outcomes. In contrast, SR encompasses a broader set of processes, including planning, goal-setting, monitoring, in addition to the execution of behavior [4,6,7]. Unlike self-control, SR is not necessarily dependent on the presence of a conflict between competing goals [4]. Given this complexity, SR has been challenging to define consistently, often falling prey to jingle-jangle fallacies, where similar terms are used interchangeably or distinct concepts are conflated (e.g., [8]). Despite definitional challenges, SR has drawn significant interest across biological, differential, developmental, and clinical psychology due to its malleability [9] and its predictive power for life outcomes such as academic achievement, social relationship quality, and substance-use behavior [10,11].

SR is commonly conceptualized through the dual-process theory [4,12,13], in particular, when considering its neural underpinnings. According to this theory, successful SR is achieved by balancing two domain-specific systems. The first one reflects a cognitive, rational, top-down, or so-called cool component that is mostly related to control processes focused on long-term goals. EFs form an integral part of this cool components, as they enable the cognitive control necessary for effective SR. However, EFs are not synonymous with SR; while critical, they are not sufficient for SR and can also support other psychological goals [5,14]. The factor structure of EF, detailed in [15], comprises three core components: cognitive flexibility or set shifting, which involves the ability to switch between tasks, mental sets or operations; working memory, which allows active updating, operating, and monitoring of mental representations; and inhibitory control, the deliberate suppression of prepotent responses. The second component of SR encompasses emotional, impulsive, bottom-up, or hot processes often oriented towards short-term rewards. Effective SR depends on the ability to appropriately balance and adapt these two components to the demands of the context.

The dual-process model of SR has been widely applied in studies investigating brain-behavior associations. The cool component has been related with prefrontal cortex activation (e.g., [16]), and the hot component was additionally found to be associated with activation in subcortical limbic structures, such as the ventral striatum and the amygdala (e.g., [17,18]). The strength of fronto-striatal connections between these regions has been linked to SR in older adults [19] and (pre-) adolescents [20–22]. Furthermore, an imbalanced trajectory describing the development of connections between striatal and prefrontal brain regions, resulting in increased reward responsivity and comparatively weaker control processes, has been suggested as the root of dysregulated behavior in adolescence [23]. This underlines the considerable importance of studying a dual-process model of SR in the field of neurobehavioral research as a prerequisite for future brain-behavior research in such large-scale datasets as the ABCD.

The measurement of self-regulation

SR encompasses a multitude of subskills that collectively define the ability to regulate oneself. Efforts to establish a comprehensive nomological network model for SR have provided valuable insights (e.g., [24,25]), but the complexity of this task remains significant. Research on the convergent validity of SR measures reveals weak correlations between different content- and methodological-domains [24,26].

SR subskills are typically assessed using different methods: some are measured with questionnaires [27,28], others are assessed by means of behavioral tasks [29] or by combining both methods [24]. These approaches are complementary, and an exhaustive model should incorporate both. Heterogeneous indicators of SR from various content domains – which are often assessed using different measurement approaches – enhance content validity. However, the consistently low correlations between survey and behavioral SR indicators [24] point to issues with convergent validity.

Each method has distinct advantages and limitations. While survey-based indicators of SR typically demonstrate high reliability and are, therefore, recommended for the study of individual differences in association with outcomes relevant for daily life [30], behavioral tasks often exhibit low reliability [30,31]. However, behavioral measures of SR are less influenced by introspection ability or socially desirable responding. They also typically assess more isolated constructs (e.g., response inhibition), which often correspond to lower variability in brain correlates. Therefore, behavioral tasks may be more appropriate for studying brain-behavior relationships [32], which is one of the main objectives of the ABCD dataset. In light of these considerations, combining task-based and survey-based SR measures from different content domains poses both opportunities and risks. To address these, we used SR measures from the ABCD study to model their covariance structure using various latent variable models, comparing several defensible approaches to account for the advantages and limitations of each method.

The measurement of self-regulation in the ABCD study

Conducting a large-scale study to apply extensive test batteries including behavioral and survey indicators of SR along with neuroimaging data acquisition is highly resource-intensive. The trade-off between resource-investment related to extensive test batteries on the one hand, and content validity of the measured constructs on the other hand, is increasingly balanced by open large-scale neuroimaging initiatives as those mentioned above. To take full advantage of these datasets, psychometric models based on the measures used in these studies need to be established. The ABCD study includes neural data and a > 8,5 h test battery of > 11,000 nine- to ten-year-old pre-adolescents at a baseline assessment. With its variety of brain measures, bio-specimens, and psychological data, the ABCD dataset offers ideal conditions for a thorough investigation of complex constructs such as SR, as well as biological and environmental determinants of individual differences in SR. Establishing psychometric models for the constructs covered in the ABCD study will ideally encourage researchers to apply consistent models across multiple studies, thus promoting a cumulative understanding of SR in adolescence.

We evaluated all available instruments of the ABCD study related to mental health and neurocognition based on their theoretical overlap with the construct of SR or its subconstructs. Potentially related constructs were identified based on [5] and [33], which include a glossary of major terms and mental processes related to SR and an overview of task and survey-based indicators of SR, respectively. We identified three questionnaires measuring hot and cool facets of self-reported impulsivity and inhibition/reward seeking, as well as caregiver-reported attention problems, and three EF tasks as potential indicators of SR.

The present study

The research question guiding this modeling endeavor asks whether SR (in the ABCD dataset) can be represented by two distinct domain-specific systems, i.e., one rational or cool component, and one emotionally-driven or hot component, as postulated by the dual-process theory. We conceptualized and tested several defensible measurement models of SR in the ABCD study within this theoretical framework. By doing so, we aim to facilitate cumulative knowledge-gain on SR based on future studies involving the ABCD data. We want to highlight that our goal was neither to test the dual-process theory of SR, nor to find the best possible model of SR or compare different theories. Rather, our aim was to test how well SR, as conceptualized by the dual-process theory, is measurable by the available indicators in the rich and influential ABCD dataset. At the same time, we aim to foster multivariate modeling in personality research by establishing constructs with heterogeneous observables and high factorial and predictive validity. Thus, we modeled SR in the ABCD dataset based on several indicators from different assessment modalities (i.e., self-reports, caregiver-report questionnaires, and behavioral tasks).

Based on the theoretical assumption that all selected measures indicate – at least to some extent – the general construct of SR, our initial model included a general factor that reflected potential common variance of all indicators in addition to the factors representing domain-specific traits. However, recognizing the challenges in achieving convergent validity across different measurement modalities – specifically self- and other-reports as compared with tasks – we progressively relaxed this assumption. In Model 2, we separated behavioral tasks from survey-based measures while retaining a general factor. Finally, we tested a model without a general SR factor, allowing for the possibility that SR might be better represented without a general construct. The three models that were tested are shown in Fig 1 and described in more detail here:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Defensible measurement models of self-regulation that were compared with each other.

(A) Model 1: bifactor model with a domain-general SR factor (G_SR) and three orthogonal nested factors (B) Model 2: bifactor model with G_SR and three orthogonal nested factors only including survey-based indicators, behavioral EF tasks are measured separately (C) Model 3: model with three correlated first order factors and no domain-general factor; Hot/Cool/EF: domain-specific factors for hot/cool/executive functions components of SR; U: UPPS-P; + u: positive urgency; -u: negative urgency; ss: sensation seeking; pl: planning; per: perseverance; BIS/BAS: Behavioral Inhibition and Behavioral Approach System Scale, inhibition; rr: reward responsivity; fs: fun seeking; d: drive; CB: Parent Child Behavior Check List; at: attention problems; NIH: NIH toolbox behavioral tasks; f: flanker task; cs: dimensional card sort task; ls: list sort working memory test.

https://doi.org/10.1371/journal.pone.0322795.g001

1. 1) We estimated a bifactor model with a domain-general SR factor (G_SR) and three orthogonal nested factors. The first two nested factors comprise the hot and cool components of SR based on survey data and are named Hot and Cool, respectively. The third nested factor reflects EF measured by behavioral tasks and is referred to as EF. Even though the Cool and EF factors are conceptually closely related according to the dual-process theory, due to differing assessment domains in the ABCD study (tasks for EFs and surveys for other cool indicators), we expected their correlation to be too small to be combined within a common factor [24].
2. 2) Given expectedly low correlations between survey- and task-based indicators of SR, the second defensible model is similar to the first one, except for the fact that the EF factor was assessed in a separate model. Thus, the bifactor model included survey-based G_SR, Hot and Cool performance. Behavioral EF tasks were measured separately. This model assumes SR performance based on surveys and behavioral tasks to be uncorrelated.
3. 3) Even though hot and cool components of SR are thought to share common processes, there is conflicting evidence whether they can be combined in a general factor. While some studies found evidence for a common factor of hot and cool processes across behavioral and survey-based indicators [29], others found no commonality between hot and cool behavioral tasks [34]. Therefore, in the third defensible model, Hot, Cool, and EF were estimated as correlated first-order factors in a model without the domain-general factor G_SR.

To test the predictive validity of the model, we further expected that SR is associated with daily-life outcomes. In children and adolescents, academic achievement is greatly influenced by SR [35] and was, therefore, treated as the outcome of interest. Also, we expected that the measurement model of SR is robust across multiple smaller samples and independent from family-membership of the participants.

Participants

All data was obtained from the Adolescent Brain Cognitive Development (ABCD) Dataset (https://abcdstudy.org), data release 4.0. The ABCD study is the largest longitudinal study on children’s development to date, including 11,876 participants at the baseline assessment measured at 21 sites across the U.S. Baseline measurements started in 2016 and included participants aged nine to ten. Ethical approval of study protocols was given by the central Institutional Review Board, University of California, San Diego and individual study sites relied on local IRB approval. Caregivers provided written informed consent and children provided written assent [36]. In the present study, SR at the baseline assessment will be considered. Permission to access the anonymized data was first granted to the authors of this study on November, 22^nd 2021 and no information that could identify individual participants were ever available to the authors.

The sampling was designed to mimic sociodemographic patterns of the U.S. population as closely as possible, while oversampling siblings and especially twins. Baseline physical characteristics of the participants are described in [36]. Exclusion criteria for the overall participation were children’s lack of English language fluency, any condition preventing neuroimaging, a gestational age < 28 weeks or a birth weight <1200 g, hospitalization of more than a month after birth complications, and severe mental illnesses [36]. One individual was excluded from the present analysis due to missing data. In the whole data matrix, there were 2,836 (1.3%) missing values. At the baseline assessment, participants and their primary care takers were invited to the research site to undergo a test battery including questionnaires, behavioral and neurocognitive tasks, multimodal neuroimaging and bio-specimen sampling.

Measurement instruments

All instruments included in the baseline assessment of the ABCD study that address a subcomponent of SR [5,33], were chosen as potential indicators of SR. Some instruments that were selected as indicators of SR were reverse-coded prior to the analysis to consistently indicate better SR at higher scores. First, the UPPS-P ((negative) urgency, premeditation, perseverance, sensation seeking – positive urgency) questionnaire [37,38] was selected, which captures impulsivity on five scales with four items each: negative urgency (e.g., ‘When I feel bad, I often do things I later regret in order to make myself feel better now.’), (lack of) premeditation (e.g., ‘I like to stop and think about things before I do it.’), (lack of) perseverance (e.g., ‘I almost always finish projects that I start.’), sensation seeking (e.g., ‘I enjoy taking risks.’), and positive urgency (e.g., ‘I tend to act without thinking when I am very, very happy.’). Next, we chose the Youth Behavioral Inhibition/Behavioral Approach System Scale (BIS/BAS, [39–41]) with BAS scale scores for fun seeking (four items, e.g., ‘I often do things for no other reason than they might be fun.’), drive (four items, e.g., ‘When I want something, I usually go all the way to get it.’), reward responsiveness (five items, e.g., ‘I get thrilled when good things happen to me.’), as well as inhibition (BIS, seven items, e.g., ‘Even if something bad is about to happen to me, I rarely experience fear or nervousness.’); and the attention scale score of the Child Behavior Checklist (CBCL; [42]; ten items, e.g., ‘Can’t concentrate, can’t pay attention for long’). Additionally, three behavioral tasks from the NIH Cognition Assessment Toolbox [43] were selected as indicators of EF. The Flanker task captures inhibitory cognitive control by presenting target and flanking arrow-stimuli that can be either congruent or incongruent regarding the direction they point at. Performance is scored based on speed and accuracy of identifying the direction of the target, independently of the flanking stimuli. The Dimensional Card Sort task captures set shifting and cognitive flexibility and presents children with an object that should be sorted to match one of two preexisting objects based on either color or shape. A total score is again calculated based on accuracy and speed. The List Sorting Working Memory Test captures working memory by presenting pictures of food or animals accompanied by their names to participants. Afterwards, participants are asked to repeat the items in order from smallest to largest. Task difficulty is modulated by progressively increasing the number of items and categories (animals and food). For more details on the tasks, please refer to [44]. All indicators of SR that were considered in the measurement of SR and outcome variables are listed in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Indicators and outcomes of self-regulation in the ABCD dataset.

https://doi.org/10.1371/journal.pone.0322795.t001

Individual scale scores from these instruments were categorized based on the dual-process theory as outlined in [4]. Measures capturing rapid, bottom-up responses to high incentive stimuli – such as sensation seeking, fun seeking, reward responsiveness and drive – alongside emotion-laden impulsivity (positive and negative urgency, inhibition) were classified as components of the hot SR system. In contrast, indicators of deliberate, top-down processes including planning, perseverance, and attention problems were attributed to the cool SR system. Given their reliance on cognitive control mechanisms, the three EF indicators were categorized as part of the cool component.

Some indicators of SR were not included in our measurement model: Firstly, the assessment protocol of the ABCD study included three tasks (Monetary Incentive Delay task, Stop-Signal task, and Emotional N-Back task) related to SR that were administered in a magnetic resonance imaging (MRI) scanner, while functional MRI data was collected. They were excluded from the present analysis due to the situational differences between these tasks and the rest of the indicators of SR, as well as limited value of some of the behavioral data (see, e.g., [45] for details). Secondly, a cash choice task was included in the ABCD protocol as a delay of gratification indicator. Although delayed discounting tasks are considered to measure SR, we did not include this task in the measurement model of SR because of its low discrimination power and lack of correlation with any other indicator of SR. Furthermore, there were conceptual flaws with the task. In contrast to previous studies showing an association between performance on a cash choice task and externalizing behavior [46], participants in the ABCD study could not expect a real payment. The cash choice task was thus only imaginary and as such potentially less sensitive to measuring individual differences in delay discounting. Grades were retrieved from the Kiddie-Structured Assessment for Affective Disorders and Schizophrenia (KSADS-5, [47,48]).

Statistical analysis

SR is commonly conceptualized as a multifaceted construct comprising domain-specific hot and cool processes. Guided by this dual-process theory, we conducted confirmatory factor analyses (CFAs) to estimate the three previously postulated models. CFAs were performed with the lavaan package [50] in the R Software for Statistical Computing (version 4.2.0, [51]). Please note that despite the confirmatory nature of the analyses, we also employed an exploratory approach to compare the three defensible models and determine the best fit for the data within the dual-process framework. Firstly, univariate distributions of the variables were visually inspected, but no univariate outliers were detected. Therefore, we assumed that the multivariate distributions were neither affected by outliers. The univariate distributions of the CBCL attention scale was skewed, thus the maximum likelihood estimator with robust standard errors was applied in the factor analysis. Given 1.3% missing values, the full information maximum likelihood estimator of lavaan was used. Latent variables in all three defensible models were scaled with reference indicators (G_SR: reward responsivity, Hot: positive urgency, Cool: planning, EF: flanker task) and the variance of the factors was freely estimated.

We additionally assessed the predictive validity of the factors on grades at the baseline assessment, as well as two years later. Since the outcome variables are ordinal, prediction models were estimated with the weighted least square mean and variance adjusted estimator of lavaan. Missing values were treated with pairwise deletion in this case. We further split the data into five subsets and performed a cross validation (CV) to test the robustness of the factor structure in smaller samples and across sampling variations. To account for the nested structure of the data due to family membership, the full sample was first split between participants with and without family members in the study. Participants were then randomly allocated to the CV sets ensuring each set contained only one member per family. This approach aimed to mitigate potential bias in the estimated parameters’ standard errors arising from family membership-related dependencies within the data. The procedure was repeated with 50 CV sets.

Model fit was assessed with alternative indices of fit, given the high sensitivity of the χ²-test to large samples as the present one. We interpreted model fit as satisfactory if the root mean square error of approximation (RMSEA) <.07 and the standardized root means square residual (SRMR) <.08. Additionally, the comparative fit indices (CFI) should be > .90 [52]. Given the very large sample size, results were interpreted in terms of effect sizes and not only statistical significance. The guidelines by Cohen (1988) were applied (r > .1 as small, r > .3 as medium, and r > .5 as large effect; [53]).

Data and code availability

This study was not preregistered. The data used in this research were sourced from the ABCD study (https://abcdstudy.org), hosted within the NIMH Data Archive. Due to data access restrictions outlined by the ABCD study, the authors are unable to publicly share the study’s dataset. However, the ABCD study embraces an open science approach and data access can be requested by any eligible researcher with a valid research use of the data (https://nda.nih.gov/nda/access-data-info). Please note that the full analysis code can be accessed at https://osf.io/nzjf7/.

Defensible models of self-regulation as measured in the ABCD dataset

We tested the three proposed defensible models described above. Model 1 included a single general SR factor (G_SR) capturing the variance shared by all indicators, as well as three orthogonal nested factors Hot, Cool, and EF. Model fit was not satisfactory (CFI = .890, RMSEA = .068, SRMR = .05). G_SR was mostly determined by emotional (positive and negative) urgency (λ_GSR= .54-.70), but also planning and perseverance (λ = .29-.31). Loadings of attention, EF, and sensation seeking on G_SR were significant, but weak (λ = .11-.19). The reference indicator (reward responsivity) loaded substantially, but negatively onto the factor (λ = -.18). All loadings onto the Hot factor were moderate to high with more variance shared across BIS/BAS scales (λ = .38 -.73) as compared to UPPS-P scales (λ = .29 -.36). The Cool factor accounted for shared variance across all its indicators, i.e., perseverance (λ = .72), planning (λ = .50), and attention (λ = .23). The EF factor also explained considerable variance in its three indicators (λ = .46-.69). The variances of Hot, Cool, and EF were substantial (SD_Hot = 1.06, SD_Cool = 1.20, SD_EF= 5.75, p < .001), but the variance of G_SR was smaller (SD_GSR = .52, p = .04). Since G_SR mainly explained variance in UPPS-P scales, we added a method factor to capture survey-specific variance of the BIS/BAS scales. After doing so the Hot factor could not be established anymore (SD_Hot = .46, p = .485) and G_SR mainly explained variance in hot indicators (λ_{hot_indicators}= .19 -.68; λ_{cool_indicators}= .12 -.23; λ_{EF_indicators}= .07 -.14). We, therefore, concluded that despite the large sample size, G_SR is rather unstable. In conclusion, in Model 1, the loading patterns of the three nested factors were plausible and substantial, but the strength of the domain-general factor G_SR was insufficient for this general factor to be established.

Since the assessment modality was shown to play a crucial role in data-driven modeling approaches of SR [24], we next changed Model 1 by extracting EF from the model. Thus, in Model 2, additionally to G_SR, we estimated Hot and Cool orthogonal nested factors (Model 2a), but EF was estimated separately (Model 2b). The fit for Model 2a was slightly improved as compared to Model 1 (CFI = .903, RMSEA = .082, SRMR = .047). Note that these models can only be descriptively compared as they are not nested in each other. The general loading pattern of G_SR in Model 2a was similar to Model 1 with the two emotional urgency scales loading most strongly (λ = .48-.62), followed by planning and perseverance (λ = .32-.33,). The only BIS/BAS scale that loaded onto G_SR was the reference indicator reward responsivity, albeit negatively (λ = -.3). The loading of the attention scale was again substantial but weak (λ = .15). The general pattern for the Hot and Cool factors was similar to Model 1. Even though the variance of G_SR was significant (SD = .87, p < .001), the same loading pattern and a G_SR-Hot trade-off was observed, similarly to Model 1. Model 2b was just identified with three indicators. Thus, the fit cannot be assessed. To obtain an estimate of model fit, a fourth indicator – the attention scale from the CBCL – was added to the model. The fit was very good (CFI = .994, RMSEA = .035, SRMR = .011). Factor loadings on EF were similar to Model 1 (λ = .47-.71). Adding the attention scale score (λ = .22) did not alter these loadings significantly.

In Model 3a, the general factor G_SR was omitted from the model and indicators of EF were included. In contrast to previous models, correlations between Hot, Cool and EF were freely estimated. The fit of Model 3a was not satisfactory (CFI = .744, RMSEA = .095, SRMR = .073). All factor loadings on the Hot factor were significant with medium to large magnitude. The strongest loadings were of fun seeking (λ = .72), drive (λ = .63), and reward responsivity (λ = .62) from the BAS scale. The UPPS-P emotional urgency scales (λ = .40-.44), inhibition (λ = .38), and sensation seeking (λ = .32) also had substantial factor loadings. The Cool factor showed medium to strong factor loadings (λ = .29-.80), as did the EF factor (λ = .48-.72). EF was correlated with the Hot factor (r = .15) and the Cool factor (r = .12), but Hot and Cool showed no association (r = -.02). All factor variances were significant (p < .001). In sum, this model showed meaningful factor loadings, but unsatisfactory model fit. Given the well-known biases associated with responding on self-report questionnaires (e.g., [54]), this finding is not surprising. In the next step we, therefore, adjusted the model to improve its fit.

Firstly, we extended Model 3a by adding an orthogonal method factor (M) capturing variance attributed to the type of survey, namely the BIS/BAS questionnaire (Model 3b; CFI = .86, RMSEA = .072, SRMR = .059). BIS/BAS scales loaded on M with medium to high magnitude (λ = .31-.75). Consequently, factor loadings of BIS/BAS scales on the Hot factor were reduced (λ = .19-.39) and the Hot factor mainly explained variance in the two emotional urgency UPPS-P scales (λ = .64-.78), but also sensation seeking (λ = .26). The general pattern of Cool factor loadings persisted (λ = .32-.67) and EF factor loadings remained unchanged. Interestingly, a correlation between the Hot and Cool factor emerged (r = .33) in addition to the correlation between Hot and EF (r = .19), but the association between Cool and EF slightly decreased (r = .11). Next, we consulted modification indices and added residual correlations to the model (Model 3c). Adding residual correlations between inhibition and positive (r = .61) and negative (r = .55) urgency; sensation seeking with fun seeking (r = .24); planning with reward responsivity (r = -.25) and attention (r = -.38); and lastly perseverance with fun seeking (r = -.16), sensation seeking (r = -.17) and reward responsivity (r = -.20) improved the model fit sufficiently (CFI = .945, RMSEA = .049, SRMR = .041). Residual correlations indicate the amount of shard variance between two indicators beyond their commonality accounted by the factors, i.e., beyond the variance that is shared among all indicators. The strongest residual correlations between inhibition and emotional urgency can be explained by the fact that items on these scales are phrased negatively (e.g., for positive urgency: ‘When I am in a great mood, I tend to do things that can cause me problems’ and for inhibition: ‘I worry about making mistakes.’). Other scales are coded in the same direction, but phrased in a positive way. The association might, therefore, stem from a negative self-image. Small negative correlations between cool (lack of planning and perseverance) and hot indicators (reward responsivity, fun seeking, sensation seeking) beyond the factors could be explained by self-report biases towards extreme answers. Whether these residual correlations are robust across smaller samples was tested in the cross validation. Notably, after adding residual correlations, the inhibition scale loaded on the Hot factor with weakly negative strength (λ = -.16). Given the model fit and loading patterns, Model 3c was retained for further analysis. The correlation between Hot and Cool was of medium strength (r = .34), while Hot and EF correlated weakly (r = .17) and EF and Cool did not correlate (r = .07). Fit indices for all defensible models are summarized in Table 2 and factor loading are visualized in the Supplementary Material (S1 Fig). The final model is depicted in Fig 2A.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Model fit of three defensible confirmatory factor analysis (CFA) models.

https://doi.org/10.1371/journal.pone.0322795.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Best fitting measurement and structural model of self-regulation in the ABCD dataset.

(A) Schematic representation of a measurement model of self-regulation (SR) in the ABCD dataset (Model 3c) with standardized factor loadings. Hot/Cool/EF: domain-specific factors for hot/cool/executive functions components of SR; M_BIS/BAS: method factor for BIS/BAS indicators; U: UPPS-P; + u: positive urgency; -u: negative urgency; ss: sensation seeking; pl: planning; per: perseverance; BIS/BAS: Behavioral Inhibition and Behavioral Approach System Scale, inhibition; rr: reward responsivity; fs: fun seeking; d: drive; CB: Parent Child Behavior Check List; at: attention problems; NIH: NIH toolbox behavioral tasks; f: flanker task; cs: dimensional card sort task; ls: list sort working memory test. Residual correlations: grey color: negative correlation, black color: positive correlation; dashed line: correlation > .3, but < .5; solid line: correlation > .5 (B) Model to test predictive validity of SR on academic success (grades at baseline and grades two years later). Regression weights are semi-standardized on the latent variables’ variance.

https://doi.org/10.1371/journal.pone.0322795.g002

Predictive power of the factors for academic achievement

We then tested the predictive validity of SR, as measured in Model 3c (i.e., excluding G_SR, but including EF), for a daily-life outcome. Academic success as indicated by grades at the time of the SR assessment (baseline) and grades two years later were included as endogenous variables in Model 4. The model fit was satisfactory (CFI = .939, RMSEA = .061, SRMR = .046). Grades at baseline and two years later were correlated (r = .46). SR factors jointly explained 26% of the variance in the grades at baseline, and 27% of the variance two years later. EF and Cool contributed to the overall prediction at both times (β_EF = .33-.35, β_Cool = .32), which highlights the relevance of EF measures as well as survey-based cognitive control for school performance. The Hot factor did not contribute to the prediction of school performance (β_Hot = .01-.05). In sum, the predictive power of the factors for academic success was highlighted by the data. Model 4 is illustrated in Fig 2.

Robustness of the model

To test the robustness of the factor structure of Model 3c in smaller, family-independent samples and across sampling variations we conducted a five-fold CV (N_CV1–5 = 2,376). The results are illustrated in Fig 3. Results of a further trial with a 50-fold CV (N_CV1–50 = 238) are reported in the Supplementary Material (S2 Fig). The measurement model of SR was robust across CV sets and trials. Therefore, the second hypothesis concerning the robustness of the model in smaller samples was supported here.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Five-fold cross validation (CV) of the measurement model of self-regulation.

Factor loadings of the full dataset per factor (full) are compared to factor loadings of cross-validation sets (CV1-CV5) to test the robustness of the factor structure over smaller samples. λ Hot/Cool/EF: factor loadings onto domain-specific factors for hot/cool/executive function components of self-regulation (SR); λ M: factor loadings onto method factor for BIS/BAS scales; residual cor: residual correlations; factor cor: correlations between factors; U: UPPS-P; pu: positive urgency; nu: negative urgency; ss: sensation seeking; pl: planning; per: perseverance; BIS/BAS: Behavioral Inhibition and Behavioral Approach System Scale; rr: reward responsivity; fs: fun seeking; d: drive; CB: Parent Child Behavior Check List; at: attention problems; NIH: NIH toolbox behavioral tasks; f: flanker task; cs: dimensional card sort task; ls: list sort working memory test.

https://doi.org/10.1371/journal.pone.0322795.g003

Limitations

We acknowledge that the design of the assessment protocol in the ABCD study was not primarily designed to psychometrically represent SR within the dual-process theory. The boundaries between subcomponents of SR are often ambiguous due to the lack of a universally accepted definition of its predictors, outcomes and integral concepts. In a study allowing for greater flexibility in variable inclusion, we might have chosen more nuanced indicators of SR in surveys and behavioral tasks to ensure balanced coverage of both hot and cool components. However, the goal of the present study was not to test the dual-process theory, but rather to see whether it is applicable to the ABCD dataset with the resources available. Furthermore, applying the dual-process theory is not the only way of modeling SR (see [4] for a review) and we do not claim superiority of the dual-process model over given alternatives. However, our study supports the relevance of the dual-process model in the context of decision making and brain-behavior relationships, which are specifically addressed in the ABCD dataset.