Externalizing problems (EP), including rule-breaking, aggression, and criminal involvement, are highly prevalent during adolescence, but the adult outcomes of adolescents exhibiting EP are characterized by heterogeneity. Although many youths’ EP subside after adolescence, others’ persists into adulthood. Characterizing the development of severe EP is essential to prevention and intervention efforts. Multiple predictors of adult antisocial personality disorder (ASPD) and legal outcomes of a large sample (N = 1205) of clinically- or legally-ascertained adolescents (ages 12–19 years) with severe EP were examined. Many psychosocial predictors hypothesized to predict persistence of EP demonstrated zero-order associations with adult outcomes, but accounted for little unique variation after accounting for baseline conduct disorder symptoms (CD) and demographic factors. Baseline measures of intelligence, which explained independent variation in legal outcomes, provided the only consistent exception to this pattern, though future work is needed to parse these effects from those of socioeconomic factors. CD severity during adolescence is a parsimonious index of liability for persistence of EP into adulthood that explains outcome variance above and beyond all other demographic and psychosocial predictors in this sample.
Citation: Border R, Corley RP, Brown SA, Hewitt JK, Hopfer CJ, Stallings MC, et al. (2018) Predictors of adult outcomes in clinically- and legally-ascertained youth with externalizing problems. PLoS ONE 13(11): e0206442. https://doi.org/10.1371/journal.pone.0206442
Editor: Geilson Lima Santana, University of Sao Paulo Medical School, BRAZIL
Received: January 18, 2018; Accepted: October 14, 2018; Published: November 1, 2018
Copyright: © 2018 Border et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Access to patient data is limited due to restrictions put in place by the Colorado Multiple Institutional Review Board (COMIRB). The IRB will not permit release of these subjects’ data to a public archive because participants were not asked to consent to such release and during the consent process subjects placed limitations on the ways their study data could be used. Qualified researchers may apply for data access at http://www.ucdenver.edu/academics/colleges/medicalschool/departments/psychiatry/Research/Subdep/Data%20Request%20from%20Published%20Study/Pages/form.aspx. Data requests will be overseen by the Principle Investigator as stated by the COMIRB (firstname.lastname@example.org).
Funding: This work was funded by National Institute on Drug Abuse (https://www.drugabuse.gov/) grants DA005131 (JKH), DA011015 (N/A), DA012845 (N/A), DA021905 (SAB), DA035804 (CJH) and National Institute of Mental Health (https://www.nimh.nih.gov/) grant MH016880 (JKH). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Externalizing problems, including legal involvement, substance use problems, and conduct disorder (CD) symptoms, are common during adolescence and frequently precede negative psychosocial and legal sequelae . Despite the high prevalence of ASB during adolescence—e.g., the estimated lifetime prevalence of CD is 9.5% —the adult outcomes of adolescents with ASB are heterogeneous; many youth desist from ASB during the transition into adulthood, whereas others persist . Researchers have defined discrete groups of youth with ASB based on their developmental trajectories (e.g., persisters versus desisters) and compared their psychosocial profiles to identify potential risk factors. These studies have typically examined constellations of risk factors exerting reciprocal influences across development, wherein distinct vulnerabilities are rarely thought to operate in mutual isolation [1,4]. Though many candidate risk factors have been identified in this fashion, this approach is not without its weaknesses. First, trajectory-based categorization of individuals requires information typically unavailable to prevention and intervention efforts (e.g., age-of-onset, repeated measures across development). Second, the fact that individuals with different trajectories exhibit mean differences in a variety of psychosocial factors does not imply that measures of said factors during adolescence will explain independent variation in adult outcomes. Thus, whether previously identified correlates of persistence consistently account for outcome variation in adult in beyond baseline CD and demographic factors—information crucial to decisions of which traits to measure in intervention/prevention settings where time and resources are limited—remains unclear. To address these gaps, the present study examined the adult outcomes of clinically- and legally-ascertained adolescents with CD symptoms and substance use problems [SUP], investigating the effects of many previously-identified predictors of persistence (e.g., verbal deficits, SUP severity, maternal ASPD, familial cohesion and conflict, and perceived peer deviance) beyond those of baseline CD and demographic factors. Analyses aimed to highlight predictors of persistence at a time point particularly relevant to intervention efforts (i.e., clinical and/or legal ascertainment).
Heterogeneity in the persistence of ASB
Moffitt advanced the highly influential “developmental taxonomic theory” of ASB outlining two distinct behavioral trajectories: adolescent-limited (AL) and life-course-persistent (LCP) ASB . Individuals with AL ASB temporarily exhibit ASB normative within the context of adolescence. In contrast, LCP youth are characterized by unique neurocognitive profiles that emerge in early childhood and interact with negative environmental influences (e.g., parental neglect) to increase the likelihood that their behaviors will persist into adulthood.
Numerous investigations have reported findings congruent with Moffitt's distinction between EP limited to adolescence versus that which persists into adulthood, though the exact number of groups with distinct trajectories (e.g., adult-onset, childhood-limited, etc.) is highly variable across studies . Early longitudinal research identified AL/LCP groupings through theory-driven categorization procedures based on individual ASB trajectories and, more recently, studies employing latent class analyses to classify individuals by trajectory have supported this distinction [5,6]. However, whether or not the results from methods that categorize individual trajectories into distinct groupings (i.e., AL vs. LCP) permit investigators to determine if qualitative differences exist between groups is a point of controversy [7,8]. Additionally, many adolescent-onset ASB youth frequently continue offending into adulthood and evidence legal and psychosocial outcomes analogous to those of their childhood-onset counterparts . Though these trajectory-based investigations have identified numerous putative risk factors for persistent ASB, their results do not directly inform choices of which traits are likely to be informative in samples of youth already facing correctional action for whom no longitudinal data are available.
Correlates of persistence
Comparisons of AL and LCP ASB suggest a variety of psychosocial correlates associated with persistence. Below, we review those available in the current investigation, for the purpose of identifying any predictors of persistence beyond baseline ASB and demographic measures.
Male gender is a robust predictor of ASB across the lifespan, and females with ASB are more likely to be classified as AL . Age provides developmental context for evaluating the severity of EP (e.g., theft of alcohol at age 12 years may indicate greater severity than the same transgression at age 18 years). Verbal deficits.
Antisocial youth often exhibit verbal IQ deficits [11–14], and LCP individuals may have greater deficits in verbal capabilities than AL individuals through a variety of pathways (e.g., academic failure, poor social capabilities) . However, the association between earlier verbal deficits and EP in adulthood is characterized by mixed findings [12,14,16,17], with some work suggesting that the degree to which verbal deficits predict the persistence of CD symptoms depends on parental antisocial personality disorder (ASPD) diagnosis (i.e., the presence of a parent with ASPD negates the potential protective effects of higher-than-average verbal capabilities) . It is currently unknown whether verbal deficits account for independent variation in EP during adulthood after adjusting for baseline severity of EP and demographic factors in adolescence, and the predictive utility of verbal deficits in clinical settings remains unclear.
Externalizing problems in adolescence.
CD and SUP during adolescence are both known to predict future conduct problems and criminal activity [4,19]. However, it is difficult to distinguish between the repeated effects of underlying traits versus the cascading effects of repeated offenses (e.g., one offense may lead to school expulsion, which may place the individual at increased risk for repeated offenses). Further, the extent to which SUP accounts for unique variation in adult EP beyond adolescent EP is unclear; commonly employed trajectory-based models of EP have suggested that both AL and LCP youth demonstrate elevated levels of SUP , but do not directly speak to the independent predictive utility of either construct [7,8]. Additionally, some research has suggested that independently examining qualitatively distinct CD symptom clusters, i.e. aggressive symptoms versus non-aggressive symptoms, may improve our ability to distinguish between persistent and adolescent-limited EP [20,21], though evidence as to whether these symptom clusters reflect distinct genetic and environmental influences is mixed [22,23].
Familial ASB and family environment.
A variety of familial factors including family conflict, family cohesion, and parental antisocial behavior have been associated with EP during adulthood [18,24–28]. The importance of patterns of familial interaction is further supported by evidence for the efficacy of family- and parenting-oriented treatments for childhood disruptive behavior disorders [29–31]. Additionally, evidence from twin research indicates that individual differences in CD and SUP across childhood and adolescence reflect substantial genetic underpinnings [32,33]. In the present study, measures of paternal ASPD were unavailable, so we restricted our focus to maternal ASPD. Though the relative importance of maternal versus paternal psychiatric disorders in predicting offspring outcomes is unknown , previous research has suggested that maternal antisocial behavior predicts future child disruptive behavior disorder outcomes and related outcomes [21,35]. Due to the multiple potential pathways linking familial variables and EP (i.e., transmission through latent genetic vulnerabilities versus transmission via environmental consequences), however, the predictive utility of familial factors, particularly among adolescents with EP, remains unclear.
Deviant peer influence.
Association with antisocial peers has been hypothesized to explain ASB in AL youth and previous work has suggested that peer deviance is associated with greater ASB during early and late adolescence [1,4,6,17]. However, whether exposure to deviant peers further predicts persistence after accounting for EP during adolescence is unclear, and gene–environment correlation may also contribute to the putatively environmental consequences of affiliation with deviant peers . Beyond elucidating the predictive utility of deviant peer affiliation, further examination of this association is crucial, as clinical and legal interventions may place individuals in further contact with deviant peers .
The present study
The factors explaining unique variance in the adult outcomes of clinically- and legally-ascertained adolescents beyond demographic factors and initial EP remain unclear. Many existing longitudinal studies following adolescents with EP through the transition into adulthood have focused on group comparisons with respect to discrete developmental trajectories. Though this approach has informed the examination of potential correlates and causes of EP persistence, it has two major shortcomings in its ability to inform intervention efforts in adolescence, when individuals are most likely to be ascertained. First, it requires longitudinal information not typically available to clinical and legal institutions. Second, it does not speak to the utility of measuring specific risk factors in high-risk youth ascertained as adolescents. In this context, the present study examined the unique contributions of all available psychosocial predictors (i.e., verbal deficits, CD, SUP, maternal ASPD, family environment, and deviant peer influence) beyond baseline demographic factors and CD symptoms to adult ASPD and involvement with the criminal justice system in a longitudinal sample of 1205 clinically- and legally-ascertained youth.
The current longitudinal, multi-site sample is composed of youth with severe EP. Between 1993 and 2007, 1517 adolescents were recruited from residential and outpatient treatment facilities for substance abuse and delinquency, criminal justice records, schools for youth with behavior problems, and alcohol and drug treatment programs [38–40].
Participants from the 1993–1997 Denver Clinical Sample (n = 244) consisted of males recruited from Denver, Colorado area residential facilities for substance abuse and delinquency; Participants from the 1997–2002 Denver Clinical Sample (n = 362) sample were recruited from residential and outpatient treatment facilities for substance abuse and delinquency; Participants from the 2001–2006 Denver Clinical Sample (n = 363) were recruited from outpatient substance abuse treatment programs; Participants from the Denver Adjudicated sample were identified through Colorado criminal justice records (n = 302); Participants from the San Diego Sample (n = 246) sample were recruited from schools for youth with behavior problems and alcohol and drug treatment programs in San Diego, California. With respect to the 1993–1997 and the 1997–2002 Denver clinical samples, participants for whom a first degree relative agreed to participate in the assessments were targeted for follow-up. With respect to the remaining samples, participants who displayed at least one CD symptom or at least one non-tobacco SUD symptom at initial assessment were targeted for follow-up. All targeted participants (N = 1205) completed between one and two follow-up assessments, each occurring approximately five years apart, with the most recent follow-up assessment occurring at an average of 9.21 years (SD = 3.41) subsequent to baseline assessment. Some participants participated in two follow-up assessments of ASPD symptoms, and repeated measures were utilized when available. See Table 1 for detailed descriptions of the five samples and characteristics of participants targeted for follow-up.
Inclusion criteria for all studies were comprised of the following: (a) absence of psychosis, intellectual disability, and imminent danger to self or others; and (b) absence of physical illness or current intoxication precluding participation in treatment or evaluation. Written consent from parent or guardian and assent from the participant was obtained for all participants under age 18 years and written consent was obtained for each adult. Participants received monetary compensation. All study procedures received institutional review board approval prior to data collection by the review boards of the University of Colorado Boulder, the University of Colorado Denver, and the University of California, San Diego.
Baseline verbal and performance capabilities were assessed using the vocabulary and block design subtests from the Wechsler Abbreviated Scale Intelligence [41,42], the Wechsler Adult Intelligence Scale , or the Wechsler Intelligence Scale for Children , depending on the sample and the version available at the time. Scaled scores relative to national norms were used.
Substance abuse/dependence vulnerability.
The Composite International Diagnostic Interview-Substance Abuse Module (CIDI-SAM)  provides symptom counts and diagnoses of abuse and dependence for 11 categories of substances according to Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) guidelines . A composite substance abuse/dependence vulnerability index was generated by dividing the total number of abuse/dependence and symptoms by the number of substances tried and can be interpreted as average abuse and dependence symptoms per substance .
Perceived peer deviance.
Perceived peer delinquency was measured with the Exposure to Delinquent Peers Measure . Participants indicated their perceptions of the proportion of friends who had participated in delinquent behaviors.
CD and antisocial personality disorder (ASPD).
Baseline lifetime CD criteria count was assessed using either the Diagnostic Interview Schedule for Children (DISC) version 2.1 or the DISC-IV, depending on sample [51,52]. Although two additional CD symptoms (a 14th and 15th) were added in the fourth iteration of the DSM, correlations between DSM-III and DSM-IV-based symptom counts among participants interviewed with the DISC-IV approached unity (r = 0.978; r = 0.963 and r = 0.972 for aggressive symptoms and non-aggressive symptom clusters, respectively). Baseline maternal lifetime ASPD criteria count was assessed using the Diagnostic Interview Schedule (DIS) version III-R or version IV [53,54]. Both past-year and lifetime ASPD criteria count were measured using the DIS-IV at follow-up. Models examining these outcomes controlled for participant age at follow-up.
Arrest after 18th birthday was assessed using a single self-report item from the DIS-IV, and all models examining this outcome were adjusted for participant age at follow-up. Participants who were 18 or older at baseline were excluded from analysis. Recent legal involvement was also obtained; in a living arrangements interview, participants indicated whether they had spent on any time parole, on probation, or incarcerated during each of the previous three to five years (since last assessment). As a small subset of participants only had data available for the past three or four years, the number of years of data availability was included as a covariate in all relevant analyses.
Data manipulation was performed in the R programming environment for GNU/Linux . Analyses were conducted in Bayesian framework utilizing Hamiltonian Markov Chain Monte Carlo sampling with weakly informative priors via the probabilistic programming language Stan and its interfaces with R [56–58].
Bivariate associations (Pearson, polyserial, and polychoric correlations) between all variables were estimated using the polycor package  (Table 2). Baseline CD symptoms, demographic variables, and outcome variables were separately modeled in the context of Bayesian generalized linear mixed models (GLMMs), with the choice of distributional family and link function based on their empirical distributions. For each outcome, we first estimated a compact model, which included age at baseline, age(s) at follow-up(s), ethnicity, sex, and baseline CD as predictors. We then estimated separate augmented models for each additional psychosocial predictor by re-estimating the compact model while including the additional predictor. This procedure was chosen for two reasons. First, as not all samples were administered every measure (Table 1), including all additional psychosocial predictors would have dramatically reduced effective sample size. Second, each augmented model reflects the additional unique information gained by considering an additional predictor on top of baseline CD symptoms and demographic figures, thus specifically informing decisions regarding the utility of including additional measures in assessments in a prevention context.
A mixed models random-intercepts framework was implemented to account for dependence among observations nested within samples, and for repeated-measures variables, within individuals. Baseline CD and lifetime ASPD were modeled with linear mixed effects regression , past year ASPD was modeled with negative-binomial mixed effects regression , and binary outcomes (arrest after 18th birthday, attrition, and legal involvement during the past five years) were modeled with logistic mixed effects regression . Each outcome was first regressed on sex, age at baseline, age(s) at follow-up(s), ethnicity, and baseline CD symptoms (compact models, Table 3). Additional psychosocial variables were then included one at a time to assess their independent contributions beyond baseline CD and demographic variables (augmented models, Table 3).
GLMMs for repeated measures outcomes (lifetime and past year ASPD symptom counts) included random intercepts for participants. All models included random intercepts for sample. Thus, at their most complex, models were of the form: where i indexes individual, j indexes sample, t indexes mean deviated age(s) at follow-up(s) T, cov indicates a vector of baseline demographic covariates, CD indicates baseline CD symptoms, PSP indicates the additional psychosocial predictor measured at baseline, Y indicates the outcome variable, f indicates the link function, and roman capitals indicate random effects. The α, β, and γ coefficients represent regression coefficients at the level of follow-up time point, individuals, and sample, respectively. For each model, only samples that included all relevant measures were included. In addition to random intercept models, we also considered models including random effects of psychosocial predictors, but this approach failed to alter results for any outcome. For clarity, we only discuss results of random intercept models.
Additionally, we conducted sensitivity analyses to examine the potential impact of missingness of observations due to study attrition, which is known to bias parameter estimates when study dropout is non-random . To this end, we constructed five replicate data sets via multivariate imputation by chain equations using predictive mean matching for continuous variables and augmented general linear models for discrete variables, using the mice package for R [63–65]. These imputed datasets are constructed under the assumption that missing data were missing-at-random (MAR) after conditioning on covariates and are heretofore referred to as the MAR replicates. MAR replicates were further manipulated by adding structured random offsets to each outcome to simulate different non-random dropout regimes (i.e., systematically increasing or decreasing imputed outcome measures among attrited individuals). This resulted in two additional sets of replicate datasets: NMAR+ replicates, which simulated a positive association between attrition and outcome severity, and NMAR- replicates, which simulated a negative association between attrition and outcome severity. Imputed data were then analyzed using the same GLMM framework as the primary analyses and posterior distributions of parameters were averaged across replicate data sets using the brms R package . For continuous outcomes (lifetime and past year ASPD), attrited participants’ MAR-imputed symptom counts were augmented with positive or negative binomial (n = 2, p = .5) random offsets corresponding to one average symptom differences (SD = .707) under the NMAR+ and NMAR- regimes, respectively. For binary outcomes (arrest after 18th birthday and recent legal involvement), attrited participants’ MAR-imputed outcomes were augmented with positive or negative Bernoulli (p = .25) random offsets under the NMAR+ and NMAR- regimes, respectively. E.g., under the NMAR+ regime, participants who were imputed as not having experienced arrest after their 18th birthday under the MAR regime then had a simulated additional 25% chance of arrest. Truncation was performed as necessary to ensure all imputed scores were plausible (e.g., negative symptom counts were not allowed). Further details and results are presented in Tables A and B in S1 Supplement, and the distributions of the observed and imputed outcomes are presented in Figures A and B in S1 Supplement.
The adult outcomes of the participants were severe but variable (Table 1). At follow-up, mean lifetime ASPD criteria count was 4.11 (SD = 1.99) and mean past year ASPD criteria count as 1.56 (SD = 1.54). Sixty-eight percent of participants who were under 18 years old at baseline reported being arrested after their 18th birthday. Forty-eight percent of participants endorsed being on parole, on probation, or incarcerated at some point during the past five years. Attrition was moderate, with 26% of participants targeted for follow-up (310 of 1205 targeted for follow-up assessment) not participating in subsequent assessments.
Every demographic and psychosocial predictor was significantly related to at least one psychiatric or legal outcome at follow-up (Table 2). However, these estimates do not account for sample differences or other demographic factors and should be interpreted with caution.
Effect sizes are presented as regression weights standardized with respect to the predictor but not to the outcome measure. Exceptions include sex and ethnicity contrasts, which are to be interpreted as adjusted mean differences for linear models, adjusted odds ratios for logistic models, and adjusted incident rate ratios for negative binomial models. We present 95% Bayesian credibility intervals (CI) for all coefficient estimates, which describe the 0.025 and 0.975 quantiles of their posterior distributions. An analogous notion to frequentist statistical significance at α = .05 can be obtained by observing whether the 95% CI covers zero or one for linear and exponentiated regression coefficients, respectively. Results for lifetime ASPD, past-year ASPD, and baseline CD are presented in Table 3; results for arrest after 18th birthday, recent legal involvement, and attrition are presented in Table 4. Corresponding sensitivity analyses are presented in Tables A and B in S1 Supplement, respectively. A visual overview of how predictor effects varied across samples is presented for lifetime and past year ASPD in Figure C in S1 Supplement and for arrest and legal involvement in Figure D in S1 Supplement. Ethnicity contrasts for all outcomes are presented in Table C in S1 Supplement.
In the compact model, sex and baseline CD explained independent outcome variance such that adjusted mean symptom counts were 0.919 greater in males (95% CI: 0.609–1.221) and 0.462 per unit standard deviation increase in baseline CD symptom (95% CI: 0.342–0.585). Additionally, average symptom count was lowest among non-Latino Caucasian participants, who displayed between 0.089 and 1.094 fewer symptoms than non-Latino African-American participants (Table D in S1 Supplement). In the augmented models, no psychosocial predictors explained unique outcome variance after accounting for baseline demographics, baseline CD, and age at follow-up (Table 3). Sensitivity analyses suggested that estimates for the effects of sex and baseline CD were robust across multiple potential patterns of differential attrition (Table A in S1 Supplement).
Past year ASPD.
In the compact model, sex and baseline CD again explained independent outcome variance (eβ = 1.264, 95% CI: 1.091–1.462; eβ = 1.141, 95% CI: 1.066–1.190; respectively). Additionally, marginal incidence rate of symptom counts was greater among African-American participants as compared to other groups (Table D in S1 Supplement). Again, no psychosocial predictors explained unique outcome variance after accounting for baseline demographics, baseline CD, and age at follow-up (Table 3). Sensitivity analyses again suggested that estimates for the effects of sex and baseline CD were robust across multiple potential patterns of differential attrition (Table A in S1 Supplement). Further, MAR sensitivity analyses, which assumed attrition and past year ASPD were conditionally independent given the covariates, suggested possible independent contributions of family conflict and family cohesion (eβ = 1.071, 95% CI: 1.017–1.127; eβ = 0.947, 95% CI: 0.868–1.009; respectively; Table A in S1 Supplement).
Arrest after 18th birthday.
In the compact model, sex (but not CD) explained independent outcome variance (eβ = 3.546, 95% CI: 2.303–5.506). Conditional marginal odds of arrest did not differ across race/ethnicity categories in the context of the compact model (Table C in S1 Supplement). Among additional predictors, vocabulary and bock design evidenced independent (negative) contributions to adjusted odds of arrest in the context of the augmented models (eβ = 0.742, 95% CI: 0.595–0.919; Table 4). Sensitivity analyses supported the contributions of sex, block design, and vocabulary across multiple non-random attrition regimes (Table B in S1 Supplement).
Recent legal involvement.
Consistent with the results for arrest, only sex evidenced partial associations in the compact model (eβ = 3.083, 95% CI: 2.042–4.693). Conditional marginal odds of recent legal involvement were substantially elevated in non-Latino African-American and non-Caucasian Latino participants relative to non-Latino Caucasian participants in the context of the compact model (Table C in S1 Supplement). Additionally, both block design and vocabulary evidenced trending associations with decreased odds of recent legal involvement (eβ = 0.832, 95% CI: 0.686–1.008; eβ = 0.835, 95% CI: 0.657–1.025; respectively; Table 4), with sensitivity analyses again supporting the contributions of sex, block design, and vocabulary across multiple non-random attrition regimes and additionally suggesting that adjusted odds decreased with age at follow-up assessment (Table B in S1 Supplement).
Among all psychosocial predictors, only vocabulary evidenced independent contributions to outcome variance such standard deviation increases in vocabulary score decreased odds of attrition by a factor of 0.784 (95% CI: 0.666–0.922), though a similar trend was present for family cohesion (eβ = 0.831, 95% CI: 0.691–1.009; Table 4). Ethnicity contrasts suggested that non-Latino African-American had greater marginal odds of attriting compared to all other racial/ethnic categories (Table C in S1 Supplement).
Cross-sectional associations with baseline CD.
To further understand the differing patterns of results between zero-order models (where most psychosocial predictors exhibited associations with adult outcomes; Table 2) and multivariate models (where relatively few psychosocial predictors explained outcome variation after controlling for baseline CD and demographic factors; Tables 3 and 4), we examined the extent to which individual psychosocial predictors explained concurrent CD symptoms after controlling for age, sex, and ethnicity. The effect of each psychosocial predictor was examined in the context of a linear mixed effects regression including random intercepts across samples.
Male participants’ marginal symptom counts were between 0.222 and 0.512 standard deviations greater than female participants’ (Table 3), though no differences were evident across racial/ethnic categories (Table C in S1 Supplement). Augmented model results (Table 3) were wholly consistent with unconditional bivariate analyses (Table 2). That is, accounting for sample, age, sex, and ethnicity did not alter evidence for contributions of substance vulnerability, family cohesion, family conflict, and perceived peer deviance, whereas block design, vocabulary, and maternal ASPD did not evidence contributions to baseline CD in either context (Tables 2 and 3).
Contributions of aggressive and non-aggressive CD symptom clusters.
Given the substantial interest in aggressive versus non-aggressive CD symptom clusters [20,22,23,66], as well as the lack of evidence for independent contributions of baseline CD to arrest after 18th birthday or legal in the context of multivariate models (Table 3) despite evidencing zero-order bivariate associations (Table 2), we examined the independent contributions of symptom clusters to each longitudinal outcome. Specifically, for each outcome, we simultaneously estimated contributions of aggressive and non-aggressive symptoms while accounting for age, sex, ethnicity, and sample. Both aggressive and non-aggressive CD symptoms evidenced independent contributions to lifetime ASPD whereas only aggressive CD symptoms evidenced independent contributions to past year ASPD (Table D in S1 Supplement). Further, whereas slopes for aggressive versus non-aggressive symptom clusters did not differ for lifetime ASPD (95% CI: -0.190–0.314), differences for past year ASPD were less certain (95% CI: -0.006–0.213). No independent contributions of either symptom cluster were apparent for arrest or legal involvement and follow-up analyses did not provide evidence for an interaction between symptom clusters across all outcomes (Table D in S1 Supplement).
Partial effects of intelligence measures.
Both principal analyses (Table 4) and sensitivity analyses (Table B in S1 Supplement) of arrest and recent legal involvement suggested possible independent contributions of design and vocabulary measures evidenced negative partial associations with current ASPD symptom count, arrest after 18th birthday, and recent legal involvement beyond baseline CD and demographic factors (Tables 3 and 4). Because these two measures are correlated (Table 2), we reran the relevant principal analyses, simultaneously modeling the partial effects of both intelligence measures beyond demographic variables and baseline CD. For both arrest after 18th birthday and recent legal involvement, vocabulary, but not block design evidenced independent contributions beyond one another, sex, age, ethnicity, and sample (Table D in S1 Supplement). However, 95% credibility intervals for slope differences covered zero in both cases (95% CI: -0.467–0.209, 95% CI: -0.507–0.162, for arrest and legal involvement, respectively). Follow-up analyses did not provide evidence for an interaction between measures across all outcomes (Table D in S1 Supplement).
Vocabulary-by-maternal ASPD interaction.
Further analyses were motivated by the lack of evidence for contributions of maternal ASPD across all outcomes (Tables 3 and 4), evidence for independent contributions of vocabulary to odds of arrest after 18th birthday and recent legal involvement, and previous research suggesting that the association between verbal deficits and future EP is moderated by parental ASPD . We thus estimated additional models including terms for maternal ASPD, vocabulary score, and their product, as well as baseline CD symptoms and demographic factors. We found no evidence of such an interaction across any of the adult outcomes (Table D in S1 Supplement).
The present study examined the degree to which multiple available theoretically-motivated predictors explained unique variance in concurrent and adult outcomes in a longitudinal, multi-site study of adolescents ascertained for severe EP. Our results pertain to the practical problem of selecting additional risk factors to measure on top of information provided by demographic variables and baseline CD to assess risk for persistence of EP into adulthood in intervention and prevention settings. We discuss the contributions of demographic predictors, baseline CD, and psychosocial predictors in turn.
Despite the initial severity of the samples, adult outcomes were variable—e.g., 51.94% of individuals reported recent legal involvement at follow-up assessment. Prominent sex differences were apparent across all outcomes other than attrition. For example, after controlling for age at baseline, age(s) at follow-up(s), ethnicity, and baseline CD, males’ odds of arrest after 18th birthday and incident rates of past-year ASPD symptoms were 3.546 and 1.263 times greater than those of females, respectively (Tables 3 and 4). Non-Latino African-American participants were broadly at greater risk for negative outcomes, after accounting for baseline CD and other predictors, though these effects varied substantially across outcomes (Table C in S1 Supplement). Reasons for these differences are unclear, but are likely to reflect systematic variation in unmeasured socioeconomic factors. Moreover, non-Latino African-American participants demonstrated elevated rates of attrition, which sensitivity analyses suggest might have influenced results (Tables A and B in S1 Supplement), complicating interpretations of effects of ethnicity on psychiatric and legal outcomes. Age at baseline assessment failed to consistently predict any outcomes variables, which is unsurprising as it is a poor proxy for age-of-onset of EP.
Baseline CD independently predicted both lifetime and past-year ASPD criteria count, which is consistent with the conceptualization of CD as a developmental precursor to ASPD. Independent associations between CD and negative legal outcomes were directionally consistent with ASPD results but subject to greater uncertainty (Tables 4, Table B in S1 Supplement). Follow-up analyses suggested that contributions of aggressive and non-aggressive symptoms to lifetime ASPD were indistinguishable, whereas only aggressive symptoms appeared to contribute to past-year ASPD. These results are at odds with a recent study reporting that only aggressive CD symptoms contributed to lifetime ASPD in a sample of 254 boys recruited from an economically distressed area in Pennsylvania . However, lifetime ASPD symptoms might be a poor indicator of persistence that simply reflected the severity of baseline CD among participants in the present study. Our results for past year ASPD are indeed congruent with greater contributions of aggressive symptoms to persistence of EP.
Despite significant zero-order associations between many psychosocial predictors and outcome variables (Table 2), as well as evidence for independent associations between these predictors and baseline CD (Table 3), few were unique predictors after accounting for demographic characteristics and baseline CD (Tables 3 and 4). Moreover, some variables associated with concurrent CD in adolescence (i.e., substance abuse and perceived peer deviance), failed to explain independent variance in any adult outcomes beyond that explained by baseline CD and demographics. On the other hand, vocabulary and block design, which both had significantly variable cross-sectional associations with CD in adolescence across samples, were the only predictors that predicted adult legal outcomes with moderate consistency. For other predictors, patterns were less clear; e.g., family conflict contributed independent variance to baseline CD and past year ASPD, but not to lifetime ASPD, with sensitivity analyses suggesting that study attrition may have attenuated coefficient estimates.
Follow-up analyses examining the simultaneous partial effects of intelligence measures suggested that, after accounting for the outcome variance explained by variation common to vocabulary and block design, deficits in vocabulary still predicted negative adult legal outcomes. One possible explanation for this divergence is that vocabulary deficits might impair individuals’ ability to effectively navigate the legal system, though we are hesitant to make such claims, given the exploratory nature of these analyses and lack of a comprehensive measure of socioeconomic status (SES) (Table 3). Further, examining the posterior distribution of vocabulary/performance slope differences did not suggest robust differences in effects.
There are several limitations in the present investigation. As mentioned above, study attrition was moderate (26%; Table 1), differed substantially between ethnic groups (Table C in S1 Supplement), and possibly biased estimates effects of family cohesion and family conflict (Table A in S1 Supplement). Second, as outcomes were measured during at most two follow-up assessments per individual, a latent growth modeling approach, which would have allowed us to differentiate between factors affecting mean levels versus changes in ASPD severity over time, was not possible. Third, not all measures were administered for every sample (Table 1), diminishing effective sample size (and hence statistical power) for a number of covariates. Fourth, despite considerable interest in the contributions in possible connections between callous-unemotional (CU) traits and EP [21,67,68], such measures were unavailable for the present sample, though the utility of CU traits in examining persistence of EP is a point of controversy [67,69,70]. Fifth, our measures of legal outcomes did not distinguish between qualitatively different types of crime (e.g., violent versus non-violent crimes), perhaps masking type-specific contributions of psychosocial predictors. For example, previous work suggests that SUD among detained adolescents increases risk for substance-related recidivism ; it is therefore possible that failure to distinguish between substance- and non-substance-related legal outcomes might partially underlie the failure to detect independent contributions of substance vulnerability in the present investigation. Sixth, and perhaps most crucially, a comprehensive measure of SES was unavailable, which was unfortunate given the known associations between SES, ethnicity, and intelligence measures. Finally, we acknowledge that collapsing across diverse samples may have reduced are ability to detect sample-specific effects of individual predictors, even after adjusting for sample differences. However, allowing slopes associated with psychosocial variables to vary by sample failed to substantially alter our conclusions (see Figures C-D in S1 Supplement for a graphic presentation). In the search for robust predictors of unique outcome variance, we have focused on variables with the potential to be valuable across a wide of prevention and intervention efforts and believe this approach to be practical to that end.
The present study examined the independent predictive contributions of a wide array of candidate risk factors for the persistence of adolescent EP into adulthood. Although we considered a large variety of theoretically-derived psychosocial predictors, we were ultimately limited to those available in the present sample and thus cannot claim to have covered the breadth of candidate risk factors suggested in the literature. Specifically, in addition to previously discussed limitations, we lacked comprehensive measures of personality traits, psychiatric comorbidity, psychopathy, emotional and behavioral regulation, parenting practices, and community level risk factors (e.g., crime rates), each of which has been suggested to increase risk for persistent EP [3,71–73]. Still, given the spectrum of predictors considered, our results inform the development of prevention, intervention, and public health initiatives, as we examined a sample of youth with severe externalizing problems who were already “naturally” ascertained by clinical or correctional programs. Individuals such as our participants, virtue of having already attracted the attention of public institutions (be they clinical or correctional), are clear targets for public health initiatives.
Developmental taxonomic theory, the dominant theoretical perspective in the longitudinal study of EP, predicts that the continuity of these behaviors into adulthood should reflect psychosocial and socioeconomic factors. Many of these psychosocial variables were associated with unadjusted legal outcomes in adulthood (Table 2) and explained unique variation in CD severity during adolescence (Table 3). However, parsimony favored models only including demographic variables and, in the case of ASPD, baseline CD. Intelligence measures were the only additional variables to consistently explain independent variation across legal outcomes, but it is unclear how these results would change in the presence of an adequate measure of SES. We speculate that unmeasured socioeconomic factors may underlie much of the observed heterogeneity in persistence of EP, but our data did not permit further investigation of this hypothesis.
We suggest targets for future research efforts based on our findings. Given that the observed negative associations between vocabulary and legal outcomes may reflect underlying socioeconomic disparities, these associations should be examined after accounting for potential socioeconomic confounds related to educational attainment. Alternatively, a discordant-sibling approach would address this question without fine-grained measurements of socioeconomic factors. Additionally, our evidence is congruent with the taxonomic theory’s notion that the stability of EP is linked to the “extremity” of EP . That is, greater severity of EP as indexed by CD symptoms was associated with negative adult outcomes above all other factors. On the other hand, our results suggest that examining underlying clusters of psychosocial and neurocognitive risk factors in adolescence beyond CD severity and demographic factors may not be useful in predicting adult outcomes. To our understanding, current empirical results remain agnostic with regard to whether persistence versus desistence of EP in adulthood reflects unique clusters of risk factors or relates monotonically to EP severity in adolescence. Particularly, the latent class modeling analyses underlying the majority of trajectory based work are fundamentally exploratory, and a posteriori comparisons of groups of individuals differentiated by these methods are difficult to interpret [7,8]. Future investigations should directly compare confirmatory discrete (i.e., class-based) and continuous (i.e., severity-based) models of persistent EP. Finally, given the increasing availability of large datasets and our growing abilities to link diverse data sources (e.g., court records and electronic health records), prediction-focused research (in contrast to explanation-focused research) is growing in feasibility . Though these approaches are not mutually exclusive, we suggest that a future emphasis on maximizing out-of-sample predictive performance with respect to persistence of EP has the potential to help determine what additional information will be most useful to clinical and legal practitioners working with at-risk adolescents.
Continued data acquisition was aided by the cooperation of Colorado Department of Corrections (Project 11–323).
- 1. Murray J, Farrington DP. Risk factors for conduct disorder and delinquency: key findings from longitudinal studies. Can J Psychiatry. 2010;55: 633–642. pmid:20964942
- 2. Nock MK, Kazdin AE, Hiripi E, Kessler RC. Prevalence, subtypes, and correlates of DSM-IV conduct disorder in the National Comorbidity Survey Replication. Psychol Med. 2006;36: 699–710. pmid:16438742
- 3. Moffitt TE. Adolescence-limited and life-course-persistent antisocial behavior: a developmental taxonomy. Psychol Rev. 1993;100: 674. pmid:8255953
- 4. Bassarath L. Conduct disorder: a biopsychosocial review. Can J Psychiatry. 2001;46: 609–616. pmid:11582821
- 5. Jennings WG, Reingle JM. On the number and shape of developmental/life-course violence, aggression, and delinquency trajectories: A state-of-the-art review. J Crim Justice. 2012;40: 472–489.
- 6. Moffitt TE, Caspi A. Childhood predictors differentiate life-course persistent and adolescence-limited antisocial pathways among males and females. Dev Psychopathol. 2001;13: 355–375. pmid:11393651
- 7. Skardhamar T. Distinguishing facts and artifacts in group-based modeling. Criminology. 2010;48: 295–320.
- 8. Walters GD. The latent structure of life-course-persistent antisocial behavior: Is Moffitt’s developmental taxonomy a true taxonomy? J Consult Clin Psychol. 2011;79: 96. pmid:21171739
- 9. Fairchild G, Goozen SH, Calder AJ, Goodyer IM. Research review: evaluating and reformulating the developmental taxonomic theory of antisocial behaviour. J Child Psychol Psychiatry. 2013;54: 924–940. pmid:23826820
- 10. Moffitt TE, Caspi A, Harrington H, Milne BJ. Males on the life-course-persistent and adolescence-limited antisocial pathways: Follow-up at age 26 years. Dev Psychopathol. 2002;14: 179–207. pmid:11893092
- 11. Isen J. A meta-analytic assessment of Wechsler’s P> V sign in antisocial populations. Clin Psychol Rev. 2010;30: 423–435. pmid:20350774
- 12. Manninen M, Lindgren M, Huttunen M, Ebeling H, Moilanen I, Kalska H, et al. Low verbal ability predicts later violence in adolescent boys with serious conduct problems. Nord J Psychiatry. 2013;67: 289–297. pmid:23167535
- 13. Coleman J. The School Years. Routledge; 2012.
- 14. Farrington DP, Hawkins JD. Predicting participation, early onset and later persistence in officially recorded offending. Crim Behav Ment Health. 1991;1: 1–33.
- 15. Brownlie E, Beitchman JH, Escobar M, Young A, Atkinson L, Johnson C, et al. Early language impairment and young adult delinquent and aggressive behavior. J Abnorm Child Psychol. 2004;32: 453–467. pmid:15305549
- 16. Lahey BB, Loeber R, Burke JD, Applegate B. Predicting future antisocial personality disorder in males from a clinical assessment in childhood. J Consult Clin Psychol. 2005;73: 389. pmid:15982137
- 17. Loeber R, Pardini DA, Stouthamer-Loeber M, Raine A. Do cognitive, physiological, and psychosocial risk and promotive factors predict desistance from delinquency in males? Dev Psychopathol. 2007;19: 867–887. pmid:17705906
- 18. Lahey BB, Loeber R, Hart EL, Frick PJ, Applegate B, Zhang Q, et al. Four-year longitudinal study of conduct disorder in boys: patterns and predictors of persistence. J Abnorm Psychol. 1995;104: 83–93. pmid:7897057
- 19. Vermeiren R, de Clippele A, Deboutte D. Eight month follow-up of delinquent adolescents: predictors of short-term outcome. Eur Arch Psychiatry Clin Neurosci. 2000;250: 133–138. pmid:10941987
- 20. Burt SA, Donnellan MB, Iacono WG, McGue M. Age-of-onset or behavioral sub-types? A prospective comparison of two approaches to characterizing the heterogeneity within antisocial behavior. J Abnorm Child Psychol. 2011;39: 633–644. pmid:21298333
- 21. Hyde LW, Burt SA, Shaw DS, Donnellan MB, Forbes EE. Early starting, aggressive, and/or callous-unemotional? Examining the overlap and predictive utility of antisocial behavior subtypes. J Abnorm Psychol. 2015;124: 329–342. pmid:25603360
- 22. Burt SA, Klump KL. Etiological Distinctions between Aggressive and Non-aggressive Antisocial Behavior: Results from a Nuclear Twin Family Model. J Abnorm Child Psychol. 2012;40: 1059–1071. pmid:22466619
- 23. L GH, C SM, E YS, P CR, Hyun RS, K HJ. Genetic and environmental influences on conduct disorder: symptom, domain and full-scale analyses. J Child Psychol Psychiatry. 2005;46: 580–591. pmid:15877764
- 24. Farrington DP. Childhood origins of antisocial behavior. Clin Psychol Psychother. 2005;12: 177–190.
- 25. Henry B, Moffitt T, Robins L, Earls F, Silva P. Early family predictors of child and adolescent antisocial behaviour: who are the mothers of delinquents? Crim Behav Ment Health. 1993;3: 97–118.
- 26. Bergman LR, Andershed A-K. Predictors and outcomes of persistent or age-limited registered criminal behavior: A 30-year longitudinal study of a Swedish urban population. Aggress Behav. 2009;35: 164–178. pmid:19177554
- 27. Rutter M. Family discord and conduct disorder: Cause, consequence, or correlate? J Fam Psychol. 1994;8: 170.
- 28. Greene RW, Biederman J, Zerwas S, Monuteaux MC, Goring JC, Faraone SV. Psychiatric Comorbidity, Family Dysfunction, and Social Impairment in Referred Youth With Oppositional Defiant Disorder. Am J Psychiatry. 2002;159: 1214–1224. pmid:12091202
- 29. Woolfenden SR, Williams K, Peat JK. Family and parenting interventions for conduct disorder and delinquency: a meta-analysis of randomised controlled trials. Arch Dis Child. 2002;86: 251–256. pmid:11919097
- 30. McCart MR, Priester PE, Davies WH, Azen R. Differential Effectiveness of Behavioral Parent-Training and Cognitive-Behavioral Therapy for Antisocial Youth: A Meta-Analysis. J Abnorm Child Psychol. 2006;34: 525–541. pmid:16838122
- 31. Grove AB, Evans SW, Pastor DA, Mack SD. A meta-analytic examination of follow-up studies of programs designed to prevent the primary symptoms of oppositional defiant and conduct disorders. Aggress Violent Behav. 2008;13: 169–184.
- 32. Tuvblad C, Narusyte J, Grann M, Sarnecki J, Lichtenstein P. The Genetic and Environmental Etiology of Antisocial Behavior from Childhood to Emerging Adulthood. Behav Genet. 2011;41: 629–640. pmid:21431322
- 33. Hopfer CJ, Crowley TJ, Hewitt JK. Review of Twin and Adoption Studies of Adolescent Substance Use. J Am Acad Child Adolesc Psychiatry. 2003;42: 710–719. pmid:12921479
- 34. Stein A, Pearson RM, Goodman SH, Rapa E, Rahman A, McCallum M, et al. Effects of perinatal mental disorders on the fetus and child. The Lancet. 2014;384: 1800–1819.
- 35. Hyde LW, Waller R, Trentacosta CJ, Shaw DS, Neiderhiser JM, Ganiban JM, et al. Heritable and Nonheritable Pathways to Early Callous-Unemotional Behaviors. Am J Psychiatry. 2016;173: 903–910. pmid:27056607
- 36. Button TM, Corley RP, Rhee SH, Hewitt JK, Young SE, Stallings MC. Delinquent peer affiliation and conduct problems: A twin study. J Abnorm Psychol. 2007;116: 554. pmid:17696711
- 37. Weiss B, Caron A, Ball S, Tapp J, Johnson M, Weisz JR. Iatrogenic effects of group treatment for antisocial youths. J Consult Clin Psychol. 2005;73: 1036. pmid:16392977
- 38. Anderson KG, Tapert SF, Moadab I, Crowley TJ, Brown SA. Personality risk profile for conduct disorder and substance use disorders in youth. Addict Behav. 2007;32: 2377–2382. pmid:17408870
- 39. Hopfer CJ, Stallings MC, Hewitt JK, Crowley TJ. Family transmission of marijuana use, abuse, and dependence. J Am Acad Child Adolesc Psychiatry. 2003;42: 834–841. pmid:12819443
- 40. Trim RS, Worley MJ, Wall TL, Hopfer CJ, Crowley TJ, Hewitt JK, et al. Bivariate trajectories of substance use and antisocial behavior: associations with emerging adult outcomes in a high-risk sample. Emerg Adulthood. 2015;3: 265–276. pmid:26889401
- 41. Wechsler D. WAIS-R manual: Wechsler adult intelligence scale-revised. Psychological Corporation; 1981.
- 42. Heaton RK, Grant I, Matthews CG. Comprehensive norms for an expanded Halstead-Reitan battery: demographic corrections, research findings, and clinical applications; with a supplement for the Wechsler Adult Intelligence Scale-Revised (WAIS-R). Psychological Assessment Resources; 1991.
- 43. Wechsler D. WAIS-III, Wechsler adult intelligence scale: Administration and scoring manual. Psychological Corporation; 1997.
- 44. Wechsler D. Wechsler intelligence test for children (WISC-IV). San Antonio TX Psychol Corp. 2003;
- 45. Moos RH, Moos BS. Family environment scale manual. Consulting Psychologists Press; 1994.
- 46. Plomin R, McClearn G, Pedersen NL, Nesselroade JR, Bergeman C. Genetic influence on childhood family environment perceived retrospectively from the last half of the life span. Dev Psychol. 1988;24: 738.
- 47. Robins L, Cottler L, Babor T. The WHO/ADAMHA CIDI-SAM interview. World Health Organiziation Geneva. 1986;
- 48. First MB. Diagnostic and statistical manual of mental disorders. DSM IV-4th Ed APA. 1994; 1994.
- 49. Corley RP, Stallings MC, Hewitt JK, Young SE, Zeiger J. Robustness of genome scan results on adolescent dependence vulnerability from the Colorado CADD. Aix-en-Provence, France; 2004.
- 50. Elliott DS, Huizinga D, Menard S. Multiple problem youth: Delinquency, drugs and mental health. 1989;
- 51. Fisher PW, Shaffer D, Piacentini JC, Lapkin J, Kafantaris V, Leonard H, et al. Sensitivity of the Diagnostic Interview Schedule for Children, (DISC-2.1) for specific diagnoses of children and adolescents. J Am Acad Child Adolesc Psychiatry. 1993;32: 666–673. pmid:8496131
- 52. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): Description, Differences From Previous Versions, and Reliability of Some Common Diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39: 28–38. pmid:10638065
- 53. Robins LN, Helzer JE, Croughan J, Williams JB, Spitzer RL. NIMH diagnostic interview schedule: Version III. Rockv MD Natl Inst Ment Health. 1981;
- 54. Robins L, Cottler L, Bucholz K, Compton W, North C, Rourke K. Diagnostic Interview Schedule for the DSM-IV (DIS-IV) Washington University School of Medicine; St. Louis MO. 2000;
- 55. R Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna, Austria; 2018. Available: https://www.R-project.org/
- 56. Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, et al. Stan: A probabilistic programming language. J Stat Softw. 2017;76.
- 57. Stan Development Team. RStan: the R interface to Stan. 2018; Available: http://mc-stan.org/
- 58. Bürkner P-C. brms: An R Package for Bayesian Multilevel Models Using Stan. J Stat Softw. 2017;80: 1–28.
- 59. Fox J. polycor: Polychoric and Polyserial Correlations [Internet]. 2016. Available: https://CRAN.R-project.org/package=polycor
- 60. Bates D, Maechler M, Bolker B, Walker S. lme4: Linear mixed-effects models using Eigen and S4. R Package Version. 2014;1: 1–23.
- 61. Christensen RHB, Christensen MRHB. Package ‘ordinal.’ Stand. 2015;19: 2016.
- 62. Rubin DB. Inference and missing data. Biometrika. 1976;63: 581–592.
- 63. White IR, Daniel R, Royston P. Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Comput Stat Data Anal. 2010;54: 2267–2275. pmid:24748700
- 64. mice: Multivariate Imputation by Chained Equations in R | van Buuren | Journal of Statistical Software. https://doi.org/10.18637/jss.v045.i03
- 65. Morris TP, White IR, Royston P. Tuning multiple imputation by predictive mean matching and local residual draws. BMC Med Res Methodol. 2014;14: 75. pmid:24903709
- 66. Rhee SH, Waldman ID. Genetic and environmental influences on antisocial behavior: A meta-analysis of twin and adoption studies. Psychol Bull. 2002;128: 490–529. pmid:12002699
- 67. Wall TD, Frick PJ, Fanti KA, Kimonis ER, Lordos A. Factors differentiating callous-unemotional children with and without conduct problems. J Child Psychol Psychiatry. 2016;57: 976–983. pmid:27133383
- 68. Andershed H, Colins OF, Salekin RT, Lordos A, Kyranides MN, Fanti KA. Callous-Unemotional Traits Only Versus the Multidimensional Psychopathy Construct as Predictors of Various Antisocial Outcomes During Early Adolescence. J Psychopathol Behav Assess. 2018;40: 16–25. pmid:29576680
- 69. Fanti KA, Colins OF, Andershed H, Sikki M. Stability and change in callous-unemotional traits: Longitudinal associations with potential individual and contextual risk and protective factors. Am J Orthopsychiatry. 2017;87: 62–75. pmid:27046166
- 70. Colins OF, Andershed H, Salekin RT, Fanti KA. Comparing Different Approaches for Subtyping Children with Conduct Problems: Callous-Unemotional Traits Only Versus the Multidimensional Psychopathy Construct. J Psychopathol Behav Assess. 2018;40: 6–15. pmid:29576679
- 71. Colins O, Vermeiren R, Vahl P, Markus M, Broekaert E, Doreleijers T. Psychiatric Disorder in Detained Male Adolescents as Risk Factor for Serious Recidivism. Can J Psychiatry. 2011;56: 44–50. pmid:21324242
- 72. Frick PJ. Current research on conduct disorder in children and adolescents. South Afr J Psychol. 2016;46: 160–174.
- 73. Dodge KA, Pettit GS. A Biopsychosocial Model of the Development of Chronic Conduct Problems in Adolescence. Dev Psychol. 2003;39: 349–371. pmid:12661890
- 74. Yarkoni T, Westfall J. Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspect Psychol Sci. 2017;12: 1100–1122. pmid:28841086