Data and cohort.

Our data came from the National Longitudinal Survey of Youth 1997 (NLSY97), an annually administered panel survey of 8,984 people aged 12 to 18 years at first survey administration in 1997 [27]. This initial cohort comprised a nationally representative cohort of 6,748 participants and a supplemental cohort of 2,236 participants designed to oversample black and Hispanic respondents, both of which were included in this study. NLSY97 had an overall retention rate of 83.2% across the 2 combined cohorts by survey administration round 14, the round from which we took all of our later outcomes, except for assets at age 30 years, as not all individuals turned 30 years on the same round. Our study cohort included NLSY97 participants who were interviewed in round 4 in 2000—the round when individuals first completed the Mental Health Inventory-5 (MHI-5)—who were aged 17 years or younger, and who completed all 3 questions in the psychological distress subscale of the MHI-5.

Exposure.

Table 1 summarizes the variables of interest. Our exposure variable came from the MHI-5 in survey round 4. The MHI-5 included 5 items on a four-point Likert scale assessing past-month mood based on the presence of psychological wellbeing and the absence of psychological distress [28]. We used the three-item psychological distress subscale of the MHI-5 in which scores range from 0 to 9, with higher scores indicating less distress. The psychological distress subscale was the only scale or subscale of the MHI-5 that was validated as predictive of clinically significant anxious or depressive symptoms and diagnosable anxiety and depression disorders at a score of 3 or less [28]. We tested a dichotomous variable of those with a score of less than or equal to 3 and those with a score of 4 or greater as well as a categorical variable of population mean psychological distress scores at values of 6, 7, and 8. The mean score in the cohort population was approximately 6.9 and scores of 7 and 8 represented potential population means in response to an effective intervention strategy. In this way, binary and categorical exposure measures facilitated comparison with results from prevention and early intervention studies on adolescent mental health, which often report impacts as changes in the population mean on a symptom rating scale (e.g., standardized mean difference) or as relative risk of incidence or remission of clinically significant challenges.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Variables included in the analysis.

https://doi.org/10.1371/journal.pmed.1004506.t001

Outcomes.

We included 5 sets of outcome variables: labor supply, income, education, health, and health insurance coverage. Each of these variables was self-reported and captured in round 14, in 2010, 10 years after the exposure variable was captured, except for financial assets, which was captured at age 30 years for all participants. We selected a timeframe of 10 years because this is the same window over which many budget analysis entities, including the CBO, score the impacts of policies. We did not apply a discount rate throughout this study, as the goal is to estimate the actual economic impacts, rather than to assess the benefits in a decision context where considerations relevant to a discount rate are present, such as opportunity cost, time preference, or diminishing marginal utility [29]. All dollars were reported as 2022 USD, adjusted for inflation using CPI-U (Consumer Price Index–Urban).

Labor supply was measured in 3 ways: past-year labor force participation in the civilian labor force, number of weeks worked in any job in the past year in the civilian labor force, and number of hours worked in any job in the past year in the civilian labor force. Labor force participation in the civilian labor force was determined by whether an individual reported working any weeks at any job in the civilian labor force or reported looking for work. These metrics provide key inputs into the labor supply components of macroeconomic models.

Income included measures of both total self-reported income from working in any job in the civilian labor force in the past year and total self-reported financial assets at age 30 years. For both, respondents could either report a specific dollar amount or a range of dollar amounts. When respondents reported their income or assets as a range, we used the central point in the range as the actual value. The income component focused only on wages from employment rather than total income, which can be an input in several different pieces of a macroeconomic model. The assets are relevant to estimating changes in the capital supply.

For our education outcomes, educational attainment was recoded as a series of binary variables of at least high school completed, at least some college completed, at least college graduation, and more than college completed.

Health included self-rated overall health and psychological distress. Self-rated overall health was assessed using a single-item measure with levels of excellent, very good, good, fair, and poor [30]. Psychological distress was measured in the same way as the independent variable with the MHI-5, although treated as a continuous variable with scores from 0 to 9.

Health insurance was captured through a self-report of primary health insurance over the past year and was dichotomized between Medicaid/Medicare and any other coverage, including no insurance. The federal government would only pay directly for healthcare utilized while an individual is insured by Medicaid or Medicare (as a result of disability, given the young age of this population), so we separated out these insurance types. In models of the impact of investments in mental health on later public spending, the distinction between healthcare costs accrued under public health insurance coverage as opposed to private would be an important consideration.

Explanatory variables.

To account for potential confounding in the relationship between adolescent psychological distress and later outcomes, we included explanatory variables in domains of basic demographics, neighborhood environment, family resources, family processes, physical health, school quality, and academic skills, driven by a theory of how mental health problems in adolescence relate to later labor outcomes and human capital formation more broadly [31]. NLSY97 round 1 measured our explanatory variables, when the most robust data in the cohort were collected. By using variables from 3 years prior to when the exposure variable was measured, we captured the developmental context for adolescents (e.g., the family arrangement they had during formative years) but not disruptions that may have occurred between rounds 1 and 4 (e.g., loss of a family member). These disruptions represent potential intervention targets that we may not want to control out of the final estimate (e.g., impacts on mental health and later outcomes from the loss of a family member could be moderated through interventions that focus on coping after bereavement).

Basic demographics included age, race and ethnicity, gender, region of the country, urbanicity, and month of birth. Race and ethnicity were coded together in the NLSY97 as black, Hispanic, mixed race (non-Hispanic), and non-black/non-Hispanic, based on caregiver report. While this coding missed different racial and ethnic interactions, it still provided valuable information about an individual’s racial and ethnic identity. Region of the country comprised 4 options based on census categories of Northeast, North Central, South, and West. Urbanicity allowed for 3 options: not in a metropolitan statistical area (MSA), in an MSA but not in the central city, and in an MSA and in the central city. We also included birth month to account for any effects that this may have on outcomes, such as the impacts of school entry at slightly different developmental stages [32].

Neighborhood environment included measures of peer behaviors and of neighborhood safety. The peer behavior variables asked the youth questions about “what percentage of kids in your grade…” and included smoking, engagement in extracurricular activities, plans to go to college, or engagement in delinquency. The neighborhood safety variable was an index that included questions that were both youth- and interviewer-reported on the availability of heat and electricity, quality of the buildings in the neighborhood, quality of the interior of the home, concerns for safety, and hearing gunshots. The basic demographic and neighborhood environment variables influenced exposures and access to opportunities across development that impact psychological distress or later economic outcomes, such as the effects of racism, community violence, or peer norms.

Family resources included variables that captured household circumstances and caregiver education, health, and income. The family arrangement variable identified the caregivers present in the youth’s life, including whether one or both biological parents were in the home or whether caregivers were adoptive, grandparents, or other individuals. Household size indicated the number of siblings and other individuals across whom resources (both financial and attentional) may be split. The age of the mother when she had her first child and the age of the mother when she had the respondent youth were also included, which offered information about resources for childrearing and birth order. Because some values of maternal birth age appeared nonsensically high or low, we recoded the variable into a factor with levels of younger than 18, 18–23, 24–29, and older than 30 to reduce bias from inaccurate responses. Caregiver(s) health was reported by a single respondent for both caregivers if they had a partner and was based on the same self-rated assessment of overall health as the health outcome variable. Caregiver(s) highest level of education was similarly reported by a single caregiver. In the NLSY97, the caregiver respondent was selected by first interviewing an individual over the age of 18 to create a household roster and then selecting a caregiver from the available adults based on a prioritized list of options, with biological mother being the first choice and a non-relative youth who acts as a father figure as a 13th choice.

Caregiver(s) (e.g., parent) income was computed based on questions about sources and amounts of annual income, as reported by a responding caregiver. As with the income-related outcome variables, if respondents reported the values as a range, we used the central point in the range. In calculating total annual income, we included income from farms or businesses, which could have negative values. When respondents reported income from farms or businesses as a loss but did not specify a value (which they were able to do if they reported income as a range), we used the median value of losses from respondents who did specify an amount. Caregivers also reported the number of months in the past year they received some form of government assistance, such as the Supplemental Nutrition Assistance Program, excluding unemployment insurance or worker’s compensation. Caregivers also answered whether the youth had experienced hard times growing up, such as having to “live in a place without water or electricity, or in a homeless shelter.” We also incorporated an index of whether the youth had an enriching environment, which included youth-reported questions on the presence of a computer in the home, a dictionary in the home, and engagement in extracurricular activities. These family resource variables captured exposures and access to opportunities and capacity for investment by adults in a child’s life, which is associated with both adolescent psychological distress and later economic outcomes.

Family process variables further built on the family resource variables by including the day-to-day developmental context for a child, which shapes a child’s psychological functioning as it relates to both psychological distress and later economic outcomes. An index of family routines measured the number of days each week the youth reported that they ate dinner with their family, housework got done, they did something fun as a family, or they did something religious as a family. We also included a youth-reported index of parental monitoring by any residential caregivers, which asked how much the caregiver knows about the youth’s close friends, the youth’s close friends’ parents, who the youth was with when not at home, and the youth’s teachers and school life. Both caregivers and youth reported on whether limits are set by caregivers, the youth, or jointly for how late the youth stayed out at night, who they could hang out with, and what media they were allowed to consume, and caregivers and youth also both reported on whether the youth broke these limits. Parenting style was coded as uninvolved, permissive, authoritarian, and authoritative, based on how youth answered 2 questions about how supportive and how strict each caregiver is. Two youth-reported variables captured the extent to which residential caregivers supported one another, if 2 residential caregivers are present, through a five-point Likert scale on how often each person in the relationship screamed when angry, compromised when in disagreement, expressed affection, insulted or criticized ideas, encouraged for important issues, or blamed for their problems.

A child’s other health challenges could have also caused mental health problems and impaired later health and economic success. We included variables on caregiver reports of the presence of different types of birth defects, chronic conditions (e.g., diabetes), learning disabilities, and intellectual disabilities. We did not include a variable of self-rated health for youth because evidence indicates that it also captures psychological distress, hence poor self-rated health can be thought of as the consequence of psychological distress [33].

We took school quality variables from four-point Likert scale measures about the extent to which a youth agrees with different statements. These statements included whether the teachers were good, whether the teachers were interested in the students, whether students disrupted learning, whether students were graded fairly, whether students cheated on tests, whether the discipline was fair, and whether the youth felt safe at school. These variables captured not only aspects of the quality of educational opportunities available to the child, but also the child’s sense of attachment to these opportunities, both of which have important implications for later development. Finally, we measured academic skills through a youth report of average grades in eighth grade—a time point several years before the exposure was measured when children are aged approximately 12 to 13 years. We coded grades in eighth grade as a factor with levels of mostly Cs and Ds (almost unsatisfactory or unsatisfactory performance), mostly Bs and Cs (satisfactory performance), mostly Bs (above average performance), and mostly As (exceptional performance). We also included a variable of youth-reported usual number of weeknights spent doing homework to incorporate a measure of motivation or caregiver influence on academic persistence. Academic skills, attitudes toward school, and psychological distress share an endogenous relationship with one another—psychological distress can impair academic success and school difficulties can promote psychological distress—while both relate to later health and economic outcomes [34]. Accounting for academic skills and attitudes toward school without teasing out the endogeneity may lead to an underestimation of the impact of psychological distress on later health and economic outcomes. We opted to completely account for academic skills and school attitudes, leading to a conservative underestimation of the psychological distress effects.

Analytic approach.

We estimated the average treatment effect (ATE) of the exposure (presence or level of psychological distress) on the outcomes of employment, income, education, and health, accounting for the explanatory variables identified. The ATE is a causal contrast, comparing expected potential outcomes for our population of interest under 2 hypothetical states of the world: if everyone had past-month clinically significant psychological distress at one time point between ages 15 and 17 years, or if no one did, without regard for whether they received treatment or subsequently sought treatment. This can be extended to estimate contrasts between different levels of psychological distress. Although this study did not test the effects of a particular intervention, the results may bear on the potential impacts of a preventive intervention that reduces the risk of developing clinically significant psychological distress in adolescence.

We used targeted maximum likelihood estimation (TMLE) with machine learning algorithms to estimate the ATE [35–37]. This built on the developing literature on the possibility of generating causal evidence with observational data when certain identifying assumptions were met, particularly with the use of machine learning–enabled methods [38–40]. TMLE offered important advantages for addressing our research question with the available data. Traditional regression approaches, such as propensity score or G-computation methods, only produced unbiased results if the exposure or outcome mechanism was consistently estimated and the functional form was correctly specified. TMLE offered doubly robust estimation of an ATE, such that it would produce unbiased estimates if either the exposure mechanism or the outcome mechanism was consistently estimated, conferring an advantage over more traditional regression approaches. TMLE also accommodated machine learning techniques to minimize bias from misspecified regressions, which was critical here insofar as we had many covariates and were not certain of the correct functional form for the outcome and exposure models (e.g., nonlinearities and interactions among them).

TMLE allowed for a causal interpretation when 4 assumptions were met. First, the psychological distress of one individual must not have affected the outcome of others (noninterference), which was likely true for NLSY97 respondents from different households, but approximately 12% of the cohort were respondents from the same household. To examine the extent to which potential interference may have biased the results, we tested the impact of including just one individual per household selected at random in a sensitivity analysis. Second, the level of measured psychological distress exposure must have led to outcomes that arise from the given level of exposure (consistency) [41]. The MHI-5 offered a clinically validated dimensional measure of psychological distress that should have consistently captured the effects of psychological distress on cognitive, affective, and social functioning that impacted later outcomes across individuals [28]. Third, all confounders must have been measured (conditional exchangeability). We included direct or indirect measures of all potential confounders dictated by the theoretical framework, including 50 explanatory variables. We had only indirect measures of cognitive skills or personality, character, motivation, and aspiration, which included academic achievement in eighth grade, number of days doing homework on weekdays, interactions with family (e.g., rule-breaking or -following), and attitudes toward school and teachers. Potential existed for unobserved confounding that fell outside of the theoretical framework and poor measurement of identified confounders, which we discuss further in our limitations section. Finally, all clusters of individuals with different sets of values for the explanatory variables must have been able to develop (or not) psychological distress at all levels, which was true in our cohort because, although the different factors may modify the risk of developing psychological distress, none eliminate the risk completely (positivity).

With TMLE and machine learning algorithms that account for nonparametric and misspecified models, we calculated the ATE for all outcomes. We implemented TMLE in R version 4.0.5 using the tmle and SuperLearner packages, which applied machine learning approaches for the prediction steps to correct for misspecified regressions in TMLE, with 10 folds for cross-fold validation steps when the exposure was treated as a binary variable and 20 folds when the exposure was treated as a categorical variable, given the low variability in the exposure at some levels [42]. When implementing SuperLearner, we tested the glm, gam, glmnet, ranger, and XGBoost libraries, which applied generalized linear models, generalized additive models, elastic net regression, random forests, and extreme gradient boosting, respectively [43]. For comparison, we also ran TMLE with parametric model specifications and without machine learning algorithms. We did not implement corrections for multiple testing, as the focus was on ATE estimation rather than hypothesis testing, and we did not assess statistical significance. We did offer 95% confidence intervals (CIs), which should be interpreted cautiously given the lack of correction for multiple testing. All code and data used in the analysis are available in the supplement as items S1 Code and S1 Data, respectively.

Missing Data. We handled missing data in several ways. For all explanatory and outcome variables with missing data (which result from refusal to answer, uncertainty, invalid skips, or non-interview in the case of outcomes), we created a separate binary variable to capture that missingness, also known as a missingness indicator, and set all missing values to the median value (using the higher value in the case of categorical variables) [44]. Note that because some SuperLearner libraries cannot accommodate unordered factor variables, all unordered factors were recoded as a set of dummy variables, so identifying a median was not an issue (we rounded up when the median was between 2 values). Data missing because a caregiver was not present (e.g., residential father’s parenting style when there is no residential father) were accounted for through both the explanatory variable indicating missingness and through the explanatory variable indicating family arrangement. The variable for missingness in each outcome was passed into the TMLE function as a separate argument (“Delta”), whereas the missingness for each explanatory variable was passed as additional explanatory variables. This approach applies the machine learning capabilities of TMLE to incorporate information about the missing data, rather than relying on model-based specification and related assumptions about the mechanism, as with other common approaches to missing data [45]. For individuals who died by round 14, we coded all outcomes at the lowest level (0 for continuous, null for binary, and the lowest level of an ordered factor) rather than considering the outcomes missing as they otherwise would be in the NLSY97.

For the 18 cases missing exposure data, we excluded these individuals (see Fig 1 for cohort flowchart). Data were missing from the exposure when individuals refused to answer questions in the MHI-5 or responded that they were unsure. Some additional data were also missing because we only included those who were interviewed in round 4. Of the total NLSY97 (including those over the age of 17 years by round 4, who would not be part of our cohort), 10.1% of data was missing as a result of non-interview in round 4. Because each round was fielded over a period of several months (and an 18-month gap occurred between rounds 1 and 2), determining exactly what proportion of the cohort under a certain age could not be interviewed in round 4 is difficult. Approximately 41% of the cohort that was interviewed in round 4 was under age 18, so, if nonresponse was distributed evenly across the population, less than 5% of the total cohort would have been excluded from the subsample analyzed here.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Cohort selection flowchart.

https://doi.org/10.1371/journal.pmed.1004506.g001

Robustness checks.

As noted, interference among siblings in the same household in our cohort may hinder causal inference. We tested the potential impacts of interference from inclusion of siblings in the same household in the cohort by repeating the analysis with siblings in the same household removed at random. We also accounted for possible bias introduced by our use of a subsample. To the extent that our choice to use the subsample of individuals who were interviewed and younger than 18 years during round 4 introduced selection bias that the measured explanatory variables do not account for, our ATE estimates from the subsample will not be reflective of the total sample (and thus the national population, given the representativeness of the NLSY97) [46]. To address this potential concern about the generalizability of the findings, we accounted for the use of a subsample by adapting an approach developed for TMLE [47]. We treated the outcomes for all individuals outside of the subsample (i.e., individuals older than age 17 in round 4) as missing, then calculated the population average treatment effects as contrasts between population mean outcomes at different levels of the exposure for the total sample. We also incorporated NLSY97 sampling weights to promote the generalizability of the estimates to the total US population in the analysis years. We calculated weighted standard deviations using provided sampling weights to determine the confidence intervals of the population average treatment effects, using the Hmisc package in R [48].

Data and sample.

We drew data from several sources. The Current Population Survey (CPS), a program of the US Census Bureau and the Bureau of Labor Statistics that samples approximately 60,000 households each month, provided nationally representative data on employment and education in the United States at different ages and years. The National Survey on Drug Use and Health, a nationally representative survey administered by the US Substance Abuse and Mental Health Services Administration, asked youth whether they experienced a major depressive episode (MDE) in the past year. Our estimates of the relationship between adolescent psychological distress and later outcomes provided the remaining parameters. Our cohorts included the total populations reflected in the 2 surveys in 2022.

Approach to forecasting changes in labor supply over time.

We estimated how prevalence of clinically significant adolescent psychological distress in 2022 would impact US labor supply growth for 2023 to 2032, holding other variables constant, using a simplified regression model: where LS represented the labor supply rate at age a and time t, LS ^o_a,t represented the labor supply rate for those without clinically significant adolescent psychological distress at age a and time t, β^m represented the reduction in labor supply for those with adolescent psychological distress at age a, m represented the proportion of potential labor force participants of age a that experienced adolescent psychological distress, β^s_a represented the reduction in the labor supply rate for those enrolled in education at age a, ED_a represented the educational enrollment rate of those age a without adolescent psychological distress, and S^m_a represented the amount that the educational enrollment rate of those age a is reduced among adolescents with psychological distress.

We estimated S_a using the formula: where ENROLL_a was the proportion of individuals enrolled in education at age a.

We calculated LS_a,t based on the labor supply for people at each age a in the 2022 CPS data. We conducted 2 separate analyses, one that represented labor supply as any work in the past year and one that represented labor supply as the number of hours worked in the past year. For m_a, we used data from the National Survey on Drug Use and Health on prevalence of past-year MDEs. While MDEs and the more general distress measured by the MHI-5 are not identical, this approach may have been able to offer a strong approximation, given that the MHI-5 is predictive of clinical depression. Note that, for simplicity, we did not allow the rate of psychological distress or the rate of enrollment in education to change with time. We calculated β^s_a based on the difference in labor supply rates between those at age a not enrolled in education and those who are. EDUC_a was the proportion of individuals of age a in the CPS data in 2022 that reported being enrolled in education.

Our previous analysis supplied β^m_a and S_m. We used the ATE estimates from our analysis of the exposure as a binary variable. We used Monte Carlo analysis to capture the uncertainty associated with using multiple ATE estimates in a single model, drawing values from a normal distribution determined by the ATE mean and standard error and conducting 1,000 runs for a simulated cohort of 1,000 agents. Because our analysis only gave us an estimate of β^m_a at a single time point, we tested 2 approaches: (1) holding β^m_a constant at the same rate of labor supply impact; and (2) starting at half of the final value and incrementing consistently over the 10-year period. LS^o_a,t and ED_a were then calculated based on these other values. For S_m, we used the ATE for completing at least high school for age 18 years, completing at least some college for ages 19 and 20 years, completing at least college for ages 21 and 22 years, and completing more than college for ages beyond 22 years.

Budget impacts of a hypothetical policy.

We then estimated how a hypothetical policy that altered the prevalence of clinically significant adolescent psychological distress would impact labor supply over 10 years, using the forecasting model. The hypothetical policy would fund the coverage and implementation of high-quality integrated mental health preventive interventions in primary care for adolescents [49,50]. Based on a recent meta-analysis, we estimated that the intervention decreased the relative risk of developing adolescent psychological distress by 0.71 (95% CI [0.51, 0.99]) (a potential overestimation because effect sizes in real-world implementation are often somewhat lower than those obtained in clinical trials) [49]. The preventive services reduced the total proportion of people with clinically significant adolescent psychological distress based on a Monte Carlo analysis that treated the distribution of relative risks from the previously cited meta-analysis as log-normally distributed. As our goal was only to test whether budget impacts were large enough to be worth systematically considering and not to evaluate a particular policy, we made some simplifying assumptions. We assumed that the policy was sufficiently funded to reach 10% of the adolescent population that would otherwise go on to develop depression, either through targeting with screening for risk factors or through broad-scale implementation, and that it was completely implemented in 2022. We did not consider the policy costs, as we focused on the magnitude of later budget effects, not cost-benefit analysis.

To estimate policy impacts, we contrasted baseline labor supply growth projections provided by the CBO for 2022 to 2032 with our projections under the scenario in which the hypothetical policy was implemented in 2022, reducing the prevalence of adolescent psychological distress and thus changing the forecasted trajectory of labor supply [51]. We assumed that the policy was completely implemented at t = 0 (2022 in this case), so that only those at a = 18 would be impacted at t = 1 and so on until t = 10 (or 2032). To determine the total impact on labor supply growth at each t, we included all potential labor force participants between the ages of 18 and 64 years, including those who did not receive the intervention resulting from the policy, due to being older than 17 at the time of implementation.

The CBO published a workbook that allows users to examine how different scenarios—such as different growth rates in labor supply—would impact the budget, based on the CBO’s economic model. We used the contrast in predicted annual labor supply growth between baseline and intervention in the context of CBO’s workbook to estimate the effects of the hypothetical policy on the federal budget [51]. This study was reported as per the Consolidated Health Economic Evaluation Reporting Standards 2022 Statement, available in the S1 Checklist [52].