Search strategy

We conducted a search for documents in French, English, Spanish or German in Web of Science^TM, Embase^®, PsycINFO^®, PubMed^®, Medline^®, and ProQuest Dissertations & Theses Global^TM, from inception to March 8, 2020. Search terms (provided in S1 Text) for four categories were used: youth, substance use, suicidality, and a prospective/longitudinal design. The search terms were combined using the Boolean operators “OR” within each category, and “AND” between categories. Cross-referencing was used by searching the reference lists of relevant articles. The retrieved titles and abstracts from the search were screened for relevance. The full-text article was evaluated for every abstract that was identified as potentially relevant.

To be included in the review, studies had to meet the following eligibility criteria: (1) quantitative empirical study; (2) text in English, French, Spanish or German; (3) includes data on the relation between SUDs and suicidality; (4) those associations are prospective or longitudinal; and (5) participants are 25 years or younger at follow-up. Exclusion criteria included: (1) experimental study; (2) literature review; (3) only a published abstract is available. In this review, “prospective designs” referred to effects that did not control for the initial levels of the outcome. “Longitudinal designs” controlled for initial levels of the outcome either by controlling for it in the analytical model, coding the outcome measure so it would reflect only new cases, or excluding participants who experienced the outcome at baseline from the analyses.

Data extraction

Study characteristics (see Results section) and data were extracted into a Microsoft Excel spreadsheet (available upon request). Information extracted from the studies included study identifiers, sample characteristics, suicide measurement, SUD measurement, design, effect sizes (see next paragraph), and methodological quality (see below). Studies were attributed a study number and a sample number identifying studies that analysed the same sample. Effects sizes, variances, and moderating variables were coded in data files for use in the analyses (see below).

We coded effect sizes and standard errors (SE) from each study using the odds ratio (OR). When studies did not include ORs but provided frequency tables, ORs were calculated from these tables. When articles did not provide necessary statistics, the information was requested from authors via electronic mail. Because the distribution of ORs is skewed, the natural log (lnOR) was used for the analyses and transformed back to ORs for reporting the results. Unadjusted ORs were used, wherever possible, to maximize comparability of effect sizes between studies on both hypotheses and to avoid comparing ORs adjusted for different covariates (see S1 Table for details on covariates in the included effect sizes).

Analyses

A key assumption in traditional meta-analytic approaches is the independence of effect sizes [22], requiring that all effect sizes be from different samples of participants. However, some studies included more than one effect size (i.e., reporting on more than one SUD or suicidality measure, both prospective and longitudinal associations, several follow-up lengths, and/or age at baseline). Furthermore, multiple studies were published for some samples. Common methods to address this issue include ignoring the dependence of the data, computing an average of dependent effect sizes, or only keeping one effect size per study [22]. However, these methods can bias results and lead to a loss of information. Therefore, a three-level meta-analysis was conducted, which allows the non-independent effect sizes to be clustered [23, 24]. Because there were both multiple effect sizes per study and multiple studies per sample, there were four levels to the data, but current meta-analytic methods are limited to three levels. Thus, the analyses were conducted within samples rather than within studies. Accordingly, the three-level random effects model examined three sources of variance: the sampling variance of the observed effect sizes (level 1), the variance between effect sizes from the same sample (level 2), and the variance between the samples (level 3).

Analyses were first conducted separately for samples examining the SPDH and the SSUDH. Random-effects models were estimated to obtain overall pooled effect sizes and their 95% confidence intervals (CIs). The distribution of effect sizes was examined using tests of heterogeneity with the Q statistic [25]. Significant heterogeneity indicates that differences across effect sizes are likely due to factors other than sampling error, such as different study characteristics. Within-sample and between-sample heterogeneity was quantified using the I² index; an I² of 25%, 50%, and 75% respectively reflects a small, medium and large degree of heterogeneity [26]. When there was significant heterogeneity, moderator analyses were conducted to explain variability in effect sizes. The moderators examined included: type of SUD, type of suicidality measure (ideation, attempt; general = measure of ideation and attempt), research design (prospective, longitudinal), type of population (community, at risk, clinical), study quality, follow-up length, proportion of males, proportion of minorities, and year of publication. Because there were few studies examining cannabis use disorder and drug use disorder, they were combined (into a drug use disorder category) for moderation analyses. The moderation analyses in the three-level model are equivalent to a meta-regression analysis, but control for the clustering of effect sizes, and allow examining moderators that vary within and/or between samples. Analyses with cannabis use disorder showed the effect size did not differ significantly from other drug use disorders (see S2 Table). To examine whether the effect size differed according to the direction of association, analyses were then conducted across all studies with the direction of association as a moderator. All analyses were conducted using restricted maximum likelihood estimation and the package metafor [27] in R version 4.0.0 [28] and RStudio version 1.2.5042 [29] on MacOS. The data files and R code are available in S1 Appendix.

Risk of bias appraisal

Methodological quality of the included studies was assessed with the Newcastle-Ottawa Quality Assessment Scale for cohort studies [30]. The scale assesses the quality of studies based on three categories: participant selection (e.g., representativeness), comparability (i.e., whether the final analysis accounted for important confounding factors), and ascertainment of outcome (e.g., adequacy of follow-up). Studies can be classified as having good, fair or poor quality (see S2 Text for the full assessment scale).

Publication bias was assessed using Egger’s regression test [31, 32], which can be conducted within multilevel random-effects models. A significant test indicates publication bias, or significant funnel plot asymmetry. Funnel plots and trim-and-fill analyses [33] cannot be conducted while taking into consideration the clustering of effect sizes but were used as a complement to Egger’s regression test by examining funnel plots with all included effect sizes and conducting trim-and-fill analyses with effect sizes aggregated to one per sample.

Included studies and quality assessment

Fig 1 summarizes the results of the literature search. The search yielded 4379 studies after cross-referencing and removing duplicates. 3872 of these studies were excluded based on the titles and abstracts, resulting in 507 full-texts screened. Among those, 27 studies fulfilled the inclusion and exclusion criteria. The required data could not be obtained for two studies [34, 35], resulting in 25 included studies (see Table 1) clustered in 23 samples, with 15 samples examining the SPDH, seven examining the SSUDH, and one examining both hypotheses. Study characteristics are presented in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flowchart of study selection process.

SPDH = Secondary psychiatric disorder hypothesis (substance use predicts suicidality); SSUDH = Secondary substance use disorder hypothesis (suicidality predicts substance use). See S1 Text for list of full-text articles excluded, with reasons for exclusion.

https://doi.org/10.1371/journal.pone.0255799.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of included studies, classified by sample.

https://doi.org/10.1371/journal.pone.0255799.t001

Complete quality assessments for each study can be found in S2 Text. For the SPDH, 7 of the studies had an overall rating of good quality, while 9 had a low rating. Five of those low-quality ratings were due exclusively to the comparability criteria (i.e., the inclusion of important confounders), with the selection and outcome criteria being rated fair or good. Study quality was slightly higher for the SSUDH, with 4 good quality studies, 3 fair quality studies, and 2 poor quality studies, both due to the comparability criteria.

Secondary psychiatric disorder hypothesis

Analyses for the SPDH included 31 effect sizes nested in 16 samples (see Fig 2 for forest plot of the results). Results showed that SUDs were associated with 2.16 times greater odds (95%CI 1.57–2.97, p < .001) of subsequent suicidal ideations/attempts (95%CI 1.57–2.97, p < .001). There was significant heterogeneity (Q = 103.29, df = 30, p < .001) of large magnitude within-samples (I² = 96%), but not between-samples (I² = 0%). However, moderation analyses (see Table 2) could not explain this heterogeneity, as none of the examined moderators were significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Secondary psychiatric disorder hypothesis forest plot.

Markers represent effect sizes, bars represent the 95% confidence interval. Pooled OR based on three-level random effects model.

https://doi.org/10.1371/journal.pone.0255799.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results of the three-level meta-analytic model for the secondary psychiatric disorder hypothesis (SPDH).

https://doi.org/10.1371/journal.pone.0255799.t002

Secondary substance use disorder hypothesis

Analyses for the SSUDH included 24 effect sizes nested in 8 samples (see Fig 3 for forest plot of the results). Results showed that suicidal ideations/attempts were associated with 2.16 times greater odds of subsequent SUDs (95%CI 1.53–3.04, p < .001). There was significant heterogeneity (Q = 63.56, df = 23, p < .001) of moderate magnitude within-samples (I² = 45%) and of small magnitude between-samples (I² = 21%). Moderation analyses (see Table 3) showed that the type of suicidality measure and follow-up length were significant moderators. The effect size was larger for suicidal attempts than for suicidal ideations (t(21) = -2.29, p = 0.03) and general suicidality (t(21) = -2.86, p = 0.01), while effect sizes for suicidal ideations and general suicidality were not significantly different (t(21) = 1.36, p = 0.19; effect sizes in Table 3). A longer follow-up length was associated with a smaller effect size (see Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Secondary substance use disorder hypothesis forest plot.

Markers represent effect sizes, bars represent the 95% confidence interval. Pooled OR based on three-level random effects model.

https://doi.org/10.1371/journal.pone.0255799.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Results of the three-level meta-analytic model for the secondary substance use disorder hypothesis (SSUDH).

https://doi.org/10.1371/journal.pone.0255799.t003

Comparison of the hypotheses

Studies for both hypotheses were analyzed together with the direction of association as a moderator. Results showed that the effect size did not differ significantly between both hypotheses (F(1,53) = 0.49, p = 0.49).

Publication bias

Egger’s regression test suggested that there was publication bias for both the SPDH (p = 0.06) and the SSUDH (p = 0.001). Funnel plots and trim-and-fill analyses (see S1 Fig) suggested that studies with negative effect sizes were missing for the SPDH (i.e., effect sizes missing on the left side of the funnel plot, suggesting the pooled effect size may be larger due to publication bias), while studies with positive effect sizes were missing for the SSUDH (i.e., effect sizes missing on the right side of the funnel plot, suggesting the pooled effect size may be smaller due to publication bias).

Sensitivity analyses

Sensitivity analyses were conducted to examine whether pooled ORs for each hypothesis were strongly influenced by a single study. Overall (pooled) ORs for each hypothesis were re-estimated, each time leaving out one study. The overall interpretation of the results was not changed by any single study, with ORs all significant and ranging from 1.96 to 2.30 for the SPDH and from 1.96 to 2.39 for the SSUDH (see S3 Table for results of each analysis).

Direction of association and age groups

Because effects were significant in both directions, results suggest that the bidirectional hypothesis may be the best fit for understanding the association between SUDs and suicidality in youth. Indeed, SUDs and suicidality may be exacerbating each other consistently across development. Another possibility is that effects change direction through developmental transactions. Unfortunately, the wide variety of average ages at study onset, age ranges and follow-up lengths in the present meta-analysis did not allow examining age as a moderator. Cross-lagged panel studies would be well-suited to clarify these bidirectional changes over time [61].

Although none of the studies included in the meta-analysis on SUDs examined bidirectional effects, two studies tested such a model using substance use frequency and suicidal ideation. One longitudinal study ranging from ages 13 to 16 years of age found that suicidal ideation at 14 years predicted alcohol use frequency at 15 years, but alcohol use did not predict suicidal ideation [62]. The other longitudinal study examined cannabis use frequency from 15 to 20 years of age and found that weekly cannabis use at 15 years predicted suicidal ideation at 17 years, but suicidal ideation did not predict cannabis use—and the association found was fully explained by alcohol, tobacco, and other drug use [63]. This raises the possibility that the SSUDH would be most relevant in early adolescence, when suicidal thoughts and behaviors are prevalent, but substance use prevalence only begins to increase, whereas the SPDH would be most relevant later in adolescence, when quantity and frequency of substance use, as well as SUDs, are higher. This should be examined in studies with an age range covering the whole adolescent period. Further, as for now, studies on bidirectionality have focused on substance use frequency and suicidal ideation, however, studies including SUDs and suicidal attempts are needed. Finally, it should be noted that although such analyses can clarify the directionality of effects, they do not demonstrate causality.

Moderators

Several moderators were examined in the present meta-analysis. Neither models were moderated by the type of SUD (i.e., general SUD, alcohol use disorder, drug use disorder). However, the effects reported for each substance could be considered general substance use effects as most studies reviewed did not control for the common variance across substances or examine specific effects. Only one study examined substance specific effects, showing that cannabis use disorder was associated with increased odds of suicide attempt even when taking into account alcohol use [40]. Thus, more studies are needed to clarify general from substance-specific effects, as well as the effects of poly-substance use. The effect size of suicidality predicting SUDs was larger for suicidal attempts than for suicidal ideations for the SSUDH, but not for the SPDH. Furthermore, a longer follow-up length was associated with a smaller effect size for the SSUDH, but not for the SPDH. However, there were longer follow-up lengths for the SSUDH (ranging from 21 to 228 months) than for the SPDH (ranging from 3 to 168 months). Accordingly, these results suggest that the effect of SUDs predicting suicidality is longer lasting than the effect of suicidality predicting SUDs, and/or the interinfluence between SUDs and suicidality gets smaller after several (7+) years. Longitudinal studies examining bidirectional effects between SUDs and suicidality at different intervals could help clarify how long the prediction of one by another lasts.

Significant moderators explained some, but not all, variance in effect sizes for the SSUDH. Large within-sample heterogeneity could not be explained for the SPDH, as no significant moderators were found. This suggests that some factors that vary within samples but could not be examined in this meta-analysis may moderate the association between SUDs and suicidality and should be investigated in future empirical studies. For example, although the percentage of males in the samples did not moderate results in this meta-analysis, effects may still differ between sexes when examined separately. While odds ratios by sex were not available in enough studies to be examined here, moderation by sex was examined in a few studies. For the SPDH, only one study examined moderation by sex, but SUDs were not associated with later suicide attempts for either sexes [53]. Studies on substance use frequency found that the prediction of suicidality by substance use frequency was significant for both sexes [64–67], and one study found that early onset cannabis and inhalant use (i.e., use before 15 years) were associated with suicidal ideations and attempts at 19 years in girls, but not in boys [68]. For the SSUDH, one study found that suicidal ideation and attempts before 18 years were associated with SUDs between 18 and 25 years for girls, but not for boys [45], while one study found the association between suicidal ideations at 15 years and drug use disorders at 18 years was not moderated by sex [57]. Thus, the moderation of these associations remains unclear and may differ by type of substance use, but results suggest that sex is an important moderator to consider in future studies. Likewise, no studies examined moderation by gender, which should also be considered in future studies, especially considering that measures of gender (i.e., of masculinity and femininity) have been shown to account for differences which may have otherwise been attributed to sex if gender had not been examined in the prediction of drug use in emerging adulthood [69].

Similarly, ethnic minority status in the samples did not moderate results in this meta-analysis, but these analyses were particularly low powered (9 studies for the SPDH, 3 studies for the SSUDH), and effects may still differ between specific ethnic groups when examined separately. Studies on SUDs did not examine this moderation, which was only tested in one study on substance use frequency and the SPDH hypothesis that found no moderation by ethnicity, with alcohol, cannabis, and other drug use frequencies predicting suicide attempts in Black, Hispanic, and white youth. Importantly, over half of the studies reviewed did not provide any information about the ethnic distribution of their sample. While studies examining the moderation of the association between SUDs and suicidality by ethnicity are needed, all studies should provide these demographics to allow for proper interpretation of the results and their generalizability.

In terms of other moderators, one of the reviewed studies also found that alcohol use disorder interacted with the quality of father-child relationship to predict suicidal ideation [41], and other family factors could be examined in future studies (e.g., parenting style, parental monitoring, parent-child communication [70, 71]). Other correlates of substance use and suicidality that could be moderators of these associations but have not been examined in the studies reviewed notably include peer affiliations [72, 73], personality [71, 74], and sexual minority status [75].

Limitations

The present review is, to our knowledge, the first meta-analysis comparing the direction of the association between SUDs and suicidality in youth. It has important strengths, including a comprehensive systematic literature search and multilevel meta-analysis, which allowed multiple effect sizes per sample to be included, made it possible to examine some potentially important moderators, and can yield more precise average effect size estimates. Still, some limitations should be noted.

First, despite the multilevel approach, some moderation analyses did not include all studies, and were thus underpowered. Many moderators included levels with fewer than five studies, which is not ideal for ensuring adequate coverage and accurate results [22]. Once more studies on the topic are published, future meta-analyses should conduct more reliable and well-powered moderation analyses. Second, over half of the studies for the SPDH and a fourth of the studies for the SSUDH were of poor quality according to the Newcastle-Ottawa criteria [30]. The lower quality of studies for the SPDH may have affected the robustness of the results comparing the directional hypotheses. However, moderator analyses indicated that results did not significantly differ between studies of good and poor quality. Still, the evaluation of study quality highlighted the need for studies of high quality, especially studies controlling for potential confounders, and studies including other substance use to disentangle substance-specific effects. Further, a majority of studies had attrition rates over 10% and used deletion techniques, and future studies should aim to use appropriate missing data treatments to avoid bias when attrition and missing data are present [76, 77]. Third, although studies from various countries were included, the majority of studies were from the western world and there were no studies from South America and Africa, which may affect the generalizability of the results. One study from South Africa found that alcohol use frequency in 10-18-year-olds did not significantly predict suicidal ideation/attempts one year later [78]. However, more studies from this region, including studies on SUDs, would be needed to properly assess whether associations are similar or different from those found in other countries. Finally, this meta-analysis only included published studies written in French, Spanish, German, or English, possibly increasing the risk of publication bias in the results. Analyses suggested that publication bias was present, and differential bias between the SPDH and the SSUDH may have impacted the moderation analyses comparing the two hypotheses.

Clinical implications

Despite its limitations, the results of the present meta-analysis suggest that the associations between SUDs and suicidality are bidirectional. Although caution is warranted in the interpretation of results, and replication of findings in future studies specifically designed to examine bidirectionality is warranted, these findings have potential but clear implications for clinical practice and policy.

First, findings support the importance of detecting and treating SUDs and other mental health problems early to prevent co-morbidity, which is associated with poorer treatment outcomes [20]. In addition to assessing substance use as a potential etiological factor in youth who present with mental health problems, clinicians should also be watchful for the initiation of substance use by their patients, which they might not be as attentive to as reflected by the publication bias towards the SPDH. While a recent meta-analysis suggested that public health programs should target adolescent cannabis use to prevent suicidality [14], the present meta-analysis highlights that these prevention efforts should target all substance use, in addition to also aiming to prevent the exacerbation of substance use by suicidality itself and its root causes. A cross-influence of SUDs with suicidality could notably be prevented by targeting the mechanisms thought to explain their associations, for example by increasing coping and problem-solving skills [17, 18]. Some programs targeting these skills have already been shown to prevent both substance use [79, 80] and suicidality [81, 82], and may also prevent co-morbidity when youth already experience one or the other, which could be examined in future experimental research.