Overview of the Brisbane longitudinal twin study (BLTS)

Ethical approval was obtained for the Brisbane Longitudinal Twin Study (BLTS) from the Human Research Ethics Committee at the Queensland Berghofer Institute of Medical Research (references: EC00278 and P1212). The study follows Strengthening the Reporting of Observational Studies in Epidemiology guidelines (S1 Checklist). Additional descriptions of the protocol, procedures and data collection processes are provided in the (S1 File). Further details and research findings (including health service use and treatment profiles) are also available in other recent cohort publications [8, 11–13]. Here we provide only a precis of key information relevant to the current study.

Essentially, the BLTS is a community-based cohort study of twins and non-twin siblings, recruited via media appeals and word of mouth from 1992 onwards. Ethnically, the cohort reflects the population structure of the greater Brisbane area at the time of recruitment, with most participants of European ancestry and minorities of predominantly Asian ancestry [11, 12]. Individuals were eligible to join the cohort from age 12 onwards with written parental consent. However, the current article focuses only on data collated between the ages of about 15–25 years when participants completed a set of mental health and family history assessments. Repeated self-rating and interview assessments were performed during this peak age range for the onset of mood and psychotic disorders, and individuals who missed a follow-up could be invited to participate at the next wave [8, 11]. Due to the nesting of the data collection within a longitudinal framework, findings from recent cross-sectional assessments can be linked to those from earlier waves [11].

Cohort eligible for this study

De-identified individual data were extracted from the BLTS dataset according to the following inclusion criteria:

1. the cohort member had completed all self-ratings of psychopathology at a follow-up undertaken between age 15–19;
2. the Composite International Diagnostic Interview (CIDI) [13] and family history of major mental disorders [14] assessments were available from the 19Up or 25Up follow-ups;
3. age at onset of self-rated psychopathology, e.g. sub-threshold syndromes, and/or of the first full-threshold CIDI episode was recorded (or could be estimated from information available in the dataset).

Our aim was to explore symptoms and subthreshold syndromes that represented antecedents, not consequences, of any full-threshold syndromes identified. As such, individuals were excluded from the analyses if the age at completion of the CIDI assessment and/or estimated age at onset of any full-threshold syndrome preceded the age at completion of the symptom self-ratings. Likewise, we excluded individuals if insufficient data were recorded and/or ages at onset could not be estimated or specific age data were missing.

Using the above criteria, we identified that 1815 individuals (out of 2540 potential participants) met all the eligibility criteria. As shown in S1 Table, the excluded group hardly differed from the included cohort, except that the former was slightly older (mean age 26.9 versus 26.4 years; p<0.03).

Assessments

1) Demography: key characteristics were recorded (see results).

2) Self-Rated Psychopathology: The box lists the 23 mood and psychotic symptoms/experiences included in three self-rating scales (Hypomanic-Like Experiences: HMLE; Psychotic-Like Experiences: PLE; Depressive-Like Experiences: DLE) [9]. The test-retest reliability of the self-ratings is good (inter-class correlations = 0.8) and the inter-rater reliability (weighted kappas) is about 0.75 for each subthreshold syndrome [15–19]. These self-report instruments were chosen as they are widely used to evaluate any psychopathology experienced by young people and the ratings can be examined from several perspectives (S1 File gives details of symptoms and rating scale properties, etc). First, the total number of items endorsed across all three rating scales can be used to estimate overall symptom load (Sx_Load). Second, item endorsements on a particular scale can be used to determine if an individual has experienced a pre-defined subset of symptoms that represent a sub-threshold syndrome (SubT) [8, 9]. Also, in this study, we used machine learning (see statistics section) to examine patterns of item endorsements across all three ratings to examine whether there is a cluster of symptoms that may represent a clinical phenotype for trans-diagnostic transition to caseness.

Box. Description of self-rated symptoms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

https://doi.org/10.1371/journal.pone.0252550.t001

3) CIDI caseness: We applied established algorithms to CIDI assessment data to determine the presence or absence of a range of DSM-IV disorders and their age at onset [13]. For this study, we extracted data regarding the presence or absence of major depressive, hypo/manic, and/or psychotic syndromes that met CIDI criteria for caseness (see S1 File). Whilst we noted the co-occurrence of these syndromes, we did not explore diagnostic subgroupings separately (as this is a trans-diagnostic study).

4) Family History of Mood and Psychotic Disorders: Psychiatric histories in 1st and 2nd degree family members were identified using an online assessment based on the Family History Screen [14]. As with CIDI ratings, this study used the dichotomous ratings regarding the presence or absence of any family history (FH) of mood and/or psychotic disorders.

Statistical analyses

Analyses were undertaken using RStudio (version 3.5.3) and SPSS (version 27.0). Additional details regarding statistical procedures are provided in the (S1 File); all reported analyses consider familial clustering [8, 11, 12].

Descriptive statistics are reported as means with standard deviations (M; SD), medians with inter-quartile ranges (IQR) and counts and percentages (for categorical variables). To explore the utility of FH, Sx_Load and SubT in differentiating between cases and non-cases, we used Bayes’ theorem to estimate the positive and negative predictive values (PPV; NPV) and their 95% confidence intervals (95% CI) [20]. Overall accuracy of each classification is reported as Area under the Curve (AUC) with 95% CI. The NPV, PPV and AUC were estimated for each variable separately (FH; Sx_Load; SubT) and then for combinations of variables (FH with Sx_Load; FH and SubT).

The above analyses used data about exposure to previously known antecedents of full-threshold disorders that are well described in the existing research literature (i.e., FH, Sx_Load, SubT). However, to examine whether a particular group of self-rated trans-diagnostic symptoms can discriminate cases from non-cases, we employed an analysis that generated a cluster of symptoms de novo. We used a machine learning approach to perform Chi-Square Automatic Interaction Detector (CHAID) analyses [21, 22]. We undertook two planned analyses of self-rated symptoms that allowed reporting of findings allow comparison with the previous analyses. The initial CHAID model included all 23 self-rated symptoms whilst the second model included FH as well as the self-rated symptoms. However, the latter analysis should be regarded as exploratory as preferably it should be undertaken in an entirely independent cohort.

Additional details about CHAID are provided in S1 File, but it is important to note that we applied recommended approaches for minimizing over-fitting of the models and for optimizing reliability and validity of the analysis. First, we created a single ‘variable of influence’ that accounted for individual differences in age, sex, and familial clustering (ie. age above or below the cohort median; male versus female; monozygotic versus other sibling status). This variable ensured that the CHAID model was adjusted for these characteristics (but avoided the need to over-ride the automatically determined maximum tree depth). Second, all CHAID analyses were cross-validated using train-test split evaluation with 50% split (i.e., the model was trained on a ~50% random sample of the cohort and then tested on the remaining half) and, in the results section, we report findings on the test set. We only report findings for the test models (as this is less vulnerable to over-fitting and offers a better estimate of validity and of performance of the machine learning algorithm). Third, we set a conservative level of statistical significance (Bonferroni adjusted p<0.01) for the inclusion of variables in the final tree structure. Fourth, the stopping criteria for tree generation were set a priori (S1 File details the rationale for the minimum node size for splitting or creation and notes maximum tree depth, etc.).

For readers less familiar with CHAID, we briefly summarize how to interpret the outputs. A CHAID analysis generates a bifurcating decision tree composed of a root node (the variable with the strongest association with the dependent variable and lowest p-value) which then branches and grows iteratively into internal and terminal nodes (the latter represent variables that carry maximum information). The order of importance of explanatory variables is represented by the tree structure and the percentages shown within the nodes are an indicator of the relevance of each characteristic as a primary predictor of CIDI status at follow-up (and PPV and NPV) [23]. In the current study, the PPV and NPV are reported for different combinations of specific symptoms identified in the classification trees. As the roots and nodes may link in different ways (following an ‘if-then’ type sequence), there may be more than one PPV or NPV associated with variables included in the tree. However, the reported AUC indicates the overall accuracy (performance) of the CHAID model. Tree building ends when p-values of all the observed independent variables are above the specified threshold for statistical significance, so the absence of any self-rated symptoms from the trees we report indicates that those items did not make a meaningful additional contribution to case classification. It should be noted that, as the tree diagrams for the ‘test set’ only include findings regarding root and terminal nodes with corrected p-values <0.01, we have not reported summary statistics for each root and node (this has been done to make the classification tree easier to interpret, but all these statistics are available upon request).

Cohort characteristics

As shown in Table 1, the sample comprised 1815 participants (Female = 1050; 58%) with a mean age of 26.4 (S = 4.2); about 55% were single and 60% were in full-time employment.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of key characteristics of the study cohort (also see S1 and S2 Tables).

https://doi.org/10.1371/journal.pone.0252550.t002

On average, individuals self-reported the co-occurrence of 5.21 symptoms (SD = 4.35; median = 5); the distribution of scores for total number of self-rated symptoms is shown in S1 Fig. The prevalence of each self-rated DLE, HMLE, and PLE symptom ranged from 1–50% (see S2 Table). For all cohort members, the three most frequently endorsed items were hypersomnia (DLE8), feelings of elation (HMLE1) and impaired sleep quality (DLE10); the three least frequently endorsed items were third party auditory hallucinations (PLE2), thought withdrawal (PLE6) and hearing voices when alone (PLE3).

About 21% (n = 383) of the cohort had symptoms that met criteria for a > = 1 SubT and 346 (19%) individuals had a FH of a mood and/or psychotic disorder. Of these individuals, 198 (11%) reported both a SubT and FH. About 31% of the study cohort (n = 568) met criteria for > = 1 subsequent full-threshold CIDI diagnosis. The median age at onset for self-reported psychopathology, such as the first SubT, was about 16 years (IQR: 14–18) and the median age at first onset of a full-threshold syndrome meeting CIDI diagnostic criteria was about 20 years (IQR: 18–23).

Early expressions of psychopathology and risk and subsequent CIDI caseness

Some data regarding the co-occurrence of FH, Sx_Load, and SubT with each other and with CIDI caseness are shown in Table 1, whilst PPV, NPV and AUC estimates are presented in Table 2 in the main text (additional raw data are provided in S3 Table in the supplementary materials). Regarding exposure to antecedents, it was additionally noted that 113 individuals identified as low risk for caseness (Low Sx_Load; SubT-; FH-) subsequently met criteria for a CIDI syndrome (6% of total cohort), whilst 78 individuals identified as high risk (High Sx_Load; SubT+; FH+) did not meet criteria for a CIDI syndrome during follow-up (4% of the total cohort) (see S3 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Classification of cohort members according to early expressions of psychopathology and risk and subsequent caseness: Positive and Negative Predictive Values (PPV; NPV), Area Under the Curve (AUC) and 95% Confidence Intervals (CI).

https://doi.org/10.1371/journal.pone.0252550.t003

As shown in Table 2, case classification according to FH demonstrated a PPV of 40.44% (95% CI: 37.13, 43.85), a NPV of 72.31% (95% CI: 70.83, 73.74) and an AUC of 62.75% (95% CI: 60.48, 64.98). For case classification according to preceding Sx_Load, the PPV was 44.60% (95% CI: 42.55, 46.78), the NPV was 81.61% (95% CI: 79.49, 83.56) and the AUC was 63.14% (95% CI: 60.87, 65.36). For case classification according to preceding SubT, the PPV was 55.19% (95% CI: 52.43, 57.93), the NPV was 77.23 (95% CI: 75.41, 78.95) and the AUC was 72.76% (95% CI: 70.80, 74.65).

When case classification was explored for the different combinations of early expressions of psychopathology and risk factors, the NPV increased for Low Sx_Load and FH- (88.33%) and the PPV increased to 52.87 for FH+ and Sx_Load and to 60.10% for FH+ and SubT+. However, Table 2 demonstrates that all NPV and PPV estimates were in the moderate range and changes in the AUC for combined models were only marginal (with some improving but some declining).

Classification tree models

Diagrams show trained classification trees run on the test set (hence sample size is about 50% of the study cohort) and comprises of roots and nodes that have statistically significant chi-square estimates and adjusted Bonferroni p values (these statistics are not shown here but are available from authors).

Test model 1: Self-rated symptoms.

The classification tree identified that six self-rated experiences can be utilized to predict CIDI caseness (see Fig 1); the overall AUC for this model was 74.31% (95% CI: 72.72, 75.87).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Classification tree for model 1: Self-rated symptoms, with adjustment for age, sex and familial clustering.

https://doi.org/10.1371/journal.pone.0252550.g001

The root node is represented by sadness/depressed mood (DLE2). The PPV for CIDI caseness was 76% (Node 12) in individuals who reported sadness (Node 2), feeling overwhelmed (DLE4; Node 6) and impaired concentration (DLE11; Node 12). In the absence of sadness (Node 1), CIDI caseness was more likely than not in those who reported paranoia (PLE5; Node 4). The NPV for not being a case was 79% (Node 7) and was best predicted by the absence of sadness (DLE2), paranoia (PLE5), or reduced need for sleep (HMLE3). Although physical activity (HMLE5) is included in the model, it appears to have limited utility for predicting caseness.

Test model 2: Self-rated symptoms combined with FH.

It was interesting to note that the classification tree shown in Fig 2 indicates that the addition of FH into the CHAID analysis did not lead to a change in the self-rated symptoms included in the diagram; the overall AUC for this model was 75.23% (95% CI: 73.18, 77.21).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Classification tree for model 2: Family history combined with self-rated symptoms, with adjustment for age, sex and familial clustering.

https://doi.org/10.1371/journal.pone.0252550.g002

As shown in Fig 2, the PPV for CIDI caseness was 74% in FH+ individuals who self-reported impaired concentration (DLE 11; Node 6) and feeling overwhelmed (DLE4; Node 14). Further, the PPV for caseness was 77% in FH- individuals who self-reported sadness (DLE2; Node 4) and paranoia (PLE5; Node 10). The NPV was 78% in FH- individuals without self-reported sadness (Node 3) or increased physical activity (HMLE5; Node 7). In Model 2, HMLE5 (physical activity) and HMLE3 (need for sleep) appear to make limited contributions of to the test model.

Limitations

We acknowledge that the findings must be replicated in independent samples and using other assessment tools and/or analyses. For example, although there is evidence to support the generalizability of our findings (i.e., prevalence rates for risk factors and CIDI diagnoses were similar to other studies) and we adjusted analyses for age, sex and familial clustering, it would be unsafe to assume this community cohort (a third of which comprised monozygotic twins) is representative of other cohorts [7, 9]. Of more concern is the possibility that CHAID analyses may ‘overfit’ the classification trees. Even though we took precautions to minimize this risk and only report the cross-validated models, we acknowledge that the findings of the second CHAID analysis (which included FH) must be treated with caution as the analysis and findings require replication in an independent sample. We also relied on self-ratings to assess expressions of psychopathology and risk. Self-report of FH can be unreliable, and unlike other similar recent studies, we did not incorporate polygenic risk scores (PRS) [9]. Also, we relied on simple categorical self-ratings of symptoms/experiences, rather than more sensitive, dimensional, or temporal evaluations. Lastly, the BLTS uses repeated cross-sectional assessments, which could be used to make more subtle examinations of longitudinal course of phenomenology using survival or temporal network analyses and other dynamic explorations of risk [30, 31].

Implications & comments

Finally, we consider the findings in the broader context of preventive psychiatry. Prevention strategies are informed by observational research on risks and adverse outcomes [2, 5, 28, 29, 35–39]. However, as highlighted by Cuijpers [25], we lack a detailed understanding of the exact pathways leading to mental disorders. Further, one risk factor alone is rarely sufficient to produce or predict disorder and most known risk factors have low specificity [2, 7–9, 25, 27–29]. To enhance options for prevention, we need greater knowledge not only of which ‘at risk’ individuals are most likely to transition to full threshold caseness, but also to develop screening procedures that capture the warning signs and symptoms during the specific timeframe when a ‘risk syndrome’ is most likely to evolve into a severe, impairing, or distressing condition [29–31]. The need to extend trans-diagnostic approaches is recognized in the literature [2, 5, 27, 39] and in published protocols for future prospective research [16, 35, 36, 38]. These all acknowledge that FH is an established marker of risk and emphasize that there is no rationale to recommend discontinuing its recording. However, they are less clear on other elements to include in screening. To date, early expressions of psychopathology or of clinical stage have either employed assessments of selected ‘clinical high risk’ syndromes [29] or, less frequently, micro-level assessments of specific symptoms as part of a disorder-specific project [37]. Unsurprisingly, low levels of symptoms have a high NPV, but the PPV of symptom load in this and other studies does not indicate that accumulation of symptoms can be translated into a risk assessment tool. SubTs have utility, but the definitions of different ‘risk syndromes’ are highly inconsistent and lack specificity; also, the increasing wish to undertake screening in community rather than clinical populations may limit their value. For example, a recent community study demonstrated that, whilst clinical high-risk states for psychosis have predictive validity for the onset of full-threshold psychotic disorders, the subthreshold antecedent conditions were relatively rare in the general population [40]. As such, the investigators suggested that a comprehensive prevention strategy with a focus on broader psychopathology may be more effective than a disorder-specific approach for achieving population-based improvements in prevention of major mental disorders [40]. Our study represents one of the first empirical reports to suggest novel, but discrete subsets of symptoms may have some utility in such a broad screening approach. We stress that the overall accuracy of the models reported here does not mean we regard the currently identified symptoms represent the optimal combination. However, a model comprising four trans-diagnostic symptoms appears to offer equivalent or better prediction of caseness than other established measures.