Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

The Reliability and Validity of Liu´s Self-Report Questionnaire for Screening Putative Pre-Psychotic States (BQSPS) in Adolescents

Abstract

The usage of rigorous analyses based on contemporary methods to enhance psychometric properties of screening questionnaires aimed to address psychotic-like experiences (PLE) is currently being encouraged. The Brief Self-Report Questionnaire for Screening Putative Pre-psychotic States (BQSPS) is a recently created tool addressing PLE beyond attenuated positive symptoms (APS). Its psychometric properties as a screening tool for first step assessment seems to be adequate, but further research is needed to evaluate certain validity aspects, particularly its dimensionality, internal structure, and psychometric properties in different populations. We assessed the reliability, construct validity, and criterion validity of BQSPS in two samples: 727 adolescents aged 13–18 years, and 245 young adults aged 18–33 years. We used exploratory structural equation modeling (ESEM), confirmatory factor analysis (CFA), and Structural Equation Modeling (SEM). The original four-factor structure was not replicated. The best fit in adolescents was obtained by a structure of three-correlated factors: social anxiety (SA), negative symptoms (NS), and positive symptoms (PS). This structure was confirmed in young adult subjects. The three-factor model reached a predictive capability with suicidality as external criterion. PLE are represented by a three-factor structure, which is highly stable between adolescent and young-adult samples. Although the BQSPS seems to be a valid tool for screening PLE, its psychometric properties should be improved to obtain a more accurate measurement.

Citation: Núñez D, Arias VB, Campos S (2016) The Reliability and Validity of Liu´s Self-Report Questionnaire for Screening Putative Pre-Psychotic States (BQSPS) in Adolescents. PLoS ONE 11(12): e0167982. https://doi.org/10.1371/journal.pone.0167982

Editor: Carrie E. Bearden, University of California Los Angeles, UNITED STATES

Received: May 2, 2016; Accepted: November 23, 2016; Published: December 14, 2016

Copyright: © 2016 Núñez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within Supporting Information files.

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Developing strategies aimed at identifying individuals at high clinical risk of first episode psychosis is one of the major current goals of psychiatric services worldwide [1]. These strategies are increasingly focused on the early detection of subjects showing subthreshold symptoms comprising positive and negative symptoms, and functional difficulties developed in the period preceding the onset of psychosis [2]. The role of these attenuated psychotic symptoms–also termed psychotic-like experiences (PLEs)–as specific predictors of psychosis remains unclear [3,4]. However, cumulative evidence shows associations between persisting PLEs and suicidality [5], higher risk of psychiatric disorders [6], and functional impairments in ultra-high- risk (UHR) populations [7,8], general adolescent populations [9], and help-seeking adolescent subjects [10].

Operational criteria and different measures aimed to enhance the early detection of people at risk of psychosis have been created and tested during the last two decades. Daneault et al. [11] identified 22 instruments with the most widely used measures being clinician administered interviews addressing previously well-validated clinical high-risk criteria [12]. The prognostic accuracy of these thorough psychometric measures is sufficient for help-seeking subjects with psychiatric symptoms [2]. But because the usage of these instruments is unsuitable either in the general population [13,14] or in primary health care settings, in recent years, several brief, easy-to-use self-report questionnaires have proliferated for screening purposes [14]. Under the general framework of the clinical high-risk approach [15], these tools have been mainly focused on predicting the transition to psychosis rather than in construct specificity per se [16].

Despite the relatively expanded usage of these screening questionnaires, different issues have recently been highlighted, that prevent against a clear recommendation of screening measures for PLE for psychosis. For instance, studies analyzing their psychometric properties, beyond sensitivity and specificity, are surprisingly scant. In fact, a recent review identified 17 screening tools, and evidence about validity and reliability was found for very few of them [13]. In addition, knowledge on the structure of PLEs and how they can accurately be captured by the available brief self-administered questionnaires is still inconclusive. The Community Assessment of Psychic Experiences (CAPE-42) [17,18] and the shorter version, CAPE-20 [7,19, 20], likely contain the most cumulative evidence about PLE factor structure, showing a similar and stable structure with three to five factors. Although some initial evidence has been reported for the new brief version (CAPE-P15) [21], the controversy remains, and mirrors the current theoretical debate about the structure of the psychopathology, organized around two main perspectives: the multidimensional [7,18,22] and the unidimensional approaches [23,24]. Both perspectives are supported by recent studies using advanced statistical techniques. Whereas the former was supported by Therman and Ziermans [25] who confirmed the three-factor structure (persecutory ideation, bizarre experiences, and perceptual abnormalities), the latter was supported by Núñez et al. [26], who found that the underlying structure of the CAPE-P15 can be adequately represented by a general factor and three separable specific traits. This suggests that there might be a common source underlying the subclinical psychotic symptoms addressed by the scale.

The need for developing shorter questionnaires with robust psychometric properties is currently well recognized [11, 13, 16]. Consequently, the usage of rigorous analyses based on contemporary methods is strongly encouraged nowadays. Recently, a self-report screening for pre-psychotic putative symptoms (BQSPS) [27] was created. Unlike other questionnaires developed to improve the predictive validity regarding the transition to psychosis, the BQSPS aims to detect early and broadly at risk mental states characterized by subtle symptoms and functional impairments [28]. Additionally, the BQSPS not only addresses attenuated positive symptoms, like most of the available screening questionnaires [16], but also includes other subthreshold clinical manifestations. This fits with evidence showing that APS are recognized as part of a late and severe stage in the development of psychotic disorders [29], and that they have not been entirely useful to predict the transition to psychosis [4, 30–32]. Moreover, it is in line with findings demonstrating that the concurrence of both positive and negative symptoms increases the risk for schizophrenia [33]. The BQSPS is a 15-item scale with four categories: interpersonal difficulty/social anxiety symptoms, self-deprecating descriptions, negative symptoms, and subthreshold psychotic-like experiences. According to the author’s analyses, the BQSPS has certain construct validity [27], and it can be useful to reliably distinguish clinical from non-clinical samples. A recent study found a moderate to large convergent validity, acceptable internal consistency for each scale, and modest test–retest reliability, recommending its usage for screening PLEs in college populations [34]. Nevertheless, and critically for the present study, factor analyses demonstrating the accuracy with which the proposed four-category model represents the data have not yet been reported. Consequently, in order to be truly useful for further investigation, and to perform screening for first step risk assessment, its internal structure must be thoughtfully tested, in a more heterogeneous sample of adolescents, arguably the most identifiable at-risk population [35].

We used exploratory and confirmatory analyses to examine the internal structure of the BQSPS. Moreover, we attempted to expand the scope of the research on and use of the BQSPS from the population studied by Liu et al. (a Chinese-speaking, relatively small sample of subjects with different at-risk levels) to a broader population of Spanish-speaking adolescents aged 13–18. Finally, based on evidence demonstrating associations between subthreshold psychotic symptoms and suicidality [36], and revealing that the former may be reliable indicators of the latter in both adolescent and adult general populations [37, 38], we tested the differential relationships between suicidality and BQSPS factors as predictive latent variables.

Material and Methods

2.1 Participants

This cross-sectional study was conducted with 727 high school students (adolescents) (women = 50.7%) aged 13–18 years (mean = 15.4 ± 1.33), recruited between April and July 2014 in six secondary schools of the city of Talca, Chile. All students between the ages of 13–18 years who were formally registered in school were invited to participate. Only two subjects declined to participate in the study, and thus, nearly 100% of the subjects attending each school were recruited. We observed that 5.9% of subjects had one or more missing values, with a distribution completely at random (MCAR, Little's test sig. = 0.142). Thus, cases with missing values were dropped out. We conducted the CFA analyses with a final sample of 684 subjects (see S1 File), and the SEM analyses with a final sample of 669 (see S2 File) subjects. To provide evidence about the generalizability of the results, we used a sample of 245 university students (young adults) (women = 75.4%) aged 18–33 years (mean = 20.4 ± 2.5). The only inclusion criterion was that the students voluntarily agreed to participate in the study. Only one subject declined to participate in the study. No missing data were observed in the university student sample.

2.2 Measures

The BQSPS [27] is a 15-item self-report questionnaire aimed to capture the early and broadly defined at risk mental status. It addresses four symptomatic categories (interpersonal difficulty/social anxiety symptoms, self-deprecating descriptions, negative symptoms, and subthreshold psychotic-like experiences). Responses to items ranged from 1 (never) to 5 (very often). Additionally, we addressed suicidality by the Okasha Suicidality Scale [39, 40]. It is a self-administered screening questionnaire with four items. Items 1, 2 and 3, address suicidal ideation, and item 4 addresses suicide attempt. In the current version, the items ranged from 1 (never) to 5 (very often).

2.3 Procedure

We translated and adapted the English version of the questionnaire to the Spanish language [41, 42] (see S1 Table).

We conducted the study in those public schools who agreed to participate after meetings with directive committees. Researchers participated in different parent’s meetings to present the research project. We also conducted the study in the Faculty of Psychology of the Universidad de Talca. We explained the project to both the directive committee and students. After its approval and once written informed consents were obtained from both the caregivers of adolescents and young adults themselves, the participants completed the questionnaires, administered in a classroom setting by trained psychologists.

Ethical approval was obtained from the Bioethics Committee of Universidad de Talca.

2.4 Statistical analysis

First, we explored the scale structure through exploratory structural equation models (ESEM) [43] with geomin oblique rotation. Based on the unidimensional solution, we estimated different models adding a latent variable in each iteration (models M1-M4). In order to find a balance between fit indices, parsimony and interpretability of the factor-loading pattern, we selected the three-factor solution, which was confirmed by confirmatory factor analysis (CFA; model M5). We additionally estimated with CFA the four-factor model (M6) proposed by Liu et al. To provide evidence about the generalizability of the results obtained in adolescents, we tested both the three and four factor confirmatory models in an independent sample composed by college students. Finally, we estimated a structural equation model with BQSPS factors as predictors, and suicidality as criterion (model M9).

Given the ordinal nature of our data, we used the Weighted Least Squares Means and Variance-adjusted estimation method, performed through MPlus v. 7.3 [43].

Results

The fit indices of tested models are shown in Table 1. Regarding the ESEM models, the RMSEA of M4 model did not substantially improve relative to the more parsimonious M3 model, with three-correlated factors: social anxiety (SA), negative symptoms (NS) and positive symptoms (PS).

thumbnail
Download:
Table 1. Fit indices of the CFA, ESEM and SEM models.

https://doi.org/10.1371/journal.pone.0167982.t001

Moreover, the load factor structure of M4 was not clearly interpretable (the loading configuration of factor 4 was not interpretable; see Table 2), unlike the better defined load pattern reached by M3. In this latter model, the items were grouped in three correlated clusters with content similarities: social anxiety (SA, 7 items), negative symptoms (NS, 4 items), and positive symptoms (PS, 4 items). The cross-loadings of ME were low, except for items 3, 4, and 13. A new estimation of M3 by CFA (M5) revealed an acceptable fit (RMSEA = .064; CFI = .927; TLI = .912), substantially better than the theoretical model proposed by Liu et al. (M6; ΔRMSEA = -0.014, ΔCFI = 0.034, ΔTLI = 0.046) [44, 45]. We replicated these results in the adult sample, where the fit of the confirmatory model of three factors (M7) was substantially better than the fit of the theoretical four-factor model (M8, see Table 1). Factor loads of M7 are depicted in S2 Table.

thumbnail
Download:
Table 2. Factor loadings of the three and four factor ESEM models (adolescent sample).

https://doi.org/10.1371/journal.pone.0167982.t002

Between-factor correlations in M5 were moderate (Fig 1). Congruence coefficients (Cr) between factor loads of M5 and M7 were high (.99, .97, and .95) [46]. The composite reliability (Cr) of SA was sufficient, but moderately low for PS and NS (SA-Cr = .80, NS-Cr = .65, PS-Cr = .60). Finally, the fit of the structural model (M9) model was good (RMSEA = .052; CFI = .976; TLI = .972). The regression path from PS toward suicide ideation was significant and low (.38). A similar result was observed for SA (.33). NS showed a non-significant negative regression path (-.05).

thumbnail
Download:
Fig 1. Parameters of model M9.

Standard errors are in parentheses. SA = Social anxiety; NS = Negative symptoms; PS = Positive symptoms. SUI = Suicidal ideation; CR = Composite reliability.

https://doi.org/10.1371/journal.pone.0167982.g001

Discussion

We examined the internal structure of early and broadly at risk mental states characterized by subtle psychotic symptoms addressed by the Brief Self-Report Questionnaire for Pre-psychotic Putative Symptoms (BQSPS) [27] in a sample of non-help-seeking adolescents aged 13–18 years. Our results did not replicate the original four-factor structure. The best fit and interpretability was obtained by a structure of three -correlated factors: social anxiety (SA), negative symptoms (NS), and positive symptoms (PS). This model was confirmed in an independent sample of young adults. SA comprises seven items addressing aspects related to anxiety about social interaction, feelings of being emotionally distant, and having few social skills. NS addresses four items referring to feelings of tiredness and lethargy and concentration difficulties. PS involves four items associated to perceptual anomalies and paranoid ideation.

We used contemporary methods suitable for analyzing some critical issues about the early detection of subthreshold psychotic symptoms,—for instance, the structure of these pre-clinical manifestations in the general population [25]. According to our knowledge, this is the first study specifically aimed at testing the structure of early and broad at risk mental states addressed by the BQSPS. Our result showing three factors distinct enough from each other fits in with the multidimensional approach according to which subthreshold psychotic symptoms should not be regarded as a homogeneous entity [47, 48].

Based on this finding, and given the adequate construct validity of the BQSPS, it could be used as a multidimensional screening for sub-psychotic experiences, in the context of new approaches not only aimed to prevent psychosis, but also other psychiatric disorders [29]. Nevertheless, there are some issues requiring clarification by further research before a clear recommendation for it usage.

First, our finding revealed a better goodness of fit for the three-factor structure relative to the four-factor structure originally proposed [27]. When accounting for these structural differences, some relevant aspects deserve mentioning. We observed that the items comprising the prior factor termed “self-deprecating descriptions” (1, 4 and 9) were subsumed in both SA (item 1) and NS factors (items 4 and 9). Moreover, SA included two additional items previously defined as negative symptoms (items 3 and 11). As outlined in Table 2, the former (“I feel lethargic whatever I do”) presents an unclear and theoretically inconsistent loading pattern (i.e., higher loading scores for the SA factor) either in the four-factor model or in the three-factor model, possibly because the item wording is confusing for adolescents. Therefore, it should be modified or excluded in future research. Additionally, our NS factor acquired a new structure (the two self-deprecation items 4 and 9, plus the original item 10). Finally, the most stable items were those addressing attenuated positive symptoms, all of them remaining as originally proposed.

Second, despite the adequate construct validity of the scale, a higher accuracy could be obtained if some items were slightly modified. Particularly, the moderately low reliability and variance explained by PS subscale could be improved by re-wording some items. For instance, it is reasonable to think that “being worried about loyalty of friends” might be an item confounding paranoid ideation with certain normal reactions of adolescence. This is supported by our results revealing a good functioning of this item in adults (λ = .609) but not in adolescents (λ = .28).

The three-factor model reached its own explicative capability on suicidality as criterion. SA and PS showed low and positive significant correlations with this criterion. The explanative contribution of SA was incremental with respect to PS, which means that including social anxiety indicators contribute to predicting suicidality beyond the generic psychotic-like symptoms. Overall, recent research examining the relationships between suicidality and psychiatric symptoms suggests the existence of differential and specific associations [49–52]. Concerning psychotic risk symptoms, Granö et al. [53] found that visual distortions explained suicidal ideation when other psychotic risk symptoms and demographic variables were controlled in a sample of help-seeking adolescents. Additionally, Fujita et al. [54] observed that auditory verbal hallucinations increase the risk for suicide attempts in a clinical sample of adolescents with suicidal ideation. The knowledge about the nature of this relation in non-clinical samples is scarce and contradictory [55]. Alternately, Koyanagi et al. [56] recently found that each psychotic experience, regardless of the type, was independently associated with suicide ideation in adults. In contrast, DeVylder and Hilimire [57] reported specific associations between auditory hallucinations and suicidal ideation in young adult subjects, and Capra et al. [58] observed that perceptual abnormalities and persecutory ideation, but not bizarre experiences were specifically associated with an increased risk for suicide in young adults. Finally, Kelleher et al. [59] found a specific association between auditory hallucinations and higher rates of suicide attempt in adolescents. Because of the usage of different domains (mainly positive-like symptoms in the case of these prior investigations, and a broader pool of subthreshold symptoms beyond PS in our research), direct comparisons should be made with caution. Nevertheless, our results showing that both PS and SA might contribute to predict suicidality, support the existence of specific patterns of relationships between subtle psychotic experiences and suicidality in adolescents. This finding should be cautiously interpreted. Given the evidence showing that PLEs can represent a severity index of non-psychotic psychopathology [32,60], relationships between suicidality and PLE could merely reflect a higher underlying risk of suicidality as a function of higher severity of psychiatric symptoms or more severe levels of mental distress. Further research with broader samples of adolescents is necessary for a better understanding of these differential relationships, probably influenced by other mediating variables [38] or explained by shared risk factors as suggested by DeVylder et al. [55].

Given some existing controversy about screening PLE in community settings [13], our results highlight the importance of properly adapting measurement instruments to different populations according to their own characteristics, and fits in with recent literature encouraging increased focus on psychometric properties of questionnaires addressing psychotic experiences [13, 61]. We think that providing accurate evidence about psychometric properties of questionnaires addressing PLE may help researchers avoid risks associated with their usage in different cultural contexts.

In summary, a three-factor model can represent PLE addressed by the BQSPS. This model was highly stable between adolescent and adult samples. Although the BQSPS seems to be a valid tool for screening PLEs, it could be improved by either rewording some items or testing new items with better psychometric properties. Additionally, the three-factor model of PLE reached certain explicative capabilities on suicidality; SA and PS being the factors with higher correlations with this criterion.

Some limitations deserve mention. First, we did not address clinical samples. To investigate the functioning of the measurement in diagnosed individuals could provide new insights about the PLE. Second, we did not use a random sampling method. Although the distribution of the age showed an equiprobable distribution in both male and female adolescents, a random sampling would be helpful to reduce a possible sampling bias. Finally, because of our cross-sectional design, causal relationships cannot be inferred from the present findings.

Supporting Information

S1 File. CFA Data Set.

https://doi.org/10.1371/journal.pone.0167982.s001

(XLSX)

S2 File. SEM Data Set.

https://doi.org/10.1371/journal.pone.0167982.s002

(XLSX)

S1 Table. Adapted items in Spanish.

https://doi.org/10.1371/journal.pone.0167982.s003

(DOCX)

S2 Table. Factor loadings of the three-factor CFA model (adult sample).

https://doi.org/10.1371/journal.pone.0167982.s004

(DOCX)

Author Contributions

  1. Conceptualization: DN.
  2. Data curation: DN VA.
  3. Formal analysis: VA.
  4. Funding acquisition: DN.
  5. Investigation: DN.
  6. Methodology: VA.
  7. Project administration: DN.
  8. Resources: DN.
  9. Supervision: DN.
  10. Validation: VA DN.
  11. Visualization: SC.
  12. Writing – original draft: DN VA SC.
  13. Writing – review & editing: DN SC.

References

  1. 1. Fusar-Poli P, Nelson B, Valmaggia L, Yung A, McGuire P. Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: Impact on psychopathology and transition to psychosis. Schizophrenia Bulletin. 2012;40(1):120–131. pmid:23180756
  2. 2. Fusar-Poli P, Cappucciati M, Rutigliano G, Schultze-Lutter F, Bonoldi I, Borgwardt S et al. At risk or not at risk? A meta-analysis of theprognostic accuracy of psychometricinterviews for psychosis prediction. World Psychiatry. 2015;14(3):322–332. pmid:26407788
  3. 3. Kline E, Thompson E, Bussell K, Pitts S, Reeves G, Shiffman J. Psychosis-like experiences and distress among adolescents using mental health services. Schizophrenia Research. 2014;152(2–3):498–502. pmid:24411529
  4. 4. Thompson A, Nelson B, Bruxner A, O'Connor K, Mossaheb N, Simmons M et al. Does specific psychopathology predict development of psychosis in ultra high-risk (UHR) patients?. Australian & New Zealand Journal of Psychiatry. 2013;47(4):380–390.
  5. 5. Kelleher I, Devlin N, Wigman T, Murtagh A, Fitzpatrick C et al. Psychotic experiences in a mental health clinic sample: implications for suicidality, multimorbidity and functioning. Psychological Medicine. 2014;44(8):1615–1624. pmid:24025687
  6. 6. Tandon N, Shah J, Keshavan M, Tandon R. Attenuated psychosis and the schizophrenia prodrome: current status of risk identification and psychosis prevention. Neuropsychiatry. 2012;2(4):345–353. pmid:23125875
  7. 7. Armando M, Nelson B, Yung A, Ross M, Birchwood M, Girardi P et al. Psychotic-like experiences and correlation with distress and depressive symptoms in a community sample of adolescents and young adults. Schizophrenia Research. 2010;119(1–3):258–265. pmid:20347272
  8. 8. Kelleher I, Keeley H, Corcoran P, Lynch F, Fitzpatrick C, Devlin N et al. Clinicopathological significance of psychotic experiences in non-psychotic young people: evidence from four population-based studies. The British Journal of Psychiatry. 2012;201(1):26–32. pmid:22500011
  9. 9. Kelleher I, Wigman J, Harley M, O'Hanlon E, Coughlan H, Rawdon C et al. Psychotic experiences in the population: Association with functioning and mental distress. Schizophrenia Research. 2015;165(1):9–14. pmid:25868930
  10. 10. Nishida A, Shimodera S, Sasaki T, Richards M, Hatch S, Yamasaki S et al. Risk for suicidal problems in poor-help-seeking adolescents with psychotic-like experiences: Findings from a cross-sectional survey of 16,131 adolescents. Schizophrenia Research. 2014;159(2–3):257–262. pmid:25315221
  11. 11. Daneault J, Stip E, Refer-O-Scope Group. Genealogy of instruments for prodrome evaluation of psychosis. Frontiers in Psychiatry. 2013;4.
  12. 12. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rössler A, Schultze-Lutter F et al. The psychosis high-risk state. JAMA Psychiatry. 2013;70(1):107. pmid:23165428
  13. 13. Addington J, Stowkowy J, Weiser M. Screening tools for clinical high risk for psychosis. Early Intervention in Psychiatry. 2014;9(5):345–356. pmid:25345316
  14. 14. Kline E, Wilson C, Ereshefsky S, Tsuji T, Schiffman J, Pitts S et al. Convergent and discriminant validity of attenuated psychosis screening tools. Schizophrenia Research. 2012;134(1):49–53. pmid:22036199
  15. 15. Cicero D, Martin E, Becker T, Docherty A, Kerns J. Correspondence between psychometric and clinical high risk for psychosis in an undergraduate population. Psychological Assessment. 2014;26(3):901–915. pmid:24708081
  16. 16. Kline E, Schiffman J. Psychosis risk screening: A systematic review. Schizophrenia Research. 2014;158(1–3):11–18. pmid:25034762
  17. 17. Stefanis N, Hanssen M, Smirnis N, Avramopoulos D, Evdokimidis I, Verdoux H et al. Evidence that three dimensions of psychosis have a distribution in the general population. European Psychiatry. 2002;17:61.
  18. 18. Konings M, Bak M, Hanssen M, van Os J, Krabbendam L. Validity and reliability of the CAPE: a self-report instrument for the measurement of psychotic experiences in the general population. Acta Psychiatrica Scandinavica. 2006;114(1):55–61. pmid:16774662
  19. 19. Barragan M, Laurens K, Navarro J, Obiols J. Psychotic-like experiences and depressive symptoms in a community sample of adolescents. European Psychiatry. 2011;26(6):396–401. pmid:21334860
  20. 20. Wigman J, van Nierop M, Vollebergh W, Lieb R, Beesdo-Baum K, Wittchen H et al. Evidence that psychotic symptoms are prevalent in disorders of anxiety and depression, impacting on illness onset, risk, and severity—implications for diagnosis and ultra-high risk research. Schizophrenia Bulletin. 2012;38(2):247–257. pmid:22258882
  21. 21. Capra C, Kavanagh D, Hides L, Scott J. Brief screening for psychosis-like experiences. Schizophrenia Research. 2013;149(1–3):104–107. pmid:23830544
  22. 22. Vollema M, Hoijtink H. The multidimensionality of selfreport schizotypy in a psychiatric population: An analysis using multidimensional Rasch models. Schizophrenia Bulletin. 2000;26(3):565–575. pmid:10993398
  23. 23. Laurens K, Hobbs M, Sunderland M, Green M, Mould G. Psychotic-like experiences in a community sample of 8000 children aged 9 to 11 years: an item response theory analysis. Psychological Medicine. 2012;42(07):1495–1506.
  24. 24. Reininghaus U, Priebe S, Bentall R. Testing the psychopathology of psychosis: Evidence for a general psychosis dimension. Schizophrenia Bulletin. 2013;39(4):884–895. pmid:22258881
  25. 25. Therman S, Ziermans T. Confirmatory factor analysis of psychotic-like experiences in a general population simple. Psychiatry Research. 2016; 235:197–199. pmid:26738980
  26. 26. Núñez D, Arias V, Vogel E, Gómez L. Internal structure of the Community Assessment of Psychic Experiences—Positive (CAPE-P15) scale: Evidence for a general factor. Schizophrenia Research. 2015;165:236–242. pmid:25935814
  27. 27. Liu Ch, Tien Y, Chen Ch, Chiu Y, Chien Y, Hsieh M et al. Development of a brief self-report questionnaire for screening putative pre-psychotic states. Schizophrenia Research. 2013;143(1):32–37. pmid:23182728
  28. 28. Keshavan M, Delisi L, Seidman L. Early and broadly defined psychosis risk mental states. Schizophrenia Research. 2011;126 (1–3):1–10. pmid:21123033
  29. 29. Seidman L, Nordentoft M. New targets for prevention of schizophrenia: Is it time for interventions in the premorbid phase? Schizophrenia Bullettin. 2015;41(4):1–6.
  30. 30. Owens D, Johnstone E. Precursors and prodromata of schizophrenia: findings from the Edinburgh High Risk Study and their literature context. Psychological Medicine. 2006;36(11):1501. pmid:16817986
  31. 31. Owens D, Johnstone E, Miller P, Macmillan J, Crow T. Duration of untreated illness and outcome in schizophrenia: test of predictions in relation to relapse risk. The British Journal of Psychiatry. 2010;196(4):296–301. pmid:20357306
  32. 32. Werbeloff N, Drukker M, Dohrenwend B, Levav I, Yoffe R, van Os J et al. Self-reported attenuated psychotic symptoms as forerunners of severe mental disorders later in life. Arch Gen Psychiatry 2012;69(5):467–475. pmid:22213772
  33. 33. Werbeloff N, Dohrenwend B, Yoffe R., van Os J, Davidson D, Weiser M. The association between negative symptoms, psychotic experiences and later schizophrenia: A population-based longitudinal study. PLOS ONE 2015;10(3):1–12.
  34. 34. Demmin D, DeVylder J, Hilimire M. Screening for subthreshold psychotic experiences and perceived need for psychological services. Early Intervention in Psychiatry. 2015.
  35. 35. Kelleher I, Connor D, Clarke MC, Devlin N, Harley M, Cannon M. Prevalence of psychotic symptoms in childhood and adolescence: a systematic review and meta-analysis of population-based studies. Psychological Medicine. 2012;9:1–7.
  36. 36. Kelleher I, Corcoran P, Keeley H, Wigman J, Devlin N, Ramsay H, et al. Psychotic symptoms and population risk for suicide attempt: a prospective cohort study. JAMA Psychiatry 2013;70(9):940–948. pmid:23863946
  37. 37. DeVylder J, Thompson E, Reeves G, Schiffman J. Psychotic experiences as indicators of suicidal ideation in a non-clinical college sample. Psychiatry Research. 2015;226(2):489–493.
  38. 38. DeVylder J, Lukens E, Link B, Lieberman J. Suicidal ideation and suicide attempts among adults with psychotic experiences: data from the Collaborative Psychiatric Epidemiology Surveys. JAMA Psychiatry. 2015a;72(3):219–225.
  39. 39. Okasha A, Lotaif F, Sadek A. Prevalence of suicidal feelings in a sample of nonconsulting medical students. Acta Psychiatrica Scandinavica. 1981;63(5):409–415. pmid:7315487
  40. 40. Salvo L., Melipillán R., Castro A. Reliability, validity and cutoff point for scale screening of suicidality in adolescents. Revista Chilena de Neuropsiquiatría. 2009;47(1):16–23.
  41. 41. International Test Commission. International Guidelines for Test Use. International Journal of Testing. 2006;1(2):93–114.
  42. 42. Muñiz J, Elosua P, Hambleton R. International Test Commission Guidelines for test translation and adaptation: Second edition. Psicothema. 2013;25(2): 151–157. ISSN 0214–9915. pmid:23628527
  43. 43. Muthén L, Muthén B. Mplus user’s guide. 7th ed. Los Angeles, CA: Muthén & Muthén; 2014.
    • 44. Chen F. Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal. 2007;14(3):464–504.
    • 45. Cheung G, Rensvold R. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal. 2002;9(2):233–255.
    • 46. Lorenzo-Seva U, ten Berge J. Tucker's congruence coefficient as a meaningful index of factor similarity. Methodology. 2006;2(2):57–64.
    • 47. Yung A, Nelson B, Baker K, Buckby J, Baksheev G, Cosgrave E. Psychotic-like experiences in a community sample of adolescents: implications for the continuum model of psychosis and prediction of schizophrenia. Australian & New Zealand Journal of Psychiatry. 2009;43(2):118–128.
    • 48. Wigman J, Vollebergh W, Raaijmakers Q, Iedema J, van Dorsselaer S, Ormel J, et al. The structure of the extended psychosis phenotype in early adolescence—a cross-sample replication. Schizophrenia Bulletin. 2011;37(4):850–860. pmid:20044595
    • 49. DeShong H, Tucker R, O’Keefe V, Mullins-Sweatt S, Wingate L. Five factor model traits as a predictor of suicide ideation and interpersonal suicide risk in a college sample. Psychiatry Research. 2015;226(1):217–223. pmid:25623017
    • 50. Nock M, Hwang I, Sampson N, Kessler R. Mental disorders, comorbidity and suicidal behavior: Results from the National Comorbidity Survey replication. Molecular Psychiatry. 2009;15(8):868–876. pmid:19337207
    • 51. Nock M, Hwang I, Sampson N, Kessler R, Angermeyer M, Beautrais A et al. Cross-National analysis of the associations among mental disorders and suicidal behavior: Findings from the WHO World Mental Health Surveys. PLoS Med. 2009;6(8):e1000123. pmid:19668361
    • 52. Chapman C, Mullin K, Ryan C, Kuffel A, Nielssen O, Large M. Meta-analysis of the association between suicidal ideation and later suicide among patients with either a schizophrenia spectrum psychosis or a mood disorder. Acta Psychiatrica Scandinavica. 2014;131(3):162–173. pmid:25358861
    • 53. Granö N, Salmijärvi L, Karjalainen M, Kallionpää S, Roine M, Taylor P. Early signs of worry: Psychosis risk symptom visual distortions are independently associated with suicidal ideation. Psychiatry Research. 2015;225(3):263–267. pmid:25595340
    • 54. Fujita J, Takahashi Y, Nishida A, Okumura Y, Ando S, Kawano M et al. Auditory verbal hallucinations increase the risk for suicide attempts in adolescents with suicidal ideation. Schizophrenia Research. 2015;168(1–2):209–212. pmid:26232867
    • 55. DeVylder J, Jahn D, Doherty T, Wilson C, Wilcox H, Schiffman J et al. Social and psychological contributions to the co-occurrence of sub-threshold psychotic experiences and suicidal behavior. Social Psychiatry and Psychiatric Epidemiology. 2015;50(12):1819–1830. pmid:26493307
    • 56. Koyanagi A, Stickley A, Haro J. Subclinical psychosis and suicidal behaviour in England: Findings from the 2007 Adult Psychiatric Morbidity Survey. Schizophrenia Research. 2015;168(1–2):62–67. pmid:26255564
    • 57. DeVylder J, Hilimire M. Suicide risk, stress sensitivity, and self-esteem among young adults reporting auditory hallucinations. Health Social Work. 2015;40(3):175–181. pmid:26285356
    • 58. Capra C, Kavanagh D, Hides L, Scott J. Subtypes of psychotic-like experiences are differentially associated with suicidal ideation, plans and attempts in young adults. Psychiatry Research 2015;228(3):894–898. pmid:26050011
    • 59. Kelleher I, Corcoran P, Keeley H, Wigman J, Devlin N, Ramsay H et al. Psychotic symptoms and population risk for suicide attempt. JAMA Psychiatry. 2013;70(9):940. pmid:23863946
    • 60. Honings S, Drukker M, Groen R, van Os J. Psychotic experiences and risk of self-injurious behaviour in the general population: a systematic review and meta-analysis. Psychological Medicine. 2015;46(02):237–251.61.
    • 61. Mark W, Toulopoulou T. Psychometric Properties of “Community Assessment of Psychic Experiences”: Review and Meta-analyses. SCHBUL. 2015;:sbv088.