Subjective cognitive complaints are prevalent in those affected by functional psychoses and a variety of possible associated factors have been investigated. However, few studies have examined these potential factors within single studies or analyses.
Patients with a history of a schizophrenia spectrum disorder (n = 115) and a non-clinical comparison group (n = 45) completed the Subjective Scale to Investigate Cognition in Schizophrenia (SSTICS) and the Brief Assessment of Cognition in Schizophrenia (BACS). The patient group also completed the Positive and Negative Syndromes Scale (PANSS), the Birchwood Insight Scale (IS), and the Hospital Anxiety and Depression Scale (HADS).
The BACS and SSTICS scores were associated in the non-clinical comparison group, but not in the patient group. In the patient group worse subjective cognition was associated positively with good insight, greater dysphoria and greater positive symptoms. Linear regression revealed that, once other variables had been accounted for, dysphoria (HADS anxiety and depression factor) was the only significant predictor of SSTICS scores.
Citation: Sellwood W, Morrison AP, Beck R, Heffernan S, Law H, Bentall RP (2013) Subjective Cognitive Complaints in Schizophrenia: Relation to Antipsychotic Medication Dose, Actual Cognitive Performance, Insight and Symptoms. PLoS ONE 8(12): e83774. https://doi.org/10.1371/journal.pone.0083774
Editor: Xiang Yang Zhang, Baylor College of Medicine, United States of America
Received: January 14, 2013; Accepted: November 12, 2013; Published: December 20, 2013
Copyright: © 2013 Sellwood et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This paper presents independent research commissioned by the National Institute for Health Research (NIHR), UK under its Programme Grants for Applied Research scheme (RP-PG-0606-1086). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Self-report of cognitive deficits in schizophrenia is potentially important. If patients can accurately report their own cognitive problems this should help in the planning of interventions related to functional outcome and individual therapy. In addition, it has been suggested that for some, subjective cognitive complaints (SCC) underlie a wide range of symptoms elicited from patients that develop later as psychosis progresses and that there is therefore an association between SCC and future psychotic symptoms . However, in a number of studies SCC and actual cognitive performance is not associated and in those where there is such an association, the relationship between these two variables is weak, , , . Moreover, specific complaints are not mirrored on relevant specific tests of neuropsychological function. This has been explained in terms of the lack of ecological validity of neuropsychological tests and patients misinterpreting terms such as “attention” and “memory”. This raises the question as to what SCC actually represents and what factors are involved in its expression. A number of factors have been investigated in this regard. Medication side effects , , , insight  and symptoms associated with psychosis , including depression,  have all been found to be related to SCC. However, no studies have looked at the relative predictive values of these variables, or whether they contribute to predicting SCC independently or not. The aim of the present study was to replicate the finding that patients' subjective experiences of cognitive functioning fail to reflect objectively assessed cognitive performance, , ,  and to determine the relative importance of possible predictors of subjective cognitive complaints in schizophrenia. Specifically, once neuropsychological performance and antipsychotic dose had been accounted for, would SCC be predicted in terms of symptoms related directly to psychosis, including insight? In addition, would anxiety and depression predict SCC over and above psychotic symptoms?
This study was conducted in accordance with guidelines set by the UK NHS national ethics procedures (National Research Ethics Service - NRES) and approved by the NHS trusts at local levels (reference numbers 09/H1012/9 for clinical group and 10/H1013/5 for comparison group). All participants (including those aged 16 to 17 years) provided written informed consent and were deemed to have capacity to provide informed consent by the clinical team working with them. Recruitment and consent procedures for all participants including those aged 16 to 17 years were approved by the NRES committees.
115 patients, 84 male and 31 female, mean age = 36.0 years (SD = 11.59), who had been diagnosed by their consultant psychiatrists as suffering from a schizophrenia-spectrum disorder according to ICD-10 criteria, aged 16–65, with a sufficient level of English literacy to complete the measures and capacity to provide informed consent, were referred from clinical teams in the North West of England. The selection criteria were purposefully broad, in order to recruit patients varying in their symptoms and duration of illness. Although those with acquired brain injury or learning disability were not excluded and presence of such a diagnosis noted, none were recruited into the sample. The majority were White British (83%). Twenty-six had no school qualifications, 35 had completed school level qualifications, 15 had experienced higher education, and the rest had obtained vocational qualifications (data missing from 3). Four were in full or part-time employment and 6 were students. Nine were in long-term relationships and 10 were divorced or separated. Diagnoses, confirmed using ICD-10 checklists, were as follows: schizophrenia (n = 75), schizoaffective disorder (n = 10), unspecified non-organic psychosis (n = 14), acute and transient psychotic disorder (n = 12), and persistent delusional disorder (n = 4). Data on medication were available for 93 patients; of these 11 were not taking antipsychotics and 82 were (mean daily dose = 351.41 mg, SD = 221.04, CPZ equivalent calculated using the method of  when possible, otherwise from the British National Formulary (Joint Formulary Committee, 2007). Duration of illness was estimated from participant's reports of age of first psychotic episode and was highly skewed (median = 140 months, range 8–522; data missing from 5 participants). Data for this latter variable were therefore log transformed for use in the analyses reported below.
The comparison group comprised a convenience sample recruited from local fire services; staff working in National Health Services via posters displayed in office areas; and acquaintances of the research assistants, using a snowballing method. The comparison participants completed the study in the same way as the patients, directly with the researcher, and were remunerated for their time.
Thirty were male and 15 female, with a mean age of 34.6 years (SD = 11.72). Seventeen had obtained school-level qualifications, 17 had experienced higher education and the rest had vocational qualifications. Fifteen were in long-term relationships and 2 were separated.
Positive and Negative Syndromes Scale (PANSS).
The PANSS  is a clinician administered thirty-item semi-structured interview consisting of seven items assessing positive symptoms (Pos; e.g. hallucinations, delusions, conceptual disorganization), seven items assessing negative symptoms (Neg; e.g. blunted affect, passive/apathetic social avoidance) and sixteen items assessing global psychopathology (e.g. depression, anxiety, lack of insight, guilt). All items are scored between 1 (not present) and 7 (severe). A number of studies have demonstrated the reliability and validity of the PANSS. PANSS raters (RB, SH) were trained using a standardised approach and had achieved good inter-rater reliability (ICC = 0.874, P<0.001).
Beck Hopelessness Scale.
The Hopelessness Scale (BHS) was developed as a self-report measure regarding pessimism about the future. There are 20 true/false items and scores of 9 or above indicate a high risk of future suicide, . It has a demonstrably high internal consistency and scores are associated with levels of depression.
Hospital Anxiety and Depression Scale.
The hospital anxiety and depression scale (HADS;  is a 14 item self-report measure. Of these items, 7 assess depression, whilst the remaining 7 assess anxiety over the preceding week. Cronbach's alpha values for anxiety and depression scales for this sample are α = 0.86 and α = 0.83 respectively. Because these scales were highly correlated (r = .67, p<000) scores were added to derive a single dysphoria score for subsequent analyses (alpha = .90) (Using separate anxiety and depression scores did not appreciably alter any of the results).
Birchwood Insight Scale.
Insight was measured with the Insight Scale (IS; , an 8-item self-report scale designed to be sensitive to change, and which captures three widely accepted dimensions of insight: perceived need for treatment, awareness of illness and relabeling of symptoms as pathological. Higher scores indicate greater levels of insight. In this study the scale had an alpha coefficient of 0.66.
Brief Assessment of Cognition in Schizophrenia (BACS)
The Brief Assessment of Cognition in Schizophrenia (BACS; ,  is a battery of neurocognitive tests that assesses domains of cognition found to be most impaired and most strongly correlated with outcome in patients with a diagnosis of schizophrenia. These include verbal memory, working memory, motor speed, verbal fluency, attention and speed of processing and executive function. The BACS has 6 subscales (list learning, digit sequencing task, token motor task, verbal fluency, Tower problem solving, and symbol coding), takes approximately 35 minutes to complete and has high levels of test re-test reliability on each of the subscales and with parallel forms for those subtests where they were available. In this study, the token motor task (in which participants are asked to pick up small tokens in both hands simultaneously and place them in a basket) was omitted after it was found to be poorly tolerated. Therefore rather than use the total BACS score the data were factor analysed and the single resultant factor used in susequent analyses. The BACS has an equivalent sensitivity to the detection of impairments as a standard, more extensive, neuropsychological test battery .
Subjective Scale to Investigate Cognition in Schizophrenia (SSTICS)
The SSTICS  is a brief, 21 item self report scale examining subjective complaints about memory, attention, praxia and executive function in patients affected by schizophrenia. In the original study it exhibited good internal consistency (α = .86) and test-retest reliability (rs = .82). In this study, the internal consistency (α) was.90 across the two groups. Specific areas of cognition covered are working memory, explicit memory (divided into episodic and semantic memory), attention, executive function and language. There are five subscales: explicit memory, episodic memory, semantic memory, working memory and attention. Stip et al. reported significant but weak associations between the memory subscales and objectively assessed mnemonic function. The 21 items take approximately six minutes to complete.
Comparison of groups
Table 1 shows mean scores of the two groups on measures of mood, subjective and objective cognition. Significant differences were observed between the groups on all of the variables with the exception of the SSTICS working memory subscale. Pearson correlation coefficients revealed a modest and marginally significant correlation between total BACS and total SSTICS scores for the non-clinical comparison group (r = −.31, p = .05, n = 41) but no correlation for the patients (r = −.05, p = .62, n = 117). Although these correlations differed in terms of significance level, they did not differ significantly from each other (z = 1.44, ns).
It might be argued that insight should moderate the relationship between BACS and SSTICS, so that only patients with good insight will correctly judge their cognitive ability. To test for this possibility, a multiple regression was carried out using centred variables, with SSTICS total scores as the dependent variable, total insight and total BACS scores entered as predictors in the first stage, and then the interaction between the BACS and insight scores entered in the second stage. When the BACS and insight variables were entered, there was a significant model (F[2,103] = 4.113, p<.02, adjusted r2 = .056) but only insight was retained as a significant predictor (β = .27, t = 2.86, p<.005). However, adding the interaction between insight and BACS scores in the second stage did not improve the model (Fchange = .55, p = .46) and the effect of the interaction was nonsignificant.
Predictors of subjective cognitive impairment in the patient group
Correlations between SSTICS and BACS subscales for the patient group are shown in Table 2 with those for the non-clinical comparison group summarized in Table 3. It can be seen that none of the SSTICS subscales were associated significantly with any of the BACS subscales in the clinical group. In addition the SSTICS total scores and single factor scores on the BACS (see below) were not associated (Pearson's r = .006; ns). One possibility was that level of education may have not only been related to BACS scores, but also the reporting of SCC. Table 4 provides a summary of correlations between the SSTICS, BACS factor scores and symptom variables. It can be seen that positive symptoms, insight and dysphoria exhibited significant correlations with the SSTICS such that better insight, worse positive symptoms and greater dysphoria were associated with poorer subjective cognitive functioning. It is noteworthy that dysphoria was associated significantly with insight and positive symptoms. Using the highly conservative Bonferroni method of correcting for multiple comparisons between the four clinical scales (PANSS positive, PANSS negative, dysphoria and insight) and the two cognitive measures (BACS and SSTICS) does not alter the overall picture (critical p = .00625).
Spearman correlations revealed that whilst the BACS factor was associated significantly with educational attainment (r = .37, p<.01) the SSTICS total scores were not (r = .03, ns). A further possible confound is that of diagnosis. Given the inclusion of participants with schizoaffective disorder and other psychoses the bivariate correlations between dysphoria and SSTICS total score was re-run with only those with a primary diagnosis of schizophrenia. The correlation remained significant (r = 0.66, p<.000, n = 72).
Linear regression was used to assess the relative value of relevant variables in predicting subjective cognition (SSTICS total scores). 13 participants were excluded from the analyses because of missing data on one or more scale. For the purpose of analysis, principle components analysis was used to derive a single factor score for the BACS (Eigenvalue = 2.991, % variance accounted for = 49.84, minimum loading = .58). Variables were entered in three stages: first BACS scores, second clinical variables (PANSS positive and negative and Birchwood Insight scores), and finally mood measures (HADS dysphoria and the Beck Hopelessness Scale scores). At each stage, the variables were entered simultaneously. Finally, the analyses were repeated with medication dose and duration of illness (log transformed) included at the first stage (this further reduced the sample size to 82 because data were missing for these variables for some of the participants.)
It can be seen from Table 5 that BACS scores did not significantly predict subjective cognitive functioning (p for initial model = .90). At the second stage, a significant model was generated (F[4,97] = 5.04, p<.001; adjusted R2 = .14) and greater insight and positive symptoms of psychosis predicted SSTICS scores. Adding the dysphoria variables improved the model (final F[6,95] = 10.78; adjusted R2 = .37, p<.001). At this stage, dysphoria was a significant predictor but the findings also suggested that negative symptoms were weakly although negatively associated with subjective cognitive function (less severe negative symptoms were associated with poorer subjective cognitive functioning).
When the analyses were repeated with duration of illness and antipsychotic medication dose included, the results were essentially unchanged except that the effect for negative symptoms was removed (final F[8,73] = 7.27, p<.001; adjusted R2 = .38) and dysphoria remained the only significant predictor (see Table 6).
Summary of results and implications
The aim of the present study was to determine the relative importance of possible predictors of subjective cognitive complaints in schizophrenia and to replicate the finding that patients' subjective experiences of cognitive functioning fail to reflect objectively assessed cognitive performance, , , . As in previous studies, SCC was not associated with cognitive test performance in patients with psychosis. However, in the non-clinical comparison group there was an association between cognitive function and SCC, indicating that healthy individuals are able to more accurately appraise their own mental functioning than patients. However, this association was relatively weak and the correlations between SCC and performance on the BACS in the clinical and non-clinical groups were not significantly different.
Some studies have shown that the association between cognitive performance and negative symptoms is mediated by defeatist beliefs , . However, SCC as measured in the present study are unlikely to play this mediational role, as they failed to correlate with either negative symptoms or actual cognition. One possible explanation is that defeatist beliefs arise when cognitive failures that the patient is unaware of impair social functioning. However, it should be noted that in the first regression analysis (Table 5) it can be seen that negative symptoms are inversely related to SCC. In the final analysis this relationship disappeared once antipsychotic dose and duration of illness had been accounted for.
The present findings call in to question the use of the SSTICS as a means of evaluating cognition in those affected by psychosis. It has been argued that some everyday cognitive failures that are recognised by patients may be too subtle to be detected by neurocognitive tests . However, other factors, included in the present study, appear to be important in determining patients' beliefs about their own cognitive functioning.
Consistent with a previous study of multiple factors , insight, anxiety and depression and positive symptoms were associated with subjective cognition. In the latter report the focus was on the role of insight; anxiety and depression were entered as covariates, leading to the relationship between insight and SCC being non-significant, but the authors did not highlight the implications of this finding. In fact, in the present study, anxiety and depression derived from the HADS were the only clear predictors of SCC once levels of medication, positive symptoms and negative symptoms had been accounted for. It is therefore likely that dysphoric symptoms account for patients' reports of cognitive problems.
It has been suggested that self-reports are affected in two ways. First, those affected by cognitive deficits may lack cognitive abilities to recognize deficits, whilst those whose abilities are unimpaired overestimate the scale of their problems. One possible explanation for the association between dysphoria and SCC is that people affected by anxiety and depression may be more likely to attribute their difficulties to poor memory or cognitive problems. This would be consistent with a substantial research literature that shows that depression is associated with a tendency to attribute negative experiences to internal causes . Patients with depression may be excessively sensitive to normally occurring cognitive failures, taking the view that their ‘brain is broken’, especially if this message has been passed on to them by clinicians. It has been found that those experiencing psychosis, as well as those at risk of psychotic symptoms, exhibit less confidence in their cognitive abilities and are more self-conscious about them compared to controls . These factors are a key aspect of certain anxiety disorders and depression where low ‘cognitive confidence’ plays a role in maintaining problems , .
In addition to the emotional aspects of SCC a further issue for investigation may be metacognition; that is the ability to recognize and manage one's own cognitive functioning. In this sense it should be related to insight. The present study suggests however, that insight has less predictive value for SCC once dysphoria is included in the equation. The same though may not be the case for broader areas of function. It is recognized for example that many patients under report problems with independent ‘real world’ functioning , but this is unlikely to be the result of depression or anxiety but is associated with insight. In fact, depression and anxiety may result from realization of difficulties associated with psychosis  and are thus possibly inversely related to impairment of insight.
There were certain limitations in the present study that should be considered. First, the non-clinical comparison group was not precisely matched to the patient group, especially in educational attainment. It was also was small (n = 45) compared with the clinical group. In addition there were some missing data, particularly concerning medication. This latter point is potentially important, because further questions may be raised about the roles of different categories of antipsychotic (e.g. first versus second generation treatments) as well as treatments for side effects.
Although the HADs is a widely used scale for determining levels of anxiety and depression, its use with participant groups affected by psychosis could be questioned. In relation to depression, scales have been developed focusing on signs and symptoms which are unlikely to be confused with antipsychotic medication side effects and negative symptoms, , . This might have given a more rigorous specification of depression over and above these confounding aspects of psychotic disorders. The choice of the HADS was based upon the practical constraints of having to assess many patients and burdening them with a minimal interview length. It should be noted that one of the BACS subtests was dropped because of some participants' limited willingness to tolerate the testing. Future work I this area should try to better disentangle depression and anxiety from the effects of the psychotic disorders of concern and the side effects of their treatment.
A further potential measurement issue relates to the breadth and refinement of other symptom clusters. For example, one possibility in future work might be the evaluation of the role of other symptom factors derived from the PANSS. For example what might be the role of the excitement and in particular cognitive/disorganized factors? This question could be answered by studies with larger participant samples with suitable levels of power allowing for the inclusion of more variables in regression analyses.
Implications and future research
The major clinical implication is that clinicians should investigate and consider the role of depression and anxiety when confronted with patients with marked SCC. SCC in the absence of formal cognitive testing should not be considered evidence of impaired cognitive functioning. In addition, others have pointed out that reports from other informants can improve the accuracy of such judgments and where possible should be incorporated into interviews regarding cognitive performance , . This is also the case in trials where interview based assessments of cognition are used as outcome variables .
Further more sophisticated methods might be used to examine the role of relevant variables in predicting SCC. For example Kim and Byun  found that, when medication doses were equated and patients divided by level of side effects, those reporting most side effects also reported higher levels of SCC. Thus there may be variables not associated with dysphoric symptoms or at least only partially so, that may explain SCC further, and which need to be considered in future studies. It would also be useful to investigate SCC longitudinally to determine, for example, whether they change over time with changing mood, and also whether they are more prominent at some stages of psychosis (e.g. acute episodes) than at others (e.g. during the prodromal period or in remission).
The authors would like to thank the RECOVERY Programme Group and the Mental Health Research Network for their support. We would also like to acknowledge our Service User Consultants (Nerys Baldwin and Sam Omar) and Stephen Pilling as well as individual members of the Service User Reference Group, Yvonne Awenat, Rory Byrne, Ellen Hodson, Martina Kilbride, Liz Pitt, Jason Price, Tim Rawcliffe and Yvonne Thomas.
Conceived and designed the experiments: AM HL RB WS. Performed the experiments: RB SH. Analyzed the data: WS RB. Wrote the paper: WS RB.
- 1. Peralta V, Cuesta MJ, de Leon J (1992) Positive versus negative schizophrenia and basic symptoms. Compr Psychiatry 33: 202–206.
- 2. Chan RC, Wang Y, Ma Z, Hong XH, Yuan Y, et al. (2008) Objective measures of prospective memory do not correlate with subjective complaints in schizophrenia. Schizophr Res 103: 229–239.
- 3. Johnson I, Tabbane K, Dellagi L, Kebir O (2011) Self-perceived cognitive functioning does not correlate with objective measures of cognition in schizophrenia. Compr Psychiatry 52: 688–692.
- 4. Seco RB, Sanz DG, Modamio MF, Rodriguez MA, Calleja RS, et al. (2010) Cognitive complaints in schizophrenia: relationship with insight and other cognitive measures. Revista De Psiquiatria Y Salud Mental 3: 55–60.
- 5. Zanello A, Huguelet P (2001) Relationship between subjective cognitive symptoms and frontal executive abilities in chronic schizophrenic outpatients. Psychopathology 34: 153–158.
- 6. Kim JH, Byun HJ (2007) Association of subjective cognitive dysfunction with akathisia in patients receiving stable doses of risperidone or haloperidol. J Clin Pharm Ther 32: 461–467.
- 7. Kim JH, Kim SY, Byun HJ (2008) Subjective cognitive dysfunction associated with drug-induced parkinsonism in schizophrenia. Parkinsonism Relat Disord 14: 239–242.
- 8. Moritz S, Woodward TS, Krausz M, Naber D (2002) Relationship between neuroleptic dosage and subjective cognitive dysfunction in schizophrenic patients treated with either conventional or atypical neuroleptic medication. Int Clin Psychopharmacol 17: 41–44.
- 9. Moritz S, Perro C, Woodward TS, Krausz M (2002) Subjective cognitive dysfunction in first-episode patients predicts symptomatic outcome: a replication. Psychopathology 35: 367–368.
- 10. van den Bosch RJ, Rombouts RP, van Asma MJ (1993) Subjective cognitive dysfunction in schizophrenic and depressed patients. Compr Psychiatry 34: 130–136.
- 11. Andreasen NC, Pressler M, Nopoulos P, Miller D, Ho BC (2010) Antipsychotic dose equivalents and dose-years: a standardized method for comparing exposure to different drugs. Biol Psychiatry 67: 255–262.
- 12. Kay SR, Fiszbein A, Opler LA (1987) The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 13: 261–276.
- 13. Beck AT, Steer M (1988) Manual for the Beck Hopelessness Scale. San Antonio: The Psychological Corporation.
- 14. Beck AT, Steer RA, Kovacs M, Garrison B (1985) Hopelessness and eventual suicide: a 10-year prospective study of patients hospitalized with suicidal ideation. Am J Psychiatry 142: 559–563.
- 15. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer RA (1990) Relationship between hopelessness and ultimate suicide: a replication with psychiatric outpatients. Am J Psychiatry 147: 190–195.
- 16. Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67: 361–370.
- 17. Birchwood M, Smith J, Drury V, Healy J, Macmillan F, et al. (1994) A self-report Insight Scale for psychosis: reliability, validity and sensitivity to change. Acta Psychiatr Scand 89: 62–67.
- 18. Keefe RS, Harvey PD, Goldberg TE, Gold JM, Walker TM, et al. (2008) Norms and standardization of the Brief Assessment of Cognition in Schizophrenia (BACS). Schizophr Res 102: 108–115.
- 19. Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, et al. (2004) The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res 68: 283–297.
- 20. Stip E, Caron J, Renaud S, Pampoulova T, Lecomte Y (2003) Exploring cognitive complaints in schizophrenia: the subjective scale to investigate cognition in schizophrenia. Compr Psychiatry 44: 331–340.
- 21. Grant P, Beck A (2009) Defeatist beliefs as a mediator of cognitive impairment, negative symptoms, and functioning in schizophrenia. Schizophr Bull 35: 798–806.
- 22. Perivoliotis D, Morrison AP, Grant PM, Beck AT (2009) Negative performance beliefs and negative symptoms in individuals at ultra-high risk of psychosis: A preliminary study. Psychopathology 42: 375–379.
- 23. Prouteau A, Verdoux H, Briand C, Lesage A, Lalonde P, et al. (2004) Self-assessed cognitive dysfunction and objective performance in outpatients with schizophrenia participating in a rehabilitation program. Schizophr Res 69: 85–91.
- 24. Bayard S, Capdevielle D, Boulenger JP, Raffard S (2009) Dissociating self-reported cognitive complaint from clinical insight in schizophrenia. Eur J Psychiat 24: 251–258.
- 25. Mezulis AH, Abramson LY, Hyde JS, Hankin BL (2004) Is there a universal positivity bias in attributions? A meta-analytic review of individual, developmental, and cultural differences in the self-serving attributional bias. Psychol Bull 130: 711–747.
- 26. Morrison AP, French P, Wells A (2007) Metacognitive beliefs across the continuum of psychosis: comparisons between patients with psychotic disorders, patients at ultra-high risk and non-patients. Behav Res Ther 45: 2241–2246.
- 27. Cartwright-Hatton S, Wells A (1997) Beliefs about worry and intrusions: the Meta-Cognitions Questionnaire and its correlates. J Anxiety Disord 11: 279–296.
- 28. Yilmaz AE, Gencoz T, Wells A (2011) The temporal precedence of metacognition in the development of anxiety and depression symptoms in the context of life-stress: a prospective study. J Anxiety Disord 25: 389–396.
- 29. Sabbag S, Twamley EM, Vella L, Heaton RK, Patterson TL, et al. (2011) Assessing everyday functioning in schizophrenia: Not all informants seem equally informative. Schizophr Res 131: 250–255.
- 30. Lysaker PH, Erickson M, Ringer J, Buck KD, Semerari A, et al. (2011) Metacognition in schizophrenia: The relationship of mastery to coping, insight, self-esteem, social anxiety, and various facets of neurocognition. Br J Clin Psychol 50: 412–424.
- 31. Lysaker PH, Salyers MP (2007) Anxiety symptoms in schizophrenia spectrum disorders: associations with social function, positive and negative symptoms, hope and trauma history. Acta Psychiatr Scand 116: 290–298.
- 32. Addington D, Addington J, Atkinson M (1996) A psychometric comparison of the Calgary Depression Scale for Schizophrenia and the Hamilton Depression Rating Scale. Schizophr Res 19: 205–212.
- 33. Addington D, Addington J, Maticka-Tyndale E (1994) Specificity of the Calgary Depression Scale for schizophrenics. Schizophr Res 11: 239–244.
- 34. Addington D, Addington J, Maticka-Tyndale E (1993) Assessing depression in schizophrenia: the Calgary Depression Scale. Br J Psychiatry Suppl: 39–44.
- 35. Lindenmayer JP, Grochowski S, Hyman RB (1995) Five factor model of schizophrenia: replication across samples. Schizophr Res 14: 229–234.
- 36. Keefe RS, Poe M, Walker TM, Kang JW, Harvey PD (2006) The Schizophrenia Cognition Rating Scale: an interview-based assessment and its relationship to cognition, real-world functioning, and functional capacity. Am J Psychiatry 163: 426–432.
- 37. Ventura J, Cienfuegos A, Boxer O, Bilder R (2008) Clinical global impression of cognition in schizophrenia (CGI-CogS): reliability and validity of a co-primary measure of cognition. Schizophr Res 106: 59–69.
- 38. Harvey PD, Ogasa M, Cucchiaro J, Loebel A, Keefe RS (2011) Performance and interview-based assessments of cognitive change in a randomized, double-blind comparison of lurasidone vs. ziprasidone. Schizophr Res 127: 188–194.