Bipolar disorder is accompanied by cognitive impairments, which persists during euthymic phases. The purpose of the present study was to identify those neuropsychological tests that most reliably tell euthymic bipolar patients and controls apart, and to clarify the extent to which these cognitive impairments are clinically significant as judged from neuropsychological norms.
Patients with bipolar disorder (type I: n = 64; type II: n = 44) and controls (n = 86) were examined with a comprehensive neuropsychological test battery yielding 47 measures of executive functioning, speed, memory, and verbal skills. Multivariate analysis was used to build a model of cognitive performance with the ability to expose underlying trends in data and to reveal cognitive differences between patients and controls.
Patients with bipolar disorder and controls were partially separated by one predictive component of cognitive performance. Additionally, the relative relevance of each cognitive measure for such separation was decided. Cognitive tests measuring set shifting, inhibition, fluency, and searching (e.g., Trail Making Test, Color-Word) had strongest discriminating ability and most reliably detected cognitive impairments in the patient group.
Citation: Sparding T, Silander K, Pålsson E, Östlind J, Sellgren C, Ekman CJ, et al. (2015) Cognitive Functioning in Clinically Stable Patients with Bipolar Disorder I and II. PLoS ONE 10(1): e0115562. https://doi.org/10.1371/journal.pone.0115562
Academic Editor: Keith Laws, University of Hertfordshire, UNITED KINGDOM
Received: June 24, 2014; Accepted: November 25, 2014; Published: January 23, 2015
Copyright: © 2015 Sparding et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: This research was supported by grants from the Swedish Medical Research Council (K2011-61X-14647-09-3, K2010-61X-21569-01-1, and K2010-61P-21568-01-4), the Swedish Foundation for Strategic Research, the Swedish Brain Foundation, and the Swedish Federal Government under the LUA/ALF agreement (ALF 20100305, ALFGBG-142041). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Even though recurrent episodes of depression and mania are the hallmarks of bipolar disorder, an association between bipolar disorder and cognitive impairment has also repeatedly been described, foremost regarding executive function, attention, processing speed, and verbal- and episodic memory [1–3]. Yet, it is undecided whether cognitive impairment is a general trait of bipolar disorder. Bipolar patients with a history of psychotic symptoms have been suggested to be more cognitively impaired than non-psychotic patients . Also, earlier studies disagree as to whether bipolar disorder I and II differ with respect to cognitive impairment. Some studies report no differences between these subtypes [5, 6], whereas others suggest that bipolar disorder I patients perform poorer than patients with bipolar disorder type II [7–9].
A recent meta-analysis of cognition in bipolar disorders showed a large degree of inconsistency across studies and that case-control differences were smaller than previously thought . Even though cognitive impairment during euthymic periods are thought to contribute to the bipolar patient’s difficulties in everyday occupational and social functioning , the clinical significance of case-control differences in cognitive function remains to be determined.
The magnitude of these case-control differences across studies assessing cognitive functioning has guided the choice of cognitive tasks recommended by the International Society of Bipolar Disorders . However, studies of cognitive functioning have an innate difficulty concerning the choice of appropriate statistical method. While not yet on par with ‘omics’ datasets, modern neuropsychological test batteries yield dozens of inter-correlated measures, which demand efficient statistical tools to extract robust trends and to avoid false-positives. Here, we therefore used orthogonal partial least-squares to latent structures (OPLS)  in order to characterize cognition in bipolar disorder. OPLS in its discriminant analysis form (OPLS-DA) splits the systematic variation in a dataset (47 neuropsychological test results for each participant in the present case) into two parts. One is predictive of class membership (i.e., bipolar I, bipolar II, healthy controls in the present case) and the other is uncorrelated or orthogonal to the classes. This partitioning greatly facilitates model interpretation and identifies the combination of neuropsychological variables that separate pre-defined groups.
The aims of this study was (i) to identify those neuropsychological tests that most reliably tell euthymic bipolar patients and healthy controls apart; (ii) to clarify the extent to which these cognitive impairments are clinically significant in a neuropsychological sense as opposed to merely statistically different from a healthy control group; (iii) to elucidate if patients with bipolar disorder I and II are cognitively dissimilar; (iv) to elucidate if the degree of cognitive impairment has bearing on psychosocial and clinical variables. We studied these questions in a clinical cohort of euthymic bipolar patients and matched healthy controls that had completed a comprehensive neuropsychological test battery. Some parts of the neurocognitive data from patients and controls enrolled in this study have been used in a previous study that used univariate analysis to compare cases and controls . Here, we the OPLS-DA procedure that we hypothesized would uncover information on neurocognitive performance that is unavailable when using univariate analyses.
The project was approved by Stockholm Regional Ethical Review Board and all participants consented to take part in the study. The data derive from St. Göran Bipolar Project, which has been described in detail previously [14–16]. Briefly, patients were recruited from the Affective Center at Northern Stockholm Psychiatry. The diagnostic instrument for bipolar disorder was the Affective Disorder Evaluation (ADE), which is a semi-structured interview that includes adapted versions of the mood and psychosis modules of the Structured Clinical Interview for DSM-IV (SCID), and was developed for the Systematic Treatment Enhancement Program of Bipolar Disorder (STEP-BD) project . These diagnoses were confirmed by a consensus panel of experienced clinicians whereby a best estimate diagnosis was reached. Thus 64 participants met the criteria for bipolar I disorder and 44 for bipolar II disorder (n = 44). Co-morbid psychiatric disorders were screened for by using the Mini International Neuropsychiatric Interview (M.I.N.I.) . To screen for alcohol and substance abuse, the self-report questionnaires Alcohol Use Disorders Identification Test (AUDIT)  and Drug Use Disorders Identification Test (DUDIT)  were used. In addition to information regarding age, sex, and level of education, records were available concerning age at first symptom, age at first psychosis (if any), lifetime history of psychosis, number of affective episodes, electroconvulsive treatments, number of sick leave-days the last year, and primary income source. Data on medication was collected at the baseline diagnostic assessment and the somatic examination. Drug information was therefore harvested from the date closest in time to the date of testing. The severity of bipolar disorder was rated using the Clinical Global Impression (CGI)  rating scale. Overall functioning was assessed with GAF .
The age- and sex-matched healthy, population-based control subjects (n = 86) were randomly selected by Statistics Sweden (SCB). The screening for past or present psychiatric disorders was performed using M.I.N.I. . The exclusion criteria for healthy controls have been described in detail previously .
Some parts of the neurocognitive data from patients and controls enrolled in this study have been used in a previous study that used univariate analysis to compare cases and controls .
Neuropsychological test procedure
Participants were assessed with 21 tests, most of which are described in detail by Lezak et al.,  tapping key aspects of cognition, including executive function and attention, processing speed, memory and verbal skills. The battery usually required two sessions with patients, whereas the controls were assessed during a single session. The following tests were used.
The Claeson-Dahl Verbal Learning and Retention Test is a word list learning task that presents 10 words for a maximum of ten learning trials. Measures of importance are the learning score, the retention score, and the recognition score.
Five stand-alone tests from the Delis-Kaplan Executive Function System (D-KEFS): the Color-Word Interference Test (condition 1: Color Naming, condition 2: Word Reading, condition 3: Inhibition, condition 4: Inhibition/Switching), the Design Fluency Test (condition 1: Filled Dots, condition 2: Empty Dots Only, condition 3: Switching, the Tower Test (total achievement Score and total rule violations), the Trail Making Test (condition 1: Visual Scanning, condition 2: Number Sequencing, condition 3: Letter Sequencing, condition 4: Number-Letter Switching and condition 5: Motor Speed), the Verbal Fluency Test (condition 1: Letter fluency, condition 2: Category Fluency and condition 3: Category Switching).
All tests comprising the Wechsler Adult Intelligence Scale (WAIS-III), except the Object Assembly Test.
The Continuous Performance Test II (CPT-II) with number of commission and omission errors and reaction time.
The Rey Complex Figure Test (RCFT) with measures of immediate and delayed recall plus recognition.
For each cognitive measure, skewness and kurtosis were determined and then when appropriate data were transformed according to the ladder of powers. The direction of the results was adjusted such that high scores represented good performance. Following unit variance scaling and mean-centering, data were modeled by means of OPLS-DA, implemented by SIMCA-P 13.0 software (Umetrics AB, Umeå, Sweden). The OPLS-DA procedure identifies correlation patterns that discriminate between pre-defined groups and assesses the relative importance of each test variable for the discrimination . Furthermore, conventional PLS calculations were performed, which apply to the two-block regression problem. Psychosocial and clinical variables (e.g., MADRS scores, number of depressions) were used to construct models of cognitive performance (47 neuropsychological measures) in patients.
Table 1 shows demographic and clinical characteristics for the bipolar disorder group and the control group. Premorbid IQ as assessed by years of education was not significantly different between the bipolar disorder group and the control group (F (2,193) = 1.02, p > 0.05, Partial Eta Squared = 0.01). As expected, patients with bipolar I disorder showed a higher occurrence of prior psychosis and treatment with antipsychotics than bipolar II patients.
The OPLS-DA procedure yielded a model that was significant by cross-validation. The predictive component accounted for 13% of the neuropsychological variation, with a prediction ability of 0.16, according to cross validation. This indicates that the case and control groups were partially separated on the basis of their cognitive performance, the overall pattern being that the two bipolar groups performing similar and somewhat poorer than controls (Fig. 1).
Each participant’s score is represented by a circle. The scores were t values on the component predictive of diagnostic group. The vast majority (97%) of the participants were within a ±2 standard deviation limit according to Hotelling’s T2. Positive values represent better overall performance.
The order of presentation of the individual neuropsychological measures in Table 2 reflects how well they contributed to the separation between the groups (i.e., the size of the OPLS-DA loadings). The tests with the strongest class discriminating ability were Trail Making Test 2 (number sequencing), Trail Making Test 3 (letter sequencing), Trail Making Test 4 (number-letter switching), Symbol Search, Verbal Fluency 2 (category fluency), Verbal Fluency 3 (category switching), Color-Word 3 (inhibition) and Color-Word 4 (inhibition/switching). Table 2 also presents the mean and 95% confidence interval (CI) for each measure. There was little CI overlap between bipolar patients and healthy controls on the top-loading measures, suggesting reliable case-control differences. By contrast, the CIs of the two patient groups (bipolar I and II) intersected in the vast majority of cases.
Table 2 also shows that the proportion of patients whose performance fell 1.25 standard deviations below controls was sizeable on the top-loading tests. For instance, 48% of patients with bipolar I disorder and 43% of patients with bipolar II disorder fulfilled this criterion for impairment on Trail Making Test 4.
A series of PLS models were created using patient data in order to assess the extent to which clinical and psychosocial variables could be understood in terms of cognitive performance. Regressing MADRS and YMRS scores separately against cognition yielded non-significant models, suggesting that the neuropsychological performance was not related to mood within the present scale interval (i.e., MADRS/YMRS scores below 14). Likewise, non-significant models were obtained when cognitive performance was regressed against GAF and CGI scores, the number of affective episodes, ADHD co-morbidity, a history of psychosis, or number of sick-leave days. Non-significant models were also obtained when regressing ongoing drug treatment (antidepressants, antipsychotic medication expressed as CPZ equivalents, lithium, and anticonvulsants) to cognitive performance in patients.
Neuropsychological test batteries generate large datasets with many variables that are challenging to summarize and evaluate using traditional statistical tools that primarily are designed to analyze ‘long-and-lean’ data tables . Some of the between subject variation in performance on a broad cognitive battery is shared by all tests, therefore the performance on a single test is related to latent mental ability required to perform all tests . Statistical tools developed to deal with large datasets with inter-correlated measures in other research areas may therefore be useful for the study of cognition in mental illness. Here, we employed the OPLS-DA to sift the results from 21 neuropsychological tests, yielding 47 performance measures, in patients with bipolar disorder and healthy controls, the aim being to identify specific tasks that detect cognitive weaknesses in the patient group.
As applied here, OPLS-DA partitioned the variation in neuropsychological performance into two components, one being associated with group membership and the other irrelevant. When examining the explained variance in more detail it was apparent that only measures from nine tests defined the class separation: Trail Making Test, Verbal/Design Fluency, Color-Word, Tower Test, Symbol Search, Block Design, Letter-Number Sequencing, and Symbol Coding. The remaining 12 tests, including measures of semantic knowledge, verbal learning and some tests of working memory/attention, did not tell the groups apart.
The combination of nine identified tests may hence be considered for identifying cognitive impairment in bipolar patients in research, and possibly in a clinical setting as well. The set of tests identified here are with some exceptions the same as, or similar to, the ones recommended for inclusion in a battery for cognition in bipolar disorder by the International Society for Bipolar Disorder (ISBD) on the basis of meta-analyses . For instance, the usefulness of the Trail Making Test A, Letter-Number Sequencing, Category Fluency and Digit Symbol Coding was confirmed in our study. Moreover, our analysis revealed the importance of TMT 4 (similar to TMT B) and Color-Word Interference Test, which are in line with the result from a recent evaluation of the clinical efficacy of the tests recommended by ISBD .
However, some of the tests proposed by Yatham et al.  received relatively low loadings in the present study and did not differentiate between patients and controls. These included tests for attention/vigilance (Continuous Performance Test), verbal learning and memory (Claeson-Dahl Verbal Learning and Retention Test), and visual learning (most measures of the Rey Complex Figure Test). Our analysis reveals that some of the tests suggested by Yatham et al.  might be redundant, being unable to differentiate between patient and controls. It could also be that patients with problems in these domains may constitute a cognitive subgroup that was insufficiently represented to gain statistical significance in the present sample.
It should be noted that the tests used in present study are diagnostically non-specific. For example, deficits on the Trail Making Test have been observed in conditions as disparate as posttraumatic stress disorder  and myotonic dystrophy type 1 . Nevertheless, the tests do go some way in directing researchers’ attention to particular cognitive domains such as, in the present case, searching, fluency, switching and inhibition, i.e., processes with a distinct ‘prefrontal’ flavour, which could be specified further using more fine-grained cognitive instruments.
As to the question whether the cognitive impairment in bipolar disorder is clinically significant, inspection of the group means and their CIs for the nine discriminatory tests confirmed their usefulness for identifying impairments in the patient groups. For a substantial minority of the patients the impairments approached clinical significance (as defined by a performance 1.25 SD below controls) on certain tests (e.g., for >40% on Trail Making Test 4 and Block Design tests). Such findings may be rooted in the fact that bipolar disorder is associated with palpable changes in brain structure, such as ventricular enlargement and shrinkage of medial temporal lobe areas . A reduction in temporal lobe gray matter has been associated with decline in intellectual function and with numbers of mood episodes in patients with bipolar disorder . However, we emphasize that the OPLS-DA-aided group separation was partial and incomplete, with non-trivial overlap in the overall cognitive performance between patients and controls (see Fig. 1). Hence, the observed group differences could not be used for predicting diagnoses on an individual level.
We investigated whether the subtypes of bipolar disorder differ in neuropsychological terms but found that patients with bipolar I disorder were indistinguishable from patients with bipolar II disorder on 16 of the 18 cognitive measures (89%) with discriminant loadings ≥ 0.15. This correspond to the results from our own group where we used measures of verbal and visual memory test and executive function to test this hypothesis using univariate statistics in the same cohort, and also the findings by a previous study by Dittman et al. , but contrasts with the results showing that bipolar I patients have more severe cognitive dysfunction compared with patients with bipolar II [7–9]. By and large, the cognitive impairments seen in patients with bipolar I and bipolar II disorder are hard to differentiate and appear to have much in common [5, 6].
Bipolar disorder can take on psychotic manifestations requiring antipsychotic medication, and is commonly accompanied with excessive alcohol/drug use  and psychosocial and occupational problems . Difficulties of this nature were observed also in the present patient group (see Table 1) and might conceivably have contributed to the observed cognitive deficits [35, 36]. Our attempts to model these relationships showed that medication alone could not explain the variance in cognitive performance, which was somewhat surprising given that our own group showed that treatment with antipsychotics was associated with worse performance on the time to draw parameter of the Rey complex figure test, number sequencing, letter sequencing and number-letter switching conditions of the Trail making test and all trials of the Verbal fluency test . An explanation might be that the multivariate modelling creates a new summarizing variable, which captures a latent structure in cognitive performance and is somewhat different from the original variables. However, results in the current study are in line with re-analysis of earlier studies suggesting that most neuropsychological tests do not exhibit any significant association with ongoing pharmacological treatment .
As to the other clinical and psychosocial background factors (viz. co-morbidity with ADHD, substance/alcohol abuse, GAF scores, number of affective episodes or sick-leave days), the associations were not stable enough to gain statistical significance. The modest associations with everyday occupational and social factors may suggest that the tests used here are of questionable ecological validity . Moreover, a meta-analysis, investigating the association between cognitive ability and everyday functioning in bipolar disorder showed results similar to that seen in schizophrenia, with small differences across cognitive domains . The strength of association differed to a greater extent with respect to the functional measurement approach. Nevertheless, the lack of associations in current study are consistent with a recent re-analysis of studies in the field showing that the majority of cognitive measures were not associated with measures of illness severity .
By including patients scoring between 0 and 13 on the MADRS/YMRS scales, the criterion for defining euthymia in the present study was more liberal than in earlier studies . This raises the possibility that residual mood symptoms significantly affected performance in the cognitive tasks . However, the shared variance between the mood scale values and overall cognitive performance was not sufficiently stable to generate a valid statistical model. In line with previous studies, [2, 40, 41] we conclude that mood have limited value in explaining the cognitive impairments in euthymic bipolar patients.
The strengths of the current study are that patients are representative of bipolar patients that receive psychiatric care. At the time of enrollment, virtually all patients with bipolar disorder in the Northern Stockholm catchment area were referred to the Affective unit for work up and treatment. Moreover, the diagnoses have been made with a best-estimate procedure by experienced clinicians specialized in bipolar disorder. The neuropsychological test battery was administered under standardized conditions. Controls were randomly selected from the population in the same catchment area and matched for sex and age rather than being university students or health care workers. A limitation to consider, however, is that information on drug use was not collected on the day of testing, but from either the baseline examination or the day of blood sampling. On the other hand, enrolled patients were in a stable euthymic mood and the overwhelming majority of patients would not have added or discontinued medication between these occasions.
We wish to thank the staff at the St. Göran bipolar affective disorder unit, including coordinator Martina Wennberg, study nurses Agneta Carlswärd-Kjellin, Lena Lundberg, and Benita Gezelius, and data managers Haydeh Olofsson and Mathias Kardell. We thank Björn Hultman and Pascal Borgström for designing the neuropsychological test battery. We also thank the patients and controls participating in this study.
Conceived and designed the experiments: TS KS EP SH ML. Performed the experiments: JO CS CJE ML. Analyzed the data: TS KS SH EJ. Wrote the paper: TS EP SH ML.
- 1. Robinson LJ, Thompson JM, Gallagher P, Goswami U, Young AH, et al. (2006) A meta-analysis of cognitive deficits in euthymic patients with bipolar disorder. Journal of affective disorders 93: 105–115. pmid:16677713
- 2. Torres IJ, Boudreau VG, Yatham LN (2007) Neuropsychological functioning in euthymic bipolar disorder: a meta-analysis. Acta Psychiatr Scand Suppl: 17–26.
- 3. Mann-Wrobel MC, Carreno JT, Dickinson D (2011) Meta-analysis of neuropsychological functioning in euthymic bipolar disorder: an update and investigation of moderator variables. Bipolar Disorders 13: 334–342. pmid:21843273
- 4. Bora E, Yucel M, Pantelis C (2010) Neurocognitive markers of psychosis in bipolar disorder: a meta-analytic study. J Affect Disord 127: 1–9. pmid:20231037
- 5. Dittmann S, Hennig-Fast K, Gerber S, Seemuller F, Riedel M, et al. (2008) Cognitive functioning in euthymic bipolar I and bipolar II patients. Bipolar Disord 10: 877–887. pmid:19594503
- 6. Palsson E, Figueras C, Johansson AG, Ekman CJ, Hultman B, et al. (2013) Neurocognitive function in bipolar disorder: a comparison between bipolar I and II disorder and matched controls. BMC Psychiatry 13: 165. pmid:23758923
- 7. Sole B, Bonnin CM, Torrent C, Martinez-Aran A, Popovic D, et al. (2012) Neurocognitive impairment across the bipolar spectrum. CNS Neurosci Ther 18: 194–200. pmid:22128808
- 8. Torrent C, Martinez-Aran A, Daban C, Sanchez-Moreno J, Comes M, et al. (2006) Cognitive impairment in bipolar II disorder. Br J Psychiatry 189: 254–259. pmid:16946361
- 9. Simonsen C, Sundet K, Vaskinn A, Birkenaes AB, Engh JA, et al. (2008) Neurocognitive profiles in bipolar I and bipolar II disorder: differences in pattern and magnitude of dysfunction. Bipolar disorders 10: 245–255. pmid:18271903
- 10. Bourne C, Aydemir O, Balanza-Martinez V, Bora E, Brissos S, et al. (2013) Neuropsychological testing of cognitive impairment in euthymic bipolar disorder: an individual patient data meta-analysis. Acta psychiatrica Scandinavica 128: 149–162. pmid:23617548
- 11. Burdick KE, Goldberg JF, Harrow M (2010) Neurocognitive dysfunction and psychosocial outcome in patients with bipolar I disorder at 15-year follow-up. Acta Psychiatr Scand 122: 499–506. pmid:20637012
- 12. Yatham LN, Torres IJ, Malhi GS, Frangou S, Glahn DC, et al. (2010) The International Society for Bipolar Disorders-Battery for Assessment of Neurocognition (ISBD-BANC). Bipolar Disord 12: 351–363. pmid:20636632
- 13. Eriksson L, Byrne T, Johansson E, Trygg J, Wikström C (2013) Multi- and megavariate data analysis: basic principles and applications. Malmö: MKS Umetrics. 500 p.
- 14. Ekman CJ, Lind J, Rydén E, Ingvar M, Landén M (2010) Manic episodes are associated with grey matter volume reduction - a voxel-based morphometry brain analysis. Acta Psychiatr Scand 122: 507–515. pmid:20712826
- 15. Ryden E, Johansson C, Blennow K, Landen M (2009) Lower CSF HVA and 5-HIAA in bipolar disorder type 1 with a history of childhood ADHD. Journal of Neural Transmission 116: 1667–1674. pmid:19756368
- 16. Ryden E, Thase ME, Straht D, Aberg-Wistedt A, Bejerot S, et al. (2009) A history of childhood attention-deficit hyperactivity disorder (ADHD) impacts clinical outcome in adult bipolar patients regardless of current ADHD. Acta Psychiatrica Scandinavica 120: 239–246. pmid:19426162
- 17. Sachs GS, Thase ME, Otto MW, Bauer M, Miklowitz D, et al. (2003) Rationale, design, and methods of the systematic treatment enhancement program for bipolar disorder (STEP-BD). Biological psychiatry 53: 1028–1042. pmid:12788248
- 18. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, et al. (1998) The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of clinical psychiatry 59 Suppl 20: 22–33;quiz 34–57. . pmid:9881538
- 19. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M (1993) Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction 88: 791–804. pmid:8329970
- 20. Berman AH, Bergman H, Palmstierna T, Schlyter F (2005) Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. European addiction research 11: 22–31. pmid:15608468
- 21. Luborsky L (1975) Clinicians’ judgments of mental health: specimen case descriptions and forms for the health-sickness rating scale. Bulletin of the Menninger Clinic 39: 448–480. pmid:1182328
- 22. W G (1976) ECDEU Assessment Manual for Psychopharmacology.
- 23. Asberg M, Schalling D (1979) Construction of a new psychiatric rating instrument, the Comprehensive Psychopathological Rating Scale (CPRS). Progress in neuro-psychopharmacology 3: 405–412. pmid:400996
- 24. Young RC, Biggs JT, Ziegler VE, Meyer DA (1978) A rating scale for mania: reliability, validity and sensitivity. The British journal of psychiatry: the journal of mental science 133: 429–435.
- 25. Jakobsson J, Zetterberg H, Blennow K, Ekman CJ, Johansson AGM, et al. (2013) Altered Concentrations of Amyloid Precursor Protein Metabolites in the Cerebrospinal Fluid of Patients with Bipolar Disorder. Neuropsychopharmacology 38: 664–672. pmid:23212456
- 26. Lezak M, Howieson D, Bigler E, Tranel D (2012) Neuropsychological Assessment: Oxford University Press Inc. 1161 p.
- 27. Deary IJ (2012) Intelligence. Annual review of psychology 63: 453–482. pmid:21943169
- 28. Van Rheenen TE, Rossell SL (2014) An empirical evaluation of the MATRICS Consensus Cognitive Battery in bipolar disorder. Bipolar Disord 16: 318–325. pmid:24119238
- 29. Koso M, Hansen S (2006) Executive function and memory in posttraumatic stress disorder: a study of Bosnian war veterans. Eur Psychiatry 21: 167–173. pmid:16139487
- 30. Winblad S, Lindberg C, Hansen S (2006) Cognitive deficits and CTG repeat expansion size in classical myotonic dystrophy type 1 (DM1). Behav Brain Funct 2: 16. pmid:16696870
- 31. Vargas C, Lopez-Jaramillo C, Vieta E (2013) A systematic literature review of resting state network-functional MRI in bipolar disorder. J Affect Disord 150: 727–735. pmid:23830141
- 32. Moorhead TWJ, McKirdy J, Sussmann JED, Hall J, Lawrie SM, et al. (2007) Progressive gray matter loss in patients with bipolar disorder. Biological Psychiatry 62: 894–900. pmid:17617385
- 33. Bizzarri JV, Sbrana A, Rucci P, Ravani L, Massei GJ, et al. (2007) The spectrum of substance abuse in bipolar disorder: reasons for use, sensation seeking and substance sensitivity. Bipolar Disord 9: 213–220. pmid:17430295
- 34. Martinez-Aran A, Vieta E, Colom F, Torrent C, Sanchez-Moreno J, et al. (2004) Cognitive impairment in euthymic bipolar patients: implications for clinical and functional outcome. Bipolar Disord 6: 224–232. pmid:15117401
- 35. Levy B, Monzani BA, Stephansky MR, Weiss RD (2008) Neurocognitive impairment in patients with co-occurring bipolar disorder and alcohol dependence upon discharge from inpatient care. Psychiatry Res 161: 28–35. pmid:18752854
- 36. Torres IJ, Kozicky J, Popuri S, Bond DJ, Honer WG, et al. (2014) 12-month longitudinal cognitive functioning in patients recently diagnosed with bipolar disorder. Bipolar Disorders 16: 159–171. pmid:24636366
- 37. O’Shea R, Poz R, Michael A, Berrios GE, Evans JJ, et al. (2010) Ecologically valid cognitive tests and everyday functioning in euthymic bipolar disorder patients. J Affect Disord 125: 336–340. pmid:20609481
- 38. Depp CA, Mausbach BT, Harmell AL, Savla GN, Bowie CR, et al. (2012) Meta-analysis of the association between cognitive abilities and everyday functioning in bipolar disorder. Bipolar Disorders 14: 217–226. pmid:22548895
- 39. Clark L, Iversen SD, Goodwin GM (2002) Sustained attention deficit in bipolar disorder. Br J Psychiatry 180: 313–319. pmid:11925353
- 40. Mann-Wrobel MC, Carreno JT, Dickinson D (2011) Meta-analysis of neuropsychological functioning in euthymic bipolar disorder: an update and investigation of moderator variables. Bipolar Disord 13: 334–342. pmid:21843273
- 41. Bourne C, Aydemir O, Balanza-Martinez V, Bora E, Brissos S, et al. (2013) Neuropsychological testing of cognitive impairment in euthymic bipolar disorder: an individual patient data meta-analysis. Acta Psychiatr Scand 128: 149–162. pmid:23617548