Introduction

This article describes the development of a core outcome set recommended for use in community-based trials for adults with bipolar, as part of the PARTNERS2 study. The PARTNERS2 study aims to help integrate primary care and community-based mental health services, further methodological detail and rationale for methodological decisions are available in the published protocol [1]. The term “bipolar” is used throughout this paper and in this research in place of “bipolar affective disorder” [2], as the preferred term of research team members with lived experience of bipolar and Bipolar UK, the leading charity in this area. This been understood in this research in accordance with the definition and scope defined in the International Classification for Diseases (ICD-10) [2] (the ICD-11 will come into effect in 2022 [3]).

Bipolar is defined as “two or more episodes in which the [person]'s mood and activity levels are significantly disturbed, this disturbance consisting on some occasions of an elevation of mood and increased energy and activity (hypomania or mania) and on others of a lowering of mood and decreased energy and activity (depression)”. Life expectancy among people with a diagnosis of bipolar is reduced when compared with the general population [4–6] and the mortality gap appears to be widening [7]. In addition, individuals with bipolar experience increased unemployment and stigma [8], and bipolar research is under-researched when compared to other mental health research [9]. There is a growing primary evidence base that indicates the efficacy of psychological treatments (e.g. non-pharmaceutical interventions such as cognitive-behavioural therapy, family interventions, and psycho-education) for bipolar and its long-term management; however, meta-analyses are undermined by poor quality evidence [10]. A better understanding of interventions and to make comparisons between them requires data from well-designed and conducted randomised controlled trials (RCTs) [11] using a unified approach to outcome selection.

Trialists evaluate the effectiveness of an intervention in a clinical trial by choosing outcomes that reflect any beneficial or harmful effects—these can be specific, such as change in weight, or broad constructs, such as pain [12]. Outcomes can be measured in several ways, including the use of laboratory findings, biomarkers, or mortality; or they can be reported by observers, clinicians, or the patient themselves. RCTs can provide robust evidence to inform clinical decision making and health care policy development [13], but the inconsistent use of highly varied trial outcomes within the same research area can undermine evidence synthesis. Additionally, for outcome data to be useful, the outcomes used must be of relevance to a range of stakeholders [14], including people with a bipolar diagnosis, carers and healthcare professionals [12, 15]. Commonly, the outcomes used in bipolar research have focused on clinical outcomes, such as change in symptoms as assessed by clinicians. However a mounting view in mental health research suggests that a broader set of outcomes may better suit the goals that people with diagnoses seek to achieve during treatment. This has been indicated in the case of schizophrenia [16, 17], and the Bipolar Priority Setting partnership suggests this is also the case for those with a bipolar diagnosis [18].

A core outcome set (COS) is a standardised collection of outcomes recommended to be reported in all controlled trials within a research area [12]. A COS represents the minimum outcomes to be measured and reported when undertaking a trial [19]. A COS for use in trials for those receiving non-pharmaceutical community-based interventions for bipolar (rather than as a hospital in-patient) could reduce reporting bias and enable evidence synthesis. The aim of this research is to develop such a COS.

This is the first study to develop a COS for community-based bipolar trials but builds upon an effort in the field to unify the outcomes and priorities within bipolar research. In 2010, the development of two “core sets” for bipolar based on the International Classification of Function, Disability and Health (ICF) guide [20] began. Focusing on functioning, the core sets use the ICF guide for bipolar, and while they may be used as outcome measures in research settings, the main intention of these core sets are for use in clinical practice. In 2015 a set of “patient important outcomes” [21] were published, aiming to investigate the relative importance of bipolar outcomes from the perspective of patients. The set of “patient important outcomes” considered the views of a single stakeholder group, those of people with a bipolar diagnosis, whereas the COS developed in our research included a range of information sources and engaged the views of different key stakeholders, allowing for the potential identification of gaps [22]. In comparison with the COS developed in our research, the ICF study constructed a shortlist of treatment outcomes relevant in the evaluation and selection of pharmacological treatments and was not conceptualised with intention for use in community-based bipolar trials. Similarly, the ICF core sets [23] were not developed with the sole intention of its use to be in community-based bipolar trials, as has the COS described here. In 2016, the James Lind Alliance (JLA) published a Priority Setting Partnership (PSP) about bipolar [24]. This provides researchers with 10 new bipolar research priorities into which the COS could be adopted and demonstrates increasing interest in bipolar research. The Royal College of Psychiatrists have also provided an overview of outcome measures for use in adult psychiatry [25].

Methods

Ethics statement

Ethical approval was sought and granted from the National Research Ethics Service (NRES) West Midlands—Edgbaston (Reference Number: 14/WM/0052). The research was registered with the COMET (Core Outcome Measures in Effectiveness Trials) Initiative [26] and is reported in adherence with the COS-STAR Statement [27] and the GRIPP2-SF [28]. The study was conceived prior to publication of the COS Standards for Development [29] recommendations but is in alignment with its standards.

A co-production approach, drawing upon the expertise of academics, healthcare professionals, people with bipolar, and carers, was using in this research. Experiential expertise from people with a bipolar diagnosis and carers was facilitated through the PARTNERS2 patient and public involvement (PPI) programme. Peer researchers with a diagnosis of bipolar were employed within the research team and three Lived Experience Advisory Panels (LEAPs) were established, consisting of an average of 5 people with schizophrenia and bipolar diagnoses and family members with experience of caring for individuals with mental health diagnoses. Research team members and advisors with lived experience advised on and provided strategies to ensure the research phases and output would be relevant and useful to those with bipolar. Research team members with lived experience were involved in all of the tasks required for the fulfilment of the research, including collection and analysis of data, in the same manner as those without lived experience. However, research team members with lived experience had an additional role whereby they would provide ongoing advice to colleagues that would ensure the research and the manner in which it was conducted remained relevant and appropriate to people with bipolar, for whom the COS would be finally intended. In this paper, references to the “research team’, this should always be taken to include those research team members with lived experience. The LEAPs discussed and advised on the phases of COS development on 8 occasions between February 2015 and November 2016. The LEAPs often discussed the same phases at multiple meetings. Work undertaken by LEAP members included commenting and advising on consent and information materials (in particular ensuring use of accessible language), recruitment strategies, and delivery of research. Meetings were held with individual members of the LEAP on two occasions to pilot the Delphi interface.

This article details the process through which an outcome longlist was developed and subjected to rating and refining via a two round Delphi survey and a stakeholder consensus meeting, resulting in a COS recommended for use in community-based bipolar trials. The COS was developed via three phases: 1) identifying a longlist of outcomes from focus group discussions, one-to-one interviews, and a rapid review, 2) refining the outcome longlist using Delphi methodology, and 3) finalising the COS in a consensus meeting as the last part of the Delphi (see Fig 1). The three phases involved input from several stakeholders (see Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Illustration of core outcome set development process.

https://doi.org/10.1371/journal.pone.0240518.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overview of stakeholder involvement per phase.

https://doi.org/10.1371/journal.pone.0240518.g002

The authors can confirm that the development of the bipolar COS reported was undertaken in adherence to the published protocol (steps 1–3), with the exception of use of 2 rounds of Delphi survey instead of 3, to mediate sample attrition. The latter steps of the published protocol, 4 and 5, will be completed and reported separately. The schizophrenia COS is still in development.

Results

Phase 3

Fifty Delphi participants were recruited. Delphi participants were recruited between September and December 2016, during which Round 1 was open. All participants participated via the online survey. Round 2 was open from December 2016-February 2017. Ninety-three individuals were contacted via known and referred contacts of the research team and a 32% (n = 30) recruitment rate was achieved. Twenty participants were recruited through the LEAPs, support groups, social media, and third party organisations.

A process of monitoring and reminders was used to ensure completion, resulting in a 76% return rate (n = 38) to Round 2 of the Delphi (see S5 File). Fifteen people with a bipolar diagnosis participated, 2 of whom identified as male and 13 as female; 12 identified as White British, 1 as British Asian or Asian, 1 as mixed heritage, and 1 did not specify. While the screening tool was used to promote sample representativeness, nobody was excluded from the Delphi on this basis (see S6 File). Four carers participated, 1 identified as male and 3 as female; 3 identified as White British and 1 as British Asian or Asian. Twenty-three healthcare professionals participated, 12 identified as male and 11 as female; 19 identified as White British, 2 as British Asian or Asian, and 2 did not specify. Eight researchers participated, all identifying as White British, 1 identifying as male and 7 as female.

Sixty-six outcomes were included in the Delphi questionnaire and 13 outcomes were added by participants during Round 1. Three outcomes were suggested by Round 1 Delphi participants and rated as important by participants in Round 2, so were included in the consensus meeting discussion. A total of 60 outcomes met the pre-specified criteria for automatic inclusion into the COS (see S7 File), 56 from original longlist and 4 added in Round 1 by Delphi participants. Fifteen of these outcomes were merged with other outcomes by the research group and then the remaining 45 outcomes were provisionally grouped into 11 outcome domains, (see S8 File) to ensure the final COS would be feasible for use in future trials.

The consensus meeting, co-chaired by an outcome measurement researcher and a peer researcher, took place in September 2017 and was attended by 14 people: 6 healthcare professionals, 5 people with a bipolar diagnosis, 2 carers and 1 researcher. Table 1 shows the results of the voting and discussion.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Final core outcome set, voting rounds and scores.

https://doi.org/10.1371/journal.pone.0240518.t001

Discussion

The final COS consists of 11 outcome domains: personal recovery; connectedness; clinical recovery of bipolar symptoms; mental health and wellbeing; physical health; self-monitoring and management; medication effects; service outcomes; service user experience of care; and use of coercion. The development of the COS has drawn upon several sources, including a rapid review of the Cochrane database and qualitative work with key stakeholders. We recommend that researchers use this COS to inform their selection of measures used in future community-based bipolar trials. There are a range of measures currently available for use, however, we would recommend that when choosing measures, teams do so with consultation from key stakeholders including methodologists, clinicians, and those with lived experience, in addition to considering psychometric properties of measures and their alignment with research aims. Further research is required to identify which measures should be recommended for each COS outcome, however these would require regular review and may vary due to the particular requirements of each study.

The longlist of 66 outcomes first included in the Delphi survey were rated highly by participants, and as a result, 56 (85%) of these were automatically included in the proposed COS arrangement discussed at the consensus meeting. This suggests the process of outcome identification and refining used in this work has generated a large number of outcomes that were relevant to the stakeholders involved. Bipolar is a condition that impacts on every aspect of a person’s life, and thus the detailed outcome identification undertaken within the COS process reflected this extensive process and all-encompassing impact. However, use of the COS would need to be feasible in trials while retaining all outcomes rated as important by participants. Grouping items into higher-level outcome domains during the consensus meeting served this purpose. The large number of items included in the final COS was discussed at length amongst the research team and at the consensus meeting. It was felt it was important to adhere to our protocol and retain items that met the pre-specified threshold, particularly as stakeholders had strongly indicated their importance. There is potential for several items to be assessed together, for example, using health related quality of life or satisfaction questionnaires. Core outcomes sets recommend ‘what’ to measure; further research is required to evaluate the measurement properties of assessment tools, map these to the outcomes identified and reach consensus on the optimal ways to assess these outcomes in a standardised way.

Ongoing validation is necessary to ensure external validity; the continued relevance and importance of the outcomes; to evaluate implementation; and engage additional stakeholders [1]. This research was undertaken with inclusion of participants based in England, Wales and Scotland. Further research is required to assess the validity of the COS in specific populations, such as black, Asian and minority ethnic (BAME) communities in the UK, and to achieve greater consensus on its applicability in international settings, involving expert panels and stakeholders from the widest possible range of nations and communities. Efforts are also required to ensure the adoption and endorsement of the COS with funders, journals, and others involved in the development, facilitation and undertaking of bipolar research. Uptake of the COS [41], its implementation [42], and the consistency of its measurement [43] can be assessed through review of future community-based bipolar trials and their publication outputs.

The strengths of this research include that stakeholders were given multiple opportunities throughout to identify and remedy any gaps in the outcomes longlist and proposed core outcome set as well as the consensus generating nature of the Delphi. The relevance and importance of the COS is greatly strengthened overall through extensive lived experience input at each stage. In addition to the contribution of people with a bipolar diagnosis and carers as research participants in the qualitative component, this work has drawn upon the expertise of peer researchers employed on PARTNERS2 and LEAPs throughout the process, including in the analysis of results. This ensures the COS has validity for people directly experiencing bipolar and seeking support and treatment, as well as for health care professionals, researchers and commissioners. Of the researchers gathering initial qualitative data, one had personal experience of bipolar; three LEAPs were consulted regularly and redefined and reworded the outcome descriptors for the Delphi survey; the Delphi and consensus meeting included equal numbers of professionals and people with lived experience of bipolar; and the consensus meeting was co-chaired by a peer-researcher with a bipolar diagnosis. Deliberations involving this full range of stakeholders on equal terms led to a more accessible and salient COS. Each outcome included in the COS is accompanied by an explanatory guide to aid interpretation and facilitate the selection of suitable measures without ambiguity about the intended meaning as understood by our participants, team members and advisors. The resulting COS aligns with accepted definitions of outcomes, indicated notably in its inclusion of outcomes relating patient experience [25].

Limitations include that the rapid Cochrane review identified outcomes used in systematic reviews which had the benefit of allowing a rapid and practical review, however the elicitation of outcomes through this methodology may not be complete as systematic reviews will not include all outcomes used in research within a given field. In addition to this, the outcomes identified in this way may not have been chosen by a range of stakeholders including those with lived experience, clinicians, and researchers.

Further limitations relate to the Delphi sample. The first is that the final sample 50 is relatively small, particularly given that four stakeholder groups were recruited. Secondly, is the issue of participant attrition between rounds during the Delphi survey. This is, however, mediated because the data collected in the first round was included in the summarised results presented to Round 2 participants regardless of whether the corresponding participant had returned, and would have been used to inform the Round 2 ratings. Additionally, Delphi participants were asked to self-assign to one of four stakeholder groups during survey registration– 1) person with a bipolar diagnosis; 2) carer; 3) health/social care professional; 4) researcher/policy maker. During the Delphi development it was indicated that participants may identify with more than one of these categories and as such, they were invited to assign to whichever group with which they identified primarily. Open text allowed participants to further elaborate about the breadth of their experience, however these were not able to be included in analyses. Further to this, while efforts were made to ensure the diversity of the sample of individuals recruited, men and those from BAME communities, specifically those of Black, African, Caribbean, Black British origin, are underrepresented. This may be addressed with additional validation exercises with further groups and communities.

This research has used robust methodology to develop a COS for community-based bipolar trials. Its adoption in future studies will enable the generation of coherent, stakeholder-relevant outcome data that may strengthen meta-analyses and promote the value of bipolar research and the integration of subsequent findings into clinical practice.

Supporting information

Acknowledgments

Thanks to Paula Williamson for her methodological advice throughout; Dr Ben Gray for his contribution into the qualitative data analysis; the Lived Experience Advisory Panels and the wider PARTNERS2 team for their input and support of this work.