Study population and data source

Connected Bradford is a health research database connecting de-identified, longitudinal, near-real-time data from different organisations across the Bradford and Airedale region of England, UK [29]. It uses a whole-system framework that links patient’s electronic healthcare records with regional databases covering housing, social welfare, crime, education, environment, and more [30]. Consent to use healthcare data for research is granted via the UK National data opt-out scheme prior to acquisition by Connected Bradford [31]. At the time of this study, the primary-care database excluded “sensitive” clinical codes (supplementary material S1 Table).

The study population is intended to represent patients within the Connected Bradford database who have or might have complex mental health difficulties. As such, it comprises anyone with mental ill-health but who does not have a diagnosis of a severe mental illness (specifically, schizophrenia or bipolar disorder). This was defined as: any person with a record in the primary-care tables within the Connected Bradford database; aged between 18 and 70, inclusive; who have been registered with their general practice for at least one year; who, additionally, either have a record of a SNOMED-CT diagnostic code of interest within ten years prior to 31^st December 2021 (see link associated with “Mental disorder | SCTID: 74732009 + child codes” in S2 Table), or who have a record of prescriptions for medicines of interest (Table 1) within ten years prior to 31^st December 2021; excluding those people who have a record of a SNOMED-CT diagnostic code of schizophrenia or bipolar disorder.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Parent SNOMED-CT clinical codes used to define the caseness of complex mental health difficulties. All child codes of parent codes were searched, too. Also showing medication names used to indicate active caseness (i.e., ten years prior to the 31^st December 2021). Medications were searched by name. The full codelist is available in supplementary material S2 Table.

https://doi.org/10.1371/journal.pone.0322771.t001

Our SNOMED-CT diagnostic codes of interest were defined by the complete list of SNOMED-CT concept descendants of ‘74732009 | Mental Disorder’ within the Clinical Finding domain. SNOMED-CT codes for schizophrenia and bipolar were excluded because these conditions indicate severe mental ill-health that is outside the scope of this study. Codes for dementia were excluded because they might warrant off-label prescribing of antipsychotics despite not being associated with personality disorder. Our medicines of interest were specified by the General Practitioner members of the research team to represent commonly-used antidepressants, anxiolytics, hypnotics, and antipsychotics. All codelists used in this study can be found by following the URLs in supplementary material S2 Table.

Data governance and management

Access to Connected Bradford database was via Google Cloud Platform. Bradford Teaching Hospitals NHS Foundation Trust is the data controller of the Connected Bradford database. The project Principal Investigators (co-authors PO and CB) hold overall responsibility for data management.

Institutional ethical approval was granted for our particular study (University of Sheffield; ref 047008), and for all research using the Connected Bradford data (NHS HRA reference: 22/EM/0127). All data were fully anonymized before access.

Disclosure refers to the re-identification of individuals, households, or organisations, should data users attempt to do so [32]. To produce descriptive statistics and feature sets, raw counts were redacted if less than or equal to seven, then rounded to the nearest ten, before any calculations.

Caseness variable definition

We distinguished cases and controls within the records of our study population. ‘Cases’ were those records in our study population that demonstrate recent, active, complex mental health difficulties, defined by having 1) at least one SNOMED-CT diagnostic code from our inclusion list, excluding those from our exclusion list from any time prior to 31^st December 2021, and 2) a prescription for medicines of interest within ten years prior to 31^st December 2021 (Table 1). ‘Controls’ were those records that did not meet at least one of these criteria. We chose our index date of 31^st December 2021 because it was the date before which prescription data had been reliably updated, at the time of analysis.

The requirement for medication within the specified period was to ensure that caseness was only met if an individual had recently been prescribed medication for their mental health. This was motivated by preliminary searches in the clinical team members’ own practices that had identified a small number of individuals with a single code for personality disorder years before evidence of current mental ill-health.

We accept that there will have been some contamination across groups because our definitions were designed to separate caseness and control-ness with the purpose of screening patients who might have unmet needs (Fig 1). We have attempted to maximise the specificity of our caseness definition in expectation that cases will be a small portion of the population [15–17].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Visualisation of groups defined by information within electronic healthcare records.

https://doi.org/10.1371/journal.pone.0322771.g001

Features and feature sets

Our aim was to identify features within patients’ primary-care electronic healthcare records that might indicate that a patient as has complex mental health difficulties. We defined features as any patient attribute available from patients’ primary-care electronic health record that is not the definition of caseness variable. At their simplest, features were binary indicators of the presence of a clinical code, e.g., “Body dysmorphic disorder (disorder)| SCTID: 8348200”. More complicated features included indicators of trends in observations or counts of features, e.g., count of psychological disorders diagnosed.

The set of features was inspired by a review of the literature and interviews with general practitioners and patients about complex mental health difficulties. The interviews were conducted as part of the wider project’s qualitative work package, as yet unpublished at the time of writing. For this sister study, we thematically analysed the interview transcripts and produced a set of themes that included concepts about what defines complex mental health difficulties and what might indicate them. We consider these definitional and indicative concepts when developing candidate feature sets that could be operationalised and queried within a primary-care electronic health record system. The source of each feature is summarised in Table S3 where ‘fs_literature’ indicates a feature inspired by our literature review, ‘fs_interviews’ indicates a feature inspired by our interview study, and ‘fs_clinician’ indicates a feature inspired by the clinical members of the research team.

We also defined the following feature families that, together, accounted for all component features:

1. “Antecedent”: a feature set representing features that generally precede the emergence of complex mental health difficulties in adults (e.g., child abuse, abandonment, etc), and administrative or clinical events recorded before the age of 30.
2. “Concurrent”: a feature set representing concerning findings or behaviours after 30 years of age, e.g., self-harm, risk, substance misuse or dependency.
3. “Service use”: a feature set representing patterns of recent use of healthcare services that indicate both intensity and variance of use, e.g., number of mental health-related SNOMED-CT codes in the patient’s record.
4. “Treatment”: a feature set representing patterns of therapy and prescriptions, e.g., repeated referrals to Improving Access to Psychological Therapy (IAPT).
5. “Inconsistency”: a feature set representing unstable or atypical attendance activity, e.g., median count of appointments not attended, or sample entropy of appointments.
6. “Patterns of prescription”: a feature set representing patterns in the prescriptions for medications of interest, e.g., the count of aborted antidepressant-medication regimes.
7. “Relevant prescriptions”: a feature set indicating the presence or absence of prescriptions for our medications of interest (see Table 1).
8. “Antipsychotic prescription”: a feature set containing only one feature that indicated the presence or absence of a prescription for antipsychotic medications. This recognises that much prescribing of antipsychotic medications is not for psychotic illness, but rather is for conditions such as personality disorder [17].

Each feature family was represented with five levels that indicated the count of component feature sets in a patient’s record. The five levels were: ‘None’, indicating that none of the component features were present; ‘Not none’, indicating that at least the lowest observable count of the component features was present; ‘Few’, indicating that a lower quantile of the component features were present; ‘Some’, indicating that a middle quantile of the component features were present; ‘Many’, indicating that a higher quantile of the component features were present. Family-specific quantiles were subjectively defined based on a compromise of the research team’s clinical judgement and properties of the distribution of counts of component feature sets in patients’ records (See UNSEEN_create_feature_sets_appendix3.ipynb in the GitHub repository).

Finally, we defined feature-family combinations to represent all possible combinations of feature families and their levels, e.g., a feature family combination representing records with no features from the Antecedent, Concurrent, Service Use or Treatment families, but with ‘Few’ features from the remaining families. An additional level was permitted for each family to represent the idea that the level was irrelevant (See UNSEEN_create_feature_sets_appendix4.ipynb in the GitHub repository). For example, “A1_C0” was given a value of TRUE when a record had features from the Antecedent family but not the Concurrent family, but “A1_Cx” was given a value of TRUE when a record had features from the Antecedent family regardless of how many Concurrent features were present. The motivation for these feature-family combinations was to represent multi-faceted, high-level, heuristic definitions of patient groups.

In summary, we looked at three levels of features - component features, feature families, and feature-family combinations - collectively called ‘feature sets’. The final number of feature sets was 510,073 (Table S3).

Ranking feature sets

All features, family feature sets, and family-combination feature sets were ranked by their two-way mutual information with the caseness variable, and scaled to the entropy of the caseness variable [28]. This provided a measure of distinguishability between cases and controls, rather than a measure that predicted either cases or controls. Our mutual-information statistic quantified the reduction in the uncertainty of caseness afforded by knowing the value of the feature set [33]. For feature sets that were continuous-valued rather than binary – e.g., sample entropy of appointments – we used the arithmetic mean of 20 runs of Ross et al.’s method to calculate mutual information [34]. We ranked feature sets by their scaled mutual information because there is no generalisable, analytical threshold to indicate high / good or low / poor mutual information. This means feature sets could only be interpreted as relatively better or worse, rather than absolutely good or bad.

The novelty of an information-theoretic approach

Statistical methods based on information theory are rare in health services research [27] (see [37] for suggested applications). We used an information-based statistic rather than a regression or classification statistic because it made fewer assumptions about the form of the relationship between feature sets and the caseness variable, e.g., directionality, linearity, distribution of residuals, relative weighting of true positives and true negatives, etc. Our mutual-information statistic quantified the reduction in the uncertainty of caseness afforded by knowing the value of the feature set.

Theoretically, mutual information is maximised for a given caseness prevalence when the value of the feature set perfectly coincides with the value of the caseness variable [33]. Information theory is suitable for medical decision making because the notion of coincidence lends itself to the binary nature of diagnoses and recording (or not) of signs and symptoms. Variance-based approaches like regression can handle this binary nature – with logistic link functions being perhaps the most-familiar trick – but, when the prevalence of the target variable is very low (as is caseness in our study), the variance of the target variable will be low, consequently. Regression analysis would have been limited in its ability to quantify the associated between the caseness of complex mental health difficulties and feature sets. This is because regression analysis would have struggled to partition / “explain” the (very little) variance of the caseness variable using the variance of the feature set. In contrast, mutual information intuitively estimated the associations by summarising the counts of patient records that showed a) evidence of both the feature set and caseness, b) no evidence of either, and c) both scenarios showing evidence of one but not the other.

Prevalence of complex mental health difficulties

We suggest our observed prevalence of 0.3% in the population (equivalent to 2.16% of our sample of records indicating mental ill-health) represents approximately a ten-fold under-diagnosis (or under-recording of diagnoses). By comparison, Huang et al. suggested a population prevalence of 6.1% for all types of personality disorders, based on a global survey using face-to-face interviews of the general population [11], and Coid et al.’s study in Great Britain suggested a prevalence of 4.4% (weighted by the estimated prevalence of psychiatric morbidity, in 2000) [12]. Williamson et al. similarly found substantial under-coding of adverse childhood events in primary-care records, again, by an order of magnitude [38].

We propose three likely explanations for our low observed prevalence. The first is missingness. Routinely-collected electronic healthcare data is notorious for its poor data quality, despite frequently calls for assessment and improvement [39,40]. Second, our record-based definition requires clinicians to have clinically coded diagnoses that clinicians sometimes find difficult to diagnose and are reluctant to diagnose [41–43]. For example, chronic depression might go unrecorded explicitly, though a clinician has noted and is treating a patient’s ongoing depression. In our study, a record of diagnoses of interest was the limiting criterion, with only 3% of our study population meeting the diagnosis criterion (See UNSEEN_caseness_cohort_breakdown.ipynb in the GitHub repository).

Thirdly, our records-based definition will have missed patients because it does not perfectly capture the essence of complex mental health difficulties. This is because the concept of patient complexity is difficult to precisely pin down [44–46]. In juxtaposition, the components used to define our caseness and feature definitions were coded in the prevailing nosological framework of the Diagnostic and Statistical Manual of Mental Disorders [47] and used the clinical-coding nomenclature of SNOMED-CT [48], which cater for complicated but not complex definitions. Accordingly, our definition was an example of a reductionist “diagnostic literalism” [49], which hinders the holistic approach to person-centred care that we think is needed to address complex mental health difficulties. Additionally, the healthcare-record management systems on which healthcare practitioners rely benefit from complicated yet reductionist conceptualisations because they suit the structured / tabular data format that enables efficient storage, access, and calculation. Thus, the prevailing nosological and technical situation might have made it difficult to perfectly capture the essence of complex mental health difficulties.

Consequently, our estimates should be interpreted as a lower bound of the prevalence within the dataset, limited by the resources and processes available from the existing paradigm. Such under-coding means automated searches of databases will struggle to identifying all patients with complex mental health difficulties. This is why we sought to find informative features of complex mental health difficulties that are well-coded in routinely-collected electronic healthcare data.

Features informativeness

We suggest that the features that were non-informative (i.e., manifesting only a single value in all records) is explained by the fact that most features were highly-specified binary variables. Despite generally poor informativeness of features, the count of psychological disorders recorded in a patient’s record had a scaled mutual information of 19.9%. It is perhaps not surprising that information is shared between the count of psychological disorders and our definition of the caseness of complex mental health difficulties because records met our definition of caseness by 1) including a diagnosis of at least one of the six high-level disorder groups and their taxonomic children, and by 2) including a prescription for at least one of the 16 medications used to treat a variety of mental ill-health disorders. Both this feature and our definition of caseness might be representations of the same underlying concept of comorbid mental disorders.

We noted that the probability and the odds of a patient record meeting our definition of complex mental health difficulties monotonically increases with every additional diagnosis recorded. Thus, the count of psychological disorders appears to be a “dose”-dependent proxy for the likelihood of meeting our definition of complex mental health difficulties. This “dose”-dependent proxy echoes tools like Charlson Comorbidity Index, INTERMED and LOCUS, which are used to stratify patients by their “complexity”, as a proxy for the level of care they are expected to require [46]. Like the count of psychological disorders, the Charlson Comorbidity Index is sum of conditions, but each condition is weighted [50]. INTERMED assesses patients’ biopsychosocial complexity [51,52], while LOCUS assesses psychiatric and chemical-dependency problems with a focus on defining the level of care needed [53]. Both INTERMED, LOCUS and others require surveys, interviews, or self-assessment but our count of psychological disorders is a simple rule easily implemented in patient management software. The count of psychological disorders might be a useful dose-dependent indicator of complex mental health difficulties that can be calculated easily and automatically in electronic health record systems.

Further support for the validity of this feature come from Newman et al. who note that comorbid mental disorders accompany “complications that challenge treatment planning, compliance, and coordination of service delivery” [54], which aligns with descriptions of complex mental health difficulties [2,3]. The idea of a general psychopathology factor is founded on psychological comorbidity, also [55]. Similar to our comments on the (in)appropriateness of the current nosological paradigm, many have argued to move beyond a simple, cumulative / additive model of patient complexity (e.g., [56]) and suggest non-linear, network-based conceptions of pathology, nosology, and emergent burden [49,57,58]. Therefore, we encourage further study of how patterns of psychological disorder might be informative of complex mental health difficulties.

Other component features, feature families, and feature-family combinations

Apart from the outstanding performance by the count of psychological disorders, three of the top-five component features were entropy measures of non-attendance patterns, from the Inconsistency family of features. One should keep in mind that chronic non-attendance would be a consistent (albeit concerning) behaviour. If we consider these Inconsistency-family features together, we might summarise that caseness was indicated by persistent inconsistency in quarterly patterns of non-attendance, but not in the frequency spectrum. Non-attendance has been associated with complex psychosocial difficulties [59], poorer social functioning [60], and lower socioeconomic status [61,62], which might reasonably contribute to the complexity of patient’s mental health difficulties. But, it must be noted that the scaled mutual information was low, at only 4.9–5.8%.

Considering feature families, the top rank of the Antipsychotic Prescription family is noteworthy by the higher-than-average positive predictive value associated with its presence in a patient’s record (despite its low class balance accuracy). Antipsychotics are mainly prescribed for schizophrenia and bipolar disorder, but we excluded all records that contained these diagnoses from our analysis. Excess antipsychotics prescriptions due to uncoded psychosis is unlikely because registries of severe mental illness are expected to be maintained by primary-care practice in the UK [63] and recorded rates in the UK are consistent with epidemiological studies [64]. Similarly, excess prescriptions for the other diagnoses that constituted our definition is unlikely because our observed prevalence of antipsychotic prescriptions was over twice the prevalence of having any diagnosis of interest (5.42 versus 26.1 per 1,000 records). Given that antipsychotics are routinely used for mood stabilisation of patients with comorbid personality disorder [65–67], we suggest that the discrepancy between our observed diagnoses and antipsychotic prescription might indicate treatment for under-coded diagnoses indicative of complex mental health difficulties.

Feature family combinations scored better than component features and feature families, though scores were low. The common sub-combination (and highest scorer) was no antipsychotic prescriptions, with 1–3 Concurrent-family features, with or without the presence of Service Use-family features. Records with this sub-combination were less likely to meet our definition of caseness. This implies that, although patients are currently experiencing some of our candidate features for complex mental health difficulties, there are not likely to meet our definition if they are not taking antipsychotic medicines.

Contribution to knowledge

Little is known about the recognition of complex mental health difficulties in primary care. Our study leveraged the existing infrastructure of electronic healthcare records and clinical coding, founded on insights from the experiences of patients and healthcare professionals gathered from its sister qualitative study. We did this with the novel application of an information-theoretic evaluation of multi-level, multi-dimensional feature sets.

The prevalence of complex mental health difficulties as per our definition was low, which made it difficult for any feature to be informative. Almost none of the features derived from information within electronic healthcare records were notably informative of our definition. These findings support the idea that complex mental health difficulties are difficult to identify and operationalise, leading to an under-recording of cases that limits the use of electronic healthcare records to support identification, study, and provision of appropriate and timely care.

The count of psychological disorders was a lone, outstanding feature with a definition similar to our definition of complex mental health difficulties, and with reasonable theoretical association. Other component features, feature families, and feature-family combinations variously but marginally indicated in favour and against caseness. We interpret these findings under the assumption that diagnoses indicating complex mental health difficulties are vastly under-coded. To have identified indicators of complex mental health difficulties in the presence of such under-coding makes us optimistic that more indicators with greater strength could be identified with better recognition and recording. This is why our mixed-methods research study entitled “Understanding Services for people with Complex Mental Health Difficulties (UnSeen)” developed recommendations and a toolkit to help practices and new services work together. We hope our findings motivate improvements in diagnostic frameworks and patient-record management systems that better handle the unseen reality of complex mental health difficulties.