The impact of loneliness and social isolation

Loneliness and social isolation are as significant predictors of mortality as smoking, obesity, or hypertension [18–20]. Social isolation is an “objective lack of relationships” [21], whereas loneliness is a “subjective, distressing feeling” when social relationships are inadequate [22].

Loneliness has been a longstanding priority for the WHO and governments, with its importance increasingly emphasised in recent years, particularly following the Covid-19 pandemic [23–28]. Yet, internationally, in recent decades, many countries have had a retrenchment in community care services towards personal care, with social support reduced [29–32]. Evidence suggests that loneliness increases long-term care use [33–37]. However, surveys often include few publicly funded care users [38–40], and records cannot be linked to detailed service use information, limiting its insights compared to administrative data.

Administrative records in England record eligibility-related social needs in structured and free text formats [41]. Distinguishing loneliness from social isolation in free text is challenging, as terminology often diverges from literature definitions. For example, “feels isolated” might refer to subjective loneliness or limited social contact. A 2024 paper by Patra et al. distinguished social support needs from psychiatric records, noting greater consistency and detail than typical social care notes [14]. Given the inconsistency in our dataset, we analyse loneliness and social isolation jointly. This combined approach is supported by findings showing both loneliness and isolation adversely affect older adults’ mortality and is common in public health and work extracting Social Determinants of Health (SDoH) from clinical notes [12,18–20,42–44].

Data collection

In England, every person requesting publicly funded care must receive an assessment under the Care Act 2014. In this paper we attempt to identify social isolation or loneliness from the free text notes of a London borough. Adult social care records are written by individuals employed by a local authority to assess needs and commission care. This generally consists of social workers, occupational therapists or care managers. Workers complete an assessment form, which is a snapshot of needs at a certain time containing both structured data and free text. Recording systems also contain case notes, which are free text fields to record ongoing work on the case over time. In Fig 1 we show how the assessment form and case notes appear to caseworkers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Example of format of structured and unstructured data.

https://doi.org/10.1371/journal.pone.0319745.g001

Ethics statement

This study uses secondary data from pseudonymised administrative social care records. We sought and were granted departmental ethics approval for the project on 30th May 2019 at the London School of Economics and Political Science (LSE), in line with LSE’s Research Ethics Policy and Procedure.

The data were pseudonymised prior to processing, including the removal or replacement of identifiable personal information such as names, addresses, email addresses, telephone numbers, unique identifiers (e.g., NHS numbers), financial information, and location details. A Data Processing Impact Assessment (DPIA) was carried out to ensure the protection of individuals’ data privacy, and no automated decision-making processes were involved. The Data Flow Diagram is set out in S1 Fig in the Data Flow Appendix.

Details of the project were made available in the local authority’s Privacy Notice and on a separate website informing individuals of and explaining the study, allowing individuals to opt out if desired. Individual consent for data use was not required, as the data were processed in line with the UK General Data Protection Regulation (GDPR) under the legal basis of legitimate interests. This legal basis allows processing of pseudonymised data for research purposes where it serves a social or public interest and individuals are informed and able to opt out. Permission for data processing was granted by the National Health Service NHS Confidentiality Advisory Group (CAG) in June 2020 (reference number 20/CAG/0043), which was renewed annually. CAG ensures that data processing complies with national regulations for handling confidential patient information in the UK.

Data extraction and characteristics

A query was written to identify all individuals aged 65 or over on August 1st 2020 who had been receiving services for at least one year since 1st January 2016. Administrative records for these individuals were then extracted from the local authority database. Identifiable free text data tokens were masked using the open-source text pseudonymisation software PSCleaner [45]. The data was then sent to an NHS Commissioning Support Unit, where identifiable structured data such as NHS numbers were removed. Finally, the data was transferred securely to the research team at the Care Policy and Evaluation Centre (CPEC) at the London School of Economics and Political Science (LSE).

The data includes all free text case notes recorded for individuals in the cohort between 2008 and 2020, as well as needs assessment and service receipt data. During this period, there were 3,046 individuals aged over 65 receiving long-term care. The data contains 10,821 assessment forms comprising 19.1 million words of free text, and 1.14 million case notes, containing 87.8 million words of free text. Case notes in the dataset encompass a wide range of updates related to the care and support of individuals. These include records of emails and telephone calls, descriptions of home visits, case screening and allocation, managerial direction, case summaries, allegations of abuse or neglect, as well as referrals to services such as occupational therapy, physiotherapy, and intermediate care. The volume of notes reflects the comprehensive documentation required in social care to capture various interactions and decisions throughout the course of care. There is significant variation in the distribution, with for example the top 50 individuals having 8 million words recorded (7.8% of the total), the same amount as the 850 individuals with the fewest. Summary statistics per person are presented in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Quantity of free text per person.

https://doi.org/10.1371/journal.pone.0313772.t001

In addition to the free text case notes, structured data fields are routinely collected during the assessment process. These fields capture key demographic and personal information that is relevant for care planning and service provision. Structured data includes information such as gender, ethnicity, age, functional ability with activities of daily living (ADLs), and whether the individual lives alone. This information is collected directly by social care professionals during initial assessments and periodic reviews as part of standard care practices. These structured data fields provide important context for understanding the care needs of individuals and were used in conjunction with the free text data in our analysis. Of the 3,046 individuals, 61.2% were women, 47.8% White British, with a median age in 2020 of 81, and median of 3 years and 6 months of services received. These characteristics are set out in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of individuals in the training and test set.

https://doi.org/10.1371/journal.pone.0313772.t002

Overview of model development and evaluation

This section outlines methods for model development and evaluation. We describe data pre-processing, manual text classification, and training machine learning algorithms. Model evaluation involves assessing performance metrics on a test set and examining construct validity by testing expected relationships with needs, demographics, and service use.

Model development

We endeavoured to use as parsimonious a model as possible, beginning with count-based vector representations of words such as document-term matrices [46] and Term Frequency Inverse Document Frequency (Tf-idf) [47]. We also used the SpaCy large pre-trained word embeddings [48], and transformer-based representations, specifically RoBERTa and DistilRoBERTa [49]. The overall process for training and comparing these models is set out in Fig 2. Unless otherwise stated, we used Python 3.9.7 in all the analysis [50].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overall training and evaluation process.

https://doi.org/10.1371/journal.pone.0319745.g002

For all approaches, we replaced the pseudonymised masks (e.g., ∗ ∗ ∗ ∗ , ∗ ∗ ∗ ∗, which had been used to mask identifiable information) with randomly generated names and locations to ensure that the language models could correctly tokenise and parse the sentences. Retaining the pseudonymisation masks could have led to issues with tokenisation, as the models may not have handled repeated placeholders effectively. For the count-based methods, we also lemmatised the text, converted it to lowercase, and removed stop words. We set out further details in Data pre-processing in S2 Fig in our Supporting Information document. We then divided the data into a training and test set, using stratified random sampling to ensure similar proportions of individuals in each set (see Table 2). Each set contained notes about 200 distinct individuals. We split each set by person to ensure that the test set did not contain sentences about individuals who are in the training set.

Human annotators manually classified 10,083 sentences in the training set and 3,573 sentences in the test set for model evaluation. These manually classified sentences represent a subset of the total number of sentences in the dataset, which exceeds 600,000. It would not have been feasible to classify every sentence manually; instead, we focused on classifying sentences most likely to inform the development of the model. We defined a set of rules for annotators to determine which sentences to classify, using binary classification (either indicative or not indicative of loneliness or social isolation). These rules covered statements such as when a person explicitly expressed feeling lonely, had little social contact, or received referrals to services like befriending. Conversely, sentences indicating practical support needs, support for safety or cognition, or day centre attendance for carer respite were classified as not indicative of loneliness or social isolation. The full set of rules is detailed in S1 Text. Our interrater reliability measures produced Cohen’s κ [51] of 0.89 (95% CI 0.84–0.94) and Krippendorff’s α [52] of 0.89 (95% CI 0.89–0.93). The maximum level of agreement in both cases is 1, and 0.89 represents excellent levels of agreement beyond chance [53,54]. The training dataset was imbalanced, with 9,383 sentences in the negative class (not indicative of loneliness or social isolation) and 700 in the positive class.

We implemented three approaches for the representation of words:

1. Count-based approaches: We split each sentence into lemmatised, word-level tokens. Each sentence was represented by a raw count of the number of times a word appears in it (a document-term matrix) [46]. We also applied Term Frequency Inverse Document Frequency (Tf-idf) [55] to transform the count matrix to a weighted representation, reducing the weighting of higher frequency words across all documents.
2. Pre-trained vectors: We used the Spacy large English model [48], which represents language through dense embeddings [56], where words which have similar semantic meanings are clustered together in vector space. We took the mean of each dimension to create a single 300-dimensional vector to represent each sentence.
3. Transformer-based approaches: We used the RoBERTa base model, which has 12 hidden layers, 768 dimensions and 12 heads [49]. This was relatively computationally expensive to fine-tune, so for comparison we also used DistilRoBERTa, which has identical parameters except it has 6 hidden layers, and is around twice as fast to train. In both cases, we used the HuggingFace implementation of each model’s tokenizer to split each sentence into sub-word tokens [57,58].

We describe these approaches in more detail in S1 Text. After pre-processing, vectorising and labelling each sentence, the problem becomes a binary classification task. For both the count and pre-trained embedding based approaches, we evaluated five classification algorithms. We used k fold cross-validation to avoid overfitting on the training set, choosing 5 folds for k as a value which tends to elicit reasonably high accuracy [59] while reducing training time compared with higher values. We used five classification algorithms: class-weighted logistic regression, bootstrap aggregation, random forest, quadratic discriminant analysis and a feed-forward neural network. Again we set these out in the Classification algorithms section of S1 Text. For the transformers approach, the HuggingFace implementation of both the RoBERTa and DistilRoBERTa models contain a classification head. We trained this final layer of the model on the labelled, tokenised sentences using the HuggingFace Transformers and PyTorch libraries [60,61]. Our final model had a training batch size of 16 sentences, with 500 warm up steps, and weight decay of 0.01. The weight decay parameter is bounded between 0 and 1, with 0.01 indicating relatively low L2 regularisation, which can help the model fit the smaller, positive classes more accurately, but can risk overfitting. The final output layer produces a predicted probability for the negative and positive classes (either indicative or not indicative of loneliness or social isolation). During training the parameters of the classification head were optimised using binary cross-entropy loss, which measures the difference between the predicted probabilities and the true labels.

Model evaluation

We evaluate the model’s accuracy by comparing performance metrics (accuracy, precision, recall, F₁) on a test set of 3,573 unseen sentences drawn from individuals not included in the training data (3026 in the negative class and 547 in the positive class). We assess construct validity of the indicator derived from the best-performing classification model by analysing associations between the NLP model output and demographic characteristics, and comparing this with associations in survey data. We also conduct logistic regression to assess whether the model’s loneliness or isolation predictions are associated with the use of services typically related to social support needs.

Construct validity: Comparison with survey data.

Using the best-performing model, we assess construct validity by classifying free text from the initial assessments of 1,331 individuals at their first contact with statutory care services. We analyse text from assessment forms and case notes within 90 days of the assessment, with the indicator of loneliness or isolation treated as binary. We derive four metrics: individuals with no positive sentences (Neither), those with a positive sentence only in their assessment (Assessment), those with a positive sentence only in case notes (Case notes) and those with both (Assessment). This is set out in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Classification outcomes across assessments and case notes.

https://doi.org/10.1371/journal.pone.0313772.t003

We then compare the results of the model predictions with pooled data from waves 6–9 (2012–2019) of ELSA [40]. This secondary dataset, collected as part of a large national survey through structured interviews and self-reported questionnaires, provides a validated source of information on the characteristics of older adults in England. We use ELSA data for all older adults who stated that they had care needs and received publicly funded care (N = 995 unique individuals with 1361 total observations over the period). We pool the results due to the low number of responses in some groups. We tabulate responses to the ELSA Center for Epidemiological Studies Depression Scale (CES-D) loneliness question [62]. We also compare our results to the three UCLA loneliness scale questions within ELSA, converting a total score of 6 or more into a binary indicator of loneliness, as in Hanratty et al. [33]. As our model measures loneliness and social isolation, in the ELSA data we also establish which individuals are socially isolated according to the Social Network Index (SNI) defined in Minicuci et al. (2016) [63].

We compare the results with ELSA graphically, by examining the proportion of people in our data and in ELSA who appear lonely or socially isolated, broken down by demographic characteristics and care needs. We also conduct a Pearson’s χ² test of independence [64] of each need or demographic factor with loneliness or isolation, to establish whether there are the same associations between our indicator of loneliness and those found in ELSA. Finally, we conduct a logistic regression of all these factors and loneliness or isolation, to establish which factors remain significant after controlling for characteristics such as living alone.

Construct validity: Predicting service receipt for loneliness or isolation.

Our data includes information on whether individuals are attending day centres, which are community-based services provided for older people at risk of loneliness or social isolation [30]. To assess whether, as we expect, our indicator is associated with day centre attendance, we generated predictions of loneliness or social isolation using our best-performing model, the RoBERTa-based language model. Next, we conducted a logistic regression to examine whether these predictions were associated with the receipt of day centre services within 90 days of the initial assessment, for the 1,331 individuals whose initial assessment could be identified. To ensure that the RoBERTa model was not simply picking up cases with more notes, or cases driven by demographic characteristics rather than actual loneliness or isolation, we included the number of notes and relevant demographic variables as controls in the logistic regression model. This allowed us to verify that the RoBERTa model’s predictions were not confounded by factors such as a greater volume of documentation or demographic differences, rather than genuine cases of loneliness or social isolation. The logistic regression model is specified in Eq (1).

(1)

Where p is the probability of receiving day services in the first 90 days, SIL is the binary prediction of social isolation or loneliness generated by our model, notes is the number of sentences written within 90 days of assessment, sex is a binary variable where 1 indicates male, ethnicity is a binary indicator of white or non-white and age is age of the person receiving care in years. Additionally, we include as continuous variables the following rank of severity of needs, where higher indicates more care needs. pc is personal care needs (the sum of mobility, toileting and dressing), memory is the score for memory and cognition, safety is the extent to which the person is aware of their own safety and risk and alone is a binary indicator of whether an individual lives alone. The demographic and needs-related scores are extracted from the structured data of the initial assessment.

Model performance on the test set

In Table 4 we detail the accuracy, precision, recall and F₁ score [65] of each model on the test set of 3,573 labelled sentences not seen by the training set. The transformer-based models considerably outperform all other models, with DistilRoBERTa achieving an F₁ score of 0.86 and RoBERTa 0.92. The pre-trained Spacy embeddings outperformed all non-transformers based approaches when classes were predicted using a feed-forward neural network, with an F₁ score of 0.61. However, using the same embeddings, the neural network only slightly outperformed logistic regression, which had an F₁ score of 0.58. The count-based approaches were not effective at prediction using any of the classification methods. High accuracy alongside low precision, recall, and F1 scores in some models reflects the imbalanced dataset, as models like random forest almost exclusively predict the majority class, inflating accuracy while failing to classify minority cases. In Fig 3, we present a confusion matrix comparing the predictions of the best-performing model against the classes defined by human annotation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Accuracy and F1 score of classification models.

https://doi.org/10.1371/journal.pone.0313772.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Confusion matrix (RoBERTa).

https://doi.org/10.1371/journal.pone.0319745.g003

Construct validity: Comparison with survey data.

The overall proportion of individuals with at least one case note indicating loneliness or social isolation according to our model is 0.44 (95% CI 0.42–0.47), and the proportion with at least one sentence indicating the same in their needs assessment is 0.43 (95% CI 0.40–0.45). This compares with a proportion of 0.38 in ELSA (95% CI 0.32–0.43) who are lonely according to the CES-D measure or SNI isolated, and 0.45 (95% CI 0.39–0.51) who are lonely according to the UCLA measure, or SNI isolated. The overall proportions are similar to the UCLA loneliness measure, and this holds for many characteristics. We present in Fig 4 a breakdown of these proportions by demographic and needs-related factors. While these similarities are reassuring, there are differences between the results of our model and ELSA. For example, the difference in loneliness between individuals who live alone and live with others is wider in ELSA than in our data. We present the results in Fig 4 in tabular form in the S1 Text document.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Proportion of lonely/isolated by demographic characteristics: Administrative and survey data.

https://doi.org/10.1371/journal.pone.0319745.g004

We set out in Table 5 the results of the χ² test of independence between loneliness or isolation and the needs-related factors in both our results and ELSA. We also present the results of the combined indicators, Either and Both . The χ² tests reveal both similarities and differences in the associations between our RoBERTa-based indicator and the ELSA measures of loneliness and social isolation (CES-D and UCLA combined with SNI). Both our indicator and the survey data show a strong association between loneliness and living alone. However, our indicator also identifies a significant link between memory issues and loneliness, which is not found in the ELSA data. Additionally, ELSA data shows that people receiving unpaid care are more likely to be lonely, a pattern not reflected in our findings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Factors in structured data associated with loneliness and social isolation: Administrative data and ELSA.

https://doi.org/10.1371/journal.pone.0313772.t005

We set out the results of the χ² test and regression of the association with needs and demographic factors in Table 5. We assessed multicollinearity using the generalised variance inflation factor (GVIF), with a maximum value of 1.3, well below the typical threshold of 4–10 [66,67]. The regression output indicates that in ELSA, living alone is by far the most significant predictor, though requiring support with shopping and presence of unpaid care are also significant. Across all four of our measures, living alone is also a very important predictor of loneliness or isolation. The coefficient is around the magnitude of that for memory, where individuals who have memory problems are more likely to be lonely or socially isolated.

This discrepancy between our results and ELSA may be due to differences in the cohorts or the nature of the data, as ELSA data is self-reported, while administrative assessments of functional ability are recorded by professionals. Although we have taken a subset of individuals from ELSA who are older people receiving local authority care, individuals in the administrative data have higher needs than those in ELSA (see Table 6). We do not consider this a barrier to comparing the datasets, but we do consider it when interpreting the results. We elaborate on this in the Discussion section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison of demographic and ADL needs between ELSA waves 6–9 and administrative data.

https://doi.org/10.1371/journal.pone.0313772.t006

Construct validity: Predicting service receipt for loneliness or isolation.

We include the results of the day centre services regression in Table 7. Accounting for the number of notes and demographic factors, the model output remains a strong predictor of whether an individual is in receipt of day centre services. The maximum GVIF for any indicator is less than 1.4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Logistic regression: Association of loneliness extracted from free text with services received for loneliness.

https://doi.org/10.1371/journal.pone.0313772.t007

Limitations

Our findings have some limitations. For pre-trained embeddings, we used mean pooling to represent sentences, but methods that summarise spans of embeddings may improve performance [75]. Similarly, count-based approaches could have benefited from using n-gram co-occurrence matrices to capture contextual relationships more effectively. These enhancements might have increased the F₁ score of simpler NLP methods. However, we ensured robust evaluation by testing a range of classifiers, including boosting, bagging, logistic regression, MLP, and random forests. Additionally, with transformer models, we achieved strong results using default parameters without hyperparameter tuning, suggesting potential for further optimisation in these approaches too.

Our binary classification of loneliness or social isolation oversimplifies the concept, as it cannot capture variations in intensity. This limitation reflects the recorded data, necessitating a pragmatic approach. However, the different measures derived from the binary indicator may hint at intensity. Another limitation is combining loneliness and social isolation, though consistent with prior research [e.g. 12]. Distinguishing these concepts is important for targeted interventions; for instance, day centres reduce social isolation but may not address emotional loneliness [as conceptualised in e.g. 76]. While social care administrative records do not allow for such distinctions, insights from other datasets could better inform interventions [see e.g. 12,14,18–20,42,43].

Another notable limitation is related to the dataset which, although large in terms of sentence count, is limited to a relatively small geographic area. Although notes in the training and test set are not about the same individual, they may have been written by the same worker. Similarly, there may be organisational culture issues which lead to individuals using similar phrases that would not be seen elsewhere. We expect that the model will not perform quite as well on free text case notes from another area, although the magnitude of the drop-off, and how many new samples need to be labelled to improve performance, is an empirical question that we hope to answer in the future.

Studies using administrative data face inherent limitations when measuring phenomena like loneliness or isolation. While Table 7 shows that workers who record loneliness or isolation are more likely to arrange social inclusion services, it is unclear how often services are provided to individuals who are actually lonely or isolated. Some loneliness or isolation may remain unrecorded, and services may also be declined. However, the observed associations with characteristics like impaired memory and living alone suggest unobserved cases are not significant enough to invalidate our results. Administrative data provides real-time information on service use and has been used to link care home admission to factors like age, gender, and disability [e.g. 77,78]. Including loneliness or isolation as a structured indicator in such models could further enhance such models.