Background

Loneliness is associated with numerous negative health outcomes and increased risk of mortality [1–3]. It is prevalent in most developed countries [4], and due to its public health implications has generated substantial research activity [5,6]. Systematic reviews of loneliness have found over 60 measurement scales [7]; however, there is no consensus on “gold standard” measures and studies may vary on or recommendations for which cut-offs they use when classifying loneliness. Lack of guidance on the selection and the use of these measures creates challenges for research, surveillance, and screening, both from a comparability perspective and for clinicians and service providers trying to choose the best available measure for their setting.

Loneliness measures include multi-item scales and single-item questions. Measures range in the number of items, dimensionality (i.e., unidimensional view of loneliness versus multidimensional), mode of administration (e.g., telephone versus in-person), and degree of subjectivity (i.e., direct or indirect measure) [8–10]. Scales, such as the UCLA Loneliness Scale, the De Jong Gierveld Loneliness Scale, are considered indirect measures because they ask multiple questions related to the concept of loneliness without directly using the word lonely [11,12]. One of the most widely used scales to assess loneliness is the 3-item Loneliness Scale, a version of the Revised UCLA Loneliness Scale [11,13]. It is used in several national surveys including the Canadian Longitudinal Study on Aging (CLSA), the Canadian Community Health Survey (CCHS), and the Health and Retirement Survey in the United States [14–17]. This shorter version was developed because the original revised 20-item UCLA scale is self-administered, lengthy, and less feasible for large population studies, particularly those that rely on telephone interviews [11]. The 3-item Loneliness Scale is one of the few loneliness scales that has been validated for telephone administration, making it particularly useful for large population-based surveys, and it is quick to complete. A psychometric comparison of the 3- and 20-item UCLA Loneliness Scales also supports the use of the 3-item scale in time- and resource-limited studies, as it shows comparable psychometric performance in terms of reliability and construct validity [18]. In addition, indirect measures like the 3-item Loneliness Scale are often preferred for respondent groups, such as older men, who are reluctant to self-identify as lonely [19]. Unlike indirect measures, single-item, direct measures specifically use the word lonely or loneliness. Direct measures are more controversial than indirect because of concerns that they elicit less truthful, more socially desirable responses because of the stigma associated with loneliness [10,20]. However, given the interest in screening for loneliness, single-item measures may prove useful in contexts where time and resources are limited such as primary care. A recent systematic review and meta-analysis examined the prevalence of loneliness across 113 countries and found that single-item direct measures were the most used assessment tools for loneliness [4].

Given the large number of different loneliness measures (both scales and single item), there have been few opportunities to compare multiple measures in the same respondents, even though the Office for National Statistics in the United Kingdom recommends using both types of capture a more complete picture of loneliness [19]. The CLSA includes the 3-item Loneliness Scale and a direct single-item question. Both scales assess loneliness by asking “how often do you feel,” using a frequency-based scaling reference. However, to date, the CLSA does not offer recommendations for users on which measure to use, and responses to each measure have not been compared. This paper aims to address this gap by descriptively comparing responses to the direct (single item question) and indirect (3-item Loneliness Scale) measures of loneliness used in the CLSA. The purpose of this project was to describe the responses to the direct and indirect measure of loneliness in the CLSA and explore how these measures compare in relation to respondents’ demographic, social, and health characteristics.

Methods

We used a cross-sectional study design to descriptively analyse population-based survey data from the CLSA [17]. We examined data from baseline (2010–2015) and the first follow-up wave of survey data collection (2015–2018), when the 3-item Loneliness Scale was first included. The CLSA consists of two cohorts that are both age- and sex-stratified random samples of community-dwelling Canadians aged 45–85 years at recruitment. The random samples were stratified at the provincial level. The Tracking Cohort was recruited from across all 10 provinces with no geographic restrictions and completed a computer-assisted telephone interview. The Comprehensive Cohort includes individuals who live within 25–50 km of a CLSA data collection site and took part in in-home interviews and provided physical measures and biological data [21]. The CLSA exclusion criteria is described elsewhere [17].

Results

The analytic sample included 43,384 respondents for the single-item loneliness measure with a mean age of 65.9 (SD 10.2) and 44,373 respondents for the 3-item scale with a mean age of 66.0 (SD 10.2). About half of the respondents (49%) were women, and more than half had less than a university education. Most respondents were identified as White, and lived in urban areas. Nearly one-quarter lived alone, and around one-fifth had four or more chronic conditions.

We found the single-item and scale had a moderate-strong positive correlation (λ = .80, p < 0.001). The prevalence of loneliness was similar for the two measures when dichotomized: 24% (n = 10,205) for the single-item measure versus 23% (n = 10,246) for the 3-item scale (S2 Table). For the ordinal measure, the prevalence of severe loneliness was also comparable: 11% (n = 4,618) on the single-item measure versus 12% (n = 5,463) on the 3-item scale. However there was variation in the number identified as moderately lonely, with 15% (n = 5,587) on the single-item measure versus 29% (n = 12,717) on the 3-item scale (S3 Table and S4 Table).

We found that 14% (n = 5,848) of the total sample were identified by both measures as lonely and 67% (n = 28,998) by both as not lonely, and 19% had a different classification across the two measures (Table 1). When stratified by gender, women followed a similar pattern to the total sample, with slightly more showing a different classification across the two measures (21%) compared to men (18%) (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparing the Dichotomous (Lonely/Not Lonely) Responses from the 3-item Scale and Single-item Loneliness Measures for the Full Sample and Stratified by Gender (Women, Men).

https://doi.org/10.1371/journal.pone.0341572.t001

CLSA respondents who were identified as lonely by both measures differed from respondents identified as not lonely on both measures in several ways. Those identified as lonely (on both measures) were slightly older, more often women, less educated, and had lower income (Table 2). Compared to not lonely (on both measures) respondents, lonely respondents had different marital status distributions. A greater proportion of lonely respondents were widowed (21% of lonely respondents versus 7% of not-lonely respondents), divorced/separated (23% versus 9%), or single/never married (16% versus 6%), and there was a large discrepancy among those who were married (39% versus 78%). In terms of health, lonely respondents had a greater proportion of 4 + chronic conditions (33% versus 16%) and functional impairment (24% versus 10%). They had lower social contact (11% versus 6%) and low social participation (28% versus 15%). Compared to not-lonely, lonely respondents reported an unmet need in the previous 12 months (16% versus 6%) and having been to the emergency department (29% versus 20%). Discordant respondents using the dichotomous measure of loneliness had a less obvious pattern of difference in their characteristics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Descriptive characteristics of concordant and discordant respondents to scale and single-item loneliness measure.

https://doi.org/10.1371/journal.pone.0341572.t002

For each measure we categorized loneliness based on severity (not lonely, moderately lonely, severely lonely). We found fair agreement between the groups (ordinal measure weighted κ = 0.382, dichotomous measure weighted κ = 0.378) and found similar patterns emerge for the concordant respondents [25] (Table 3). Results of the McNemar-Bowker test found that for all groups there was significant (p < 0.001) asymmetry, indicating that individuals were re-classified differently across the two measures. The patterns of disagreement between the 3-item and single-item scales suggest that the two tools may not be interchangeable in identifying the severity of loneliness, as individuals were often classified as severely or moderately lonely based on the single-item cut-off, but were classified as not lonely by the 3-item scale. This misalignment was consistent across the full sample as well as in women and men, suggesting that the differences in classification are not driven by gender alone.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparing the loneliness severity from the 3-item scale and single-item loneliness measure for the full sample and stratified by gender (Women, Men).

https://doi.org/10.1371/journal.pone.0341572.t003

Compared to not lonely respondents, severely lonely respondents were slightly older (66.8 years versus 65.6), women, less educated, widowed, and living alone. Discordant respondents who were severely lonely in the single-item, direct measure but not lonely in the scale were older (69.5 years of age) than the other discordant and concordant groups (Table 4). Moderately lonely respondents (single-item) who were identified as not lonely in the scale, were married (67%), higher income (70%), high social contact (53%), and few unmet needs (8%) (S3 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Descriptive characteristics of discordant respondents by loneliness severity for the 3-item scale and single-item measure.

https://doi.org/10.1371/journal.pone.0341572.t004

Discussion

We compared the two measures of loneliness in the CLSA and found they demonstrated a similar prevalence of loneliness, and they comparably classified respondents as either lonely or not lonely. Respondents who were identified as lonely by both the scale and single-item were more likely to be women, less educated, and have lower income. Women tend to experience higher levels of loneliness partly due to their longer life expectancy and greater likelihood that older women in North America live alone, which increases their risk of loneliness [26]. Lower income is also associated with loneliness, as financial constraints can limit access to social and community activities and reduce opportunities for meaningful engagement with others [26]. Similarly, individuals with lower levels of education are more likely to face socioeconomic disadvantages that restrict social participation and may contribute to social withdrawal [27]. Together, our findings are consistent with the extant literature on factors related to loneliness in older adults [26–28].

The demographic characteristics of the discordant groups were not strikingly different, aside from marital status and living arrangement (living alone). Our findings reinforce the broader recommendation outlined by the Office of National Statistics that recommends both indirect and direct measures, can provide complementary information [19]. The limited agreement we observed highlights that each measure captures somewhat different expressions or interpretations of loneliness. For this reason, we suggest that, where feasible, future CLSA users collect and examine both measures, as doing so allows them to better understand how question framing influences reported loneliness. At the same time, using both measures in subsequent analyses requires careful methodological consideration. Although collecting both is valuable, they should not be included simultaneously in statistical models due to their conceptual overlap. Instead, we recommend selecting one measure at a time, based on analytic objective. The single-item question may be more appropriate when parsimony is essential to the models. The multi-item scale may be preferable when researchers seek a more nuanced assessment of loneliness. Providing both measures allows for descriptive comparisons and subgroup exploration.

Loneliness is a complex concept and one that does not have an agreed upon definition [29]. It is likely that disparate responses that we observed reflect the different ways that the questions and response categories tap into different conceptualizations of loneliness and participant comfort with attributing the term loneliness to their experience. The 3-item Loneliness Scale in the CLSA was based on the 20-item UCLA loneliness scale [11]. The developers of the 3-item scale selected the three items that reflect an individual’s feeling of aloneness, rejection, and withdrawal which the developers that underlies the concept of loneliness. On the other hand, the single-item question that we examined in our study does not allude to the concepts that constitute loneliness (without ever using the word lonely), rather it poses a different challenge where it directly uses the word lonely, but then leaves what exactly loneliness means up to the interpretation of that respondent and further asks that they ascribe that label to their experience. Cultural and social factors contribute to our understanding of the concept of loneliness and this inevitably will influence our willingness or reticence to take on the label [30,31]

In our sample, participants that self-identified as severely lonely using the single-item measure but were not considered lonely using the scale were older (69 + years of age) when we compared all concordant and discordant groups. Those that were moderately lonely in the single item but not the scale did not have common characteristics we see in lonely older adults, instead they were married, had a high income and high social contact. Criticisms of single-item, direct measures of loneliness often note that these questions will elicit socially desirable responses and that specific groups of respondents (e.g., older men) will be less likely to describe themselves as lonely. However, comparably little is known about those respondents who do willingly describe themselves as lonely but do not meet the criteria for moderate or severe loneliness in widely used scales. Our study found that these individuals have various supports, both financial and social, that some older adults do not and yet they are still lonely.

Our findings demonstrate why using two measures of loneliness can be useful, particularly for those aiming to do subsequent qualitative research to understand the context and rationale behind the responses. Although population-based surveys offer an important opportunity to examine the experience of a broad swath of older people, they cannot tease out the individual experience and how this may influence responses to single-item or multi-item scale questions. Our findings point to critical areas for future consideration for those interested in loneliness. For example, future researchers could design a sequential explanatory mixed-methods study to explore why certain respondents are classified as severely lonely on the 3-item scale but do not self-identify as lonely on the single-item question. Qualitative studies could navigate how participants interpret the wording and questions of each measure, how they understand loneliness and being lonely and what personal or cultural factors influence their willingness to express their true feeling of loneliness.

Meanwhile, the misalignment we observed in the two measures may be related to the time-based parameters in the question stems and response categories. An important component of loneliness is its temporal nature, with some research suggesting that those who are temporarily or situationally lonely (e.g., after loss of a spouse, moving to a new area) are more likely to recognize and communicate their loneliness compared to those that have been chronically lonely over a period of years [31,32]. The single-item question in the CLSA includes specific temporal boundaries, asking participants if they felt lonely over a period of days, whereas the scale does not specify a unit of time. The differences we observed in the responses to the scale versus single-item may be due to these temporal differences in the response categories. To further explore these patterns, qualitative interviews could investigate how respondents think about the “timing” of loneliness. For example, researchers could ask participants whether their feelings change or fluctuate during a day or over days or across years, and how these temporal differences shape their responses to each measure. Chronically lonely individuals have experienced loneliness over years and their loneliness can lead to an affective flattening, and an overall change in moods and attitudes which inhibits their ability to recognize their own loneliness [32]. Studies describe the ability to perceive and self-ascribe loneliness as attributable to a very specific group of lonely individuals and not all those that are labelled as lonely [31].

Overall, the discordant responses we observed are informative and highlight meaningful variability in how older adults interpret and express loneliness. These differences become clearer when the conceptual focus and temporal framing of the two measures are considered. Using both measures can therefore be valuable for researchers who are willing to explore the underlying reasons for these divergent response patterns and to gain deeper insight into how individuals understand and express their loneliness.

Limitations

We examined data from a large population-based survey; however, as is the case in many national surveys, the respondents are relatively homogenous (e.g., White, urban-dwelling, English speaking). While we aimed to examine differences in the measures across the spectrum of gender identities, sample size in the gender diverse group was small and made it difficult to explore differences in responses for this group. This study was cross-sectional and only included responses from baseline and the first follow-up survey. It is unclear if these comparisons will remain consistent across the further waves of data collection. Although we selected cut-offs consistent with prior research, alternative thresholds may classify respondents differently and could yield different patterns of association. We recognize that we may lose valuable information by dichotomizing scores; however, it was necessary for the purposes of our study to describe if participants were broadly identifiable as lonely or not lonely by both scale and single-item measures.

Conclusions

Our findings suggest that the use of multiple measures of loneliness is advisable for CLSA users because they each identify critical features of respondent experience. We observed variation in respondent characteristics by comparing responses across different response categories and based on temporal periods described in question stems. We highlight future areas for research, particularly as it relates to respondent understanding of the term lonely in the single-item, direct measure and how this understanding may vary by different factors.

Supporting information

Acknowledgments

This research was made possible using the data/biospecimens collected by the Canadian Longitudinal Study on Aging (CLSA). This research has been conducted using the CLSA dataset Baseline Tracking Dataset version 3.6, Baseline Comprehensive Dataset version 4.2, Follow-up 1 Tracking version 2.1, and Follow-up 1 Comprehensive version 3.0 under Application Number 20CA004. The CLSA is led by Drs. Parminder Raina, Christina Wolfson and Susan Kirkland. The time and commitment of the participants to the CLSA study platform is gratefully acknowledged, without whom this research would not be possible.