Participants who provided search terms for analysis

The participants were all aged 18 or over, lived in the UK, had been in contact with mental health services in the preceding year and had experienced suicidal thoughts or behaviours and engaged in SRIU during this time. Characteristics of the full sample are described in detail elsewhere [1]. Participants were asked to write down the search terms (queries) they had used when seeking suicide-related content online [1]. Of 533 participants who engaged in suicide-related internet use, 196 (37%) provided their search terms. This group included 10 participants who reported encountering suicide-related content accidentally but still provided search terms. Overall, participants provided 520 usable entries. On average, each participant contributed 2.6 search terms (range = 1–25), and each search term was treated as an independent unit of analysis. Characteristics of participants who provided search terms were examined.

Search term selection

The process of selecting search terms for extracting Google Trends data involved both qualitative and quantitative approaches, which comprised the following steps:

1. Compilation and coding of search terms
   1. Firstly, search terms were ordered by frequency. Those that were used by at least two participants were designated base search terms. Three of these base terms were excluded as they were not related to the topic of suicide.
   2. All search terms (base search terms and others) were coded and organised into seven broad categories: a) non-specific suicide-related queries, b) symptoms or diagnosis, c) seeking help queries, d) pro-suicide queries, e) suicide methods, f) procedural queries, and g) characteristics of suicide (see Table 1 in Results section). The coding was carried out by LB and reviewed by SF and SI.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Categories of search terms and their descriptions, including the number of unique search terms and the frequency with which participants reported those terms.

https://doi.org/10.1371/journal.pone.0344926.t001

1. Identification of synonyms and related phrases

Synonyms, abbreviations, alternative spellings, and related phrases for base search terms 1.1.) were searched for within their categories 1.2.). These were then combined into finalised search terms using Google Trends Boolean operators (i.e., ‘+’ for combining and ‘–‘ for excluding terms). For example, for the base search term ‘overdose’ (searched for by 9 participants), its category e) (suicide methods) was explored for related content and identified two participants who searched for the abbreviation ‘od’. This additional search term was combined with the base search term as follows: ‘overdose + od’. This was carried out by LB with input from SI and SF.

1. Search term optimisation via related topics and related queries features

The finalised search terms were entered into the Google Trends website to examine the results from the “related topics” and “related queries” features, which provide insight into topics, phrases, or questions commonly searched alongside the selected terms [11]. In some cases, search terms were revised based on this examination; for example, ‘toxicity’ was modified to ‘toxicity -dog’ to exclude irrelevant results related to common searches for the toxicity of substances for dogs. The final number of search terms for which data was extracted was 36.

Obtaining and preparing Google Trends data

Monthly Google Trends data for 36 selected search terms were collected for England, Wales, and Scotland, covering the period from January 1, 2011, to December 31, 2022. Google Trends data was not obtained for the whole of the UK as suicide mortality data was not available on the level of month for Northern Ireland (see next paragraph). The start year was chosen because it marks when over 85% of the UK population was using the internet and Google held a 92% search engine market share, indicating that from this point onward, Google Trends data would be sufficiently representative of UK online activity [12,13]. The final year, 2022, was selected as it was the most recent year with comprehensive mental health patient suicide statistics available [14]. Steps were taken to address potential bias introduced by the Google Trends sampling process. Namely, Google Trends time series for any given search term are based on a sample rather than the whole search volume, and each sample somewhat differs from another, raising concerns of replicability [15]. To address this, the data were retrieved on 15 separate days in March 2025 and averaged to reduce random fluctuations, following the recommendations of Medeiros and Pires [15].

Before averaging, the obtained data were normalised in three steps: firstly, for each search term, the values for each devolved nation were weighted by the respective population percentage of each nation, producing a composite index. Population percentages were determined using yearly population estimates for each year from 2011 to 2022. Weighting Google Trends data by population size was appropriate given the broadly similar internet infrastructure across the UK [16], online searching behaviour [17], and mental health related help-seeking practices [18].

Secondly, a maximum value for each search term was identified. Finally, the composite index value was divided by this maximum value and multiplied by 100 to restore the traditional 0–100 scale characteristic of Google Trends data. After normalisation, data were averaged, creating 36 unique Google Trends time series. Time series with more than 15% data points missing and/or 4 consecutive months having missing data were not analysed. After the pattern of missingness was analysed, four search terms were removed due to having more than 15% of missing data (1 seeking help, 1 suicide method, 1 pro-suicide and 1 of a non-specific suicide-related category). Time series with 0.1 to 15% missing data points were imputed using linear interpolation [19,20]. This was the case in search volumes for nine other search terms (28.1%).

Mental health patient and general population suicide time-series

Actual monthly numbers of mental health patients who died by suicide were obtained from the National Confidential Inquiry into Suicide and Safety in Mental Health (NCISH). Note that the more detailed NCISH methodology is available elsewhere [21]. NCISH collects data on mental health patients who die by suicide in the UK through a three-stage process. Firstly, the national mortality data on all deaths classified as suicide or undetermined intent are collected from the national statistical bodies of the UK’s devolved nations. Secondly, individuals who had contact with specialist mental health services within 12 months prior to death are identified via the healthcare providers in the area where the person lived and died. Finally, detailed clinical and socio-demographic information is collected via questionnaires completed by the patient’s supervising clinician. Note that NCISH cases are simply a subset of ONS/NRS cases; ONS/NRS establishes whether a death is a suicide, NCISH then establishes mental health service contact for those cases. Due to the lower data completeness in recent years caused by the time associated with legal processes, identifying patient cases, and obtaining information via questionnaires, monthly data for the years 2020, 2021 and 2022 for England and 2021 and 2022 for Wales and Scotland were projected. These projections were based on the number of unreturned questionnaires, the rate of returns in previous years, and the accuracy of estimates in previous years. This is a standard process for reporting patient suicide figures from recent years [14]. Due to the data on the month of death not being available for Northern Ireland, only data for England, Wales and Scotland were analysed. Monthly general population suicide data is routinely provided to NCISH by the Office for National Statistics and National Records Scotland. Rather than rates, actual suicide counts were used in the analysis since the suicide data already represented complete Great British population of suicide deaths, both in general population and in mental health patients. Notably, suicide counts and suicide rates are highly correlated in our dataset (r = 0.97). Therefore, the correlation between the weighted GT composite and absolute suicide counts result in the same statistical inference to what we would obtain using rates. Data were accessed for research purposes on the 1^st of March 2025. Authors had no access to information that could identify individual participants during or after data collection.

Statistical analysis

Characteristics of participants who provided search terms have been described using frequencies and proportions (for categorical) and median and interquartile range (IQR) (for continuous variables, visual analogue scales). Chi-square tests were used for comparisons of categorical variables, while the Wilcoxon test was employed for comparisons of visual analogue scale data between participants who did and those who did not provide search terms. Associations between Google Trends and mental health patient deaths by suicide time-series were determined using transfer function modelling, as recommended by Tran et al [9]. This was done on both on both full (2011−2022) and sensitivity time period (2011−2020). In the first step, explanatory time-series (Google Trends time-series) was modelled using autoregressive integrated moving average (ARIMA) models; if there was a seasonal component, seasonal ARIMA (SARIMA) was used. Modelling included explanatory time-series outliers, if present. Types of outliers considered were innovational outliers (IO), additive outliers (AO), level shifts (LS), temporary changes (TC) and seasonal level shifts (SLS) [22]. This modelling was iterative, and adequate models were determined based on a) corrected Akaike Information Criterion (AICc) and Bayesian Information Criterion (preferring models with comparatively lower values), b) inspection of the autocorrelation functions (ACF) and the partial autocorrelation functions (PACF) of the model, and c) testing for the presence of remaining autocorrelation of residuals using Ljung-Box statistics. The identified adequate model was then fitted to dependent time series (time series of mental health patient/general population suicide). Finally, the cross-correlation function (CCF) was used to determine the correlation between prewhitened explanatory and dependent time series. CCF was examined for lags from −3 to +3. In order to correct for multiple testing, the Bonferroni correction was applied (p < 0.05/number of tests). All analyses were carried out in RStudio [23], with Google Trends obtained using package gtrendsR [24] and analyses carried out using packages tsoutliers [25] and forecast [26].

Ethical approval

Ethical approval for collecting and analysing participants’ suicide-related search terms was obtained from the University of Manchester Research Ethics Committee 1 on the 3rd of April 2023 (ref: 2023-16133-28055). Suicide data is routinely obtained by the National Confidential Inquiry into Suicide and Safety in Mental Health with approval from the North West – GM South REC (reference: ERP/96/136) and Section 251 Approval under the NHS Act 2006 (originally Section 60 of the Health and Social Care Act 2001) (reference: 23/CAG/0024) for England and Wales and Public Benefit and Privacy Panel for Health and Social Care (HSC-PBPP) (reference: 2021−0114) for Scotland.

Characteristics of participants who provided search terms

Participants who provided search terms were mostly aged between 18 and 34 years (129, 65.9%), female (133, 67.9%) and White (164, 83.7%). Over a third were lesbian, bisexual, or gay (72, 36.7%) and a further 13.3% [26] did not identify with the sex they were assigned at birth. Over half were employed (105, 53.6%), and 61.7% (121 participants) had a university degree or higher qualification. The most common diagnoses were affective disorder (139, 70.9%) and anxiety disorder (133, 57.7%), and 158 (80.6%) had been prescribed psychotropic medication. Participants who provided search terms did not differ from other participants who engaged in SRIU on sociodemographic or clinical characteristics.

Participants who provided search terms most commonly experienced thoughts of suicide every day or almost every day (78, 39.8%) and 89.3% (175) disclosed their thoughts of suicide to someone offline (i.e., a mental health professional, friend, family member). A similar proportion of those who did and who did not provide search terms had attempted suicide in the last year (33.2% v. 31.5%), but those who had provided search terms reported more planning involved in their suicide attempt (median = 7.7, IQR = 5.0–9.0 v. median = 6.0, IQR = 4.0–8.0, p = 0.002).

Participants who had provided search terms mostly searched online for suicide-related content/information (174, 88.8%) and used the internet to interact/connect with others (99, 50.5%), similar to participants who did not provide search terms. The most common motivations to engage in SRIU were seeking help (107, 54.6%) and searching for methods of suicide; the latter was more common for those who provided search terms than in other patients who engaged in SRIU (136, 69.4% v. 182, 54.0%, p = 0.001). Furthermore, 26% [51] of participants who provided search terms were searching for a specific suicide-related site and 66% (130) had engaged in SRIU at least once a month. While engaging in SRIU, the majority (171, 87.2%) had seen suicide prevention messages. Nearly half (83, 48.5%) who had seen prevention messages engaged with them (i.e., called a helpline or visited an advertised prevention website). Finally, participants who provided search terms found their SRIU overall neither helpful nor harmful (median = 5.0, IQR = 4.0–6.4), similar to other participants who had engaged in SRIU. Full comparison between participants who did and those who did not provide search terms is available in Tables 1–4 in S1 File

Characteristics of search terms

Table 1 categorises the suicide-related search terms reported by participants, including the number of unique search terms extracted per seven categories and the frequency with which participants reported those terms. Most of the unique search terms extracted were categorised as suicide methods-related (11, 30.6%). Participants mostly reported searching for non-specific suicide-related queries (127, 40.7%), followed by terms related to suicide methods (90, 28.8%).

Models and cross-correlation coefficients

Table 2 shows the type of SARIMA or ARIMA models that were fitted to explanatory time-series (Google Trends data) for which cross-correlation coefficients on at least one lag were significant. ARIMA models account for time-dependent patterns in the data, while SARIMA models additionally model recurring seasonal effects. The exact search terms are not provided in the table due to ethical considerations regarding the public dissemination of potentially harmful search terms related to suicidality (see [27]).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Fitted Models of Time Series Data with Significant Cross-Correlation Coefficients.

https://doi.org/10.1371/journal.pone.0344926.t002

Of 448 lags (lags from −3 to +3 for each time series) whose cross-correlation coefficient significance was tested, only 18 (2.0%) were significant for both mental health patient and general population suicide. Overall, these cross-correlational coefficients were of medium (.30 ≤ |r| < .50, 22%) and small (0.10 ≤ |r| < 0.30, 78%) effect sizes. Most of the significant coefficients were present for categories of non-specific suicide-related queries (6/18, 33.3%), followed by characteristics of suicide (4/18, 22.2%) and seeking help queries (4/18, 22.2%). Notably, none of the pro-suicide queries yielded significant cross-corelations with actual suicide time series.

Over the full period, there were two significant concurrent (lag 0) positive cross-correlations between search volumes and deaths by suicide of mental health patients time-series; one for a term related to a characteristic of suicide (minimum pain suicide, 0.26) and one for a non-specific suicide-related term (generic suicidal ideation, 0.32). This indicates that increases in mental health patient deaths co-occurred with increases in search volume for these terms. Additionally, there was a significant negative cross-correlation at lag −2 for a suicide method-related search term (referencing self-poisoning, −0.24), suggesting that an increase in search volume for this term was followed by a decrease in mental health patient deaths two months later. Over the full period, there were three concurrent positive cross-correlations between suicide-related search terms time series and general population suicides; one each for search volumes related to a characteristic of suicide (pain-free suicide, 0.25), suicide method (referencing hanging, 0.24), and seeking help (generic suicide support, 0.30). An additional positive cross-correlation was significant at lag 2 (generic suicide term, 0.27).

Over the shortened (sensitivity) period (2011–2020), there were four concurrent significant positive cross-correlations; an increase in search volume for non-specific suicide-related queries (generic suicidal ideation search term, 0.38), characteristics of suicide (minimum pain suicide, 0.25), procedural (‘how to’ suicide query, 0.24) and symptoms or diagnosis-related queries (mentions of anxiety, 0.27) co-occurred with an increase in mental health patient deaths. Two more significant negative cross-correlation coefficients, one for non-specific suicide-related query (expression of intent to die by suicide, −0.25) and one for a seeking help related query (references suicide prevention charity, −0.25), were present at lag −3. This indicates that an increase in search volume for these terms was followed by a decrease in mental health patient deaths three months later. Positive significant cross-correlations in the sensitivity period between search volumes and general population suicides were present at lag 0 for a characteristic of suicide (minimum pain suicide, 0.25) and a help-seeking query (generic suicide support, 0.25) as well as at lag 1 for a non-specific suicide related query (generic suicide methods, 0.25) and a help-seeking query (expression of desire to stop being suicidal, 0.27). Finally, there was one negative cross-correlation between the search volume for a non-specific suicide-related search term and suicides in the general population at lag −3 (expression of intent to die by suicide, −0.31).

Strengths and limitations

To the authors’ knowledge, this is the first study to [1] examine associations between search volume of suicide-related terms and mental health patient deaths by suicide and [2] use search terms provided by the people who actually engaged in SRIU rather than search terms generated a priori by the researchers. Mental health patients, however, are at significantly higher risk for suicide than the general population [42], and their behaviours and needs may differ substantially. Furthermore, by analysing search terms provided by people who actually engaged in suicide-related internet use (SRIU), it was ensured that the data reflected the real language, intentions, and lived experiences of individuals at risk. It also allowed for the identification of terms that may be relevant but are overlooked in traditional research designs. This is particularly important because the effectiveness of online interventions can depend on accurately matching the language and intent of users in distress [43]. Another methodological strength was the use of robust methodology and analysis. The models were carefully selected, accounting for seasonality, outliers, and autocorrelation and the use of multiple data retrievals and weighted averaging added further methodological rigour.

There are also some limitations to the current study. Firstly, it is important to note that associations do not mean causation; both search activity and suicide deaths may be driven by common underlying or external factors [9]. Secondly, chosen statistical approach, which tested multiple lags per time series, required the use of Bonferroni correction to control for multiple comparisons. This conservative adjustment may have limited the detection of smaller, yet potentially meaningful, effects. [44]. Thirdly, it is acknowledged that using UK-wide search data and suicide time series would be optimal. This was not possible since the available suicide data for Northern Ireland did not include month of death. Recent mental health patient suicide data were partially projected due to delays in legal process and data collection. Although NCISH projections are consistently closely correlated with actual numbers with 0–5% margin of errors [45] and sensitivity analyses restricted to earlier years were conducted, some residual bias cannot be ruled out.

While the decision to use search terms provided directly by participants who engaged in SRIU enhanced ecological validity, the characteristics of participants may have amplified the observed associations between certain search volumes. Therefore, the findings are especially relevant to digitally engaged individuals with frequent suicidal ideation and may be less generalisable to those whose suicidal behaviour is more impulsive or who are less active online.

There are some limitations applicable to all research using Google Trends to explore suicidality. Averaging Google Trends data collected across 15 different days helped minimise random fluctuations; however, it could not completely remove sampling bias or limits to replicability. Google frequently updates its algorithms and moderation practices, including the introduction of protective measures for suicide-related searches and these changes are not transparent to researchers. This means older research may underestimate the role of internet searches in suicidality because fewer people used the internet and/or Google in the past and newer research is complicated by Google’s evolving moderation and intervention strategies, which may suppress or redirect certain queries. While this evolution in Google’s policies is largely positive, it makes current data less reflective of organic search behaviour and, potentially, suicide behaviour. It is worth noting that, with these changes, the content shown to the user following a suicide-related search are probably different today compared to earlier periods. Also, it is important to note that Google Trends was not developed as a research tool and the algorithms Google uses to process, sample, and present search data are proprietary. Therefore, researchers cannot know how search volumes are sampled, how outliers or anomalies are handled, or how changes to Google’s algorithm might affect results over time.

Conclusion and future directions

A recent paper by Rushendran and Chitra [46] states that without knowing who is searching for suicide-related content and why, it is impossible to reliably infer population-level distress or suicide risk from search volumes alone. This study provided some evidence as to who is searching, but the ‘why’ is still largely unknown. Even seemingly clear ‘how to’ queries, when considered motivation-wise, may stem from personal crisis, academic research, professional interest, or even curiosity after media coverage. Therefore, future research in this field should investigate intent as well as the content of the suicide-related searches. Studies could also consider analysing the content elicited by the search term studied, which could provide additional insights into why certain search terms are preventative or risk related. For instance, the information and resources in search results may influence user perceptions and subsequent actions, potentially explaining observed associations. However, Google does not provide access to historical search result pages, meaning that researchers cannot retrospectively analyse what users would have seen at a given point in the past. Consequently, these studies should be designed prospectively and search result pages would need to be systematically captured at regular intervals. While online searching for suicide-related information is most likely a prevalent behaviour, this study corroborates conclusions from Tran and colleagues [9] that Google Trends data is likely not suitable for monitoring and prediction of actual suicide behaviours on a population level. However, this approach can still provide valuable insights and help generate hypotheses on suicide-related internet use, as long as the interpretations are tempered by explicit acknowledgement of its limitations.