Correction
1 Mar 2018: Marchant A, Hawton K, Stewart A, Montgomery P, Singaravelu V, et al. (2018) Correction: A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLOS ONE 13(3): e0193937. https://doi.org/10.1371/journal.pone.0193937 View correction
Figures
Abstract
Background
Research exploring internet use and self-harm is rapidly expanding amidst concerns regarding influences of on-line activities on self-harm and suicide, especially in young people. We aimed to systematically review evidence regarding the potential influence of the internet on self-harm/suicidal behaviour in young people.
Methods
We conducted a systematic review based on an electronic search for articles published between 01/01/2011 and 26/01/2015 across databases including Medline, Cochrane and PsychInfo. Articles were included if: the study examined internet use by individuals who engaged in self-harm/ suicidal behaviour, or internet use clearly related to self-harm content; reported primary empirical data; participants were aged under 25 years. New studies were combined with those identified in a previous review and subject to data extraction, quality rating and narrative synthesis.
Results
Forty-six independent studies (51 articles) of varying quality were included. Perceived influences were: positive for 11 studies (38191 participants); negative for 18 studies (119524 participants); and mixed for 17 studies (35235 participants). In contrast to previous reviews on this topic studies focused on a wide range of internet mediums: general internet use; internet addiction; online intervention/treatment; social media; dedicated self-harm websites; forums; video/image sharing and blogs. A relationship between internet use and self-harm/suicidal behaviour was particularly associated with internet addiction, high levels of internet use, and websites with self-harm or suicide content. While there are negative aspects of internet use the potential for isolation reduction, outreach and as a source of help and therapy were also identified.
Conclusions
There is significant potential for harm from online behaviour (normalisation, triggering, competition, contagion) but also the potential to exploit its benefits (crisis support, reduction of social isolation, delivery of therapy, outreach). Young people appear to be increasingly using social media to communicate distress, particularly to peers. The focus should now be on how specific mediums’ (social media, video/image sharing) might be used in therapy and recovery. Clinicians working with young people who self-harm or have mental health issues should engage in discussion about internet use. This should be a standard item during assessment.
A protocol for this review was registered with the PROSPERO systematic review protocol registry: (http://www.crd.york.ac.uk/prospero/display_record.asp?ID=CRD42015019518).
Citation: Marchant A, Hawton K, Stewart A, Montgomery P, Singaravelu V, Lloyd K, et al. (2017) A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS ONE 12(8): e0181722. https://doi.org/10.1371/journal.pone.0181722
Editor: Kim-Kwang Raymond Choo, University of Texas at San Antonio, UNITED STATES
Received: March 6, 2017; Accepted: July 6, 2017; Published: August 16, 2017
Copyright: © 2017 Marchant et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files. Relevant included studies are all referenced. Searches are available from the authors.
Funding: This work was supported by the Health and Care Research Wales(Grant: SC-14-11; https://www.healthandcareresearch.gov.wales). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: KH is a National Institute for Health Research Senior Investigator and is also supported by Oxford Health NHS Foundation Trust. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Internet use has a mixed effect on children and young people’s (CYP) well-being, with evidence of increased self-esteem and perceived social support alongside harmful effects such as increased exposure to graphic content and cyber-bullying [1]. High profile cases of cyber-bullying and suicide over the past decade [2] and reports of suicide clusters facilitated by social media [3] have resulted in researchers increasingly focusing efforts on understanding the relationship between internet use and both self-harm and suicide. An early review examining the use of dedicated self-harm support sites in young people found high levels of support available online alongside normalisation of self-harm behaviour [4]. A more recent systematic review relating to self-harm and internet use in young people additionally found evidence of information sharing of methods of self-harm and concealment, a reduced sense of isolation and reinforcement of positive behaviours such as help seeking [5]. The authors concluded that the internet exerts both positive and negative influences on self-harm and provides an opportunity for intervention, which is supported by several more recent studies [6, 7].
While research appears to be beginning to answer some questions regarding the role of the internet for young people who self-harm a number of questions remain unanswered. Authors of the previous review highlighted that research to the date of their review (26th December 2011) described internet use only in relation to use of forums or general use [5]. This is an important limitation since specific internet pathways may represent increased risk. Internet addiction and pro-suicide websites have been suggested as high risk factors facilitating suicidal behaviours, particularly in isolated and susceptible individuals [8]. Additionally, a recent study examining changes in online suicide-related content showed that the results of searches for self-harm have changed over time, with an increasing presence of graphic imagery [6]. The role of such images has been examined in relation to self-harming behaviours [9], as well as relevant content of videos on a popular sharing website [10]. A recent study has also showed that the nature of sites returned varied according to the suicide/self-harm-related search terms use and almost half contain video content. Negative influences such as providing information on methods, encouraging self-harm behaviours and images considered evocative by researchers were common. However, more positive influences including advice on how to seek help was given on over half of identified sites [7]. Further research has highlighted the positive and negative influences of the internet, the potential benefit of directing individuals to healthier online behaviour and the discrepancy in perspectives regarding online activities between adolescent culture and that of many mental health professionals [11].
The previous review also identified a possible relationship between study design and perceived outcome. Qualitative and mixed methods studies tended to report a positive influence of internet use and quantitative designs tended to find a negative impact. Therefore the design and quality of individual studies may have had an impact on perceived outcomes. Following their systematic review [5] the authors highlighted the need for more rigorous research to clarify the positive and negative influences and to focus on the mediating and moderating factors in order to optimise the benefits whilst minimising the potential harm of internet use in young people who self-harm.
The primary aim of this study was to systematically review all research relating to the potential influence of the internet on self-harm/suicidal behaviour in young people, with a particular focus on identifying the factors which determine whether the internet is perceived as positive or negative in its potential influence.
Method
Search strategy and selection criteria
This study was a systematic review. The previous electronic literature search, conducted by some of the authors, was up to 26th December 2011 [5]. For the present review, an electronic literature search was conducted (AM) for all articles published between 1st January 2011 and 26th January 2015. A range of databases were searched including: CINAHL; Cochrane Library; EMBASE (excluding Medline journals); HMIC; Medline; NICE; Prospero; PsycINFO; PubMed; SCOPUS. Additional searches were conducted in health improvement sources, topic specific websites (American Association of Suicidology, British Psychological Society, CEBMH, Centre for Mental Health, DH, DHSPSS-NI, MFH, NHS Scotland, Royal College of Psychiatrists, Welsh Government) and meta-search engines (Google/ Google scholar) for these dates only.
The search terms employed in the original review (which included ‘Self harm’, ‘Suicid*, ‘Internet’, ‘Children’, ‘Young People’) were updated to account for the rapidly changing nature of the internet and memes (an image, piece of text, idea that spreads rapidly) in young people. The full details of search terms and sources searched are included in S1 Table. Reference lists of all review articles were manually screened for potential eligible papers. The research team included a number of experts in the field who reviewed searches for any potentially unidentified citations. Experts in the field were also contacted to assist in identification of literature. These included those known to members of the team and others identified from existing literature. They were contacted by email regarding any additional studies and, where relevant, data or full text copies of articles were also requested by email.
Articles were included if they examined internet use by individuals who experienced suicidal ideation, self-harm, or internet use which was clearly related to self-harm content. Any type of online media or activity was considered for inclusion. We included articles that presented primary empirical data and were published in journals. Participants had either to all be aged less than 25 years or have a mean age of 25 years or less. If ages were not stated, participants had to be described as children, adolescents or young adults. Where articles examined more than one age group, only data for the age group fitting these criteria were analysed. Results were not restricted on the basis of location; however, only English language papers were included.
In order to be consistent with the previous review [5] review articles, news articles, single case studies, editorials, comments, conference abstracts and other grey literature, while searched for (e.g. CINAHL, HMIC), were not included in the final analysis. While the inclusion of grey literature can reduce the impact of publication bias, it may also introduce its own set of biases. These include the absence of peer review and the potential that the availability of data would impact on overall results [12]. Included articles from the previous review on this topic 1991–2011 [5] were added to the newly identified eligible articles to provide a complete overview of available evidence. Studies relating to cyber-bullying and self-harm from both the new and existing literature were excluded and will be reviewed in a separate report. This decision was taken due to the number of known quantitative studies related to this specific topic and because it would allow for a more thorough discussion of the literature with the potential for meta-analysis (protocol available from http://www.crd.york.ac.uk/prospero/display_record.asp?src=trip&ID=CRD42017056487).
Two independent reviewers (AJ, AM) manually screened titles. Any disagreements were resolved by consensus. Titles that clearly had no relevance, book chapters, case reports, conference abstracts, comments, editorial, journal notes, grey literature and news sources were excluded at title screen, although reference lists were manually screened for relevant studies. A record was kept of all discarded articles, including the reason for exclusion. Duplicates were removed. The remaining titles with abstracts were then screened for eligibility by the same two researchers. Full text articles were obtained where suitability could not be determined based on the title and abstract. Two researchers independently reviewed the remaining citations (AJ, AM). Any disagreements that could not be resolved through consensus were discussed with a third expert reviewer (KH).
A protocol for this review was registered with the PROSPERO systematic review protocol registry (http://www.crd.york.ac.uk/prospero/display_record.asp?ID=CRD42015019518)
Data analysis
The data extraction sheet from the previous review [5] was adapted and used to record specific findings from both newly identified articles and those from the previous review (S2 Table). Additional fields were added to account for the greater level of detail in more recent research articles and to allow comparison of internet medium, outcome measured and study design. Studies were divided between four reviewers (AM, NP, AS, VS) and pairs of reviewers’ independently extracted data for each study. Any inconsistencies in data extraction and quality scores were clarified by consensus with at least two study authors. Articles were amalgamated and grouped according to internet medium studied and perceived influence. Positive influences were defined as results indicating perceived reduction of psychological distress, reduced suicidal ideation and self-harm, advice on how to seek help and encouragement to do so. Negative influences were defined as results indicating: increased psychological distress, self-harm or suicidal ideation; information on methods of self-harm/suicide was provided; self-harm behaviours were encouraged. Mixed influences were recorded where a report included both positive and negative influences. Internet media were grouped according to their stated description within articles. These media categories were inductively generated following initial reading and data extraction of papers and were cross checked by two members of the study team (AM and AJ).
Quality of included articles was assessed according to the Critical Appraisal Skills Programme (CASP) [13] as performed previously [5]. This tool assesses various aspects of study design including the study sampled, data collection methods, study design and the clarity and appropriateness of results and conclusions. It also includes items related to potential sources of bias such as from the study population or design.
Due to the range of research questions, methods used, populations and outcomes studied there was a high level of clinical and methodological heterogeneity across studies, precluding any meaningful combination of study results through meta-analysis. Therefore, a narrative synthesis was employed. Based on published guidance [14] this narrative synthesis examined a number of key aspects. Comparisons across studies were made regarding the way in which the relationship between self-harm/suicide and internet use had been identified and analysed; relationships between study results were examined and compared across the studies; the influence of heterogeneity was further explored including theoretical variables, differences in baseline characteristics of populations, measures employed and outcomes studied.
Results
Fig 1 shows the results of the search strategy and screening process. A total of 51 articles (from 46 independent studies) were included in the review. A summary of included articles by internet medium and perceived influence can be seen in Table 1. Studies were based in the USA (n = 10), UK (5 studies, 9 articles), Canada (n = 5), Japan (n = 3), Korea (n = 3), Australia (n = 2), New Zealand (n = 2), Sweden (n = 2), China (n = 1), Germany (n = 1), Israel (n = 1), Northern Ireland (n = 1), South Africa (n = 1), Taiwan(n = 1) and Turkey (n = 1), with the remaining 7 studies (8 articles) from multiple countries. A total of 192,950 individuals participated. Forty-four of the 51 articles had more than 50% female participants. Eleven studies [15–25] examined content of forum posts or websites in which participants were described only in terms of demographics of site users. One study [26] examined rates of suicide by age group with no further description of the number of participants. Using the CASP quality score 17 articles were assessed as high quality, 19 as medium quality and 15 as low (S3 Table). The quality of articles varied by study design with a greater proportion of quantitative (13/21) than qualitative ones (0/18) rated as high quality. Number of studies, number of articles, study design and quality varied across internet media (Table 2).
The sampling of participants varied greatly between studies, each introducing potential selection bias. For example, study participants were recruited via dedicated online support forums [51–59, 61]; emergency departments [49]; other healthcare settings [44–46]; through digital metrics [27]; and through large school-based and community surveys [28–31, 33, 36–42].
A number of outcomes were assessed in the studies, including levels of self-harm and suicidal behaviours, mental disorders, internet addiction, levels of loneliness and insomnia, the potential to recruit participants for research [15] and the nature of online information seeking [27] (S4 Table). Measures ranged from study-specific self-report questionnaires or content analysis themes to validated scales to assess suicidal behaviours, internet use, mental disorders and well-being e.g. Internet Addiction Scale, Beck Depression Inventory, Strength and Difficulties Questionnaire, Suicidal Ideation Questionnaire.
Perceived influences were: positive, 15 articles, 11 independent studies, n = 38,191 participants; negative, 19 articles, 18 independent studies, n = 119,524 participants; mixed, 17 articles, 17 independent studies, n = 35,235 participants. Table 3 summaries the mechanism of perceived influence by internet medium.
General internet use
Twelve studies examined the influence of general internet use [3, 20, 26–35]. Papers were categorised as examining general use if they spoke generally about internet use, did not detail the type of internet use or examined a number of different mediums. Two studies identified positive influences [27, 28]. One of high quality utilised digital to metrics to demonstrate high levels of engagement reported in online discussions about mental health by young people [27]. The second, a low quality study, utilised a community survey and showed lower levels of depression associated with frequent video game use. No association was found between media use and suicidal ideation [28].
Seven studies identified negative influences [3, 26, 29–33]. Studies rated as high quality utilised survey data and showed that high internet users and non-internet users were at higher risk of suicidal ideation and attempted suicide when compared with moderate internet users [31]. An ‘invisible risk’ group of individuals, who spent a lot of time online but did not engage in other risky behaviours (e.g. smoking) and had similar prevalence of suicidal thoughts as the ‘visible’ risk group, was identified in another high quality school based survey [29]. A relationship between internet use and self-harm was reported in several high quality papers. 15% of girls and 26% of boys reported that either the internet or social media had influenced their self-harm [33]. Risky online behaviours such as having a close relationship with someone met online (38% of those who reported self-harm compared with 10% of those who did not) was reported in a set of telephone interviews [32]. This relationship was further supported by an additional low quality school-based survey where suicidal ideation was found to be significantly associated with accessing suicide or self-injury information online (OR 5∙11; 95% CI 0∙35–0∙75), anxiety about getting email replies (OR 2∙06; 95% CI 1∙33–3∙20) and hurtful experiences online (OR 1∙71; 95% CI 1∙03–2∙84) [30]. In addition to the relationship between online behaviours and self-harm it was found that the internet and social media had facilitated the spread of information in a potential suicide cluster, and may have linked suicides otherwise unconnected by school or district [3]. Furthermore, a relationship between internet searches for specific suicide methods and the suicide rate in young people was found in one low quality study [26].
Three studies showed mixed influences [20, 34, 35]. In one small but high quality set of structured interviews participants found some interactive media supportive, while use of violent methods of self-harm were often preceded by viewing suicide related content online [34]. In a medium quality study aiming to examine self-harm content across internet mediums, large dedicated self-harm social media groups, 2290 videos related to self-harm on YouTube and high levels of views of videos about self-harm (the top video being viewed 339646 times) were identified. Peer-driven websites were found to contain warnings of triggering content, including stories and image galleries (not always present). These sites were accessed more frequently than professionally-driven websites [20]. Participants partaking in structured interviews reported that online sources of information related to suicide were quite common, and while social networking sites were frequently cited as sources these were not linked to increases in suicidal ideation, whereas online discussion forums were [35].
Internet addiction
Seven studies [36–42] examined the relationship between internet addiction and self-harm. For the purpose of this review internet addiction was distinguished from general internet use if papers specifically referred to internet addiction or pathological internet use. There is no agreed definition of internet addiction in the current literature. It ranges from an impulse control disorder likened to pathological gambling [36–38, 41] to assessments of level of functional impairment or level of use [39, 40, 42]. All seven studies were rated as of high or medium/high quality, utilised school-based surveys with validated outcome measures and all found a significant relationship between internet addiction and self-harm/ suicidal behaviour. However, the direction of causality remained unclear. Five of these studies found exclusively negative results [36–40]. When a pathway model was employed problematic internet use was found to predict suicidal ideation alongside suicidal ideation predicting problematic internet use [40]. Two studies showed mixed effects [41, 42]. Potential mechanisms of positive influences included low levels of loneliness and higher levels of life satisfaction in individuals with possible internet addiction and a potential protective influence of low levels of internet use compared with none [42] (Table 3).
Online intervention/treatment
Six studies were of help administered online with the aim of reducing self-harm/suicidal behaviour, often alongside other manifestations of psychological distress such as depression and anxiety [43–48]. Participants came from a range of settings including schools [43], healthcare settings [44, 46] and universities [47, 48]. Positive influences were identified in four small studies. The successful administration of a program of treatment for depression was demonstrated in one high quality study [46]. Medium quality studies showed the successful implementation of an online monitoring tool for depression and suicidality [44], and for cognitive behaviour therapy [43]. All three of these articles showed a reduction in either self-harm or suicidal ideation. The fourth low/medium quality paper demonstrated that participants positively viewed the idea of mental health services delivered online via questionnaire data and structured interviews [45]. Two much larger studies rated as high [48] and medium quality [47] showed mixed influences, and in the first of these participants did not report negative experiences following the completion of online surveys related to self-harm Participants responding to distressed emails solved problems personally rather than suggesting professional help, with females being more likely to offer help than males [47].
Social media
Four studies examined the role of social media, including one content analysis paper [21, 24, 49, 50]. For this review social media studies were defined as studies focused on social networking sites such as Facebook and Twitter. Three studies showed negative influences, the studies being rated as low [21, 49] or low/medium quality[24]. It was found that young people attending emergency departments are increasingly using social media to communicate distress prior to hospital attendance for self-harm, particularly to a peer rather than to an adult [49]. Content analysis of open self-harm groups on Facebook revealed glorification and normalisation of self-harm [21]. While Facebook is a moderated site, groups are not moderated in the same way as support forums, where there are often rules on appropriate content. Analysis of comments on MySpace revealed suicidal thoughts, behaviours and intentions indicating significant amounts of hopelessness and despair [24].
One article rated medium/high quality reported the results of an internet survey (n = 1000) and contained mixed results [50]. Tweeting ‘want to die’ or ‘want to commit suicide’ was significantly related to suicidal ideation and behaviour. However, no association was found with simply having a twitter account or tweeting frequently [50].
Forums
Fourteen articles [15–19, 51–59] related to forums (five content analysis articles). For this review forum use was defined as the use of dedicated support forums, separate from social media sites. The setting of five of these was the ‘Sharptalk’ forum purposely built for research [53–57]. Each article examined a different aspect of forum use, was of low or medium quality and reported a positive influence. There was some reluctance on the part of health professionals in this study to participate actively in forums due to a reported lack of confidence and concerns relating to workload and duty of care [54]. In total, nine papers (five independent studies) reported positive results; one high quality content analysis [16] and four medium and low quality small studies [15, 51–57]. Five studies reported both positive and negative results; one high quality [58] and four medium and low quality [17–19, 59]. All the studies recruited via dedicated discussion forums. One study gathered data via in-depth email interviews [51], all but one of the other papers analysed the content of posts and four studies additionally collected data via questionnaires [53–59]
Positive influences identified in high quality papers include the potential for isolation reduction and informal support [16], and a significant reduction in suicidal thoughts following forum use, although it was noted that a causal relationship could not be directly inferred [58]. Well-moderated self-harm forums appeared to be viewed positively by young people as an online community providing continued support. One study (n = 20) found that levels of forum involvement were associated with lower levels of distress, but this level of distress did not decrease over the three month study period [52]. The potential for support and isolation reduction was supported by a number of small medium and low quality papers [15, 16, 18, 19, 51–57, 59].
Negative influences include the use of forums for destructive means, such as finding a suicide partner and exchange of potentially harmful information found in one high quality paper [58] and four low and medium quality papers [17–19, 59]. Direct encouragement to go ahead with suicide plans, including detailed suggestions of method, and validation for reasons given for a planned suicide were also reported [19]. While some if the discourse was judged to be potentially harmful, no association was found between worsened suicidal thoughts and forum use [59].
Website with suicide/self-harm content
Two articles reported on websites with dedicated self-harm/suicide content. A medium quality content analysis study of websites indicated negative influences, including that dedicated self-harm websites may normalise and reinforce self-harm [25]. A large high quality cross-cultural survey reported that individuals use these websites to gain help and support. While some participants reported a reduction in self-harm a small number reported increased self-harm associated with these websites.
Video/image sharing
Four articles focussed on video/image sharing [10, 22, 23, 61]. One medium/high quality content analysis of comments on self-harm videos identified negative influences [22]. Comments on such videos illustrate a strong motivation for sharing experiences online, rarely mention recovery and may contribute to the maintenance of self-harm. The other three papers were of medium and low quality and identified mixed results [10, 23, 61]. Videos with self-harm content on YouTube were found to frequently contain explicit imagery with factual and educational tones [10]. Videos raising awareness of lesbian gay bisexual or trans (LGBT) suicides highlighted that difficulties in life can be overcome; however, negative connotations were also present [23]. Some participants recruited from an online self-harm community reported that viewing of self-harm acted as a deterrent or alternative to self-harm, whilst others reported a triggering effect and sense of competition [61].
Blogs
Two articles reported the content analysis of the same set of pro-anorexia blogs [62, 63]. Both were rated as low quality and reported negative influences in the form of sharing potentially harmful information related to self-harm and suicide methods and means of concealment.
Across different mediums several studies focused on groups often hidden from traditional research and service provision. For example, three studies focused on populations of LGBT young people [15, 18, 23]. The first was an analysis of the content of YouTube videos raising awareness of the level of suicide in LGBT youth [23]. The other two papers showed that online discussion forums had the potential to engage this hard to reach group and to produce novel, unmediated data [15]. This approach was utilised to gather data on LGBT young people’s perspective on seeking help for suicidal feelings. LGBT individuals found it challenging to articulate emotional distress and seek help from family members or professionals. They were most comfortable communicating online, particularly in dedicated LGBT forums [18]. A further two papers focused on young people with eating disorders [62, 63]. In these two studies a set of eating-disorder blogs were analysed, which showed that discussion of self-harm and suicidal thoughts was common.
Discussion
This systematic review is an update of a previous smaller review exploring the relationship between internet use and self-harm [5] and incorporates previous evidence, thus also updating the field. A total of 51 articles (representing 46 independent studies) were included, with 192950 individual participants, together with some reports of content analytical studies. While a comparable number of articles included positive (15), negative (19) and mixed (17) influences of the internet on self-harm behaviour, articles demonstrating negative influences included more participants (n = 119524) than those with positive (n = 38191) and mixed influences (n = 35235). On balance, considering the quality of studies and numbers of participants assessed, there is significant potential for harm of online behaviour in relation to self-harm and suicidal behaviour, but also potential benefits that merit exploitation.
Results of this review are largely supportive of the findings of the previous review with comparable proportions of studies with positive and negative results [5]. However, in recent years this field of research has developed substantially allowing greater examination of details such as medium of internet use and for a greater range of positive and negative influences to be identified. Studies on general internet use, internet addiction and online interventions/treatment presented the strongest evidence, with mostly high or medium quality research. High internet use and internet addiction appear to have largely negative influences. Twelve studies examined general internet use, half of which were of high quality. The latter group of studies demonstrated that high levels of internet use (more than two [31] or five [29] hours per day) were associated with suicidal ideation [29, 31]. Further low and medium quality research suggested that self-harm and suicidal ideation were related to searching online for suicide information [30] and that searches for specific methods were related to rates of suicide in young people [26]. Online media appeared to facilitate the spread of information, linking otherwise unconnected suicides, making it difficult to recognize and manage a suicide cluster [3]. Research examining internet addiction represented the most homogenous group of studies; all employed high/medium quality quantitative methodologies in the form of cross sectional school-based surveys and validated outcome measures. All studies examining internet addiction found a relationship between internet addiction and self-harm or suicidal behaviour. The direction of causality of the negative influence of internet addiction and self-harm/suicidal behaviour was unclear. Positive influences included lower levels of loneliness [41] and a potential protective influence of low levels of internet use when compared with no internet use at all [31, 42]. Other studies have found that low and high internet use is associated with higher adolescent health risks than moderate use [31]. The internet may provide an opportunity to be part of a community online when this is lacking in everyday life. The potential of the internet as a medium to deliver interventions to address suicidal behaviours and self-harm was examined in six studies of medium to high quality, with mixed results but generally being viewed positively by participants.
Studies exploring the other mediums (social media, forums, videos/ images sharing, blogs) were smaller, of lower quality and with more mixed results. Only one paper related to social media was rated as medium/high quality. Distressed online posts were found to be related to suicidal ideation and behaviour, but there was no evidence suggesting that simply using social media presents a risk [50]. The remaining low and medium/low quality research found that young people are increasingly using social media to communicate distress, particularly to peers [49]. This is in keeping with more recent research showing that self-harm and suicide-related internet use prior to ED attendance is higher among children and young people than in adults, and is additionally related to higher suicidal intent [64]. While social media is being utilised here to communicate distress, this does not indicate a causal relationship. Glorification and normalisation of self-harm was found in two of the four social media studies [21, 24]. Forums were viewed positively as a source of peer support. There was evidence of a reduction in suicidal thoughts following forum use in one high quality study [58] but also evidence of normalisation of self-harm, encouragement to go ahead with suicide plans and discussion of how to conceal self-harm, supported by one high quality and a number of medium and low quality studies [16, 17, 19, 50, 58]. The impact of forum use on levels of self-harm remains unclear. Videos were highly viewed/shared, largely factual or educational and often contained graphic imagery, but rarely with warnings of such content [10]. One high quality study found that comments on videos may contribute towards the maintenance of self-harm and suggests a strong motivation for sharing experiences online. It remains unclear whether this sharing of experiences has a positive or negative impact on individuals, with its influence likely to vary with individual circumstances [22]. In the remaining (low quality) study some participants reported an alleviating effect of images, others a triggering one [61]. Sharing of potentially harmful information related to self-harm and suicide was reported in two small low quality studies of the same set of pro-anorexia blogs [62, 63].
The variation in results between mediums may be partially attributable to study design and participant samples. Research examining general internet use and internet addiction was largely based on school surveys employing validated outcome measures and found largely negative influences of internet use. In contrast, research examining forum use recruited almost exclusively from online discussion forums, undertook content analysis of forum posts, sometimes alongside questionnaire data, and often found positive influences. The contrasting results between studies recruiting from general population and self-selecting participants are not unexpected. Studies with more diverse participant samples and employing validated outcome measures may assist in clarification of the effects of different types of medium.
Research in this area may be biased towards measuring certain effects and outcomes e.g. a positive effect for interventions, or a negative effect for internet addiction. The choice of outcome measures may reflect the expected effects of the internet and fail to capture the full complexity of the experience of individuals. This is a risk in all research but may be a particular issue in this field. For example, studies examining internet addiction report largely negative results. However one study found that internet addicted individuals reported high levels of life satisfaction and low levels of loneliness that may not have been identified had these measures not been included [41]. In future investigations researchers should aim to ensure that outcome measures capture both possible positive and negative outcomes in order to give a complete and unbiased picture. Such steps can be seen in more recent research. For example an online treatment for depression resulted in symptom reduction but examination of possible negative influences demonstrated that individuals with lower education were at higher risk of symptoms becoming more severe [65]. Another recent study found both positive and negative influences of websites detailing suicide methods [66].
In keeping with the previous review [5] the majority of studies finding solely negative influences utilised quantitative methodologies. Mixed methods and qualitative studies tended to report more mixed results. It is unclear if this is a discrepancy between what participants report and actual outcomes or if quantitative data are failing to capture the complexity of the issue. This is further compounded by the mostly low quality of included qualitative studies. High quality qualitative and quantitative research is needed to establish whether differences in the influence of various mediums are partially attributable to study design.
Strengths and limitations
The potential for publication bias exists in any review of literature and should be considered when interpreting the results. Steps were taken to minimise any bias as much as possible including conducting an extensive search of multiple databases including grey literature databases and topic specific websites, reviewing reference lists and contacting experts in the field. However, only English language publications were included. It was noticeable that males remain under-represented in studies. Further examination of any gender differences is a potential avenue for future research.
While the decision to review literature related to cyber-bullying in a separate review will have had an influence on the proportion of articles reporting positive and negative results, it will allow for a more in-depth discussion of this important topic. The inclusion of this large body of research would have created an unwieldy review or necessitated a more cursory discussion of internet mediums. Literature related to cyber-bullying is unlikely to report any positive outcomes and had it been included it would have added to studies reporting a negative influence. This could potentially be viewed as an important bias in this review. However, results of this review highlight that the internet has the potential for both positive and negative influences dependent on the way in which it is used. Identifying both beneficial and harmful mediums and directing individuals towards healthy online behaviour should be considered of greater importance than weighing up whether the internet is simply ‘good’ or ‘bad’.
While this review summarises data from an extensive search, there will be further literature published since the search was conducted. Based on the number of additional articles in the two years since the review by Daine et al [5] it is likely that this will represent a considerable body of research. Summarising it is likely to be beyond the scope of one review. Because of the rapidly increasing body of research, authors conducting future systematic reviews on this topic may consider reviewing by specific internet medium (forums, videos etc.) to ensure the task is manageable and the resultant reviews are of sufficient depth in each area to allow identification of key messages for clinicians and policy makers. While a range of internet mediums were included in this review, recent research has expanded even further, for example examining behaviours such as online gambling [67].
The quality of qualitative studies in this review appeared mostly to be low. While this may be reflective of quality, the appropriateness of using checklists to assess qualitative research has been questioned due to the diversity of approaches in collecting, analysing and interpreting data [68, 69]. This may be particularly problematic with research into internet use where there is considerable heterogeneity in methodology across study designs. It is unlikely that a single checklist would be suitable to capture all aspects of study quality across the range of study designs, populations and outcomes. Authors of future reviews in this area might consider the selection of quality outcome measures that fully capture the quality of various study designs.
It is not always possible in studies examining forum use or comments on videos/photos to accurately determine the characteristics of individuals and young people may misrepresent their age when creating profiles. Therefore ages of participants were not always clearly reported but were inferred by study authors based on demographics of typical users of sites or based on profile information.
It was a challenge to categorise findings as negative or positive in some studies [5]. The sharing of experiences online, for example, while therapeutic for some, may be destructive for others. The expression of distress online could be viewed as negative on the one hand, but as an opportunity for intervention on the other. This is an area of research where outcomes are not always clear cut. It was not always easy to understand whether certain factors acted as mediators or moderators of distress, nor the long-term implications. For example, individuals with a history of self-harm were more likely to report discomfort following online questionnaire completion related to self-harm, but it also caused them to think more deeply about their lives which may have been positive [48]. While it has been possible here to broadly discuss the influence of various mediums and to identify those that may represent a higher risk, the impact of various aspects of internet use is likely to vary between individuals and over time and should therefore always be assessed on an individual basis.
A wider range of internet mediums was examined than was possible previously [5]. This has resulted in a varied pool of participants, including those recruited from discussion forums, schools and healthcare settings. The majority of quantitative studies employed validated outcome measures, in contrast to literature identified in the previous review [5], allowing some comparison of results across studies. However, the considerable range in methodology, population and outcomes studied means that, at this time, it is not possible to conduct a meaningful statistical meta-analysis.
Implications
The volume of self-harm videos shared on YouTube and the high number of views and comments have led to suggestions of developing videos to emphasise help and recovery [10, 22]. The introduction of psycho-educational prevention programmes in schools concerning appropriate responses to distressed posts on social media and digital citizenship may mitigate some of the negative influences of the internet [20, 47, 49]. The internet is a potential tool for outreach by health professionals. Research suggests some disconnect between healthcare professionals and media usage [11]. This was only explicitly discussed in one set of studies reviewed here in which health professionals expressed discomfort about engaging with young people in an online setting and had concerns over duty of care [54]. This could be addressed through further training and encouragement of clinicians working with young people who self-harm or have mental health issues to engage in discussion about internet use. This should be a standard item during assessment. It could include the asking about the role of images/videos [61] and designing treatment plans to maximise beneficial online behaviours and reduce associated harms [20].
Suggestions have been made for the implementation of guidance to individuals and service providers such as avoiding details of method and including warnings of graphic content on web pages [20]. Stricter regulations could be modelled on the initiative in Australia, where pro-suicide sites were banned in 2006 [70]. Several major social media platforms (Tumblr, Pintrest, Instagram, Facebook) have responded to concerns and implemented policies regarding posts related to self-harm. Such content may not be searchable, is banned or brings up links to counselling and prevention resources [71]. The potential to access groups online largely hidden from the health service, such as those for LGBT individuals or those with eating disorders for both interventions and research may improve access to care and allow representation in research that has not been possible previously. Any online sign-posting or interventions should not be limited to platforms exclusively dedicated to self-harm but should also extend to other groups at potential risk. This is supported by recent research finding that experiences of victimisation are associated with entering pro self-harm and pro-suicide websites [72].
Conclusions
Research concerning the internet and self-harm in young people is rapidly evolving in an attempt to keep pace with the continually changing nature of its use. On balance, considering the quality of studies and numbers of participants assessed in this review, there is significant potential for harm to result from online behaviour in relation to self-harm and suicidal behaviour (normalisation, triggering and competition between users, a source of contagion and harmful information for vulnerable individuals), but also the potential to exploit its benefits (a sense of community, crisis support and reduction of social isolation). The focus should now be on a range of internet mediums including social media, video/image sharing, and the potential for the internet to be used in therapy and recovery.
There were a number of innovative suggestions from research teams responsible for publications, including educational programmes in schools to teach young people how to respond to distressed posts/messages on social media and repeated calls for clinicians to be aware of internet use. The internet may also represent an under-utilised setting to access ‘hidden’ at risk groups, giving them a voice both in research and in practice.
Supporting information
S4 Table. Outcomes studied and measures used.
https://doi.org/10.1371/journal.pone.0181722.s004
(DOCX)
Acknowledgments
KH is a National Institute for Health Research Senior Investigator and is also supported by Oxford Health NHS Foundation Trust.
Author Contributions
- Conceptualization: AJ AM NP AS VS KD KH KL PM.
- Data curation: AM AJ NP.
- Formal analysis: AJ AM KH.
- Funding acquisition: AJ.
- Investigation: AJ AM NP AS VS KD.
- Methodology: AJ AM NP AS VS KD KH KL PM.
- Project administration: AJ AM.
- Resources: AM AJ NP.
- Validation: AJ AM NP AS VS KD KH KL PM.
- Visualization: AM AJ.
- Writing – original draft: AM AJ.
- Writing – review & editing: AJ AM NP AS VS KD KH KL PM.
References
- 1. Best P, Manktelow R, Taylor B. Online communication, social media and adolescent wellbeing: A systematic narrative review. Children and Youth Services Review. 2014;41:27–36.
- 2. Lester D, McSwain S, Gunn JF. Suicide and the internet: the case of Amanda Todd. International Journal of Emergency Mental Health. 2013;15(3):179–80. pmid:24558746
- 3. Robertson L, Skegg K, Poore M, Williams S, Taylor B. An adolescent suicide cluster and the possible role of electronic communication technology. Crisis. 2012;33(4):329–45.
- 4. Messina ES, Iwasaki Y. Internet use and self-injurious behaviors among adolescents and young adults: An interdisciplinary literature review and implications for health professionals. Cyberpsychology, Behavior, and Social Networking. 2011;14(3):161–8.
- 5. Daine K, Hawton K, Singaravelu V, Stewart A, Simkin S, Montgomery P. The power of the web: a systematic review of studies of the influence of the internet on self-harm and suicide in young people. PloS one. 2013;8(10):e77555. pmid:24204868
- 6. Biddle L, Derges J, Mars B, Heron J, Donovan JL, Potokar J, et al. Suicide and the Internet: Changes in the accessibility of suicide-related information between 2007 and 2014. Journal of Affective Disorders. 2016;190:370–5. pmid:26546772
- 7. Singaravelu V, Stewart A, Adams J, Simkin S, Hawton K. Information-seeking on the internet. Crisis. 2015;36(3):211–9. pmid:26088826
- 8. Durkee T, Hadlaczky G, Westerlund M, Carli V. Internet pathways in suicidality: a review of the evidence. International Journal of Environmental Research and Public Health. 2011;8(10):3938–52. pmid:22073021
- 9. Baker TG, Lewis SP. Responses to online photographs of non-suicidal self-injury: A thematic analysis. Archives of Suicide Research. 2013;17(3):223–35. pmid:23889572
- 10. Lewis SP, Heath NL, St Denis JM, Noble R. The scope of nonsuicidal self-injury on YouTube. Pediatrics. 2011;127(3):e552–e7. pmid:21339269
- 11. Lewis SP, Heath NL, Michal NJ, Duggan JM. Non-suicidal self-injury, youth, and the Internet: What mental health professionals need to know. Child and Adolescent Psychiatry and Mental Health. 2012;6(1):13. pmid:22463379
- 12.
Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions: John Wiley & Sons; 2011.
- 13.
Critical Appraisal Skills Programme UK. Critical Appraisal Skills Programme Making sense of evidence. 2012. Available: http://www.casp-uk.net accessed 2015 June 01.
- 14.
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme. Lancaster: Institute of Health Research; 2006
- 15. McDermott E, Roen K, Piela A. Hard-to-reach youth online: Methodological advances in self-harm research. Sexuality Research and Social Policy. 2013;10(2):125–34.
- 16. Whitlock JL, Powers JL, Eckenrode J. The virtual cutting edge: the internet and adolescent self-injury. Developmental Psychology. 2006;42(3):407–17. pmid:16756433
- 17. Franzen AG, Gottzén L. The beauty of blood? Self-injury and ambivalence in an Internet community. Journal of Youth Studies. 2011;14(3):279–94.
- 18. McDermott E. Asking for help online: Lesbian, gay, bisexual and trans youth, self-harm and articulating the ‘failed’ self. Health: 2015; pmid:25413341
- 19. Westerlund M. Talking suicide. Nordicom Review. 2013;34(2):35–46.
- 20. Duggan JM, Heath NL, Lewis SP, Baxter AL. An examination of the scope and nature of non-suicidal self-injury online activities: Implications for school mental health professionals. School Mental Health. 2012;4(1):56–67.
- 21. Zdanow C, Wright B. The representation of self injury and suicide on emo social networking groups. African Sociological Review/Revue Africaine de Sociologie. 2012;16(2):81–101.
- 22. Lewis SP, Heath NL, Sornberger MJ, Arbuthnott AE. Helpful or harmful? An examination of viewers' responses to nonsuicidal self-injury videos on YouTube. Journal of Adolescent Health. 2012;51(4):380–5. pmid:22999839
- 23. Grzanka PR, Mann ES. Queer youth suicide and the psychopolitics of “It Gets Better”. Sexualities. 2014;17(4):369–93.
- 24. Cash SJ, Thelwall M, Peck SN, Ferrell JZ, Bridge JA. Adolescent suicide statements on MySpace. Cyberpsychology, Behavior, and Social Networking. 2013;16(3):166–74.
- 25. Lewis SP, Baker TG. The possible risks of self-injury web sites: a content analysis. Archives of Suicide Research. 2011;15(4):390–6. pmid:22023646
- 26. Hagihara A, Miyazaki S, Abe T. Internet suicide searches and the incidence of suicide in young people in Japan. European archives of psychiatry and clinical neuroscience. 2012;262(1):39–46. pmid:21505949
- 27. Carew C, Kutcher S, Wei Y, McLuckie A. Using digital and social media metrics to develop mental health approaches for youth. Adolescent Psychiatry. 2014;4(2):116–21.
- 28. Casiano H, Kinley JD, Katz LY, Chartier MJ. Media use and health outcomes in adolescents: findings from a nationally representative survey. Journal of Canadian Academy of Child and Adolescent Psychiatry. 2012;21(4):296–301.
- 29. Carli V, Hoven CW, Wasserman C, Chiesa F, Guffanti G, Sarchiapone M, et al. A newly identified group of adolescents at “invisible” risk for psychopathology and suicidal behavior: findings from the SEYLE study. World Psychiatry. 2014;13(1):78–86. pmid:24497256
- 30. Katsumata Y, Matsumoto T, Kitani M, Takeshima T. Electronic media use and suicidal ideation in Japanese adolescents. Psychiatry and Clinical Neurosciences. 2008;62(6):744–6. pmid:19068014
- 31. Kim JY. The nonlinear association between Internet using time for non-educational purposes and adolescent health. Journal of Preventive Medicine and Public Health. 2012;45(1):37–46. pmid:22389757
- 32. Mitchell KJ, Ybarra ML. Online behavior of youth who engage in self-harm provides clues for preventive intervention. Preventive Medicine. 2007;45(5):392–6. pmid:17599399
- 33. O'Connor RC, Rasmussen S, Hawton K. Adolescent self-harm: a school-based study in Northern Ireland. Journal of Affective Disorders. 2014;159:46–52. pmid:24679388
- 34.
Collings S, Fortune S, Steers D, Currey N, Hawkton K, J W, et al. Media influences on suicidal behaviour: an interview study of young people in New Zealand. The National Centre of Mental Health Research Information and Workforce Development. 2011.
- 35. Dunlop SM, More E, Romer D. Where do youth learn about suicides on the Internet, and what influence does this have on suicidal ideation? Journal of Child Psychology and Psychiatry. 2011;52(10):1073–80. pmid:21658185
- 36. Kaess M, Durkee T, Brunner R, Carli V, Parzer P, Wasserman C, et al. Pathological Internet use among European adolescents: psychopathology and self-destructive behaviours. European Child and Adolescent Psychiatry. 2014;23(11):1093–102. pmid:24888750
- 37. Kim K, Ryu E, Chon M-Y, Yeun E-J, Choi S-Y, Seo J-S, et al. Internet addiction in Korean adolescents and its relation to depression and suicidal ideation: a questionnaire survey. International Journal of Nursing Studies. 2006;43(2):185–92. pmid:16427966
- 38. Lam LT, Peng Z, Mai J, Jing J. The association between internet addiction and self-injurious behaviour among adolescents. Injury prevention. 2009;15(6):403–8. pmid:19959733
- 39. Lin IH, Ko CH, Chang YP, Liu TL, Wang PW, Lin HC, et al. The association between suicidality and Internet addiction and activities in Taiwanese adolescents. Comprehensive psychiatry. 2014;55(3):504–10. pmid:24457034
- 40. Park S, Hong K-EM, Park EJ, Ha KS, Yoo HJ. The association between problematic internet use and depression, suicidal ideation and bipolar disorder symptoms in Korean adolescents. Australian and New Zealand journal of psychiatry. 2013;47(2):153–9. pmid:23047959
- 41. Aktepe E, Olgaç-Dündar N, Soyöz Ö, Sönmez Y. Possible internet addiction in high school students in the city center of Isparta and associated factors: a cross-sectional study. The Turkish Journal of Pediatrics. 2013;55(4):417–25. pmid:24292036
- 42. Messias E, Castro J, Saini A, Usman M, Peeples D. Sadness, suicide, and their association with video game and internet overuse among teens: results from the youth risk behavior survey 2007 and 2009. Suicide and Life-Threatening Behavior. 2011;41(3):307–15. pmid:21463355
- 43. Hetrick S, Yuen HP, Cox G, Bendall S, Yung A, Pirkis J, et al. Does cognitive behavioural therapy have a role in improving problem solving and coping in adolescents with suicidal ideation? The Cognitive Behaviour Therapist. 2014;7:1–15.
- 44. Hetrick SE, Dellosa MK, Simmons MB, Phillips L. Development and pilot testing of an online monitoring tool of depression symptoms and side effects for young people being treated for depression. Early Intervention in Psychiatry. 2015;9(1):66–9. pmid:24612591
- 45. Mar MY, Neilson EK, Torchalla I, Werker GR, Laing A, Krausz M. Exploring e-Mental Health Preferences of Generation Y. Journal of Technology in Human Services. 2014;32(4):312–27.
- 46. Saulsberry M, Marko-Holguin M, Blomeke K, Hinkle C. Randomized clinical trial of a primary care internet-based intervention to prevent adolescent depression: One-year outcomes. Journal of Canadian Academy of Child and Adolescent Psychiatry. 2013;22(2):106–17.
- 47. Barton AL, Hirsch JK, Lovejoy MC. Peer Response to Messages of Distress Do Sex and Content Matter? Crisis. 2013;34(3):183–91. pmid:23195459
- 48. Whitlock J, Pietrusza C, Purington A. Young adult respondent experiences of disclosing self-injury, suicide-related behavior, and psychological distress in a web-based survey. Archives of Suicide Research. 2013;17(1):20–32. pmid:23387400
- 49. Belfort E, Mezzacappa E, Ginnis K. Similarities and differences among adolescents who communicate suicidality to others via electronic versus other means: a pilot study. Adolescent Psychiatry. 2012;2(3):258–62.
- 50. Sueki H. The association of suicide-related Twitter use with suicidal behaviour: a cross-sectional study of young internet users in Japan. Journal of Affective Disorders. 2015;170:155–60. pmid:25240843
- 51. Baker D, Fortune S. Understanding self-harm and suicide websites: a qualitative interview study of young adult website users. Crisis. 2008;29(3):118–22. pmid:18714907
- 52. Barak A, Dolev-Cohen M. Does activity level in online support groups for distressed adolescents determine emotional relief. Counselling and Psychotherapy Research. 2006;6(3):186–90.
- 53. Jones R, Sharkey S, Ford T, Emmens T, Hewis E, Smithson J, et al. Online discussion forums for young people who self-harm: user views. The Psychiatrist Online. 2011;35(10):364–8.
- 54. Owens C, Sharkey S, Smithson J, Hewis E, Emmens T, Ford T, et al. Building an online community to promote communication and collaborative learning between health professionals and young people who self‐harm: an exploratory study. Health Expectations. 2012;18(1):81–94. pmid:23075133
- 55. Sharkey S, Smithson J, Hewis E, Jones R, Emmens T, Ford T, et al. Supportive interchanges and face-work as' protective talk'in an online self-harm support forum. Communication & medicine. 2012;9(1):71–82.
- 56. Smithson J, Sharkey S, Hewis E, Jones RB, Emmens T, Ford T, et al. Membership and boundary maintenance on an online self-harm forum. Qualitative Health Research. 2011;21(11):1567–75. pmid:21715606
- 57. Smithson J, Sharkey S, Hewis E, Jones R, Emmens T, Ford T, et al. Problem presentation and responses on an online forum for young people who self-harm. Discourse Studies. 2011;13(4):487–501.
- 58. Eichenberg C. Internet message boards for suicidal people: A typology of users. CyberPsychology & Behavior. 2008;11(1):107–13.
- 59. Sueki H, Eichenberg C. Suicide bulletin board systems comparison between Japan and Germany. Death Studies. 2012;36(6):565–80. pmid:24563935
- 60. Harris IM, Roberts LM. Exploring the use and effects of deliberate self-harm websites: an Internet-based study. Journal of Medical Internet Research. 2013;15(12):e285. pmid:24362563
- 61. Sternudd HT. Photographs of self-injury: Production and reception in a group of self-injurers. Journal of Youth Studies. 2012;15(4):421–36.
- 62. Castro TS, Osorio A. Online violence: Not beautiful enough… not thin enough. Anorectic testimonials in the web. PsychNology Journal. 2012;10(3):169–86.
- 63. Castro TS, Osorio AJ. "I love my bones!" Self-harm and dangerous eating youth behaviours in Portuguese written blogs. Young Consumers: Insight and Ideas for Responsible Marketers. 2013;14(4):321–30.
- 64. Padmanathan P, Carroll R, Biddle L, Derges J, Potokar J, Gunnell D. Suicide and self-harm related internet use in patients presenting to hospital with self-harm: a cross-sectional study. The Lancet. 2016;388, Supplement 2:S2.
- 65. Ebert D, Donkin L, Andersson G, Andrews G, Berger T, Carlbring P, et al. Does Internet-based guided-self-help for depression cause harm? An individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychological Medicine. 2016;46(13):2679. pmid:27649340
- 66. Mok K, Jorm AF, Pirkis J. The perceived impact of suicide-related internet use: A survey of young Australians who have gone online for suicide-related reasons. Digital Health. 2016;2:
- 67. Lloyd J, Hawton K, Dutton WH, Geddes JR, Goodwin GM, Rogers RD. Thoughts and acts of self-harm, and suicidal ideation, in online gamblers. International Gambling Studies. 2016;16(3):408–23.
- 68. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? British Medical Journal. 2001;322(7294):1115–7. pmid:11337448
- 69.
Green J, Thorogood N. Qualitative methods for health research: Sage; 2013.
- 70. Pirkis J, Neal L, Dare A, Blood RW, Studdert D. Legal Bans on Pro‐Suicide Web Sites: An Early Retrospective from Australia. Suicide and Life-Threatening Behavior. 2009;39(2):190–3. pmid:19527159
- 71. Dyson MP, Hartling L, Shulhan J, Chisholm A, Milne A, Sundar P, et al. A systematic review of social media use to discuss and view deliberate self-harm acts. PloS one. 2016;11(5):e0155813. pmid:27191728
- 72. Minkkinen J, Oksanen A, Kaakinen M, Keipi T, Räsänen P. Victimization and Exposure to Pro-Self-Harm and Pro-Suicide Websites: A Cross-National Study. Suicide and Life-Threatening Behavior. 2017;47(1):14–26. pmid:27094352