There has been no study that has allowed clear conclusions about the impact of suicide-related or mental health consultation-related internet use.
To investigate the impacts of suicide-related or mental health consultation-related internet use.
We conducted prospective observational longitudinal study with data collection at baseline screening (T0), 1 week after T0 (T1) and 7 weeks after T0 (T2). Participants with a stratified random sampling from 744,806 internet users were 20–49 years of age who employed the internet for suicide-related or mental health consultation-related reasons and internet users who did not. The main outcome was suicidal ideation. Secondary outcome measures comprised hopelessness, depression/anxiety, and loneliness.
The internet users who had employed the internet for suicide-related or mental health consultation-related reasons at T0 (n = 2813), compared with those who had not (n = 2682), showed a significant increase in suicidal ideation (β = 0.38, 95%CI: 0.20–0.55) and depression/anxiety (β = 0.37, 95%CI: 0.12–0.61) from T1 to T2. Those who disclosed their own suicidal ideation and browsed for information about suicide methods on the web showed increased suicidal ideation (β = 0.55, 95%CI: 0.23–0.88; β = 0.45, 95% CI: 0.26–0.63, respectively). Although mental health consultation with an anonymous other online did not increase suicidal ideation, increased depression/anxiety was observed (β = 0.34, 95%CI: −0.03–0.71).
Citation: Sueki H, Yonemoto N, Takeshima T, Inagaki M (2014) The Impact of Suicidality-Related Internet Use: A Prospective Large Cohort Study with Young and Middle-Aged Internet Users. PLoS ONE 9(4): e94841. https://doi.org/10.1371/journal.pone.0094841
Editor: Mitsuhiko Yamada, National Institute of Mental Health, Japan
Received: November 27, 2013; Accepted: March 20, 2014; Published: April 16, 2014
Copyright: © 2014 Sueki et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by Japan Internet Safety Promotion Association (http://good-net.jp) and the Japanese Ministry of Internal Affairs and Communications, and Grant-in-Aid for Scientific Research (KAKENHI) (C) in Japan. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Suicide is a critical global issue with a global mortality rate of 16 per 100,000 . More specifically, suicide rates in young people have risen . Suicide is among the top 20 leading causes of death globally for all ages and among the three leading causes of death among those aged 15–44 years in some countries . Previous studies have shown that many suicidal people do not seek help and treatment . Reasons for not seeking help have been reported as stigma and temporal/financial constraints  . As a consequence, the internet may be useful for providing information and help for those who are suicidal, especially young and middle-aged persons, because it is anonymous, low cost, and easy to use . Previous studies have discussed how the internet has both suicide-preventive and suicide-inducing effects –. Information about suicide methods was possibly categorized by expert consensus as pro-suicidal  , and provision of consulting about mental health (e.g., email-based crisis intervention) as anti-suicidal –.
However, there has been no large-sized prospective cohort study or randomized controlled trial that has allowed any causal conclusions about the impact of suicidality-related internet use. Cross-sectional studies have reported an association of internet usage with suicidal ideation  . In addition, a previous study reported the association of suicide information availability on the internet with suicide methods used among those who died by suicide, based on inquests . Previous studies did not reach a clear conclusion about the effects of suicidality-related internet use .
Therefore, we investigated whether suicidality-related internet use (disclosing one's suicidal ideation, mental health consultation, and browsing for information about suicide methods) is related to changes in suicidal ideation and other mental health status items related to suicide (hopelessness, depression/anxiety, and loneliness) in a prospective observational longitudinal study. The hypothesis tested was that suicidality-related internet use affects the users' suicidal ideation and the other mental health scores.
A survey was made of internet users who were between 20 and 49 years of age with Japanese literacy. Minors were excluded from our research target groups for ethical reasons. We did not include those in their 50 s or older because of the finding that this group is seldom included in surveys of suicidality-related internet users .
At the screening survey, we excluded individuals who had planned or attempted suicide within the past month to avoid any encouragement of suicidal behaviours. In addition, participants with incomplete or untrustworthy answers (e.g., answers including incomprehensible character strings in open-response questions) were also excluded because these answers would not be reliable.
The study was a prospective observational longitudinal study. The baseline screening survey (T0 survey) and two waves (T1 [1 week after T0] and T2 [7 weeks after T0]) of follow-up surveys were conducted with members of comprehensive internet survey panels through a major Japanese internet survey company (Cross Marketing Inc., Tokyo, Japan) (see Figure 1). The T0 survey was based on a target population of those from 20 to 49 years of age distributed according to the demographics of the census data of 2005 in Japan , with stratified random sampling of 744,806 internet panel participants (about 20,000 internet panel participants in each of the groups of ages 20–29, 30–39, and 40–49 years). The stratified variables were age, sex and geographic region of residence. The sample size chosen was based on an expected response rate of 10% or less . This sample size would have over 90% power to detect an expected regression coefficient of 0.3 in the multivariate regression analysis.
At the T0 survey, we asked the following four questions and defined the participants who endorsed any of them as the group with suicide-related or mental health consultation-related internet use: “Q1: Over the past month, have you disclosed your wish to commit a suicide to an anonymous other on the Internet?”, “Q2: Over the past month, have you consulted with an anonymous other about your mental health on the Internet?”, “Q3: Over the past month, have you browsed for information concerning suicide methods on the Internet?”, and “Q4: Have you ever disclosed your wish to commit suicide to an anonymous other on the Internet?”.
In the T1 survey, a questionnaire was given to the participants who answered “Yes” to any of the questions Q1, 2, 3 and 4 (the group with suicide-related or mental health consultation-related internet use) at the T0 survey. Also, a random sample was taken of the participants who answered “No” to all four questions at the T0 survey and defined as the control group. In the T2 survey, a questionnaire was distributed to all participants who completed the T1 survey.
The study was approved by the ethical review board at The University of Tokyo, Japan (Registration number: 10–24, http://www.u-tokyo.ac.jp/ja/administration/lifescience/), and complied with the ethical guidelines for epidemiology research by the Ministry of Health, Labour and Welfare, Japan. We briefed survey participants on the possibility that viewing or responding to the questionnaire might lead to a mood change before they consented to participate in the study, and then obtained informed consent from participants online. In addition, links to websites containing professional support resources were shown to participants occasionally during the time they were completing the questionnaire.
All participants answered a self-administered questionnaire on the internet containing questions about suicide-related or mental health consultation-related internet use, suicidal ideation, hopelessness, depression/anxiety, loneliness, and coping with stress (see Table 1).
At the T0 survey, Yes-No questions were asked about experience with suicide-related or mental health consultation-related internet use: disclosing one's suicidal ideation within a month (Q1), mental health consultation within a month (Q2), browsing for information about suicide methods within a month (Q3), and lifetime experience of disclosing one's suicidal ideation (Q4). All items were made by reference to our previous related study about suicide-related internet use .
The Scale for Suicide Ideation designed by Beck and his colleagues  was used to measure suicidal ideation at T1 and T2. This scale is a 19-item clinical research instrument designed to quantify and assess suicidal intention. A score of 0–2 is given to each response, with higher points representing increased levels of suicidal ideation. It should be noted that the original scale was designed to be given by a trained administrator, while the version used in this study (Japanese version) is a self-rating scale with questions modified in light of the Japanese social-cultural environment (e.g., the item about gun-related suicide was eliminated). The Japanese version of this scale, consisting of 13 items (score range = 0–26), was confirmed for reliability (Cronbach's α = 0.85) and validity through a survey with 344 college students .
The Beck Hopelessness Scale was used to measure hopelessness at T1 and T2 .This scale is a 20-item true-false self-report instrument that assesses the degree to which a person holds negative expectations about the future. The items are summed to obtain a total hopelessness score (range = 0–20). The Japanese version of this scale showed a high degree of reliability (Kuder–Richardson Formula 20 = 0.86) and validity through a survey with 160 college students .
The K6 (six items) was used to measure depression/anxiety tendency at T1 and T2 . The K6 is an abridged version of the Kessler Psychological Distress Scale (K10), a scale based on the item response theory for effectively detecting mental disorders. For each item, responses were rated on a 5-point scale ranging from 1 point for “Not at all” to 5 points for “Always.” Scores could vary from 6 to 30. The Japanese version was developed and has been shown to be equal in screening performance to the original .
Loneliness was measured by Ochiai's Loneliness Scale (nine items) at T1 and T2 . Responses to questions (e.g., “I don't think there is anyone sympathetic to whom I can turn to for advice”) were rated using a 5-point scale ranging from 5 points for “Yes” to 0 points for “No.” The validity of this scale has been confirmed on the basis of its association with the Revised UCLA Loneliness Scale . In addition, it was confirmed to have a high reliability (r: correlation coefficient, 0.66<r<0.83) according to the test-retest method at intervals of 1 month and 6 months.
The Coping Scale (14-item) was used to measure stress coping at T1 . This scale was designed to measure an individual's coping ability with the most important stressor that he/she is experiencing at a given moment in the simplest manner possible. It consists of three subscales: five items for problem-focused type coping (item example: making an effort to change the present situation) and three items for emotion-focused type coping (item example: encouraging oneself), both as positive coping, as well as six items for avoidance/escape type coping, as passive coping (item example: trying not to think of the future). A question was asked for each item, and the response to the question was rated by using a 3-point scale from 0 points for “I don't do that at all” to 3 points for “I always do that.” The reliability and validity of this scale was confirmed through a survey of over 599 college students. Cronbach's α values for the problem-focused, emotion-focused, and avoidance/escape subscales were 0.75, 0.76, and 0.72, respectively.
We analysed the data for the impact of suicide-related or mental health consultation-related internet use from those who responded and gave complete answers in the surveys. The descriptive statistics show the differences in characteristics of participants between the group with suicide-related or mental health consultation-related internet use and the control group. We checked the impacts of non-responders and missing cases for the analysis. In a comparison of the characteristics between participants with and without suicide-related or mental health consultation-related internet use experience, the t-test was employed for continuous data and the chi-square test was used for binary data.
For our primary analyses to examine the relationships of suicide-related or mental health consultation-related internet use with users' suicidal ideation and the other mental health scores, multivariate regression models were performed. The aim was to investigate whether those with suicide-related or mental health consultation-related internet usage showed changes in scores for suicidal ideation, hopelessness, depression/anxiety, and loneliness from T1 to T2, adjusting for the participants' characteristics, coping skills, and other mental health scores at T1. This is because these mental health scores, demographics and coping skills have been reported previously to have an influence on both suicidal ideation and internet usage  .
In this analysis, there were two types of independent variables of suicide-related or mental health consultation-related internet use. First, we analysed the models where the independent variable was whether or not participants endorsed any of the four questions about suicide-related or mental health consultation-related internet use. Second, the models for each of the four types of suicide-related or mental health consultation-related internet use (disclosing one's suicidal ideation within a month, mental health consultation, browsing for information about suicide methods, and disclosing one's suicidal ideation up until a month ago) were analysed separately with each suicide-related or mental health consultation-related internet use as the independent variable in the model. We examined three versions of the model: Model 1 (minimally adjusted) controlling for T1 mental health scores (suicidal ideation, hopelessness, depression/anxiety, and loneliness); Model 2 controlling for T1 mental health scores and the participants' characteristics (educational background, marital status, household income, drinking alcohol, smoking, psychiatric hospital visit, and time spent online per day); and Model 3 (fully adjusted) controlling for T1 mental health scores, the participants' characteristics, and coping skills (problem-focused, emotion-focused and avoidance/escape). In these three models, we added potential confounders in the order corresponding to the strength of the relationship to suicidal ideation, because robustness of the results needed to be checked.
There were strong observed confounders between the two groups. Therefore, we adjusted for these confounders. The methods for control of confounding are of various types, including restriction, matching, stratification and regression modeling. Multivariate regression modelling is more flexible and can fully adjust strong observed confounders for targeted populations with high validity. Other methods like matching have some limitations from a modern epidemiological perspective. In the other methods, the numbers of adjustment factors are limited. Second, these methods carry risks of over or under adjustments. Third, adjusted results using these methods would not be representative of the targeted population. Only regression modeling can overcome these limitations.
In the regression model, we selected confounding factors in the model on the basis of prior knowledge of risk factors, because the confounders could not be identified in the observed data . However, the knowledge from previous studies was limited. Therefore, we performed three models with potential confounders to check the sensitivity of the confounders, step-by step. Model 3 (the full model) allowed adjustment for all observed confounders.
Regression coefficients (β) and their 95% confidence intervals (CI) were calculated in the models. The p-values presented are for two-tailed tests. The analysis was performed using SPSS software (SPSS 19.0 for Windows; SPSS Inc., Chicago, IL). The present study is in accordance with the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement.
Characteristics of the participants
The primary number of analysed participants was 5495: 43.3% were women and the mean age was 35.5 years (standard deviation = 7.8, range = 20–49); 2813 participants were the group with suicide-related or mental health consultation-related internet use and 2682 were the control group (see Figure 1). No participants were excluded because of their suicidality. There were no missing values for any variable in the final data set.
Characteristics of the participants are shown in Table 2. Significant differences in the proportions of age, marital status, household income, drinking alcohol, smoking, and hospital visit were found between the group with suicide-related or mental health consultation-related internet use and the control group at the T0 survey. There were no differences of gender and education. Compared with the control group at T0, the group with suicide-related or mental health consultation-related internet use were likely to have lower scores for coping with stress and higher scores for mental health problems at T1. The proportion of lifetime suicidal behaviours of the group with suicide-related or mental health consultation-related internet use was significantly greater than that of the control group at T1.
Drinking alcohol, smoking and hospital visit had some differences between dropouts and completed responders in the group with suicide-related or mental health consultation-related internet use. However, in the control group, there were no differences in drinking alcohol, smoking and hospital visit.
Relationships between suicide-related or mental health consultation-related internet use and mental health score changes from T1 to T2
As seen in Table 3, the regression models show the relationships between suicide-related or mental health consultation-related internet use and changes of suicidal ideation, hopelessness, depression/anxiety, and loneliness from T1 to T2. Statistically significant positive coefficients were found for suicidal ideation (β = 0.38 [95%CI: 0.20 to 0.55]) and depression/anxiety (β = 0.37 [95%CI: 0.12 to 0.61]). The scores for hopelessness and loneliness did not show any effects of suicide-related or mental health consultation-related internet use.
Table 4 reports the regression coefficients of each suicide-related or mental health consultation-related internet use. Disclosing one's suicidal ideation up until a month ago (β = 0.37, 95%CI: 0.17 to 0.57), disclosing one's suicidal ideation within the last month (β = 0.55, 95%CI: 0.23 to 0.88), and browsing for information about suicide methods within a month (β = 0.45, 95%CI: 0.26 to 0.63) significantly increased suicidal ideation from T1 to T2. Internet mental health consultation with an anonymous other within the last month did not increase suicidal ideation. We obtained a similar result in the other three models for suicidal ideation.
Those who disclosed their suicidal ideation up until a month ago showed increased depression/anxiety (β = 0.30, 95%CI: 0.02 to 0.58) from T1 to T2. Those with mental health consultation within a month (β = 0.34, 95%CI: −0.03 to 0.71) and browsing for information about suicide methods within a month (β = 0.26, 95%CI: −0.01 to 0.53) showed increased depression/anxiety also. The scores for hopelessness and loneliness did not show any effects of suicide-related or mental health consultation-related internet use.
We described the differences between primary analysed cases and non-responders and missing cases. There were few differences in background of the participants. However, the main results of our analysis were still robust. (data was not shown).
Disclosing one's suicidal ideation and browsing for information about suicide methods increased suicidal ideation. Mental health consultation with anonymous others by internet users did not increase suicidal ideation but increased depression/anxiety. Our large-sized prospective longitudinal study confirmed the effects of suicide-related internet use on suicidality. Those with suicide-related or mental health consultation-related internet use showed increased suicidal ideation and depression/anxiety, and the three types of suicide-related internet use were each independently related to these same changes in mental health scores. As in previous studies  , our results extended and strengthened the causal link between browsing for information about suicide methods online and increased suicidal ideation. Those who disclosed suicidal ideation, which was considered to be suicide-preventive in some previous studies  , showed increased suicidal ideation. Those who had mental health consultation with an anonymous other showed no effect on suicidal ideation, but had increased depression/anxiety. We showed a negative effect of disclosing one's suicidal ideation online, which dovetailed with the indication that there was the potential harm of a downward depressive spiral in online anonymous communities consisting of users with depressive tendencies .
The effect size of suicide-related or mental health consultation-related internet use might be interpreted as relatively small. Each suicide-related or mental health consultation-related internet use increased suicidal ideation scores only by 0.37–0.55 points during a 6-week observed period. The observation was over a relatively short period. Also, as we have presented in the backgrounds of the group with suicide-related or mental health consultation-related internet use, they have a high probability of being diagnosed with mental disorders associated with suicide, as indicated by the mean scores of K6, the screening scale for detecting CIDI/DSM-IV mood and anxiety disorders. A previous study showed that increasing suicidal ideation was significantly associated with a diagnosis of a principal mood disorder, a diagnosis of a personality disorder and previous suicide attempts . Under such circumstances, those with suicide-related or mental health consultation-related internet use may show more serious consequences such as self-harm, repeated suicidal behaviours and completed suicide in the long term.
Strengths and limitations
Our internet survey was the first large-scale prospective longitudinal study with a control group chosen by stratified random sampling. There is almost no prospective research in the related area . Most of the previous studies dealing with the effects of suicide-related internet use were based on internet suicide pacts or used a cross-sectional study design   . The sampling process in this study was based on the demographic composition of the latest census to minimize sampling and representative bias. In addition, our research clarified the level of suicidal risk of the group with suicide-related or mental health consultation-related internet use, which has not been previously examined.
There were, however, some limitations to our study. First, an online panel survey has some biases such as coverage bias . Second, the T0 participation rate in the survey was relatively low (14.5%). However, the proportion of participation was nearly equal to that of other internet research  and the population characteristics of this study were also similar to the latest census data. Third, we showed only short-term results. The long-term effects of suicide-related or mental health consultation-related internet usage are not clear. Fourth, we did not examine the effects of professional mental health services online. This is because there are not yet such validated services in Japan. Fifth, we did not define the contents of the websites or the services rendered in consequence of disclosing suicidal ideation or in seeking mental health consultation with anonymous others. Finally, it is impossible to know which websites participants actually visited and whether the websites they visited were topically related to suicide or not from our research. As an alternative, we asked about the actions that suicidal internet users frequently perform instead of identifying the websites that the participants actually visited in the questionnaire. Because a website may have a function of both prevention and promotion of suicide , it is not possible to verify whether the internet use of such a specific website enhances suicide prevention or not. It is useful for us to identify which actions may predict increases or decreases in the internet user's suicidality in order to utilize online helping resources. Therefore the four questions about suicide-related internet usage were made by reference to our previous study about the internet and suicide. However, these items were not used frequently and did not cover all the actions performed on websites that might be suicide-related.
Increased suicidal ideation was observed in the young and middle-aged with suicide-related or mental health consultation-related internet use. Those who disclosed their suicidal ideation up until a month ago, disclosed their suicidal ideation within the previous month, and browsed for information about suicide methods within the previous month had significantly increased suicidal ideation 7 to 8 weeks later. Meanwhile, mental health consultation via the internet was not useful but may have worsened depression/anxiety. Therefore, regulation of information regarding suicide on the internet should be promoted as a health policy. As a next challenge, new and more effective interventions for vulnerable populations with suicide-related internet use should be developed.
Conceived and designed the experiments: HS NY TT MI. Analyzed the data: HS NY MI. Wrote the paper: HS NY MI.
- 1. World Health Organization (2013) Suicide prevention (SUPRE). Available: http://www.who.int/mental_health/prevention/suicide/suicideprevent/en/2013. Accessed 2013 Jun 17.
- 2. Hawton K, Saunders KEA (2009) Psychiatric service development and suicide. Lancet 373: 99–100.
- 3. Barnes LS, Ikeda RM, Kresnow M (2001) Help-Seeking behavior prior to nearly lethal suicide attempts. Suicide Life Threat Behav 32: 68–75.
- 4. Deane FP, Wilson CJ, Ciarrochi J (2001) Suicidal ideation and help-negation: not just hopelessness or prior help. J Clin Psychol 57: 901–914.
- 5. Pagura J, Fotti S, Katz LY, Sareen J (2009) Help seeking and perceived need for mental health care among individuals in Canada with suicidal behaviors. Psychiatric Services 60: 943–949.
- 6. Gilat I, Shahar G (2007) Emotional first aid for a suicide crisis: comparison between Telephonic hotline and internet. Psychiatry 70: 12–18.
- 7. Biddle L, Donovan J, Hawton K, Kapur N, Gunnell D (2008) Suicide and the Internet. BMJ 336: 800–802.
- 8. Luxton DD, June JD, Fairall JM (2012) Social media and suicide: a public health perspective. Am J Public Health 102: 195–200.
- 9. Sisask M, Varnik A (2012) Media roles in suicide prevention: a systematic review. Int J Environ Res Public Health 9: 123–138.
- 10. Westerlund M, Hadlaczky G, Wasserman D (2012) The representation of suicide on the Internet: implications for clinicians. J Med Internet Res 14: : e122. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3510719/. Accessed 2014 Feb 10.
- 11. Alao AO, Yolles JC, Armenta WR (1999) Cybersuicide: the Internet and suicide. Am J Psychiatry 156: 1836–1837.
- 12. Rajagopal S (2004) Suicide pacts and the internet. BMJ 329: 1298–1299.
- 13. Grohol JM (2008) Suicide and the Internet: study misses internet's greater collection of support websites. BMJ 336: 905.
- 14. Mehlum L (2000) The Internet, suicide, and suicide prevention. Crisis 21: 186–188.
- 15. Thoër C (2013) Expressing, communicating and discussing suicide: nature, effects and methods of interacting through online discussion platform. In: Mishara B, Kerkhof A, editors. Suicide prevention and new technologies: evidence based practice. Palgrave macmillan. pp. 24–38.
- 16. Eichenberg C (2008) Internet message boards for suicidal people: a typology of users. Cyberpsycholo Behav 11: 107–113.
- 17. Harris KM, McLean JP, Sheffield J (2009) Examining suicide-risk individuals who go online for suicide-related purposes. Arch Suicide Res 13: 264–276.
- 18. Gunnell D, Bennewith O, Kapur N, Simkin S, Cooper J, et al. (2012) The use of the Internet by people who die by suicide in England: a cross sectional study. J Affect Disord 141: 480–483.
- 19. Daine K, Hawton K, Singaravelu V, Stewart A, Simkin S, et al. (2013) The power of the Web: a systematic review of studies of the influence of the Internet on self-harm and suicide in young people. PLoS ONE 8: e77555.
- 20. Japanese Ministry of Internal Affairs and Communications (2008) Census in 2005. Available: http://www.stat.go.jp/data/kokusei/2005/. Accessed 2013 Jun 17.
- 21. American Association for Public Opinion Research. AAPOR Releases report on online survey panels. Available: http://www.aapor.org/AAPOR_Releases_Report_on_Online_Survey_Panels1.htm. Accessed 2013 Jun 17.
- 22. Sueki H (2013) The effect of suicide-related Internet use on users' mental health: a longitudinal study. Crisis 34: 348–353.
- 23. Beck AT, Kovacs M, Weissman A (1979) Assessment of suicidal intention: the scale for suicide ideation. J Consult Clin Psychol 47: 343–352.
- 24. Otsuka A, Seto M, Kanno J, Agari I (1998) Development of the suicide ideation scale for Japanese and a study of the factors related to suicide ideation. Jap J Counsel Sci 31: 247–258.
- 25. Beck AT, Weissman A, Lester D, Trexler L (1974) The measurement of pessimism: the hopelessness scale. J Consult Clin Psychol 42: 861–865.
- 26. Tanaka E, Sakamoto S, Ono Y, Fujihara S, Kitamura T (1996) Hopelessness in a community population in Japan. J Clin Psychol 52: 609–615.
- 27. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, et al. (2002) Short screening scales to monitor population prevalences and trends in nonspecific psychological distress. Psychol Med 32: 959–976.
- 28. Furukawa TA, Kawakami N, Saitoh M, Ono Y, Nakane Y, et al. (2008) The performance of the Japanese version of the K6 and K10 in the World Mental Health Survey Japan. Int J Methods Psychiatr Res 17: 152–158.
- 29. Ochiai Y (1983) Development of loliness scale by Ochiai. Jap J Educ Psychol 31: 332–336.
- 30. Russell D, Peplau LA, Cutrond CE (1980) The revised UCLA loneliness scale. J Pers Soc Psychol 39: 472–480.
- 31. Ozeki Y (1993) Revised stress self-rating scale for university student: for transactional analysis. Bulletin of the Graduate School of Comparative Culture, Kurume University 1: 95–114.
- 32. Gould MS, Greenberg T, Velting DM, Shaffer D (2003) Youth suicide risk and preventive interventions: a review of the past 10 years. J Am Acad Child Adolesc Psychiatry 42: 386–405.
- 33. Rothman KJ, Greenland S, Lash TL (Eds) (2008) Modern epidemiology. Lippincott Williams & Wilkins.
- 34. Takahashi Y, Uchida C, Miyaki K, Sakai M, Shimbo T, et al.. (2009) Potential benefits and harms of a peer support social network service on the Internet for people with depressive tendencies: qualitative content analysis and social network analysis. J Med Internet Res 11: : e29. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2762850/. Accessed 2013 Jun 17.
- 35. Beck AT, Brown GK, Steer RA (1997) Psychometric characteristics of the Scale for Suicide Ideation with psychiatric outpatients. Behav Res Ther 35: 1039–1046.
- 36. Carli V, Durkee T, Wasserman D, Hadlaczky G, Despalins R, et al. (2012) The association between pathological Internet use and comorbid psychopathology: a systematic review. Psychopathol 46: 1–13.
- 37. Sueki H, Eichenberg C (2012) Suicide bulletin board systems comparison between Japan and Germany. Death Stud 36: 565–580.