This study had two main goals: to examine the structure of co-occurring peer bullying experiences among adolescents in South Korea from the perspective of victims and to determine the effects of bullying on suicidal behavior, including suicidal ideation and suicide attempts, among adolescents.
This study used data gathered from 4,410 treatment-seeking adolescents at their initial visits to 31 local mental health centers in Gyeonggi Province, South Korea. The structure of peer bullying was examined using latent class analysis (LCA) to classify participants’ relevant experiences. Then, a binomial logistic regression adjusted by propensity scores was conducted to identify relationships between experiences of being bullied and suicidal behaviors.
The LCA of experiences with bullying revealed two distinct classes of bullying: physical and non-physical. Adolescents who experienced physical bullying were 3.05 times more likely to attempt suicide than those who were not bullied. Victims of (non-physical) cyber bullying were 2.94 times more likely to attempt suicide than were those who were not bullied.
Citation: Roh B-R, Yoon Y, Kwon A, Oh S, Lee SI, Ha K, et al. (2015) The Structure of Co-Occurring Bullying Experiences and Associations with Suicidal Behaviors in Korean Adolescents. PLoS ONE 10(11): e0143517. https://doi.org/10.1371/journal.pone.0143517
Editor: Ulrich S. Tran, University of Vienna, School of Psychology, AUSTRIA
Received: April 13, 2015; Accepted: November 5, 2015; Published: November 30, 2015
Copyright: © 2015 Roh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: This study used data from 31 mental health centers of Gyeonggi province in Korea. Due to ethical restrictions and patient confidentiality, this data cannot be made freely publicly accessible. However, persons may request access to this data for research purposes, by contacting Dr. Hyun Ju Hong (email: firstname.lastname@example.org).
Funding: The Hallym University Research Fund (2013) www.hallym.ac.kr. Hallym University (HJH). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exists.
School bullying is a worldwide problem among adolescents, affecting the academic achievement, social skills, and psychological well-being of targeted individuals. Although school bullying is not a well-defined term in academic literatures, previous studies on bullying has suggested measurements, prevalence and characteristics differed by gender, age, and the nations.
Olweus identified three core elements of bullying: 1) aggressive behavior that involves unwanted negative actions; 2) behaviors that are repeated over time; and 3) an imbalance of power or strength. Adolescents may be involved in bullying as victims, perpetrators, or victim-perpetrators. Bullying can be categorized with respect to various expressions of aggression as direct (overt) bullying and indirect (covert, relational) bullying. Direct bullying includes physical and verbal aggression. Indirect bullying includes aspects of relational or social isolation such as ignoring, excluding, and backbiting. Direct bullying is more frequently observed among boys, whereas indirect bullying is more frequent among girls [2–5].
Being the victim of school bullying has been associated with multiple negative mental and physical health consequences[3,4], and repeated experiences of school bullying are directly or indirectly connected to depressive tendencies and suicidal behavior[6,7]. One study found that students who have experienced school bullying are twice as likely to report suicidal ideation and 40% more likely to suffer from depressed thoughts compared with those who have not had these experiences. Similarly, in research among suburban populations in the US, bullying was found to be associated with suicidal ideation and suicide attempts [9–11]. Kim and Leventhal ] review of 37 studies found a consistent association between being bullied and suicidal thoughts among youth. Both perpetrators and victims of bullying are at higher risk for depression, suicidal ideation, and suicide attempts than are adolescents not involved in bullying [5,13,14].
Most studies pertaining to bullying and suicide among adolescents have investigated the various effects of bullying among victims, perpetrators, and victim-perpetrators. Many studies have focused on combinations of various types of bullying using standardized questionnaires, but have not dealt with the effects of these bullying types in victim’s perspective on suicide. In the few studies that have investigated the effects of different types of bullying on suicide [5,15–17], bullying was classified differently depending on the measurement instrument. In reality, victims of bullying are more likely to experience various forms of bullying simultaneously than to experience a single form at a time. Until now, no studies have examined the patterns of co-occurring peer bullying experiences from the perspective of individual victims. Such research is important for the following two reasons. First, understanding bullying from the vantage point of individual victims will result in more realistic definitions and conceptualizations. Second, information regarding the effects of specific bullying types on mental health can lead to more appropriate interventions.
This study was designed to meet two goals. First, we sought to examine the structure of peer bullying in school environment in South Korea using a relatively large sample of adolescents who visit to local mental health center for further treatment or evaluation. To this end, we used latent class analysis (LCA) to analyze the structures of bullying from the victims’ perspective and to categorize experiences of being bullied. Second, we attempted to identify the effects of these experiences on suicidal behaviors, including suicidal ideation and suicide attempts, by controlling for potential confounding factors. We hypothesized that several distinctive bullying victimization types would be identified. For example, overt bullying is more physical, and covert bullying is more social and non-physical. We hypothesized that each type would exert a distinct effect on suicidal ideation and attempts among adolescents.
This study was conducted through the Gyeonggi provincial mental health center at 31 local mental health centers in collaboration with the Gyeonggi Provincial Office of Education. Gyeonggi Province is 10,172.07 km2 in size and comprises industrial, urban, and rural areas; the province had a population of 12,789,445 in 2014. The study used data obtained during the initial assessments of adolescents who visited 31 local mental health centers in Gyeonggi province, Korea, in 2012. About three-fourths of the sample was referred by schools through the Children and Adolescents’ Mental Health Promotion Project (CAMHP); the remainder were self-referred or referred by a source in the community. The first step of CAMHP was to screen for middle and high school students at high risk for emotional and behavioral problems using a screening questionnaire. Individual assessment and treatment were subsequently provided at local mental health centers or psychiatric clinics. When individuals visited local mental health center, they were asked to complete several questionnaires and participate in an interview conducted by a mental health professional. A total of 4,410 participants provided data (age range: 12–19 years). The protocol for this study was approved by the Institutional Review Board of Hallym University Sacred Heart Hospital. Written informed consent for participation in the study was obtained from parents and the adolescents after the nature of the study was fully explained.
We asked adolescents and their parents to complete questionnaires requesting demographic information including sex, age, parents’ education, family structure, and socioeconomic status according to social welfare status. Family structure was categorized as follows: two parents, single parent, and other. Social welfare status was categorized as follows: families living below the national poverty line receiving public assistance and others.
Participants completed the self-administered questionnaire about their experiences of being the victims of various types of bullying during the past 12 months, providing yes/no responses to the questions. Participants were asked whether they had been the targets of bullying in the form of “physical bullying (hitting), verbal bullying (threatening or intimidation), extortion (having their belongings taken), ostracism by a group, harassment (being forced to do work), sexual bullying, and cyber bullying.”
These bullying types were chosen by our research team in consideration of the Korean situation. These seven types of bullying experiences were thought to be the most frequent forms of bullying in Korean schools. Ostracism, a common form of bullying in Korea, is a kind of social bullying in which victims are excluded from all kinds of social relationships and treated as if they were invisible, typically by peer groups rather than by an individual. Participants were permitted to check multiple items if appropriate.
Suicidal ideation and behavior were assessed with self-reports. Participants who provided affirmative responses to the statement “I have said that I wanted to die during the past 3 months” were classified as having suicidal ideation. Those providing affirmative answers to “I have attempted suicide during the past 3 months” were classified as having made a suicidal attempt.
We assessed emotional and behavioral problems using the Strengths and Difficulties Questionnaire (SDQ). The SDQ is designed to screen for mental health problems among children and adolescents and is used in both community and clinical settings throughout the world (http://www.sdqinfo.org). This tool contains 25 items, 20 of which are related to problem areas such as peer relationships, hyperactivity and inattention, conduct disorders, and emotional symptoms. The remaining five items comprise a prosocial behavior scale. We used these five clinical variables to calculate the propensity scores for suicidal behaviors. The psychometric properties of the Korean version of the SDQ (SDQ-Kr) for parents have been demonstrated to be acceptable compared with other translated versions. The self-rated SDQ-Kr is currently undergoing standardization, and we used the preliminarily translated form of the self-rated SDQ-Kr. With regard to internal consistency, we calculated the Cronbach’s α of five scales: peer problems, 0.64; hyperactivity/inattention, 0.76; conduct disorder, 0.47; emotional symptoms, 0.77; and prosocial behaviors, 0.62. When comparing to western countries, the alpha for hyperactivity/inattention, emotional symptoms and prosocial behaviors subscales is similar. On the other hand, peer problems subscale is higher, and conduct problems subscale is lower [19,20]. However, all α coefficients appears to be higher than China..
We used LCA to examine co-occurrences among various types of bullying experience and to understand the structure of bullying using the poLCA package in R. LCA identifies classes of individuals with similar experience patterns as measured by dichotomous variables. The latent class model with the best explanatory power is selected according to the parsimony of the model. For this purpose, we determined the best-fitting model by comparing the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) among models. Lower values for both criteria represent a better-fitting model, taking parsimony into account. The meaning of the latent classes is interpreted based on the estimated item-response probabilities. To identify demographic and clinical differences among these classes, we used ANOVA and a chi-squared test.
To examine the relationship between latent classes and suicidal behaviors, we calculated propensity scores weighting the binominal logistic regression using the twang package in R. Propensity scores are primarily used to estimate the probability of a treatment or intervention (bullying experiences are considered treatments for the purposes of our study) for each case by accounting for the covariates that predict them. This method has advantages in controlling for confounding variables and selection bias and for comparing different treatment effects in a quasi-experiment.
Our assumption that there were more than two bullied groups (i.e., non-bullied groups and various types of bullied groups), allowed us to use multinomial logistic regression analysis to estimate multiple propensity scores. Propensity scores estimate the average treatment effect (ATE). In this study, the ATE of experiences with bullying type A compared with experiences with bullying type B represents a comparison of the mean outcomes of both A and B in the total population under examination. The propensity scores for each type of bullying experience were estimated using the regression formula for the reference group, namely the non-bullied group. Sex, age, family type (two parents or other), paternal education, social welfare status, hyperactivity, emotional problems, conduct problems, peer relationship problems, and the prosocial subscale of the SDQ-Kr were treated as confounding variables in this calculation. Because it was possible that more than three groups would be compared simultaneously in this study, a boosted multinomial logistic regression analysis was used for the propensity scores. Finally, we calculated propensity scores using weighted binomial logistic regression analysis. In this stage, we examined the relationship between experiences of being bullied and suicidal behaviors, considering any compounding effects among them.
The average age of participants was 14.3 years; 43.2% of participants were male and 56.8% were female. A total of 77.1% of participants lived with two parents. With regard to suicidal behaviors, 39.6% of our participants (n = 1,746) reported that they had experienced suicidal ideation, and 6.5% of this group (n = 287) had actually attempted suicide. The relatively high proportion of those with suicidal ideation may be partially related to our inclusion of a high-risk group in our sample. A total of 851 adolescents (19.3%) responded that they had experienced at least one of the types of bullying listed on the questionnaire, while the remaining 3559 adolescents (80.7%) responded that they had not experienced any type of bullying. The most commonly experienced type of bullying was verbal bullying (10.8%), which showed a relatively high prevalence, followed by ostracism (7.9%), physical bullying (6.1%), cyber bullying (3.0%), harassment (2.8%), extortion (1.5%) and sexual bullying (0.5%).
Fig 1 represents a scree plot displaying AIC and BIC values for the LCA models. The scree plot shows differences in AIC and BIC values according to the number of classes of bullying experienced. The dotted line indicates changes in BIC, and the solid line indicates changes in AIC. Largest declines in the BIC or AIC is usually used as optimal model selection criteria. Two-class solution of majority of non-experienced (92.7%) and experienced (7.3%) was the largest decline in the BIC and AIC. However, we chose an alternative solution that includes more detailed bullying classes to properly classify bullied experiences. The two indicators show slightly different results. According to the BIC, the lowest value indicates that a three-class solution had the best fit and the four-class solution had even better model fit according to the AIC. Therefore, we chose the three-class solution considering the simplicity.
Fig 2 shows the LCA classes and the distribution of bullying experiences. A total of 4.7% of participants were categorized into Class 1, which includes primarily physical and verbal behaviors that we labeled as “physical.” Class 2, which involves verbal bullying, ostracism, and cyber bullying, but not physical bullying, was labeled as “non-physical” and included 3.5% of the total sample. Class 3, labeled “non-experienced” includes participants who had not experienced any kind of bullying. Verbal bullying was the most common experience among members of Classes 1 and 2. However, participants in Class 2 reported more covert and social features and had no experience with physical bullying.
Table 1 represents the association between bullying experiences and reconstructed bullying classes. Most notably, physical bullying was shown to be high in the physical class, at 80.2%. For both the physical and non-physical classes, verbal bullying exhibited the highest proportion among the seven types of bullying experiences, 97.3% and 93.0%, respectively. In the non-physical class, forms of ostracism and cyber bullying exhibited higher proportions (87.3% and 45.8% respectively) than in the physical class.
The clinical and demographic characteristics of each bullying class are shown in Table 2. Males who had experienced being bullied were most frequently categorized into the physical class (68.1%), whereas bullied females were most frequently categorized into the non-physical class (80.3%). Additionally, both of the groups that had experienced bullying were significantly different from the non-experienced group with respect to the prevalence of economic hardship (10.4% in the non-physical bullying class and 13.4% in the non-physical bullying class, compared with 7.2% in the non-experienced class). From a clinical point of view, scores on all four clinical subscales of the SDQ-Kr except for the prosocial behavior subscale (i.e., hyperactivity, emotional problems, conduct problems, peer problems) were significantly higher for the groups that had experienced bullying compared with the non-experienced group; no significant clinical differences were found between the physical and non-physical classes.
Table 3 shows the results of the binary logistic regression analysis examining the relationship between bullying class and two suicidal behaviors. The non-adjusted regression analysis without propensity score-weighted suicidal ideation showed that both the physical class (OR = 3.43, p < .01) and the non-physical class (OR = 2.24, p < .01) were more likely to report suicidal ideation compared with the non-experienced group. However, the regression coefficients of the adjusted analysis weighted for propensity scores were lower and not statistically significant. This result indicates that the association between peer bullying and suicidal ideation was weak and that various confounding variables may have been involved.
In terms of suicide attempts, the regression coefficients between bullying experience and suicide attempts for both the physical and non-physical classes were statistically significant in the non-adjusted model. Both the physical class (OR = 3.21, p < .001) and the non-physical class (OR = 3.28, p < .328) were more likely to attempt suicide than was Class 3. Even after weighting by propensity scores, these associations remained statistically significant. Both the physical class (OR = 3.06, p < .05) and the non-physical class (OR = 2.94, p < .01) were more likely than the reference group to attempt suicide after adjustment. This result reflects the strong association of both classes of victimization with suicide attempts even after controlling for confounding variables.
This study identifies patterns of co-occurring peer bullying experiences from the perspective of victims, and the association of these experiences on suicide in a relatively large sample of treatment-seeking adolescents. Experiences of being bullied were classified into two distinct classes. The most noticeable difference between the physical class and the other classes involved the presence or absence of a physical bullying experience. The non-physical class included more of the social aspects of experiences with bullying, but both classes were associated with suicide attempts, with similar effect sizes. These patterns were kept after controlling compounding factors using propensity scores.
This study found that about 20% of the participants reported being the victims of some form of bullying. This prevalence is found to be lower than the 25% found in a Taiwanese sample or the 37.3% observed in a US sample, or the 23% found in another Korean sample. These differences may reflect differences in the methods used to assess bullying, the clinical characteristics of the samples, or other characteristics such as gender or age. In a US study, bullying of students is defined as “when another student, or a group of students, say or do nasty or unpleasant things to the victim. Although they described additional criteria of bullying, this broad definition might have a possibility for overestimation. We evaluated the being bullied experiences using self-administrated yes/no questionnaire according to participant’s subjective perception of bullying during the past 12 months, whereas other studies defined the bullying of victims by selecting several items of the standardized bullying questionnaires based on frequency such as ‘ often’ or’ two or three times a month’. Such simplicity and dependence of retrospective self assessment during relatively long period of our study might underestimate actual condition of bullying experiences. However, the prevalence revealed to be quite similar with another Korean study with middle school students .
Although it is common to classify bullying victims with respect to physical or verbal bullying [26–28], more than one type of bullying may be experienced by a victim, and the specific clustering of experiences of being bullied may depend on various factors associated with the victims or perpetrators. Zhang et al. reported that the correlation coefficient between the two bullying types was statistically significant, with adolescents who were the targets of one type of bullying being at higher risk of experiencing another type of bullying. The present study clearly differentiated between two classes of bullying, with the presence of experiences of being physically bullied being the main determinant of this classification; however, verbal bullying was a common experience in both classes.
Several studies involving adolescents have investigated the association between various types of bullying in victims of suicide. One Dutch study reported that suicidal ideation was related with being bullied both directly and indirectly, with the associations being stronger for indirect than for direct bullying. An Italian study investigated the effects of direct and relational victimization at school on suicidal ideation. The study showed that both direct and relational victimization at school were positively associated with suicidal thoughts, with relational victimization more strongly associated with suicide. A recent Taiwanese study classified bullying as either active (physical bullying, stealing belongings, etc.) or passive (social exclusion, name-calling, etc.), and found that both types were significantly associated with various kinds of mental health problems, including suicidal ideation and attempts . Our classification of bullying shares some similarities with the classification of bullying as direct versus indirect; however, our classification is more realistic in that if was achieved by means of more case-based methodological approach with classification, and it therefore better reflects the complexities of real situations.
Cyber bullying is the newest form of bullying behavior and has expanded the temporal and spatial boundaries of bullying, allowing adolescents to be bullied 24 hours a day regardless of their location. A meta-analysis has suggested that cyber bullying is strongly related to suicidal ideation. Surveys among US middle school students have found that cyber bullying is more strongly associated with depression and suicidality than is traditional bullying. Smith et al reported that many cyber bullying victims were also victims of traditional bullying. Our bullying assessment included cyber bullying, and we found that it was common in both the physical and non-physical classes, though more prevalent in the non-physical than in the physical class. Our result suggests that cyber bullying is not a separate type of bullying, but is more associated with non-physical bullying that includes indirect or relational forms of bullying rather than physical bullying. We need to pay more attention to negative consequences including suicidal risk of cyber bullying.
School bullying is currently a serious social problem in Korea. Following the aggressive efforts of Korean society and the government to improve the monitoring and reporting of bullying, the overall rate of school bullying in Korea, particularly physical bullying, has decreased in recent years. Nevertheless, cyber and verbal bullying are harder for adults to detect, and there is objective evidence that these forms of bullying are increasing.Given the reality of this situation and the association of non-physical forms of bullying on suicidal behavior, there is an urgent need to protect students from bullying at school, an effort that must involve collaborations among schools, families and communities in carefully and comprehensively redefining the assessment of bullying. Schools and mental health professionals need to be sensitive to this “hidden” type of non-physical bullying and devote more attention to it.
We expected distinct types of bullying to exert distinct association on suicidal ideation and attempts among adolescents, but there were no significant differences in suicidal ideation and attempts between the two bullying classes in this cross-sectional study. In fact, we found no significant differences in victim characteristics between these classes, with the exception of gender. This suggests that there may be significant differences in characteristics of perpetrators. Aside from the dyadic relationship of victim and perpetrator, social and cultural factors may have an effect on co-occurring bullying experiences. Additionally, further longitudinal research with more detailed assessment methods may reveal significant differences between the two classes of bullying with respect to clinical outcomes for victims. A longitudinal study involving Swedish adolescents demonstrated that direct and indirect victimization predicted different profiles of psychological difficulties.
Studies conducted throughout the world have pointed to the dangers of bullying by emphasizing maladaptive outcomes for victims, including anxiety, loneliness, sadness, overcompliance, insecurity, and frustration [34–37]. In addition to these internalizing behaviors, the target of bullying may display externalizing problems such as impulsiveness and hyperactivity[38,39]. The present study showed significantly higher mean scores on the clinical subscales of the SDQ-Kr in both of the groups who experienced bullying relative to the group who did not, but we cannot determine whether this characteristic is a cause or a consequence of bullying.
The association between bullying and suicidality is receiving serious attention [13,17,40], as experiences of bullying result in more severe suicidal behaviors among adolescents [5,10,41]. As shown in our study, suicidal ideation was not related to either physical or non-physical bullying after adjustment was made for confounding effects. However, experiences with both bullying classes were strongly associated with suicide attempts. These results suggest that bullying is directly associated with more risky suicidal behaviors. Moreover, non-physical bullying had almost the same association with suicide attempts as physical bullying. This study supports the notion that being a victim of bullying is an important risk factor for attempted suicide regardless of the specific nature of those bullying experiences. A recent meta-analysis reported a significant positive association between suicidality and bullying victimization, with ORs of approximately 2–4. The ORs for the association between suicide attempts and bullying victimization in this study were approximately 3, similar to previous results.
Although different classes of bullying showed similar risk factors for suicide attempts, our results showed significant clinical differences between two classes. A previous study has reported that different types of bullying were associated with different mental health problems. These results suggest that different intervention strategies might be required for the two distinct classes of bullying and for the various bullying behaviors within each general type. Future research is required to gather data on the clinical characteristics and prognoses of the various types of bullying, with the ultimate aim of developing specific intervention strategies for victims suffering from psychiatric problems.
A further strength of the present study was the use of up-to-date statistical methodology methods in the analysis. The precise classification of bullying types and the determination of their association with suicidal behaviors enabled us to identify differences among bullying subgroups. LCA allowed us to identify the characteristics and complex patterns of each subgroup, which in turn may facilitate the development of more effective interventions targeted at specific groups. In psychiatry, LCA has been used to analyze symptom data from other neuropsychiatric disorders, most notably attention deficit–hyperactivity disorder [43–47] and obsessive–compulsive disorder [48,49]. Whereas a factor analysis looks for the underlying structure among specific variables, LCA can be used to find latent homogeneous groups of individuals, providing an additional dimension of analysis . This has the advantage of potentially refining our understanding of adolescents’ experiences of being bullied and may improve our ability to identify the nature of bullying, provide specific targeted interventions, and predict outcomes. A large sample is required for LCA to provide a valid classification solution. Our study had a relatively large sample, which yielded explanatory power and the possibility of parsimony. Additionally, we conducted binomial logistic regression weighted by multi-group propensity scores to more precisely estimate the association of being bullied and to compare the effect size of the bullying types. The use of propensity scores is not yet a common practice in psychiatric research, though its application is increasing. Our data indicate that the use of propensity scores and related techniques may contribute to mental health studies with highly multi-dimensional compound datasets.
For careful interpretation of association between bullying victimization and suicidal behavior, we might consider bullying assessment method. A recent meta-analysis study suggests that bullying assessment method affects the association between bully-victim status and suicidal ideation . When definitional and non-behavioral assessment methods were used, the largest effect size should have been shown . In this study, we did not provide the accurate definition of bullying to the participants but assessed their experiences about being bullied through subjective perception of bullying by simple questions. We assume that the participants were well aware of the definition of bullying because every student should get regular bullying prevention education in the school and take part in the national survey for school bullying with providing accurate definition (e.q imbalance of power and repetition, specific types of bullying) of bullying twice a year in Korea. Although our assessment might reflect realistic perception of bullying among Korean adolescents, we cannot disregard the possibility of a significant error in estimation of actual victimization prevalence. A previous study reported that among students who self-reported being victims of bullying, approximately half were confirmed as actual victims. Student might misconstrue ordinary peer conflict as bullying . Even the most widely adopted tool for bullying assessment, the Olweus Bully/Victimization Questionnaire, reported different responses with the California Bullying Victimization Scale that did not use the term “bullying,”, but included items that asked about its defining characteristics (repetition, intentionality, power imbalance). Therefore, simple assessment of bullying depending on self-report might be a major limitation of our study for proper interpretation. If we had assessed bullying with accurate definition and standardized measurement, larger effect size or different results might have been reported.
This study also has additional limitations. First, it used a clinical sample of individuals who had been referred to treatment due to psychological and behavioral issues and who were therefore a relatively high-risk group. Thus, our findings may not generalize to the present population of South Korea. Second, the prevalence of the classes identified by LCA may have been affected by our sample selection despite its size. Third, low reliability coefficient of conduct subscale was reported in the SDQ-Kr. Forth, the evaluation of suicidal behavior was self-administrated rather than based on clinical interviews or a standardized assessment. For assessing suicidal ideation, we only checked verbal expression of suicidal ideation, without including unexpressed suicidal ideation. There might be significant difference between talking about suicide and actual suicidal ideation. Finally, it is difficult to infer causal relationships from a cross-sectional study. More focused and longitudinal research would allow for more accurate identification of any causal association between peer bullying and suicidal behaviors.
The authors are thankful to excellent staffs of the Gyeonggi provincial mental health center at 31 local mental health centers of Gyeonggi province, South Korea. Also the authors are thankful to the adolescents and their parents who participated in the study.
Conceived and designed the experiments: SO SIL KH YMS JS EJP HY HJH. Performed the experiments: SO SIL KH YMS JS EJP HY HJH. Analyzed the data: BRR. Wrote the paper: BRR YY AK HJH.
- 1. Wang J, Iannotti RJ, Nansel TR (2009) School bullying among adolescents in the United States: Physical, verbal, relational, and cyber. Journal of Adolescent health 45: 368–375. pmid:19766941
- 2. Olweus D (1993) Bullying at School. What we know and whal we can do. Blackwell. Oxford.
- 3. Olweus D (1992) Victimization among schoolchildren: intervention and prevention.
- 4. Rigby K (2000) Effects of peer victimization in schools and perceived social support on adolescent well-being. Journal of adolescence 23: 57–68. pmid:10700372
- 5. Van der Wal MF, De Wit CAM, Hirasing RA (2003) Psychosocial health among young victims and offenders of direct and indirect bullying. Pediatrics 111: 1312–1317. pmid:12777546
- 6. Rigby K (1997) What children tell us about bullying in schools. Children Australia 22: 28.
- 7. Rigby K (1998) The relationship between reported health and involvement in bully/victim problems among male and female secondary schoolchildren. Journal of Health Psychology 3: 465–476. pmid:22021407
- 8. Family MoGEa (2010) Survey on the Actual Condition of School Violence. Ministry of Gender Equality and Family.
- 9. Kim YS, Leventhal BL, Koh Y-J, Boyce WT (2009) Bullying Increased Suicide Risk: Prospective Study of Korean Adolescents. ARCHIVES OF SUICIDE RESEARCH 13: 15–30. pmid:19123106
- 10. Kaltiala-Heino R, Rimpelä M, Marttunen M, Rimpelä A, Rantanen P (1999) Bullying, depression, and suicidal ideation in Finnish adolescents: school survey. BRITISH MEDICAL JOURNAL 319: 348–351. pmid:10435954
- 11. Liang H, Flisher AJ, Lombard CJ (2007) Bullying, violence, and risk behavior in South African school students. Child abuse & neglect 31: 161–171. pmid:17313977
- 12. Kim YS, Leventhal B (2008) Bullying and suicide. A review. International journal of adolescent medicine and health 20: 133–154. pmid:18714552
- 13. Brunstein Klomek A, Marrocco F, Kleinman M, Schonfeld IS, Gould MS (2007) Bullying, depression, and suicidality in adolescents. Journal of the American Academy of Child & Adolescent Psychiatry 46: 40–49.
- 14. Mills C, Guerin S, Lynch F, Daly I, Fitzpatrick C (2004) The relationship between bullying, depression and suicidal thoughts/behaviour in Irish adolescents. Irish Journal of Psychological Medicine.
- 15. Yen C-F, Yang P, Wang P-W, Lin H-C, Liu T-L,Wu Y-Y, et al. (2014) Association between school bullying levels/types and mental health problems among Taiwanese adolescents. Compr Psychiatry 55: 405–413. pmid:24529472
- 16. Lundh LG, Daukantaite D, Wangby-Lundh M (2014) Direct and indirect aggression and victimization in adolescents—associations with the development of psychological difficulties. BMC Psychol 2: 43. pmid:25566388
- 17. Baldry AD, & Winkel F. W. (2003) Direct and vicarious victimization at school and at home as risk factors for suicidal cognition among Italian adolescents. Journal of Adolescence 26: 703–716. pmid:14643741
- 18. Joung-Sook A, Sung-Kuyn Jun, Jun-Kyu Han, Kyung-Sun Noh, Robert Goodman (2003) The Development of a Korean Version of Strengths and Difficulties Questionnaire. J Korean Neuropsychiatr Assoc 42: 141–147.
- 19. Bourdon KH, Goodman R, Rae DS, Simpson G, Koretz DS (2005) The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. J Am Acad Child Adolesc Psychiatry 44: 557–564. pmid:15908838
- 20. Goodman R (2001) Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry 40: 1337–1345. pmid:11699809
- 21. Du Y, Kou J, Coghill D (2008) The validity, reliability and normative scores of the parent, teacher and self report versions of the Strengths and Difficulties Questionnaire in China. Child Adolesc Psychiatry Ment Health 2: 8. pmid:18445259
- 22. Linzer DA, Lewis JB (2011) poLCA: An R package for polytomous variable latent class analysis. Journal of Statistical Software 42: 1–29.
- 23. McCaffrey DF, Griffin BA, Almirall D, Slaughter ME, Ramchand R,Burgette LF. (2013) A tutorial on propensity score estimation for multiple treatments using generalized boosted models. Statistics in Medicine 32: 3388–3414. pmid:23508673
- 24. Shetgiri R, Lin H, Flores G (2012) Identifying children at risk for being bullies in the United States. Acad Pediatr 12: 509–522. pmid:22989731
- 25. Young Shin Kim BLL, Yun-Joo Koh, and Boyce W. Thomas (2009) Bullying Increased Suicide Risk: Prospective Study of Korean Adolescents. Archives of Suicide Research 13: 15–30.
- 26. Crick NR, Bigbee MA (1998) Relational and overt forms of peer victimization: a multiinformant approach. Journal of consulting and clinical psychology 66: 337. pmid:9583337
- 27. Zhang L, Osberg L, Phipps S (2014) Is all bullying the same? Archives of public health 72: 19. pmid:24991409
- 28. Yen C-F, Huang M-F, Kim YS, Wang P-W, Tang T-C, Yeh Y-C, et al. (2013) Association between types of involvement in school bullying and different dimensions of anxiety symptoms and the moderating effects of age and gender in Taiwanese adolescents. Child abuse & neglect 37: 263–272. pmid:23484950
- 29. van Geel M, Vedder P, Tanilon J (2014) Relationship Between Peer Victimization, Cyberbullying, and Suicide in Children and Adolescents: A Meta-analysis. JAMA pediatrics.
- 30. Wang J, Nansel TR, Iannotti RJ (2011) Cyber and traditional bullying: Differential association with depression. Journal of Adolescent Health 48: 415–417. pmid:21402273
- 31. Hinduja S, Patchin JW (2010) Bullying, cyberbullying, and suicide. Archives of Suicide Research 14: 206–221. pmid:20658375
- 32. Smith PK, Mahdavi J, Carvalho M, Fisher S, Russell S, Neil T (2008) Cyberbullying: Its nature and impact in secondary school pupils. Journal of Child Psychology and Psychiatry 49: 376–385. pmid:18363945
- 33. Klomek AB, Sourander A, Kumpulainen K, Piha J, Tammonen T, Moilanen I, et al. (2008) Childhood bullying as a risk for later depression and suicidal ideation among Finnish males. Journal of Affective Disorders 109: 47–55. pmid:18221788
- 34. Frost (1991) 'A primary school approach: What can be done about the bully?' In M Elliott (ed.) Bullying: A practical guide to copig for schools.: Longman, Harlow.
- 35. MJ HDB (2000) 'Twenty years' research on peer victimization and psychosocial maladjustment: A meta-analytic review of cross-sectional studies'. Journal of Child Psychology and Psychiatry 41: 441–455. pmid:10836674
- 36. Q BTL (2005) Cyber-harassment: A study of a new method for an old behavior. Journal of Educational COmputing Research 32: 265–277.
- 37. Patchin JW, Hinduja S (2006) Bullies move beyond the schoolyard a preliminary look at cyberbullying. Youth violence and juvenile justice 4: 148–169.
- 38. Camodeca M. Goossens (2003) Links between social information processing in middle childhood and involvement in bullying. Aggressive Behavior 29.
- 39. Johnson HR TM, Wilkinson S, Walsh L. Balding & Wright V (2002) Vulnerability to bullying: Teacher-reported conduct and emotional problems, hyperactivity, peer relationship difficulties, and prosocial behaviour in primary school children. Educational Psychology 22: 553–556.
- 40. Kim YS, Koh Y., & Leventhal B (2005) School bullying and suicidal risk in Korean middle school students. Pediatrics 115: 357–363. pmid:15687445
- 41. Roland E (2002) Bullying, depressive symptoms and suicidal thoughts. Educational Research 44: 55–67.
- 42. Melissa K. Holt AMV-K, Joshua R. Polanin , Kristin M. Holland , Sarah DeGue, Jennifer L. Matjasko , Misty Wolfe, Gerald Reid (2015) Bullying and Suicidal Ideation and Behaviors: A Meta-Analysis. Pediatrics 135: e496–e509. pmid:25560447
- 43. Hudziak JJ, Heath AC, Madden PF, Reich W, Bucholz KK, Slutske W, et al. (1998) Latent class and factor analysis of DSM-IV ADHD: a twin study of female adolescents. J Am Acad Child Adolesc Psychiatry 37: 848–857. pmid:9695447
- 44. Neuman RJ, Hudziak JJ, Heath A, Reich W, Bucholz KK, Madden PAF, et al. (2001) Latent class analysis of ADHD and comorbid symptoms in a population sample of adolescent female twins. J Child Psychol Psychiatry 42: 933–942. pmid:11693588
- 45. Neuman RJ, Todd RD, Heath AC, Reich W, Hudziak JJ, Bucholz KK, et al. (1999) Evaluation of ADHD typology in three contrasting samples: a latent class approach. J Am Acad Child Adolesc Psychiatry 38: 25–33. pmid:9893413
- 46. Lubke GH, Muthen B, Moilanen IK, McGough JJ, Loo SK, Swanson JM, et al. (2007) Subtypes versus severity differences in attention-deficit/hyperactivity disorder in the Northern Finnish Birth Cohort. J Am Acad Child Adolesc Psychiatry 46: 1584–1593. pmid:18030080
- 47. Rasmussen ER, Neuman RJ, Heath AC, Levy F, Hay DA, Todd RD (2002) Replication of the latent class structure of Attention-Deficit/Hyperactivity Disorder (ADHD) subtypes in a sample of Australian twins. J Child Psychol Psychiatry 43: 1018–1028. pmid:12455923
- 48. Delucchi KL, Katerberg H, Stewart SE, Denys DA, Lochner C, Stack DE, et al. (2011) Latent class analysis of the Yale-Brown Obsessive-Compulsive Scale symptoms in obsessive-compulsive disorder. Compr Psychiatry 52: 334–341. pmid:21145539
- 49. Scherrer JF, Xian H, Slutske WS, Eisen SA, Potenza MN (2015) Associations Between Obsessive-Compulsive Classes and Pathological Gambling in a National Cohort of Male Twins. JAMA Psychiatry.
- 50. Cornell DM, S. (2011) Counselor confirmation of middle school student self-reports of bullying victimization. Professional School Counseling 14: 261–270.
- 51. Green JG, Felix ED, Sharkey JD, Furlong MJ, Kras JE (2013) Identifying bully victims: definitional versus behavioral approaches. Psychol Assess 25: 651–657. pmid:23244644