Suicides by carbon monoxide poisoning resulting from burning barbecue charcoal reached epidemic levels in Hong Kong and Taiwan within 5 y of the first reported cases in the early 2000s. The objectives of this analysis were to investigate (i) time trends and regional patterns of charcoal-burning suicide throughout East/Southeast Asia during the time period 1995–2011 and (ii) whether any rises in use of this method were associated with increases in overall suicide rates. Sex- and age-specific trends over time were also examined to identify the demographic groups showing the greatest increases in charcoal-burning suicide rates across different countries.
Methods and Findings
We used data on suicides by gases other than domestic gas for Hong Kong, Japan, the Republic of Korea, Taiwan, and Singapore in the years 1995/1996–2011. Similar data for Malaysia, the Philippines, and Thailand were also extracted but were incomplete. Graphical and joinpoint regression analyses were used to examine time trends in suicide, and negative binomial regression analysis to study sex- and age-specific patterns. In 1995/1996, charcoal-burning suicides accounted for <1% of all suicides in all study countries, except in Japan (5%), but they increased to account for 13%, 24%, 10%, 7%, and 5% of all suicides in Hong Kong, Taiwan, Japan, the Republic of Korea, and Singapore, respectively, in 2011. Rises were first seen in Hong Kong after 1998 (95% CI 1997–1999), followed by Singapore in 1999 (95% CI 1998–2001), Taiwan in 2000 (95% CI 1999–2001), Japan in 2002 (95% CI 1999–2003), and the Republic of Korea in 2007 (95% CI 2006–2008). No marked increases were seen in Malaysia, the Philippines, or Thailand. There was some evidence that charcoal-burning suicides were associated with an increase in overall suicide rates in Hong Kong, Taiwan, and Japan (for females), but not in Japan (for males), the Republic of Korea, and Singapore. Rates of change in charcoal-burning suicide rate did not differ by sex/age group in Taiwan and Hong Kong but appeared to be greatest in people aged 15–24 y in Japan and people aged 25–64 y in the Republic of Korea. The lack of specific codes for charcoal-burning suicide in the International Classification of Diseases and variations in coding practice in different countries are potential limitations of this study.
Charcoal-burning suicides increased markedly in some East/Southeast Asian countries (Hong Kong, Taiwan, Japan, the Republic of Korea, and Singapore) in the first decade of the 21st century, but such rises were not experienced by all countries in the region. In countries with a rise in charcoal-burning suicide rates, the timing, scale, and sex/age pattern of increases varied by country. Factors underlying these variations require further investigation, but may include differences in culture or in media portrayals of the method.
Please see later in the article for the Editors' Summary
Every year, almost one million people die by suicide globally; suicide is the fifth leading cause of death in women aged 15–49 and the sixth leading cause of death in men in the same age group. Most people who take their own life are mentally ill. For others, stressful events (the loss of a partner, for example) have made life seem worthless or too painful to bear. Strategies to reduce suicide rates include better treatment of mental illness and programs that help people at high risk of suicide deal with stress. Suicide rates can also be reduced by limiting access to common suicide methods. These methods vary from place to place. Hanging is the predominant suicide method in many countries, but in Hong Kong, for example, jumping from a high building is the most common method. Suicide methods also vary over time. For example, after a woman in Hong Kong took her life in 1998 by burning barbecue charcoal in a sealed room (a process that produces the toxic gas carbon monoxide), charcoal burning rapidly went from being a rare method of killing oneself in Hong Kong to the second most common suicide method.
Why Was This Study Done?
Cases of charcoal-burning suicide have also been reported in several East and Southeast Asian countries, but there has been no systematic investigation of time trends and regional patterns of this form of suicide. A better understanding of regional changes in the number of charcoal-burning suicides might help to inform efforts to prevent the emergence of other new suicide methods. Here, the researchers investigate the time trends and regional patterns of charcoal-burning suicide in several countries in East and Southeast Asia between 1995 and 2011 and ask whether any rises in the use of this method are associated with increases in overall suicide rates. The researchers also investigate sex- and age-specific time trends in charcoal-burning suicides to identify which groups of people show the greatest increases in this form of suicide across different countries.
What Did the Researchers Do and Find?
The researchers analyzed method-specific data on suicide deaths for Hong Kong, Japan, the Republic of Korea, Taiwan, and Singapore between 1995/1996 and 2011 obtained from the World Health Organization Mortality Database and from national death registers. In 1995/1996, charcoal-burning suicides accounted for less than 1% of all suicides in all these countries except Japan (4.9%). By 2011, charcoal-burning suicides accounted for between 5% (Singapore) and 24% (Taiwan) of all suicides. Rises in the rate of charcoal-burning suicide were first seen in Hong Kong in 1999, in Singapore in 2000, in Taiwan in 2001, in Japan in 2003, and in the Republic of Korea in 2008. By contrast, incomplete data from Malaysia, the Philippines, and Thailand showed no evidence of a marked increase in charcoal-burning suicide in these countries over the same period. Charcoal-burning suicides were associated with an increase in overall suicide rates in Hong Kong in 1998–2003, in Taiwan in 2000–2006, and in Japanese women after 2003. Finally, the annual rate of change in charcoal-burning suicide rate did not differ by sex/age group in Taiwan and Hong Kong, whereas in Japan people aged 15–24 and in the Republic of Korea people aged 25–64 tended to have the greatest rates of increase.
What Do These Findings Mean?
These findings show that charcoal-burning suicides increased markedly in several but not all East and Southeast Asian countries during the first decade of the 21st century. Moreover, in countries where there was an increase, the timing, scale, and sex/age pattern of the increase varied by country. The accuracy of these findings is likely to be limited by several aspects of the study. For example, because of the way that method-specific suicides are recorded in the World Health Organization Mortality Database and national death registries, the researchers may have slightly overestimated the number of charcoal-burning suicides. Further studies are now needed to identify the factors that underlie the variations between countries in charcoal-burning suicide rates and time trends reported here. However, the current findings highlight the need to undertake surveillance to identify the emergence of new suicide methods and the importance of policy makers, the media, and internet service providers working together to restrict graphic and detailed descriptions of new suicide methods.
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001622.
- A PLOS Medicine research article by Shu-Sen Chang and colleagues investigates time trends and regional patterns of charcoal-burning suicide in Taiwan
- The World Health Organization provides information on the global burden of suicide and on suicide prevention (in several languages); it also has an article on international patterns in methods of suicide
- The US National Institute of Mental Health provides information on suicide and suicide prevention
- The UK National Health Service Choices website has detailed information about suicide and its prevention
- MedlinePlus provides links to further resources about suicide (in English and Spanish)
- The International Association for Suicide Prevention provides links to crisis centers in Asia
- The charity Healthtalkonline has personal stories about dealing with suicide
Citation: Chang S-S, Chen Y-Y, Yip PSF, Lee WJ, Hagihara A, Gunnell D (2014) Regional Changes in Charcoal-Burning Suicide Rates in East/Southeast Asia from 1995 to 2011: A Time Trend Analysis. PLoS Med 11(4): e1001622. doi:10.1371/journal.pmed.1001622
Academic Editor: Phillipa J. Hay, University of Western Sydney, Australia
Received: October 8, 2013; Accepted: February 19, 2014; Published: April 1, 2014
Copyright: © 2014 Chang et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: SSC was supported by The University of Hong Kong Seed Funding Programme for Basic Research (grant number 201203159017) and the Chiang Ching-kuo Foundation, Taiwan (grant number RG014-P-12). YYC was supported by the National Science Council, Taiwan (grant number 101-2314-B-532-005-MY2) and the National Health Research Institute, Taiwan (grant number NHRI-EX100-10024PC). PSFY was supported by a Public Policy Research Grant (HKU7030-PPR-12) and a General Research Fund (HKU-784210M). DG is a UK National Institute for Health Research Senior Investigator. The funders had no role in study design, the collection, analysis, and interpretation of data, writing of the article, and the decision to submit the article.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: ICD, International Classification of Diseases; ICD-9, ninth revision of International Classification of Diseases; ICD-10, tenth revision of International Classification of Diseases; WHO, World Health Organization
Globally, suicide is amongst the leading causes of premature mortality; in 2010, it was the fifth leading cause of death in women and the sixth in men among individuals aged 15–49 y . Time trends in suicide rates may be influenced by a number of factors including socio-economic changes, the prevalence of mental illness or distress, and certain types of media reporting ; there is growing evidence that changes in the popularity and availability of lethal suicide methods could also have a marked impact on time trends in overall suicide rates –.
Previous studies of method availability and suicide have mostly focused on the impact of restricting access to methods , such as detoxification of domestic gas , bans on sales of toxic pesticides ,, and legal changes in firearms regulations ,. However, many suicides using these methods had already occurred before the implementation of restrictions, highlighting the potential importance of surveillance for the emergence of new suicide methods at an early stage to enable public health action to prevent an increase of suicide by new methods ,.
In 1998–2000 there was a rapid rise in suicide by carbon monoxide poisoning from the inhalation of barbecue charcoal gas in Hong Kong and Taiwan –. Suicides by this method used to be very rare, but within 5 y charcoal burning became the second most common method of suicide in these two countries. Although cases of charcoal-burning suicide have been reported in other neighbouring East/Southeast Asian countries such as China , Japan , Macao , Malaysia , Singapore , and the Republic of Korea , to the best of our knowledge, there has been no systematic investigation of regional patterns and time trends in the use of this method and the association between time trends in charcoal-burning suicide and overall suicide rates in affected countries.
We used data from eight East/Southeast Asian countries (Hong Kong, Taiwan, Japan, the Republic of Korea, Singapore, Malaysia, the Philippines, and Thailand) to investigate time trends in charcoal-burning suicide across different countries and the association between changes in charcoal-burning suicide and overall suicide rates for the years 1995–2011. We also examined sex- and age-specific time trends to identify the demographic groups showing the greatest increases in charcoal-burning suicide rates across different countries. Our overall aim was to establish what can be learnt from the changing incidence of charcoal-burning suicide in this region to inform the prevention of the future emergence of novel suicide methods. Specifically, the objectives of this analysis were to investigate (i) time trends and regional patterns of charcoal-burning suicide throughout East/Southeast Asia during the period 1995–2011 and (ii) whether any rises in use of this method were associated with increases in overall suicide rates. Sex- and age-specific trends over time were also examined to identify the demographic groups showing the greatest increases in charcoal-burning suicide rates across different countries.
The study used only aggregate secondary data that were available openly; no identifiable personal data were used in the study. Ethical approval was thus not required.
To investigate time trends in charcoal-burning suicide in East/Southeast Asia we first systematically identified countries with data available in the World Health Organization (WHO) Mortality Database , which provides the most comprehensive standardised national mortality statistics for countries around the world. Figure 1 shows a flow chart summarising how we identified data for the study countries. In brief, we first identified 19 countries that were classified as in the East/Southeast Asia region by the United Nations (eight in East Asia and 11 in Southeast Asia) . The WHO Mortality Database contained suicide data for nine of these countries, but only six had method-specific data available. We then extracted complete method-specific suicide data by sex, age (5-y bands), and year for Japan and the Republic of Korea for the period 1995–2011, and for the years available for Hong Kong (2001–2011), Malaysia (2000–2008), the Philippines (1995–2003, 2008), and Thailand (1995–2000, 2002–2006). Data for the 3-y period (1995–1997) prior to 1998, when the first widely publicised suicide by charcoal burning occurred ,, were used to assess the baseline rates. We then supplemented the WHO data by extracting relevant suicide data from the national death registers for Hong Kong (1995–2011) and Taiwan (1995–2011), as well as from published mortality statistics for Singapore (1996–2011), although only sex- and method-specific, but not age-specific, data were available for Singapore . The main analysis therefore included five countries (Hong Kong, Taiwan, Japan, the Republic of Korea, and Singapore) with complete data for the years 1995/1996–2011; suicide data for these countries are thought to be relatively reliable . In contrast, the quality of suicide data for countries with incomplete time series (Malaysia, the Philippines, and Thailand) is thought to be relatively poor –. The WHO Mortality Database also provided population data; when these were incomplete or unavailable, relevant data were extracted from the United Nations population database .
There is no specific code for charcoal-burning suicide in the International Classification of Diseases (ICD) . We used the following ICD codes to extract data for charcoal-burning suicide: E952 in the ninth revision of ICD (ICD-9) and X67 in the tenth revision of ICD (ICD-10). These codes include suicides by poisoning using gases other than domestic gas and are not restricted to charcoal-burning suicides. Previous studies showed that more than 90% of deaths in this category were charcoal-burning suicides in Hong Kong  and Taiwan ; these studies used data from 2002 or before, but results were similar when we examined data for 2010, the most recent year for which detailed cause-of-death information was available—99% and 85% of suicides coded as ICD-10 X67 were charcoal-burning suicides in Hong Kong and Taiwan, respectively. We did not have access to data from other countries to investigate this further. For simplicity, we refer to ICD-9 E952 and ICD-10 X67 deaths as charcoal-burning suicides throughout the paper, although not all cases used this method. For example, in Japan some of deaths with these ICD codes were due to hydrogen sulphide poisoning when the use of this method rapidly increased in 2008 . Suicides by all other methods were identified using the following codes, excluding charcoal-burning suicides as defined above: ICD-9 E950–E959 and ICD-10 X60–X84. Previous studies have indicated that many deaths categorised as undetermined intent are likely to be misclassified suicides –, including charcoal-burning suicides ; therefore, we included undetermined deaths in the main analysis. The following codes were used to identify such deaths: (i) ICD-9 E982 and ICD-10 Y17 for deaths by charcoal burning and (ii) ICD-9 E980–E989 and ICD-10 Y10–Y34, excluding the previously defined codes, for suicides by other methods. Accidents by non-domestic gas (carbon monoxide included) were excluded, as one analysis showed that they had different demographic characteristics from those of suicides, suggesting that they were not misclassified suicides . In sensitivity analyses we used data for certified suicides only (Tables S1 and S2).
Age-standardised suicide rates for people aged 15 y or above were calculated using the WHO world standard population , when age-specific data were available. We used joinpoint regression (a segmented line regression or piecewise linear regression) to investigate time trends in charcoal-burning suicide and their associations with changes in overall suicide rates; this approach has been widely used to study time trends in mortality from causes such as cancer  and suicide ,–. Analyses were carried out using the the Joinpoint Regression Program (version 4.0.4, US National Cancer Institute) ; this software has been used by the US National Cancer Institute in the analyses of cancer trends published in their regular reports. In joinpoint regression analysis, suicide time trends are characterised by contiguous linear segments and “join points” (points at which trends change). The grid search method proposed by Lerman  was used to fit the segmented line regression; this method creates a “grid” of all possible locations for join points and tests the sum of squared errors for each one to find the best possible fit. This method allows the join points to occur only exactly at observed time points, not between time points. To identify the combination of join points and linear segments that fits the time series data best, a sequence of Monte Carlo permutation tests was used when comparing pairs of models differing by one join point . The sequence started by comparing the model with zero join points (i.e., a straight line with no change in trend) and that with one join point, and it ended when there was no statistical evidence that more joint points fit the data better or when reaching the maximum number of join points allowed. The number of join points was limited to a maximum of three according to the Joinpoint Regression Program guidelines based on the number of data points in the datasets . The overall probability of type 1 error (i.e., concluding that there are one or more join points when there are in fact none) was maintained at 0.05. For the majority of models the age-standardised rate was used as the dependent variable, and thus the regression coefficient indicated the mean increase in rate per year within segments identified by joinpoint regression analysis. In Hong Kong, there were zero values for charcoal-burning suicide in some years, causing some problems in estimating the standard errors of age-standardised rates; therefore, crude rates with variance based on the Poisson distribution were modelled, with the suicide counts adjusted by the addition of a small constant, i.e., 0.5 . Crude rates for Singapore were modelled similarly, as age-specific suicide data were unavailable. In a sensitivity analysis the autocorrelation was accounted for by estimating the regression coefficients using weighted least squares in all models .
Negative binomial regression models were used to estimate the annual rate of change in charcoal-burning suicide rates by sex for all ages and four age groups (15–24, 25–44, 45–64, and 65+ y), for the time periods (i) when there were rises in rates, i.e., from the start of the rise to its peak or to the most recent year when data were available, and (ii) when there were reductions in rates, i.e., from the peak to the most recent year when data were available. The starting and peak years were identified in the joinpoint regression analyses and by visual inspection of the graphs of time trends in charcoal-burning suicide. Negative binomial regression models were used because there was evidence for over-dispersion in the Poisson regression analyses of the data. We calculated incidence rate ratios assuming a linear change in rates. To examine whether the annual rate of change in charcoal-burning suicide differed between sexes or amongst people of different age groups, interaction terms between sex and year or between age group and year were included in the models. Negative binomial regression models were estimated using Stata version 12 (StataCorp).
Figure 2 shows time trends in overall suicide, charcoal-burning suicide, and suicide by all other methods in five East/Southeast Asian countries with complete data for 1995/1996–2011. Charcoal-burning suicide rates increased in all countries over the study period, but the magnitude of the rise varied by country. Rates were generally very low in 1995/1996, ranging from 0.1 to 1.1 per 100,000 (Table 1), whilst in 2011 they increased to values ranging from 0.4 per 100,000 (Singapore) to 4.8 per 100,000 (Taiwan). In 1995/1996, charcoal-burning suicides accounted for only 0.4%, 0.6%, 4.9%, 0.7%, and 0.8% of all suicides in Hong Kong, Taiwan, Japan, the Republic of Korea, and Singapore, respectively. These figures rose to 13.4%, 24.1%, 9.7%, 7.0%, and 4.9%, respectively, in 2011. The magnitude of increase over the period 1995/1996–2011 was most marked in Taiwan (absolute increase in percentage, 23.5%; 65-fold increase in rate) and Hong Kong (13.0%; 27-fold), followed by the Republic of Korea (6.2%; 28-fold), Japan (4.8%; 3-fold), and Singapore (4.0%; 4-fold). However, it should be noted that these estimates are sensitive to baseline rates. Rises in charcoal-burning suicide were seen in both males and females (Figure 2; Table 1). The patterns for data based only on certified suicides were similar (Figure S1; Table S1). In Malaysia, the Philippines, and Thailand, data were less complete than in the five countries reported above (Table 2). Although there was no indication of a marked rise in charcoal-burning suicide rate in these three countries, a very slight rise in Malaysia was observed (the rate increased from 0.06 per 100,000 in 2000 to a peak of 0.30 per 100,000 in 2003). For the most recent year with data available, the rate per 100,000 was 0.13 in Malaysia (2008), 0.01 in the Philippines (2008), and 0.00 in Thailand (2006).
Joinpoint Regression Analysis
Joinpoint regression analysis showed that the rise in charcoal-burning suicide began after 1998 (95% CI 1997–1999) in Hong Kong, 1999 (95% CI 1998–2001) in Singapore, 2000 (95% CI 1999–2001) in Taiwan, 2002 (95% CI 1999–2003) in Japan, and 2007 (95% CI 2006–2008) in the Republic of Korea (Table 3; Figure 3; see Figure S2 for method-specific graphs). Changes in time trends in Singapore were less certain, as they were mostly driven by the zero value in 1999; furthermore, a sensitivity analysis based only on certified suicides showed a continuously upward trend in the Singaporean charcoal-burning suicide rate over the entire study period, without any changes in time trends (Figure S3; Table S2). Analysis using Japanese data for 1995–2007 (excluding data for 2008–2010, when there was a rapid increase in suicide by hydrogen sulphide poisoning ) showed the same join points. Sensitivity analyses incorporating autocorrelation showed similar findings in all of the five countries (Table S3).
Red arrows indicate the years when charcoal-burning suicides started to increase.
If the year following the join point when the time trend turned upward is defined as the year when the increase in charcoal-burning suicide began (indicated by arrows in Figure 3), Hong Kong first experienced a rise in charcoal-burning suicide in 1999, followed by Singapore in 2000, Taiwan in 2001, Japan in 2003, and the Republic of Korea in 2008. Joinpoint analysis showed that the upward trend in charcoal-burning suicide changed and turned downward around 2001 (95% CI 1999–2004) in Hong Kong and 2006 (95% CI 2004–2008) in Taiwan (Table 3; Figure 3), although inspection of time trends showed that Hong Kong's charcoal-burning suicide rate started to fall only after 2003 (Table 1; Figure 2). Similar declines in other methods of suicide in Hong Kong and Taiwan were also observed after 2003.
Combined numbers of charcoal-burning suicides for the five study countries reached a peak in 2009 (n = 6,759). When we excluded data for 2008–2010, because in Japan these may have included not only charcoal-burning suicides but also some suicides by hydrogen sulphide poisoning , the peak occurred in 2005 (n = 6,267). Compared to the baseline levels in the years before charcoal-burning suicide began to increase in individual countries (1998 for Hong Kong, 1999 for Singapore, 2000 for Taiwan, 2002 for Japan, and 2007 for the Republic of Korea), by 2011 there were 49,738 more charcoal-burning suicides in total across the region. The distribution of these cases was 32,636 in Japan (2003–2011), 11,306 in Taiwan (2001–2011), 2,506 in Hong Kong (1999–2011), 3,127 in the Republic of Korea (2008–2011), and 163 in Singapore (1999–2011), corresponding to an annual excess of 3,626, 1,028, 193, 782, and 14 suicides, respectively.
Association with Overall Suicide Rates
Graphical examinations of time trends in suicide are presented in Figure 2. Table 3 and Figure 3 present joinpoint regression analyses. In Hong Kong, the rise in charcoal-burning suicide rate was associated with an increase in overall suicide rate over the period 1998–2003. Similarly, in Taiwan, the rise in charcoal-burning suicide in 2000–2006 was related to an increase in overall suicide rate over the same period. Some evidence of method substitution was found in Japanese males—the rise in charcoal-burning suicide was accompanied by a fall in suicide by other methods (Figure 2). In Japanese females, the increase in charcoal-burning suicide starting from 2003 was followed by an increase in the overall suicide rate, suggesting that the rise in charcoal-burning suicide may have contributed to the rise in the overall rate. In the Republic of Korea, the increase in charcoal-burning suicide was quite recent and was not associated with a rise in the overall suicide rate, as the magnitude of increase was relatively small. There was no evidence for an association of time trends in the rate of charcoal-burning suicide with changes in the overall suicide rate in Singapore.
Sex- and Age-Specific Patterns
Figure 4 shows sex- and age-specific time trends in charcoal-burning suicide in Hong Kong, Taiwan, Japan, and the Republic of Korea. Males aged 25–44 y tended to show the highest rates compared to other sex/age groups, except that Japanese males aged 45–64 y had rates similar to those of males aged 25–44 y. In contrast, females aged 65+ y tended to have the lowest rates, except in the Republic of Korea, where females aged 45–64 y had the lowest rates. Negative binomial regression analysis showed that there was no statistically significant age or sex interaction in Hong Kong, Taiwan, Japan, or the Republic of Korea (p for interaction >0.05) (Table 4). Although the overall age interaction effects were not statistically significant, in Japan, males and females aged 15–24 y tended to have the greatest rates of increase (p for interaction = 0.064), whilst in the Republic of Korea, males and females aged 25–44 y and 45–64 y tended to have the greatest rates of increase (p for interaction = 0.096). When charcoal-burning suicide rates decreased in Hong Kong and Taiwan, there was no statistical evidence of sex or age differences in the annual rates of reduction (p for interaction = 0.53–0.86).
Charcoal-burning suicides increased markedly in some East/Southeast Asian countries (Hong Kong, Taiwan, Japan, the Republic of Korea, and Singapore) in the first decade of the 21st century, but such rises were not experienced by all countries in the region. In countries with a rise in the charcoal-burning suicide rate, the timing, scale, and sex/age pattern of the increase varied by country. A rise in charcoal-burning suicides was first seen in Hong Kong (1999), followed by Singapore (2000), Taiwan (2001), Japan (2003), and the Republic of Korea (2008), although the evidence for a definite starting year for Singapore was limited because of relatively small suicide numbers. Evidence for an association between the charcoal-burning suicide rate and the overall suicide rate varied by country—an association was found in Hong Kong, Taiwan, and Japan (for females), but not in Japan (for males), the Republic of Korea, and Singapore. Combined numbers of charcoal-burning suicides for the five study countries reached a peak in 2009, resulting in around 6,800 deaths. Compared to the baseline levels prior to the increase in charcoal-burning suicide in individual countries, by 2011 there were nearly 50,000 excess suicides by charcoal burning in total. Annual rates of changes in charcoal-burning suicide rates did not differ by sex/age group in Taiwan and Hong Kong, whilst people aged 15–24 y in Japan and people aged 25–64 y in the Republic of Korea tended to have the greatest rates of increase.
Our data showed that the increases in charcoal-burning suicide were associated with various levels of changes in overall suicide rates across the East/Southeast Asian countries studied. The increase in charcoal-burning suicide first started in Hong Kong and Taiwan, where it was associated with a rise in overall suicide rates, followed by an increase in Japan, where the rise in overall suicide was less obvious, and then in the Republic of Korea, where the increase in charcoal-burning suicide was very recent, and further investigation of its association with any changes in overall suicide rate is needed. In contrast, Singapore had a much smaller rise in charcoal-burning suicide than other countries did.
Strengths and Limitations
To the best of our knowledge, this is the first systematic investigation of the rise in the popularity of charcoal burning as a method of suicide across East/Southeast Asian countries. There are several limitations to this study. First, we were not able to differentiate charcoal-burning suicides from deaths using other sources of non-domestic gas, such as car exhaust fumes or hydrogen sulphide. However, previous studies found that charcoal-burning suicide accounted for over 90% of all suicides from non-domestic gas poisoning in Hong Kong  and Taiwan , and there has been no report of a prominent rise in gassing suicide using car exhaust fumes in the East/Southeast Asian countries studied. One recent investigation of Japanese suicide data collected by the police showed that there were 2,726 charcoal-burning suicides in 2007, accounting for 8.2% of all suicides ; this finding is similar to our results based on certified suicides coded as poisoning using all non-domestic gases (3,044 charcoal-burning suicides, accounting for 9.9% of all suicides). A marked rise in suicide by hydrogen sulphide poisoning may have accounted for part of the increase in suicide by non-domestic gas poisoning in Japan during the time period 2008–2010 ,; however, the rise in charcoal-burning suicide in 2003 was 5 y before the increase in hydrogen sulphide suicide and was consistent with previous reports of the emergence of charcoal-burning suicide in Japan .
Second, the quality of suicide data may vary by country and change over time. One important factor of under-reporting is the misclassification of suicides as deaths of other causes, such as death of undetermined intent ,–. Our main analyses included both suicides and deaths coded as undetermined intent, and findings were similar when data only for certified suicides were used. In addition, death investigation systems, procedures, and practices differ across countries; this may have influenced the reliability of suicide statistics. In Hong Kong and Singapore, which both follow the British common law system, the cause of death for all unnatural deaths is determined by the coroner's court based on evidence collected by the police ,. However, the use of codes of undetermined intent appeared to be more common in Singapore than in Hong Kong (the undetermined death/suicide ratio was 0.02 in Hong Kong and 0.31 in Singapore in 2011), suggesting greater under-reporting of suicide in Singapore. In Taiwan, Japan, and the Republic of Korea, which all use the continental law system, there is no coroner's inquest for suicide, but all unnatural deaths have to undergo a medico-legal investigation to rule out the possibility of homicide ,. Suicide estimates in these five countries are considered to be reliable according to the rating scheme of the WHO . Suicide statistics are subject to under-reporting and misclassification in Malaysia, the Philippines, and Thailand, where the quality of suicide registration is not satisfactory –.
Third, we did not include data for some East/Southeast Asian countries where cases of charcoal-burning suicide were also reported recently, such as China  and Macao ; detailed method-specific data for suicide were unavailable for these countries. Last, we did not include any covariates in our models to investigate other factors that may have influenced suicide rates. However, the main aim of this study was to identify changes in charcoal-burning suicide rates and their associations with overall suicide rates; the simple linear trend models used in this study, i.e., joinpoint regression and negative binomial models, are appropriate for this purpose.
Possible Explanations of International Variations
Several factors may have contributed to differences in the timing, scale, and sex/age pattern of rises in charcoal-burning suicide amongst East/Southeast Asian countries. How and the extent to which the media reported the first or first few cases of charcoal-burning suicide may have played an important role in the adoption pattern of the method. It has been reported that extensive media reporting of cases was followed by an increase in charcoal-burning suicide in Hong Kong ,, Taiwan , Japan ,, and the Republic of Korea . Hong Kong and Taiwan share the same written language (traditional Chinese characters), and media reports of charcoal-burning suicide in one country could be easily understood by people living in the other country. In Macao and Singapore, where traditional Chinese characters are also commonly used, emerging cases of charcoal-burning suicide were reported as early as 2000–2001 ,. In contrast, Japan and the Republic of Korea use very different languages, and charcoal-burning suicides emerged some years later than those in the aforementioned Chinese societies. Malaysia, the Philippines, and Thailand also use different languages. In Chinese populations, it is commonly believed that preserving the appearance of the body is important for a better next life , and in Japan, death with a “peaceful face” is considered a good death ; these considerations may have made charcoal burning an appealing suicide method in these populations.
Variations in familiarity with coal or charcoal and their accessibility are also possible contributors to the difference between countries in the uptake of this suicide method. In Hong Kong and Taiwan, barbecue charcoal is readily accessible at local shops; such easy availability and familiarity with barbecue charcoal may have contributed to the rapid increases in charcoal-burning suicide in these countries. In contrast, charcoal-burning suicides were usually referred to in Japan as suicides using “rentan” , which was a general term for different kinds of fuels made of coal or charcoal, including the old-fashioned coal briquette, which is not readily available nowadays in Japan . Similarly in the Republic of Korea, the highest profile reported case of charcoal burning used an equivalent rentan-type coal briquette (“yeontan”). The availability of these briquettes has declined in both Japan and the Republic of Korea over the last two decades ,. Overall, there seems to be no evidence of a rapid increase in the availability of barbecue charcoal in the countries we studied over the past decade.
In Hong Kong, charcoal-burning suicides emerged in 1998–1999, following the Asian economic crisis in 1997–1998, which was shown to have a strong impact on Hong Kong's economy and suicide patterns . The economic recession, which led to an increased number of people troubled by debt problems, may have had some role in the increase in charcoal-burning suicide . However, although the Asian economic crisis was also associated with a rise in overall suicide rates in Japan and the Republic of Korea , rises in charcoal-burning suicide did not occur in these two countries until several years later. In Taiwan, the rise and fall of charcoal-burning suicide did not seem to be associated with economic conditions ,. Furthermore, although economic slowdowns may be accompanied by rises in suicide , the impact is not likely to be method-specific (i.e., affecting only trends in charcoal-burning suicide but not suicide using other methods).
Our data showed that—for both the time periods when charcoal-burning suicide rates rose and fell—the rate of change did not differ by sex or age group in Taiwan or Hong Kong. In contrast, young males and females in Japan as well as middle-aged males and young females in the Republic of Korea appeared to have the most rapid increase in charcoal-burning suicide compared to other age groups, although the statistical evidence for age interaction was not significant (p for interaction = 0.064–0.096). This may be related to the characteristics of the initial cases. For example in Japan, young people and teenagers were involved in several widely reported suicide pacts of charcoal-burning suicide ,,.
Our results have several implications for international and regional suicide prevention strategies. First, our findings indicate that it is important to undertake surveillance to identify the emergence of new suicide methods. Such surveillance may include investigating the characteristics of suicide attempts in which such methods are first adopted, and the channels through which information about the new method spreads. Such information will help identify potential measures that may prevent or stop the use of new dangerous methods at an early stage. Second, it is crucial to work with the media and policy makers to restrict graphic descriptions of novel suicide methods and technical information about how to use them. Similarly, it is also important to work with internet service providers to regulate online content containing the details of dangerous methods, which may contribute to the increasing use of new suicide methods not only within but also across country boundaries. Both such measures will limit the cognitive availability of new, high-lethality methods . A recent study of charcoal-burning suicides in England showed that over one-third of the individuals obtained information about the method from the internet . Third, because of cultural similarity and geographical closeness, charcoal-burning suicide may become popular in other East/Southeast Asian countries where charcoal-burning suicide is currently rare. Related agencies and researchers in this area should be mindful of this possibility; they should share and exchange information about surveillance and appropriate policy responses to newly emerging suicide methods. Finally, in areas where the new methods are already widely used, we should consider a range of intervention strategies such as means restriction , gate keeper training , and improving mental health and help seeking in high-risk groups .
Simplified Chinese translation of the abstract by SSC.
Traditional Chinese translation of the abstract by SSC.
Japanese translation of the abstract by AH.
Korean translation of the abstract by WJL.
Time trends in suicide rates: overall suicide, charcoal-burning suicide, and suicide by other methods (using certified suicide cases).
Time trends in suicide rates: charcoalburning suicide, suicide by other methods, and overall suicide, with linear trends from joinpoint regression analysis. Arrows indicate the years when charcoal-burning suicides started to increase.
Time trends in suicide rates by method, with linear trends from joinpoint regression analysis (using certified suicide cases). Arrows indicate the years when charcoal-burning suicides started to increase.
Number, percent, and rate per 100,000 of charcoal-burning suicide (based on certified suicide cases only).
Summary of the mean annual increases in suicide rate (based on certified suicide cases only) per 100,000 and join points for time trends in five East/ Southeast Asian countries, 1995–2011.
Summary of the mean annual increases in suicide rate per 100,000 and join points for time trends, accounting for autocorrelation, in five East/Southeast Asian countries, 1995–2011.
Conceived and designed the experiments: SSC YYC PSFY WJL AH DG. Performed the experiments: SSC YYC PSFY WJL AH DG. Analyzed the data: SSC PSFY. Contributed reagents/materials/analysis tools: SSC YYC PSFY DG. Wrote the first draft of the manuscript: SSC YYC. Contributed to the writing of the manuscript: SSC YYC PSFY WJL AH DG. ICMJE criteria for authorship read and met: SSC YYC PSFY WJL AH DG. Agree with manuscript results and conclusions: SSC YYC PSFY WJL AH DG.
- 1. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, et al. (2012) Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380: 2095–2128.
- 2. Hawton K, van Heeringen K (2009) Suicide. Lancet 373: 1372–1381. doi: 10.1016/s0140-6736(09)60372-x
- 3. Gunnell D, Fernando R, Hewagama M, Priyangika WD, Konradsen F, et al. (2007) The impact of pesticide regulations on suicide in Sri Lanka. Int J Epidemiol 36: 1235–1242. doi: 10.1093/ije/dym164
- 4. Thomas K, Chang SS, Gunnell D (2011) Suicide epidemics: the impact of newly emerging methods on overall suicide rates- a time trends study. BMC Public Health 11: 314. doi: 10.1186/1471-2458-11-314
- 5. Yip PS, Caine E, Yousuf S, Chang SS, Wu KC, et al. (2012) Means restriction for suicide prevention. Lancet 379: 2393–2399. doi: 10.1016/s0140-6736(12)60521-2
- 6. Kreitman N (1976) The coal gas story. United Kingdom suicide rates, 1960–71. Br J Prev Soc Med 30: 86–93. doi: 10.1136/jech.30.2.86
- 7. Chang SS, Lu TH, Eddleston M, Konradsen F, Sterne JAC, et al. (2012) Factors associated with the decline in suicide by pesticide poisoning in Taiwan: a time trend analysis, 1987–2010. Clin Toxicol 50: 471–480. doi: 10.3109/15563650.2012.688835
- 8. Lubin G, Werbeloff N, Halperin D, Shmushkevitch M, Weiser M, et al. (2010) Decrease in suicide rates after a change of policy reducing access to firearms in adolescents: a naturalistic epidemiological study. Suicide Life Threat Behav 40: 421–424. doi: 10.1521/suli.2010.40.5.421
- 9. Rodriguez Andres A, Hempstead K (2011) Gun control and suicide: the impact of state firearm regulations in the United States, 1995–2004. Health Policy 101: 95–103. doi: 10.1016/j.healthpol.2010.10.005
- 10. Liu KY, Beautrais A, Caine E, Chan K, Chao A, et al. (2007) Charcoal burning suicides in Hong Kong and urban Taiwan: an illustration of the impact of a novel suicide method on overall regional rates. J Epidemiol Community Health 61: 248–253. doi: 10.1136/jech.2006.048553
- 11. Lin JJ, Lu TH (2008) Suicide mortality trends by sex, age and method in Taiwan, 1971–2005. BMC Public Health 8: 6. doi: 10.1186/1471-2458-8-6
- 12. Chang SS, Gunnell D, Wheeler BW, Yip P, Sterne JA (2010) The evolution of the epidemic of charcoal-burning suicide in Taiwan: a spatial and temporal analysis. PLoS Med 7: e1000212. doi: 10.1371/journal.pmed.1000212
- 13. Xiong F, Guo Y, Liu X, Chen S, Cai J (2012) [Charcoal burning suicide occurring indoor in summer time: a case report.]. Fa Yi Xue Za Zhi 28: 310.
- 14. Kamizato E, Yoshitome K, Yamamoto Y, Iwase T, Tsuda T, et al. (2009) Factors affecting the choice of suicide method in Okayama: a database analysis from a forensic perspective. Acta Med Okayama 63: 177–186.
- 15. Ho CW, Ho CY, Wong F, Choi KM, Law CC (2008) Suicide and attempted suicide in Macao: a retrospective study. Health Sci J Macao 8: 92–94.
- 16. Free Malaysia Today News (2013 January 2) Family in Pengerang commits suicide. Available: http://www.freemalaysiatoday.com/category/nation/2013/01/02/family-in-pengerang-commits-suicide/. Accessed 31 July 2013.
- 17. Chia BH, Chia A, Ng WY, Tai BC (2011) Suicide methods in Singapore (2000–2004): types and associations. Suicide Life Threat Behav 41: 574–583. doi: 10.1111/j.1943-278x.2011.00055.x
- 18. Huh GY, Jo GR, Kim KH, Ahn YW, Lee SY (2009) Imitative suicide by burning charcoal in the southeastern region of Korea: the influence of mass media reporting. Leg Med (Tokyo) 11: S563–S564. doi: 10.1016/j.legalmed.2009.01.099
- 19. World Health Organization (2012) WHO Mortality Database. Available: http://www.who.int/healthinfo/statistics/mortality_rawdata/en/index.html. Accessed 29 July 2013.
- 20. United Nations Statistics Division (2011) Composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings. New York: United Nations.
- 21. Chung WS, Leung CM (2001) Carbon monoxide poisoning as a new method of suicide in Hong Kong. Psychiatr Serv 52: 836–837. doi: 10.1176/appi.ps.52.6.836
- 22. Chan KP, Yip PS, Au J, Lee DT (2005) Charcoal-burning suicide in post-transition Hong Kong. Br J Psychiatry 186: 67–73. doi: 10.1192/bjp.186.1.67
- 23. Immigration and Checkpoints Authority Singapore Government (2013) Register death. Available: http://www.ica.gov.sg/page.aspx?pageid=188#statistics. Accessed 22 June 2013.
- 24. Hendin H, Vijayakumar L, Bertolote JM, Wang H, Phillips MR, et al. (2008) Epidemiology of suicide in Asia. Geneva: World Health Organization.
- 25. Maniam T (1995) Suicide and undetermined violent deaths in Malaysia, 1966–1990: evidence for the misclassification of suicide statistics. Asia Pac J Public Health 8: 181–185. doi: 10.1177/101053959500800307
- 26. Redaniel MT, Lebanan-Dalida MA, Gunnell D (2011) Suicide in the Philippines: time trend analysis (1974–2005) and literature review. BMC Public Health 11: 536. doi: 10.1186/1471-2458-11-536
- 27. Thomyangkoon P, Leenaars AA, Wasserman D (2005) Suicide in Thailand, 1977 to 2002. Arch Suicide Res 9: 361–368. doi: 10.1080/13811110500248066
- 28. United Nations Population Division (2013) World population prospects, the 2012 revision [database]. Available: http://esa.un.org/wpp/ASCII-Data/DISK_NAVIGATION_ASCII.htm. Accessed 31 July 2013.
- 29. Hitosugi M, Nagai T, Tokudome S (2009) Proposal of new ICD code for suicide by charcoal burning. J Epidemiol Community Health 63: 862–863. doi: 10.1136/jech.2008.086025
- 30. Lin JJ, Chen LH, Huang SM, Lu TH (2008) Problems in estimating the number of suicides by charcoal burning in Taiwan. J Epidemiol Community Health 62: 566. doi: 10.1136/jech.2007.065771
- 31. Morii D, Miyagatani Y, Nakamae N, Murao M, Taniyama K (2010) Japanese experience of hydrogen sulfide: the suicide craze in 2008. J Occup Med Toxicol 5: 28. doi: 10.1186/1745-6673-5-28
- 32. Chang SS, Sterne JA, Lu TH, Gunnell D (2010) ‘Hidden’ suicides amongst deaths certified as undetermined intent, accident by pesticide poisoning and accident by suffocation in Taiwan. Soc Psychiatry Psychiatr Epidemiol 45: 143–152. doi: 10.1007/s00127-009-0049-x
- 33. Gunnell D, Bennewith O, Simkin S, Cooper J, Klineberg E, et al. (2013) Time trends in coroners' use of different verdicts for possible suicides and their impact on officially reported incidence of suicide in England: 1990–2005. Psychol Med 43: 1415–1422. doi: 10.1017/s0033291712002401
- 34. Linsley KR, Schapira K, Kelly TP (2001) Open verdict v. suicide—importance to research. Br J Psychiatry 178: 465–468. doi: 10.1192/bjp.178.5.465
- 35. Ohberg A, Lonnqvist J (1998) Suicides hidden among undetermined deaths. Acta Psychiatr Scand 98: 214–218. doi: 10.1111/j.1600-0447.1998.tb10069.x
- 36. Ahmad O, Boschi-Pinto C, Lopez AD, Murray CJL, Lozano R, et al. (2000) Age standardization of rates: a new WHO standard. Geneva: World Health Organization.
- 37. Oliver SE, May MT, Gunnell D (2001) International trends in prostate-cancer mortality in the “PSA ERA”. Int J Cancer 92: 893–898. doi: 10.1002/ijc.1260
- 38. Vichi M, Masocco M, Pompili M, Lester D, Tatarelli R, et al. (2010) Suicide mortality in Italy from 1980 to 2002. Psychiatry Res 175: 89–97. doi: 10.1016/j.psychres.2008.12.008
- 39. Chang SS, Gunnell D, Sterne JAC, Lu TH, Cheng ATA (2009) Was the economic crisis 1997–1998 responsible for rising suicide rates in East/Southeast Asia? A time-trend analysis for Japan, Hong Kong, South Korea, Taiwan, Singapore and Thailand. Soc Sci Med 68: 1322–1331. doi: 10.1016/j.socscimed.2009.01.010
- 40. Tamosiunas A, Reklaitiene R, Virviciute D, Sopagiene D (2006) Trends in suicide in a Lithuanian urban population over the period 1984–2003. BMC Public Health 6: 184. doi: 10.1186/1471-2458-6-184
- 41. Stark C, Stockton D, Henderson R (2008) Reduction in young male suicide in Scotland. BMC Public Health 8: 80. doi: 10.1186/1471-2458-8-80
- 42. Gagne M, Robitaille Y, Hamel D, St-Laurent D (2010) Firearms regulation and declining rates of male suicide in Quebec. Inj Prev 16: 247–253. doi: 10.1136/ip.2009.022491
- 43. Bando DH, Brunoni AR, Bensenor IM, Lotufo PA (2012) Suicide rates and income in Sao Paulo and Brazil: a temporal and spatial epidemiologic analysis from 1996 to 2008. BMC Psychiatry 12: 127. doi: 10.1186/1471-244x-12-127
- 44. Reseland S, Bray I, Gunnell D (2006) Relationship between antidepressant sales and secular trends in suicide rates in the Nordic countries. Br J Psych 88: 354–358. doi: 10.1192/bjp.188.4.354
- 45. US National Cancer Institute Surveillance Research Program Statistical Methodology and Applications Branch and Data Modeling Branch (2013) Joinpoint Regression Program, version 4.0.4 [computer program]. Available: http://surveillance.cancer.gov/joinpoint/. Accessed 24 February 2014.
- 46. Lerman PM (1980) Fitting segmented regression models by grid search. J R Stat Soc Ser C Appl Stat 29: 77–84. doi: 10.2307/2346413
- 47. Kim HJ, Fay MP, Feuer EJ, Midthune DN (2000) Permutation tests for joinpoint regression with applications to cancer rates. Stat Med 19: 335–351. doi: 10.1002/(sici)1097-0258(20000215)19:3<335::aid-sim336>3.0.co;2-z
- 48. Chang SS, Page A, Gunnell D (2011) Internet searches for a specific suicide method follow its high-profile media coverage. Am J Psychiatry 168: 855–857. doi: 10.1176/appi.ajp.2011.11020284
- 49. Law CK, Yip SFP (2008) Suicide in Hong Kong. In: Yip SFP, editor. Suicide in Asia: causes and prevention. Hong Kong: Hong Kong University Press. pp. 49–60.
- 50. Chia B-H, Chia A (2008) Suicide in Singapore. In: Yip SFP, editor. Suicide in Asia: causes and prevention. Hong Kong: Hong Kong University Press. pp. 101–120.
- 51. Chen YY, Chen F, Gunnell D, Yip PS (2013) The impact of media reporting on the emergence of charcoal burning suicide in Taiwan. PLoS ONE 8: e55000. doi: 10.1371/journal.pone.0055000
- 52. Lee DT, Chan KP, Yip PS (2005) Charcoal burning is also popular for suicide pacts made on the internet. BMJ 330: 602. doi: 10.1136/bmj.330.7491.602-b
- 53. Ozawa-De Silva C (2010) Shared death: self, sociality and internet group suicide in Japan. Transcult Psychiatry 47: 392–418. doi: 10.1177/1363461510370239
- 54. Fu KW, Chan CH (2013) A study of the impact of thirteen celebrity suicides on subsequent suicide rates in South Korea from 2005 to 2009. PLoS ONE 8: e53870. doi: 10.1371/journal.pone.0053870
- 55. Long SO (2001) Negotiating the “good death”: Japanese ambivalence about new ways to die. Ethnology 4: 271–289. doi: 10.2307/3773877
- 56. Kamijo Y (2006) [Why should we discuss CO poisoning now?]. Chudoku Kenkyu 19: 11–12.
- 57. Wikipedia Japan (2013) [Rentan.] Available: http://ja.wikipedia.org/wiki/%E7%B7%B4%E7%82%AD. Accessed 22 June 2013.
- 58. Wikipedia (2013) Yeontan. Available: http://en.wikipedia.org/wiki/Yeontan. Accessed 22 June 2013.
- 59. Korea Energy Statistics Institute (2012) Yearbook of energy statistics. Gyeonggi-do (Republic of Korea): Korea Energy Statistics Institute.
- 60. Chang SS, Sterne JA, Huang WC, Chuang HL, Gunnell D (2010) Association of secular trends in unemployment with suicide in Taiwan, 1959–2007: a time-series analysis. Public Health 124: 49–54. doi: 10.1016/j.puhe.2009.11.005
- 61. Chen YY, Yip PS, Lee C, Fan HF, Fu KW (2010) Economic fluctuations and suicide: a comparison of Taiwan and Hong Kong. Soc Sci Med 71: 2083–2090. doi: 10.1016/j.socscimed.2010.09.043
- 62. Hitosugi M (2005) Rapid response: trend in suicide pacts made via the Internet in Japan. BMJ 330: 602.3 Available: http://www.bmj.com/content/330/7491/602.3?tab=responses. Accessed 27 February 2014.
- 63. Hagihara A, Miyazaki S, Abe T (2012) Internet suicide searches and the incidence of suicide in young people in Japan. Eur Arch Psychiatry Clin Neurosci 262: 39–46. doi: 10.1007/s00406-011-0212-8
- 64. Florentine JB, Crane C (2010) Suicide prevention by limiting access to methods: a review of theory and practice. Soc Sci Med 70: 1626–1632. doi: 10.1016/j.socscimed.2010.10.001
- 65. Chen YY, Bennewith O, Hawton K, Simkin S, Cooper J, et al. (2013) Suicide by burning barbecue charcoal in England. J Public Health (Oxf) 35: 223–227. doi: 10.1093/pubmed/fds095
- 66. Yip PS, Law CK, Fu KW, Law YW, Wong PW, et al. (2010) Restricting the means of suicide by charcoal burning. Br J Psychiatry 196: 241–242. doi: 10.1192/bjp.bp.109.065185
- 67. Wong PW, Liu PM, Chan WS, Law YW, Law SC, et al. (2009) An integrative suicide prevention program for visitor charcoal burning suicide and suicide pact. Suicide Life Threat Behav 39: 82–90. doi: 10.1521/suli.2009.39.1.82
- 68. Yip PS, Lee DTS (2007) Charcoal-burning suicides and strategies for prevention. Crisis 28 (Suppl 1) 21–27. doi: 10.1027/0227-5910.28.s1.21