Figures
Abstract
Background and objective
Non-suicidal self-injury (NSSI) is a psychological disorder that the sufferer consciously damages their body tissues, often too severe that requires intensive care medicine. As some individuals hide their NSSI behaviors, other people can only identify them if they catch them while injuring, or via dedicated questionnaires. However, questionnaires are long and tedious to answer, thus the answers might be inconsistent. Hence, in this study for the first time, we abstracted a larger questionnaire (of 662 items in total) to own only 22 items (questions) via data mining techniques. Then, we trained several machine learning algorithms to classify individuals based on their answers into two classes.
Methods
Data from 277 previously-questioned participants is used in several data mining methods to select features (questions) that highly represent NSSI, then 245 different people were asked to participate in an online test to validate those features via machine learning methods.
Results
The highest accuracy and F1 score of the selected features–via the Genetics algorithm–are 80.0% and 74.8% respectively for a Random Forest algorithm. Cronbach’s alpha of the online test (validation on the selected features) is 0.82. Moreover, results suggest that an MLP can classify participants into two classes of NSSI Positive and NSSI Negative with 83.6% accuracy and 83.7% F1-score based on the answers to only 22 questions.
Conclusion
While previously psychologists used many combined questionnaires to see whether someone is involved in NSSI, via various data mining methods, the present study showed that only 22 questions are enough to predict if someone is involved or not. Then different machine learning algorithms were utilized to classify participants based on their NSSI behaviors, among which, an MLP with 10 hidden layers had the best performance.
Citation: Farajzadeh N, Sadeghzadeh N (2023) NSSI questionnaires revisited: A data mining approach to shorten the NSSI questionnaires. PLoS ONE 18(4): e0284588. https://doi.org/10.1371/journal.pone.0284588
Editor: Muhammad Fazal Ijaz, Sejong University, REPUBLIC OF KOREA
Received: November 17, 2022; Accepted: April 2, 2023; Published: April 21, 2023
Copyright: © 2023 Farajzadeh, Sadeghzadeh. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The first dataset which is used in the manuscript is available in "Group Separation and Classification of Non-Suicidal Self-Injury in a University Student Population." Researchers may access it by emailing the address: "scholarship@uwindsor.ca" to request it. The second dataset, which was collected as a part of conducting this study, can be downloaded from: https://datadryad.org/stash/share/F6EfdEsxOqBdDzlEfZZQFHkjWNBz4vkKgVjjMYLcT0c.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Non-suicidal self-injury (NSSI) is a psychological disorder that causes the sufferer to intentionally damage the tissues of their body, which are often severe and require intensive care medicine, without intending to be dead [1]. People of any age can be involved in this disorder; however, it is more prevalent in adolescents and youngsters [2]. It is said that the onset age of the involvement in the disorder usually varies from 12 to 14 [3–7]. Although NSSI does not intend to end one’s life, people who are involved in NSSI show a higher desire for committing suicide [5, 8] as their pain tolerance increases and they can be more ruthless with themselves [9]. Hence, NSSI is the second potential leading cause of death among people who are between 15 to 19 years old, and the 10th among those between 10 to 14 years old [10]. Clearly, this disorder has many negative impacts on the sufferers’ lifestyles, and therefore, identifying people who could be or become involved with NSSI can play an important role in controlling, treating, and preventing it in their older ages [11–13].
Although ample research has been conducted to identify people with NSSI planning and those without it, recent analyses show that our ability in doing so has been quite limited since only a few factors in people’s lives have been considered, not all of them [14]. Consequently, the prognosis of NSSI is still not accurate [15]. NSSI and suicidal behaviors can be differentiated by (but not limited to) nine major factors: intent, lethality, chronicity, methods and functions, cognitions, reactions, aftermath, demographics, and prevalence [5]. While NSSI is primarily intended to eliminate 1) negative emotion, 2) body alienation (dissatisfaction), and 3) dissociation, suicide tends to be a way out of all of them at once when the injurers are not satisfied with the deceptive temporary pain caused by injuries, nor are they cured via therapies [1, 5, 16–23].
NSSI is a complex concept due to various intertwined factors that contribute to its conception in one’s mind, and this complexity hinders psychologists from a clear understanding of the factor(s) they should fix in an injurer’s life [24, 25]. This makes it more likely that the majority of the injurers will still be involved in NSSI in different ways as no one can thoroughly take into consideration their needs and motivations [24, 25]. Researchers in this field have come up with a comprehensive definition of NSSI in the last 10 to 15 years; this indicates that historically, knowledge about NSSI was limited and just started to develop [26].
Despite these shortcomings, some dedicated psychological questionnaires help to identify individuals who are currently involved in NSSI (see Measures). However, it is shown that due to some latent relations among the questions, the combination of them describes involvement in NSSI better than using them separately [27]. One of the limitations the questionnaires convey is the numerous questions that a person must answer to be diagnosed. Answering a large number of questions is clearly very time-consuming and tedious so the results are likely inaccurate due to the frustration of the participants while answering them [27]. Therefore, many people answer them carelessly or inattentively. One reason that we still have to stick to the questionnaires is that some injurers hide their behaviors so that they are not objected to [7]. Therefore, if we are to identify such injurers without any questionnaires, we can only do so if they are caught injuring or they make blunders.
Hence, it is needed that academia extracts smaller subset(s) of the original questionnaire to 1) not let the participants become tired so that the answers tend to be more consistent, 2) omit the questions that explicitly ask about the participants’ involvement with NSSI so that participants will not answer with lies, and 3) omit the questions that may shift the attention of psychologists from actual facts about the participants’ involvement with NSSI to less important ones.
Traditionally, psychologists subjectively related one’s lifestyle and mentality to their involvement with NSSI, whereas Artificial Intelligence (AI)-based approaches hold promising results while noticing NSSI involvement [28]. Furthermore, AI approaches enable us to simultaneously consider multiple different factors, such as age, gender, etc., at the same time [29]. AI and machine learning fields of study try to enable computers to mimic living species’ neural systems so that they can also perceive and/or behave as living species do [30–32]. For example, an artificial spider walking like a real one [33], or an AI-based model locating cancer nuclei on histopathology images like an expert does [34].
AI-based diagnosis methods have seen particular attention in medical and healthcare systems as they can often become a life-saving tool for people [34–40]. For example, since data from Alzheimer’s Disease (AD) are multimodal and time series in nature, it is a burden for specialists to handle all the data that their patients provide; thus, they may not be able to correctly decide on the outcomes [38]. Hence, state-of-the-art AI-backed approaches can be utilized to predict AD progression 2.5 years in the future, based on the multimodal time-series data [38]. In another related work, a mobile application was introduced to detect skin disease via AI [39]. The method used MobileNet-V2 and Long Short-Term Memory (LSTM) to make an algorithm that could understand complex patterns on skin and maintain stateful information for precise predictions.
As a combination of wearable sensors and social networking, a healthcare monitoring framework based on the cloud environment and a big data analytics engine is available that stores and analyzes healthcare data provided by sensors or social networks [40]. The system classifies the healthcare data to predict drug side effects and abnormal conditions in patients. That also classifies patients’ health conditions using their healthcare data that are related to diabetes, blood pressure, mental health, and drug reviews. In another similar work, Bluetooth Low Energy (BLE)-based sensors are to gather users’ vital signs data such as blood pressure, heart rate, weight, as well as blood glucose and to send them to smartphones. Then, AI-based methods are to provide early prediction of diabetes given the user’s sensor data as input [37].
Therefore, healthcare data classification via AI provides the quickest and the most accurate automated approaches for medical centers as well as in-home diagnosis systems. These methods remove the subjectivity of specialists and try to understand and map the input values to unbiased outputs [35]. In the current case, for instance, similar methods can input data representing people’s behaviors, and decide whether they are involved in NSSI or not [28]. These methods could potentially mitigate some of the issues described earlier and can be delivered relatively quickly.
Considering these, for the first time, we aimed to summarize 17 different questionnaires (of 662 questions in total) to only 22 questions (items) using feature selection and data mining techniques. Then, we build an AI approach that classifies people with either NSSI positive or negative with 83.6% accuracy by deciding on the answers to these questions. The model can be used as a decision support system (DSS) to help psychologists quickly identify their patients with acceptable accuracy. In summary, this paper has five aspects of novelty:
- Reduces the 667-item questionnaire for NSSI identification to only a 22-item one.
- Proposes the fastest approach for identifying people involved in NSSI via Machine Learning and data mining techniques.
- Validates the abstracted questionnaire via two different statistical populations.
- Gives insight into utilizing Data Mining methods to abstract large questionnaires.
- Provides a public dataset for future studies in similar cases.
The remainder of the paper is organized as follows: information about participants, measures, and data preparation, as well as the utilized machine learning and data mining methods are available in Materials and methods. The results of the feature selection, abstracted questionnaire, and validation are available in Results, while the corresponding discussions are available in the Discussion section. Finally, to conclude the paper, the Conclusion section is available to summarize the paper and mention a limitation and a prospect.
Materials and methods
Participants
The present study builds on the research conducted by Jann MacIsaac [27]. Their participants were students of the University of Windsor, Canada, recruited by Jann MacIsaac. Their data collection process began in September 2017 and ended in April 2018. Initially, 314 participants were selected for their study, but 277 participants actually were included in the final dataset. Among them, 194 people did not have NSSI disorder, 44 were NSSI-Distal, and 39 were NSSI-Proximal. All of these 277 participants’ data is used in the current feature selection phase to choose the questions which are highly related to NSSI. Note that the dataset by Jann MacIsaac was not publicly available at the time of conducting this research (2019–2022). Thus, further information about it is not provided. Please contact them for more information.
After selecting features (questions) from the provided data in the first phase, an online website is created to test and validate the chosen features. Thus, 409 different participants from Azarbaijan Shahid Madani University, Tabriz, Iran are recruited. They were familiarized with the procedure in a virtual meeting and verbally agreed to participate (all data are kept anonymous with no traceable information. Those who did not consent to participate, did not provide any data; hence, the need for documented consent was not required as participants could deny participation if they did not consent to participate). All procedures performed in this study are in accordance with the ethical standards of the Institutional Review Board (IRB) of Azarbaijan Shahid Madani University and are reviewed and approved before the study began. After filtering the careless and inattentive participants, data from 245 participants is included in the second phase, whose ages vary from 18 to 35 (M = 27.88, SD = 9.26). The sample consisted of 152 females (62.04%) and 93 males (37.96%), whose demographics are presented in Table 1.
Measures
Jann MacIsaac included 19 questionnaires in their datasets, descriptions of which are as follows:
- Demographic questionnaire: The demographic information of all participants including age, gender, race/ethnicity, marital status, university year of enrollment, faculty, employment status, current residence, GPA, and meditation.
- Deliberate Self-Harm Inventory (DSHI): A 16-item self-report questionnaire to assess whether participants have ever performed a particular self-injurious behavior or not [6].
- Inventory of Statements About Self-Injury (ISAS): A two-part questionnaire, the first section of which assesses the lifetime frequency of 12 NSSI behaviors performed intentionally (i.e., on purpose) and without suicidal intent. Its second section assesses 13 potential functions of NSSI: affect-regulation, anti-dissociation, anti-suicide, autonomy, interpersonal boundaries, interpersonal influence, marking distress, peer-bonding, self-care, self-punishment, revenge, sensation seeking, and toughness [41].
- Risk-Taking 18 (RT-18): An 18-item questionnaire that is used to assess adults’ overall risky behaviors. This questionnaire has two levels: risk-taking and risk assessment. RT-18 sums all of the answers of a patient; the higher they score, the higher their level of risk-taking or risk assessment [42].
- Difficulties in Emotion Regulation Scale 18 (DERS-18): An 18-item questionnaire designed to assess clinical problems related to emotion regulation. The questionnaire consists of six main subgroups: awareness of personal emotions, transparency about personal emotions, acceptance of personal emotions, access to emotion regulation strategies, ability to participate in purposeful behaviors when exposed to negative emotions, and ability to manage impulses generated during negative emotions [43].
- Toronto Alexithymia Scale 20 (TAS-20): A 20-item questionnaire that assesses difficulty identifying and describing between feelings and bodily sensation, difficulty describing feelings, reduced daydreaming, and externally-oriented thinking [44].
- Positive and Negative Affect Schedule (PANAS): A 20-item self-report questionnaire designed to assess both positive affect and negative affect moods [45].
- UPPS-P Impulsive Behavior Scale (UPPS-P): A 59-item questionnaire that measures five distinct facets of impulsivity: negative urgency, positive urgency, sensation seeking, lack of perseveration, and lack of premeditation [46, 47].
- NIH Flanker Inhibitory Control and Attention Test: Assesses participants’ executive function and attention [27].
- Mindful Attention and Awareness Scale (MAAS): A 15-item questionnaire that assesses individual differences in the frequency of mindful states over time [48].
- Patient Health Questionnaire-9 (PHQ-9): A nine-item questionnaire that assesses participants’ depression [49].
- Generalized Anxiety Disorder-7 (GAD-7): A seven-item questionnaire that assesses participants’ general anxiety [50].
- Perceived Stress Scale (PSS): A 14-item questionnaire that assesses the level of tolerance of individuals (as a representative of perceived stress), when they are in a stressful situation [51].
- Big Five Inventory (BFI): A 44-item questionnaire that examines a person’s personality in five aspects: agreeableness, conscientiousness, extraversion, openness, and neuroticism [52].
- Connor-Davidson Resilience Scale-25 (CD-RISC-25): A 25-item questionnaire that assesses participants’ ability to deal with stress and adversity [53].
- Social Provision Scale (SPS): A 24-item questionnaire that assesses participants’ level of social understanding. It describes six different social functions or provisions that may be received from relationships with others: guidance, reliable alliances, reassurance of worth, attachment, social integration, and the opportunity for nurturance [54].
- Inventory of College Students’ Recent Life Experiences (ICSRLE): A 49-item questionnaire that assesses participants’ recent stressful events [55].
- Self-Compassion Scale Short-Form (SCS-SF): A 12-item self-report questionnaire that assesses participants’ feelings of self-compassion [56].
The datasets also contain validation questions (i.e., reversed items) to test the accuracy of the participants’ answers. The total number of available features (questions) is 662.
Data filtering and pre-processing
In the original dataset, the optional questions, descriptive questions (i.e., “how is your hometown”), and the non-quantitative ones (such as those of NIH) are discarded. This way, the 662 features were reduced to 403. The analyses for the second phase of the test included filtering the outliers and calculating IRV (intra-individual response variability) index [57] across the 28 items (including the reversed items; see Validation and building a DSS model) to which participants responded. All of the analyses are performed using Python and libraries such as Pandas (https://pandas.pydata.org), SciPy (https://scipy.org), Matplotlib (https://matplotlib.org), Pingouin (https://pingouin-stats.org), and Seaborn (https://seaborn.pydata.org). Respondents with very low IRV values were excluded from analyses. Additionally, we excluded the participants whose differences between the reversed items and their original analogies were more than or equal to two. Lastly, we excluded any participants that took more than 20 or less than five minutes to complete the whole experiment, as our piloting suggested that the average time to complete the test was approximately 10 minutes.
Feature selection and supervisor algorithms
The popular evolutionary algorithms, such as Genetics, Ant Colony (AC), and Particle Swarm (PS) algorithms as well as Linear Support Vector Classifier (LSVC), are used to select features [58] while nine machine learning algorithms, random forests (RF), support vector machine (SVM) with regularization parameter (C) = 1.1, decision tree (DT), k-nearest neighbor (kNN), perceptron, multilayer perceptron (MLP) with 10 hidden layers, the neurons of which are 80% more than the input size, linear discriminant analysis (LDA), AdaBoost (AB), and Long-Short Term Memory (LSTM) network, were to supervise them. Other default parameters are not changed. Lastly, the output neurons of each model conform to the number of classes (see Feature selection for more information).
The former algorithms (feature selectors) select varying subsets of questions in each epoch and have them learned by the latter algorithms. Supervisors (latter algorithms) are trained on the answers to those questions learning to map them to a particular NSSI class. It is clear that if the supervisors achieve acceptable evaluation results on a particular subset, the items in the subset highly correlate with NSSI; and thus, can be excellent representatives of NSSI involvement [58].
The LSTM model has four memory layers with 22 inputs and 22 outputs each, followed by three fully-connected layers with 128, 256, and 64 neurons in their hidden layers. Its last layer has two or three neurons (based on the output classes; see Feature selection for more information). Except for the last layer which has the Soft-Max activation function, others have ReLU.
Evaluation metrics
To evaluate the efficiency of the selected features, the Precision, Recall, F1-score, and Accuracy metrics are used, which are extracted as follows [59, 60]:
(1)
(2)
(3)
(4)
(5)
(6)
In the above relations, |C| represents the total classes, which in our case, either C = {No NSSI, NSSI Distal, NSSI Proximal} or C = {NSSI Neg., NSSI Pos.} (see Feature selection for more information). i shows the class, the metrics of which are being calculated. TPi indicates the number of cases that belong to the ith class and are classified correctly, TNi indicates the number of cases that do not belong to ith class and were not classified as class i either. In contrast, FPi indicates the number of cases that did not belong to class i but were incorrectly classified as class i, and finally, FNi indicates the number of cases that belong to class i but were classified incorrectly. Precision shows how much we can rely on the model when it classifies participants, while recall measures the ability of the model to find all the correct cases in a given dataset. Finally, the F1-score is the weighted harmonic mean of the precision and recall, and it is beneficial while finding the best trade-off between the two quantities [61]. Note that as the approach is a classification problem, statistical tests do not increase the analytical deduction of the study as the lists being tested are comprised of only two or three different discrete numbers. Statistical tests are informative in regression problems that the values are continuous. Hence, the values in the current task can never meet normality (as required by the Central Limit Theorem) to be evaluated via statistical tests.
Results
Feature selection
First, we tested our approach on three classes (No NSSI, NSSI Distal, and NSSI Proximal) without and with data augmentation. We initially tested a different number of features (questions), but only the best and second-best performing ones are reported in Table 2. Also, among the mentioned feature selectors, only Genetics and LSVC had promising results; therefore, others are omitted from comparisons. As it is evident, the DT with the genetics algorithm obtained the highest overall efficiency whose accuracy, precision, recall, and F1-score are 58.9%, 50%, 47.5%, and 48.7% respectively. To improve the results, we augmented data using SMOTE synthetic method [62] then selected features, and illustrated its results in Table 3. In this case, the most efficient result was that of the genetics algorithm with an SVM model, the accuracy, precision, recall, and F1-score of which are 71.4%, 55.1%, 56.5%, and 55.7% respectively. According to the F1-scores, all of these approaches are behaving randomly and are not reliable at all.
To overcome the random behavior, the three classes are reduced to only two, being NSSI Positive (the combination of NSSI Distal and NSSI Proximal) and NSSI Negative. Note that the results of future experiments are only reported based on the genetics algorithm since it was more accurate than LSVC (and others). Table 4 compares the models’ efficiencies considering the initial features and selected features. As is evident, RF obtained the highest efficiency with an accuracy, precision, recall, and F1-score of 80.0%, 73.1%, 76.5%, and 74.8% respectively. To see how data augmentation methods contribute to the accuracy, Table 5 compares the effects of ADASYN, SVM-SMOTE, and SMOTE data augmentation methods, all of which led the feature selector to select 45, 34, and 19 questions respectively. It is evident from the table that ADASYN with DT achieved an accuracy, precision, recall, and F1-score of 81.6%, 81.0%, 80.2%, and 80.6% respectively, whereas those of SVM-SMOTE with LDA were 76.2%, 75.8%, 73.2%, 74.5% respectively; those of SMOTE with LDA were 71.4%, 71.2%, 68.5%, and 69.8%.
The abstracted questions
According to the results in Tables 4 and 5, it is possible to reduce the original questionnaire to a 19- or 22-item one. As the latter showed a better performance compared to the former, we selected the latter as our abstracted questionnaire. Although the former has fewer items, the difference (three questions) is not significant enough to lead the participants to answer carelessly. Hence, we chose performance over conciseness. Although there are statistical ways to see which questionnaire is better, since machine learning algorithms can consider relations between items better than the statistical methods, we opted for the modern (machine learning) solution for this task and thus relied on the provided evaluation metrics. The items in the abstracted questionnaire are selected from nine questionnaires: Demographic Information Questionnaire, TAS [44], PANAS [45], UPPS-P [46, 47], PSS [51], CD-RISC [53], SPS [54], ICSRLE [55], SCS-SF [56]; and are available in Table 6. Although the set of 45 items better represents NSSI behaviors (according to Table 5), its participants may still respond carelessly due to 45 tedious questions; hence this one is also discarded. Despite showing a 74.8% F1-score, the 22 items are still more accurate than a subjective psychologist who is exposed to fatigue and has to assess many participants who might not be honest with their responses in a large and tedious questionnaire.
Since the previous methods divided data to test splits to select features, it is not clear whether the algorithms are under- or over-fitted. Therefore, we used 5-fold cross-validation on the 22 questions with the three best-performing algorithms (also with and without data augmentation) to assure consistency and presented the results in Table 7. Evident from the table is, the results are consistent with the previous table (particularly Table 4) as the difference between them is marginal. That is, the best-performing algorithm is still RF whose evaluation results are similar to the previous table that indicates the significant consistency of the selected features.
Validation and building a DSS model
To validate the abstracted questionnaire, we designed an online website to ask participants to provide data. The website had a description section to make participants familiar with its questionnaire and its aims. We added six reversed items between the abstracted 22 items to assure the answers’ consistency and to be able to filter careless participants. Finally, the Cronbach alpha of the pruned responses is 0.82.
Based on the pruned data, we train eight machine learning models which are to classify participants into two classes (NSSI Positive and NSSI Negative) based on their scores (answers) to each question. The reversed items are not used in this phase to see how well the 22 items can represent NSSI involvement. The performances of the models are available in Table 8 and are calculated using 5-fold cross-validation. Note that we tested the four data augmentation algorithms for each of the methods in Table 8; however, only the best-performing one is reported per each algorithm. It is evident that overall, its results are consistent with those in Tables 4 and 7 since the three tables have negligible differences from each other. This endorses that the selected features are consistence in both statistic populations (the initial dataset and the collected one) and can represent NSSI involvement among different nations. However, it was interesting to see the MLP algorithm could outperform its previous model (that in Table 7) and rank first with an accuracy, precision, recall, and F1-score of 83.6%, 83.6%, 83.9%, and 83.7% respectively. The MLP model has 10 hidden layers and 40 neurons in each layer. Moreover, the RF algorithm shows a very narrow difference from that in Table 7, indicating its overall stability through different datasets. Since MLP outperformed other methods, only its results are visualized in the following. Thus, Fig 1 shows average F1-score fluctuations per epoch in the 5-fold cross-validation, while Fig 2 depicts the normalized average confusion matrix of the MLP model in the same setting.
Discussion
NSSI is a psychological disorder that a person intentionally harms themselves without intending to die [1]. In the present study, an attempt has been made to provide a simpler and more efficient method for predicting NSSI disorder using several basic classification algorithms. For this purpose, after conducting various experiments using data mining and feature selection, an abstracted questionnaire was extracted which is comprised of only 22 questions, only three percent of the initial 662 questions, and an abstract of 17 different questionnaires. Answering these questions only takes about five to six minutes (without including reversed items) which potentially increases confidence and accuracy since participants are not bored while answering. Consequently, a machine learning algorithm can classify a participant with 83.6% accuracy. Our study lets psychologists find people involved in NSSI and start psychotherapeutic treatments as soon as possible. Another convincing reason for considering this study worldwide would be its independence on the questions that potentially could be answered with lies. Some paramount examples of such questions would be those asking participants’ involvement in deliberate self-injuring. The majority of people who are involved in NSSI are shown to be hiding this trait as they wish not to be objected to. Hence, the inclusion of such questions may alter the results and thus, they had better not be included.
Here, we mainly used traditional machine learning methods instead of more complex deep learning-based ones since the former methods are shown to be more robust when they are to consider statistical aspects or relations between different components of the input data on the one hand, and can perform better than the latter approaches when the data is sparse and the inter-item variety is not significant on the other hand [53–60]. Additionally, the used data in this approach are self-explanatory and do not need feature extraction phase(s). Therefore, there is no necessity to use deep learning methods as they usually rely on larger datasets for better performance. However, since the questions were shown to the participants in a static order, we thought that using an LSTM could be beneficial as it can relate the bias of previous questions to the answer of the current one. Yet, results suggested that it was not the case in this approach. We did not also use CNNs because they are inspired by neural behaviors in the eyes’ receptive field and are usually well-performed on visual tasks, such as image detection, image classification, etc. [63]. Additionally, they multiply adjacent values of the input to form filters; thus, the importance of the values will be diminished if used in cases like the one in the present study [63].
As seen in Tables 2 and 4, the inclusion of the whole dataset even confuses a machine learning method. Therefore, psychologists, who are exposed to heavy workloads and often fatigue, are not immune to the confusion and thus, may decide subjectively. However, using feature selection and machine learning algorithms, we proved that the inclusion of a more refined set of questions can be less confusing and aid people in making more accurate decisions. Although some may say that the achieved level of accuracy in the proposed research is still far from an accurate decision made by a psychologist, considering the delivery time and convenience of the provided abstracted questionnaire, it suffices the needs of the current state of psychological expectations. Furthermore, it is still more precise than a subjective psychologist who is exposed to fatigue and has to assess many participants who might not be honest with their responses in (a) large and tedious questionnaire(s).
As supported by other studies, distress (Items 3, 6, 7, 8, 12, 14, 20) [64–69], depression and substance use disorder (SUD) (Items 3, 4, 5, 8, 9, 12, 15, 18, 19, 20, 21) [1, 67, 70–72] are strong predictors of NSSI. Despite existing in questionnaires previously, Emotional Reactivity (ER) has been recently proven to be a contributing factor for NSSI identification (Items 12, 16, 21, 22) [73]. Dissatisfaction with one’s body and trauma (Items 10 and 13) [1, 5, 16, 17, 67, 74] as well as motivation and activity (Items 11, 17, 18) are shown to be correlated with NSSI [13, 75]. Gender (Item 2) is shown to be a strong factor contributing to NSSI and suicidal attempts among people [67, 72]. Age is another factor to be taken into consideration when identifying individuals with NSSI [3–7]; however, our feature selector found that in our dataset, the academic year correlates with NSSI better than the age factor (Item 1). Hence, mapping latent features of the ages to the academic years, our method used the latter data to predict NSSI involvement. All in all, it is evident that the chosen questions are backed by previous studies, all of which provide the grounds to increase reliance on the proposed abstracted questionnaire. Additionally, as mentioned, since the results were consistent among two different populations, psychologists can have more confidence when diagnosing patients using our method.
Three pioneering studies used data mining techniques to predict NSSI, two of which aimed to identify subgroups of individuals who engaged in NSSI by identifying splits on related variables [28]. The first used a machine learning technique to examine splits in the number of NSSI acts during the previous year as predicted by participant-reported psychological difficulties. Results demonstrated significant splits between zero and one-or-more past year NSSI acts on the one hand, and between five and six-or-more past year NSSI acts on the other hand. This suggested that participants reporting six or more past-year NSSI acts may represent a more severe group of self-injurers [1]. Another study built on the previous one and examined splits in NSSI behavior age of onset predicted by prior Self-Injurious Thoughts and Behaviors (SITB), including NSSI characteristics (i.e., number of NSSI-related hospital visits, NSSI frequency, etc.), suicidal ideation, suicidal planning, and suicidal attempts. Results suggested there was a potential subgroup in the data representing those with an earlier age of onset (i.e., approximately 12 or younger); this subgroup reported a greater number of NSSI methods, NSSI frequency, and NSSI-related hospital visits, in addition to an increased likelihood of having suicidal planning [71]. The final study employed two machine learning techniques to identify important indicators of NSSI frequency, both explaining a significant proportion of variance in NSSI frequency (R2 = 0.48 and 0.46, respectively). Models indicated that the number of NSSI methods was the most important indicator of lifetime NSSI frequency; after removing the number of methods from the models, suicidal planning and depressive symptoms emerged as the most important in the prediction of NSSI frequency [3].
Previously, state-of-the-art studies provided time-series analysis to determine whether a person is involved in NSSI or not. For instance, within a sample of 1,021 high-risk self-injurious and/or suicidal individuals, Huang et al. [76] examined the accuracy of three different complex model types in predicting NSSI across 3, 14, and 28 days. In another study, Marti-Puig et al. [77] built a mobile application to collect data so that later they could classify NSSI in young adults focusing on their emotions only. After the data collection phase, they used the data as a time series to test one’s involvement in NSSI. It is clear that such approaches require multiple records of data for their decision-making process, none of which are convenient when injurers are to be identified in only one and the first session of their therapies. Moreover, the former approach only achieves an 84.0% F1-score after processing data entries of the last three days and the latter achieves merely a 22.9% F1-score after processing the data of the last 15 days of a person. Additionally, they ask the user to clarify whether they have been involved in NSSI as the ground truth data. Yet, users may deliberately lie about this question in particular, if they do not want to be objected as mentioned previously. However, our method achieves an 83.7% F1-score after processing only 22 items without requiring the patient to clarify their NSSI behaviors. These automatically increase the reliance on our approach and make it one of the most convenient methods of identifying those involved in NSSI.
Some other successful machine learning-based methods are to provide insight into key factors when NSSI involvement is being decided by a psychologist. For example, Gradus et al. [72] developed gender-stratified classification trees and random forests using 1,458 predictors, including demographic factors, family histories, psychiatric and physical health diagnoses, surgery, and prescribed medications. They found that SUD, prescribed psychiatric medications, previous poisoning diagnoses, and stress disorders were important factors for predicting suicide attempts among men and women. Wallace et al. [67] in 2021 tested classification trees that evaluated 298 potential correlates of NSSI and suicidal ideation across self-identified women and men. Psychopathology, poorer psychological well-being, and other SITBs emerged as important correlates for all participants. Trauma, disordered eating, and heavy alcohol use were salient among women, whereas alcohol use norms were important correlates among men. In a similar study, Yang et al. [78] in 2022 proposed an SVM model which deduced adolescents’ gender, paranoid and histrionic personality traits, suffring physical abuse in childhood, emotional non-acceptance, and education level were associated with an increased risk of NSSI. These may alert psychologists to pay more attention to the mentioned factors. Nevertheless, solely outlining key factors may not be reasonable enough to make psychologists opt for such methods and decide on a diagnosis. Perhaps, using the factors all together to also classify the trait would be more acceptable among psychologists, which requires more complicated processes on data. Automated methods, on the other hand, can quickly do the computation to provide an outcome on the diagnosis. This also contributes to the convenience of such methods while saving some time and energy for the psychologist.
Although our baseline study is conducted on a relatively small statistical population without a broader validation, studies involving self-reports that lack additional objective components still suggest promising aspects of novelties for future research. For instance, predicting a marker that had previously not been researched extensively, Twivy et al. [79] found that differences in affective flexibility towards emotional stimuli may be a positive indicator of anxiety. Soroski et al. [12] built an online website to collect the voices of their participants to propose an Alzheimer’s detection system. Sanders and Nosofsky [11] crowdsourced participants to reconstruct psychological feature space for natural object detection in machine learning applications. That said, a need for a universal measure free from self-reporting constraints is demonstrated, both for the reliability and validity of current and future research. This can be investigated using such pilot studies which may lack additional objectiveness yet fillip the stagnant realms of research.
Currently, other means of determining human cognition and emotions are under review to act as possible alternatives to subjective self-reports. A report studying adults with schizophrenia spectrum disorders found important indicators in these models include duration of illness, number of hospitalizations, emotional and physical abuse in childhood, as well as drug usage or abuse [80]. Furthermore, computational models are concurrently being explored as a possible avenue to examine gender-specific risk profiles for suicide attempts potentially providing an objective lens through which to take into consideration SUD treatment, prescribed psychiatric medications, previous poisoning diagnoses, and stress disorders [72].
Future studies may aim to recruit non-university participants from other countries, although the strength of our study was the generalizability of the sample as we recruited NON-WEIRD (Western, Educated, Industrialized, Rich, and Democratic) [81] participants. Moreover, despite reducing the number of questions, the system still requires around five minutes (reversed items not included) of a person so that it can decide on a result. Therefore, in the future, other cognitive constructs could be used to make the NSSI classification more automated, such as via visual tasks (i.e., eye tracking, visual search, etc.).
Conclusion
People who are involved in NSSI disorder tend to deliberately damage their body tissues to dispel the negative psychological effects of their lives. Since they may hide their behaviors, their parents or caretakers can only notice their behaviors if they catch the injurers while injuring themselves. Another alternative is via dedicated questionnaires. However, they are boring and open to errors and lies. Therefore, we abstracted an original questionnaire of 662 items to own only 22 questions via utilizing data mining techniques, and trained machine learning algorithms to classify participants into two classes of NSSI Positive and NSSI Negative with 83.6% accuracy. Since answering these questions only takes about five to six minutes, the accuracy and confidence will be automatically increased.
That said, the limitation of the current study is that it has been evaluated on only two statistical populations. Further evaluation requires recruiting more participants from different nations to examine the generalizability of such approaches. As a future study, the approach can be further expanded by cognitive models. That is, cognitive models may be able to map particular aspects of complex human behaviors (i.e., their walking style, pauses in their speech) to NSSI. Hence, to leverage Artificial Intelligence, academia may further examine such models for more ubiquity in different places.
Acknowledgments
We would like to thank Jann MacIsaac for providing his dataset thus enabling us to conduct this research.
References
- 1. Ammerman BA, Jacobucci R, Kleiman EM, Muehlenkamp JJ, McCloskey MS. Development and validation of empirically derived frequency criteria for NSSI disorder using exploratory data mining. Psychol Assess. 2017;29: 221. pmid:27176128
- 2. Lloyd-Richardson EE, Lewis SP, Whitlock JL, Rodham K, Schatten HT. Research with adolescents who engage in non-suicidal self-injury: ethical considerations and challenges. Child Adolesc Psychiatry Ment Health. 2015;9: 37. pmid:26417391
- 3. Ammerman BA, Jacobucci R, Kleiman EM, Uyeji LL, McCloskey MS. The relationship between nonsuicidal self-injury age of onset and severity of self-harm. Suicide Life Threat Behav. 2018;48: 31–37. pmid:28160318
- 4. Nock MK. Why Do People Hurt Themselves?: New Insights Into the Nature and Functions of Self-Injury. Curr Dir Psychol Sci. 2009;18: 78–83. pmid:20161092
- 5. Muehlenkamp JJ. Self-injurious behavior as a separate clinical syndrome. American Journal of Orthopsychiatry. 2005;75: 324–333. pmid:15839768
- 6. Gratz KL. Measurement of Deliberate Self-Harm: Preliminary Data on the Deliberate Self-Harm Inventory. J Psychopathol Behav Assess. 2001;23: 253–263.
- 7. Chapman AL, Gratz KL, Brown MZ. Solving the puzzle of deliberate self-harm: The experiential avoidance model. Behaviour Research and Therapy. 2006;44: 371–394. pmid:16446150
- 8. Chan MKY, Bhatti H, Meader N, Stockton S, Evans J, O’Connor RC, et al. Predicting suicide following self-harm: systematic review of risk factors and risk scales. British Journal of Psychiatry. 2018/01/02. 2016;209: 277–283. pmid:27340111
- 9. Sornberger MJ, Heath NL, Toste JR, McLouth R. Nonsuicidal self-injury and gender: Patterns of prevalence, methods, and locations among adolescents. Suicide Life Threat Behav. 2012;42: 266–278. pmid:22435988
- 10. Wang Y, Tang J, Li J, Li B, Wan Y, Mellina C, et al. Understanding and Discovering Deliberate Self-Harm Content in Social Media. Proceedings of the 26th International Conference on World Wide Web. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee; 2017. pp. 93–102.
- 11. Sanders CA, Nosofsky RM. Training Deep Networks to Construct a Psychological Feature Space for a Natural-Object Category Domain. Comput Brain Behav. 2020;3: 229–251.
- 12. Soroski Thomas and da Cunha Vasco T and N-MS and GS and LC and HA et al. Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis. JMIR Aging. 2022;5: e33460. pmid:36129754
- 13. Boone SD, Brausch AM. Physical Activity, Exercise Motivations, Depression, and Nonsuicidal Self-Injury in Youth. Suicide Life Threat Behav. 2016;46: 625–633. pmid:26970091
- 14. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol Bull. 2017;143: 187. pmid:27841450
- 15. Fox KR, Franklin JC, Ribeiro JD, Kleiman EM, Bentley KH, Nock MK. Meta-analysis of risk factors for nonsuicidal self-injury. Clin Psychol Rev. 2015;42: 156–167. pmid:26416295
- 16. Muehlenkamp JJ, Brausch AM. Body image as a mediator of non-suicidal self-injury in adolescents. J Adolesc. 2012;35: 1–9. pmid:21777971
- 17. Lewis SP, Heath NL. Nonsuicidal self-injury among youth. J Pediatr. 2015;166: 526–530. pmid:25596101
- 18. del Carpio L, Paul S, Paterson A, Rasmussen S. A systematic review of controlled studies of suicidal and self-harming behaviours in adolescents following bereavement by suicide. PLoS One. 2021;16: e0254203–. Available: pmid:34242305
- 19. Knipe D, John A, Padmanathan P, Eyles E, Dekel D, Higgins JPT, et al. Suicide and self-harm in low- and middle- income countries during the COVID-19 pandemic: A systematic review. PLOS Global Public Health. 2022;2: e0000282–. Available: pmid:36962383
- 20. Bayliss LT, Christensen S, Lamont-Mills A, du Plessis C. Suicide capability within the ideation-to-action framework: A systematic scoping review. PLoS One. 2022;17: e0276070–. Available: pmid:36301944
- 21. Whitlock J. Self-Injurious Behavior in Adolescents. PLoS Med. 2010;7: e1000240–. Available: pmid:20520850
- 22. Tang J, Ma Y, Guo Y, Ahmed NI, Yu Y, Wang J. Association of Aggression and Non-Suicidal Self Injury: A School-Based Sample of Adolescents. PLoS One. 2013;8: e78149–. Available: pmid:24205132
- 23. Hamza CA, Willoughby T. Nonsuicidal Self-Injury and Suicidal Behavior: A Latent Class Analysis among Young Adults. PLoS One. 2013;8: e59955–. Available: pmid:23544113
- 24. Klonsky ED, Olino TM. Identifying clinically distinct subgroups of self-injurers among young adults: a latent class analysis. J Consult Clin Psychol. 2008;76: 22. pmid:18229979
- 25. Nock MK, Prinstein MJ. Contextual features and behavioral functions of self-mutilation among adolescents. J Abnorm Psychol. 2005;114: 140. pmid:15709820
- 26. Klonsky ED, Victor SE, Saffer BY. Nonsuicidal Self-Injury: What We Know, and What We Need to Know. The Canadian Journal of Psychiatry. 2014;59: 565–568. pmid:25565471
- 27. MacIsaac J. Group Separation and Classification of Non-Suicidal Self-Injury in a University Student Population. University of Windsor (Canada). 2019.
- 28. Burke TA, Ammerman BA, Jacobucci R. The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. J Affect Disord. 2019;245: 869–884. pmid:30699872
- 29. McArdle JJ, Ritschard G. Contemporary Issues in Exploratory Data Mining in the Behavioral Sciences. Taylor \& Francis; 2013. Available: https://books.google.com/books?id=OV9tAAAAQBAJ
- 30.
Neocleous C, Schizas C. Artificial Neural Network Learning: A Comparative Review BT—Methods and Applications of Artificial Intelligence. In: Vlahavas IP, Spyropoulos CD, editors. Berlin, Heidelberg: Springer Berlin Heidelberg; 2002. pp. 300–313.
- 31. Schulz H, Behnke S. Deep learning. KI-Künstliche Intelligenz. 2012;26: 357–363.
- 32. Farajzadeh N, Sadeghzadeh N, Hashemzadeh M. PMG-Net: Persian music genre classification using deep neural networks. Entertain Comput. 2023;44: 100518. https://doi.org/10.1016/j.entcom.2022.100518
- 33. Alshurafa NI, Harmon JT. Artificial spider: eight-legged arachnid and autonomous learning of locomotion. Unmanned Systems Technology VIII. 2006. pp. 481–490.
- 34. Farajzadeh N, Sadeghzadeh N, Hashemzadeh M. A fully-convolutional residual encoder-decoder neural network to localize breast cancer on histopathology images. Comput Biol Med. 2022;147: 105698. pmid:35714505
- 35. Gupta A, Katarya R. Social media based surveillance systems for healthcare using machine learning: A systematic review. J Biomed Inform. 2020;108: 103500. pmid:32622833
- 36. Kumari N, Acharjya DP. Data classification using rough set and bioinspired computing in healthcare applications—an extensive review. Multimed Tools Appl. 2022.
- 37. Alfian G, Syafrudin M, Ijaz MF, Syaekhoni MA, Fitriyani NL, Rhee J. A personalized healthcare monitoring system for diabetic patients by utilizing BLE-based sensors and real-time data processing. Sensors. 2018;18: 2183. pmid:29986473
- 38. El-Sappagh S, Ali F, Abuhmed T, Singh J, Alonso JM. Automatic detection of Alzheimer’s disease progression: An efficient information fusion approach with heterogeneous ensemble classifiers. Neurocomputing. 2022;512: 203–224. https://doi.org/10.1016/j.neucom.2022.09.009
- 39. Srinivasu PN, SivaSai JG, Ijaz MF, Bhoi AK, Kim W, Kang JJ. Classification of skin disease using deep learning neural networks with MobileNet V2 and LSTM. Sensors. 2021;21: 2852. pmid:33919583
- 40. Ali F, El-Sappagh S, Islam SMR, Ali A, Attique M, Imran M, et al. An intelligent healthcare monitoring framework using wearable sensors and social networking data. Future Generation Computer Systems. 2021;114: 23–43.
- 41. Klonsky ED, Glenn CR. Assessing the functions of non-suicidal self-injury: Psychometric properties of the Inventory of Statements About Self-injury (ISAS). J Psychopathol Behav Assess. 2009;31: 215–219. pmid:29269992
- 42. de Haan L, Kuipers E, Kuerten Y, van Laar M, Olivier B, Verster JC. The RT-18: a new screening tool to assess young adult risk-taking behavior. Int J Gen Med. 2011/08/12. 2011;4: 575–584. pmid:21887111
- 43. Victor SE, Klonsky ED. Validation of a Brief Version of the Difficulties in Emotion Regulation Scale (DERS-18) in Five Samples. Journal of Psychopathology and Behavioral Assessment 2016 38:4. 2016;38: 582–589.
- 44. Bagby RM, Parker JDA, Taylor GJ. The twenty-item Toronto Alexithymia scale—I. Item selection and cross-validation of the factor structure. J Psychosom Res. 1994;38: 23–32. pmid:8126686
- 45. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54: 1063. pmid:3397865
- 46.
Lynam DR, Smith GT, Whiteside SP, Cyders MA. The UPPS-P: Assessing five personality pathways to impulsive behavior. West Lafayette, IN: Purdue University. 2006;10.
- 47. Dugré JR, Giguére C-É, du Sert O, Potvin S, Dumais A, Signature, Consortium. The Psychometric Properties of a Short UPPS-P Impulsive Behavior Scale Among Psychiatric Patients Evaluated in an Emergency Setting. Front Psychiatry. 2019;10: 139. pmid:30967798
- 48. Brown KW, Ryan RM. The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol. 2003;84: 822. pmid:12703651
- 49. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16: 606–613. pmid:11556941
- 50. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Arch Intern Med. 2006;166: 1092–1097. pmid:16717171
- 51. Cohen S, Kamarck T, Mermelstein R. Una medida global de estrés percibido. Journal de Salud y conducta social. 1983;24: 385–396.
- 52. John OP, Donahue EM, Kentle RL. Big five inventory. J Pers Soc Psychol. 1991.
- 53. Connor KM, Davidson JRT. Development of a new resilience scale: The Connor-Davidson resilience scale (CD-RISC). Depress Anxiety. 2003;18: 76–82. pmid:12964174
- 54. Cutrona CE, Russell DW. The provisions of social relationships and adaptation to stress. Advances in personal relationships. 1987;1: 37–67.
- 55. Kohn PM, Lafreniere K, Gurevich M. The Inventory of College Students’ Recent Life Experiences: A decontaminated hassles scale for a special population. J Behav Med. 1990;13: 619–630. pmid:2077142
- 56. Raes F, Pommier E, Neff KD, van Gucht D. Construction and factorial validation of a short form of the self-compassion scale. Clin Psychol Psychother. 2011;18: 250–255. pmid:21584907
- 57. Dunn AM, Heggestad ED, Shanock LR, Theilgard N. Intra-individual Response Variability as an Indicator of Insufficient Effort Responding: Comparison to Other Indicators and Relationships with Individual Differences. J Bus Psychol. 2018;33: 105–121.
- 58. Kuhn M, Johnson K. Applied predictive modeling. Applied Predictive Modeling. 2013; 1–600.
- 59. Toğaçar M, Ergen B, Cömert Z. Tumor type detection in brain MR images of the deep model developed using hypercolumn technique, attention modules, and residual blocks. Med Biol Eng Comput. 2021;59: 57–70. pmid:33222016
- 60. Toğaçar M, Cömert Z, Ergen B. Intelligent skin cancer detection applying autoencoder, MobileNetV2 and spiking neural networks. Chaos Solitons Fractals. 2021;144: 110714. https://doi.org/10.1016/j.chaos.2021.110714
- 61. Grandini M, Bagli E, Visani G. Metrics for Multi-Class Classification: an Overview. arXiv; 2020.
- 62. Chawla N v Bowyer KW, Hall LO Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research. 2002;16: 321–357.
- 63. Matsugu M, Mori K, Mitari Y, Kaneda Y. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks. 2003;16: 555–559. pmid:12850007
- 64. Handley TE, Hiles SA, Inder KJ, Kay-Lambkin FJ, Kelly BJ, Lewin TJ, et al. Predictors of Suicidal Ideation in Older People: A Decision Tree Analysis. The American Journal of Geriatric Psychiatry. 2014;22: 1325–1335. pmid:24012228
- 65. Burke TA, Jacobucci R, Ammerman BA, Piccirillo M, McCloskey MS, Heimberg RG, et al. Identifying the relative importance of non-suicidal self-injury features in classifying suicidal ideation, plans, and behavior using exploratory data mining. Psychiatry Res. 2018;262: 175–183. pmid:29453036
- 66. Ghinea D, Edinger A, Parzer P, Koenig J, Resch F, Kaess M. Non-suicidal self-injury disorder as a stand-alone diagnosis in a consecutive help-seeking sample of adolescents. J Affect Disord. 2020;274: 1122–1125. pmid:32663940
- 67. Wallace GT, Conner BT, Shillington AM. Classification trees identify shared and distinct correlates of nonsuicidal self-injury and suicidal ideation across gender identities in emerging adults. Clin Psychol Psychother. 2021;28: 682–693. pmid:33169471
- 68. Hughes CD, King AM, Kranzler A, Fehling K, Miller A, Lindqvist J, et al. Anxious and Overwhelming Affects and Repetitive Negative Thinking as Ecological Predictors of Self-Injurious Thoughts and Behaviors. Cognitive Therapy and Research 2019 43:1. 2019;43: 88–101.
- 69. Bock RC, Berghoff CR, Baker LD, Tull MT, Gratz KL. The Relation of Anxiety to Nonsuicidal Self Injury Is Indirect Through Mindfulness. Mindfulness 2021 12:8. 2021;12: 2022–2033.
- 70. Kuroki Y. Risk factors for suicidal behaviors among Filipino Americans: A data mining approach. American journal of orthopsychiatry. 2015;85: 34. pmid:25110976
- 71. Ammerman BA, Jacobucci R, McCloskey MS. Using Exploratory Data Mining to Identify Important Correlates of Nonsuicidal Self-Injury Frequency. Psychol Violence. 2017/08/17. 2018;8: 515–525. pmid:30393574
- 72. Gradus JL, Rosellini AJ, Horváth-Puhó E, Jiang T, Street AE, Galatzer-Levy I, et al. Predicting Sex-Specific Nonfatal Suicide Attempt Risk Using Machine Learning and Data From Danish National Registries. Am J Epidemiol. 2021. pmid:33877265
- 73. Kiekens G, Claes L. Non-Suicidal Self-Injury and Eating Disordered Behaviors: An Update on What We Do and Do Not Know. Curr Psychiatry Rep. 2020;22: 68. pmid:33037934
- 74. Blashill AJ, Fox K, Feinstein BA, Albright CA, Calzo JP. Nonsuicidal self-injury, suicide ideation, and suicide attempts among sexual minority children. J Consult Clin Psychol. 2021;89: 73. pmid:33705164
- 75. Kamen DG. How can we stop our children from hurting themselves? Stages of change, motivational interviewing, and exposure therapy applications for non-suicidal self-injury in children and adolescents. International Journal of Behavioral Consultation and Therapy. 2011;5: 106.
- 76. Fox KR, Huang X, Linthicum KP, Wang SB, Franklin JC, Ribeiro JD. Model complexity improves the prediction of nonsuicidal self-injury. American Psychological Association. 2019;87: 684–692. pmid:31219275
- 77. Marti-Puig P, Capra C, Vega D, Llunas L, Solé-Casals J. A Machine Learning Approach for Predicting Non-Suicidal Self-Injury in Young Adults. Sensors. 2022;22. pmid:35808286
- 78. Yang J, Chen Y, Yao G, Wang Z, Fu X, Tian Y, et al. Key factors selection on adolescents with non-suicidal self-injury: A support vector machine based approach. Front Public Health. 2022. pmid:36438278
- 79.
Twivy E, Grol M, Fox E. Individual differences in affective flexibility predict future anxiety and worry. 2020;35: 425–434. https://doi.org/10.1080/02699931.2020.1843407
- 80. Hettige NC, Nguyen TB, Yuan C, Rajakulendran T, Baddour J, Bhagwat N, et al. Classification of suicide attempters in schizophrenia using sociocultural and clinical features: A machine learning approach. Gen Hosp Psychiatry. 2017;47: 20–28. pmid:28807134
- 81. Rad MS, Martingano AJ, Ginges J. Toward a psychology of Homo sapiens: Making psychological science more representative of the human population. Proceedings of the National Academy of Sciences. 2018;115: 11401–11405. pmid:30397114