https://orcid.org/0009-0004-8839-1321
Figures
Abstract
The advent of social media has led to an increased concern over its potential to propagate hate speech and misinformation, which, in addition to contributing to prejudice and discrimination, has been suspected of playing a role in increasing social violence and crimes in the United States. While literature has shown the existence of an association between posting hate speech and misinformation online and certain personality traits of posters, the general relationship and relevance of online hate speech/misinformation in the context of overall psychological wellbeing of posters remain elusive. One difficulty lies in finding data analytics tools capable of adequately analyzing the massive amount of social media posts to uncover the underlying hidden links. Machine learning and large language models such as ChatGPT make such an analysis possible. In this study, we collected thousands of posts from carefully selected communities on the social media site Reddit. We then utilized OpenAI’s GPT3 to derive embeddings of these posts, which are high-dimensional real-numbered vectors that presumably represent the hidden semantics of posts. We then performed various machine-learning classifications based on these embeddings in order to identify potential similarities between hate speech/misinformation speech patterns and those of various communities. Finally, a topological data analysis (TDA) was applied to the embeddings to obtain a visual map connecting online hate speech, misinformation, various psychiatric disorders, and general mental health.
Author summary
The advent of social media has led to an increased concern over its potential to propagate hate speech and misinformation, which, in addition to contributing to prejudice and discrimination, has been suspecting of playing a role in increasing social violence and crimes in the United States. While literature has shown the existence of an association between posting hate speech and misinformation online and certain personality traits of posters, the general relationship between online hate speech, misinformation, and general mental health remains unclear. In this study, we collected thousands of posts from misinformation, hate speech, and psychiatric disorder communities on the social media site Reddit. We then used machine-learning methods to analyze the speech patterns of posts from each community and then identify which psychiatric disorder communities’ speech patterns are most similar to those of hate speech or misinformation. We found that speech patterns in hate speech communities were most similar to those seen in communities for Antisocial Personality Disorder, Borderline Personality Disorder, Narcissistic Personality Disorder, Schizoid Personality Disorder, and Complex Post-Traumatic Stress Disorder. For misinformation communities, their most similar communities were our control communities, though there was a degree of similarity with anxiety disorder communities.
Citation: Alexander AW, Wang H (2025) Topological data mapping of online hate speech, misinformation, and general mental health: A large language model based study. PLOS Digit Health 4(7): e0000935. https://doi.org/10.1371/journal.pdig.0000935
Editor: Amara Tariq, Mayo Clinic Arizona, UNITED STATES OF AMERICA
Received: May 31, 2024; Accepted: June 24, 2025; Published: July 29, 2025
Copyright: © 2025 Alexander, Wang. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The dataset and code used in this study is available at: www.github.com/Andrew7178/TopologicalDataMappingOnlineHateSpeechMisinformationGeneralMentalHealth.
Funding: AWA was a Burroughs Wellcome Fund Scholar supported by a Burroughs Wellcome Fund Physician Scientist Institutional Award (G-1020069) to the Texas A&M University Academy of Physician Scientists (https://www.bwfund.org/funding-opportunities/biomedical-sciences/physician-scientist-institutional-award/grant-recipients/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. HW received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The advent of social media has led to an increased concern over the potential for social media forums to contribute to hate speech, defined in the literature as “the use of offensive language, focused on a specific group of people who share a common property” [1]. Online hate speech contributes to prejudice and discrimination in society when offline, and on a more severe level, social media’s allowances for hate speech have led to companies such as Facebook being accused by the United Nations of contributing to genocide [2,3]. Hate speech on social media has also been suspected of playing a role in social violence and crimes in the United States. A positive correlation between violent political rhetoric and mass shootings has been established [4].
Likewise, social media has proven to be a potent vector for misinformation. Misinformation, defined as information contrary to the consensus of the scientific community, has been considered a contributing factor to hate speech on social media [5,6]. Additionally, it has contributed to confusion and distrust in several areas of medicine, including vaccines, drugs, pandemics, eating disorders, and medical treatments, yielding another vector by which it may negatively impact society [6].
Hate speech could be argued to be driven in part by a lack of empathy towards others, as could misinformation in certain contexts. With this in mind, there have been studies investigating the potential for relationships between hate speech/misinformation and the Dark Triad personality traits, with positive correlations being established [7,8]. The Dark Triad personality traits of Narcissism, Machiavellianism, and Psychopathy have a substantial degree of overlap of characteristics with the Cluster B personality disorders defined by the Diagnostic and Statistical Manual of Mental Disorders 5th Edition (DSM-5), suggesting the potential that these disorders could have a connection to hate speech/misinformation as well [9,10].
The general relationship of online hate speech and misinformation with psychiatric disorders and/or the overall mental health of individuals is certainly intricate and remains elusive, however. While the Dark Triad personality traits are substantially correlated to the Cluster B personality disorders, they are not necessarily the exact same constructs [9]. Therefore, they may ultimately have different relations to hate speech and misinformation. Furthermore, the Cluster B personality disorders cover only a small portion of the disorders listed in the DSM-5, meaning there could be additional psychiatric disorders that may have a connection with hate speech or misinformation [10]. Finally, how are hate speech and misinformation related to the general mental state and psychological wellbeing of individual posters? If it were possible to identify hidden psychiatric disorders or traits of them with misinformation or hate speech in the overall context of mental health, the gained knowledge could be used to implement more general and effective counter strategies against hate speech or misinformation.
Artificial intelligence (AI) and machine learning (ML) provide tools to tackle these problems [11–13]. In particular, large language models (LLMs), as exemplified by GPT3 and ChatGPT [14–16] have been demonstrated to be especially powerful in understanding natural languages and performing various natural language processing tasks. One of the key features of LLMs is to learn, via massively pre-training a transformer neural network [17,18] with a context-dependent representation, called an embedding, of any text input. In the case of GPT3, such an embedding is a 1536-dimensional real-valued vector. These embeddings approximate a computer-friendly representation of the semantics of the text input, and therefore can be used for downstream tasks such as text classification, summarization and question-answering. For example, in so-called zero-shot classification [19], a LLM can be used to classify any input by simply calculating and comparing the vector distance of the input and a pre-specified class label.
The objective of this work is to utilize LLMs to study the relevance of online hate speech and misinformation in the context of the general mental health of posters. In particular, we collect thousands of online posts from a carefully selected group of communities in Reddit. We then derive the embeddings of these posts using GPT3. We explore the correlation of hate and misinformation with specific psychiatric disorders using zero-shot classification. Finally, to gain a deeper insight on hate speech and misinformation in the general context of mental health, we carry out a topological data analysis (TDA) on these embeddings to obtain an overarching topological mapping of the dataset [20–22]. The map provides a snapshot of the shape of the mental health space and relationships between psychiatric disorders, hate speech, and misinformation within that space.
Methodology
Social media selection
We opted to utilize the social media site Reddit as our source to draw embeddings from. Reddit has had a substantial number of communities associated with misinformation and hate speech in its past and present, making it an excellent repository of this data for our purposes [23]. Additionally, Reddit is split into “subreddits”, which are effectively communities dedicated to a specific topic [24]. This is beneficial to our study, as it allows for the existence of communities dedicated to specific psychiatric disorders, which would generally self-select for members with the corresponding disorder. Furthermore, Reddit has the advantage of anonymity; Reddit usernames are not tied to an individual’s personal name, and identifying information is only divulged on Reddit by a user if they wish to divulge it [24]. This is of substantial value when collecting data on psychiatric disorders, where stigmatization could prevent individuals from open discussion if anonymity was unavailable [25].
Community selection
A total of 54 communities were selected that could bear relevance to four different classes: Hate Speech, Misinformation, Psychiatric Disorders (which had subclasses for each psychiatric disorder represented by the communities we obtained embeddings from), and Control (Table 1). Hate speech and misinformation communities on Reddit were selected based on either their designation as such in prior literature or records of ban or quarantine by Reddit administration for reasons pertaining to hate, violence, or misinformation [26–42]. If a community could be considered as spreading both hate speech and misinformation, we categorized it by personal assessment as to which was the more dominant theme of the community. Communities identified as representing a psychiatric disorder were primarily based on their own declaration as such and personal assessment of posts to ascertain that this was the genuine goal of the community. Communities were considered to be control communities if they did not meet the criteria for any of the other classes.
Embedding types: Individual user post embeddings and distilled embeddings
We distinguished two types of embeddings: “Individual User Post” (IUP) embeddings, created based on each individual post, and “distilled” embeddings, created based on an aggregate of posts from multiple users within a single community. The reason to derive distilled embeddings is to obtain an averaged representation of a community that dilutes the influence of an individual’s unique speech patterns. The unique speech patterns of the individual users, being unique, would fail to stand out after aggregation, whereas the characteristic speech patterns for a community would become more influential due to their presence in a large number of users. In this sense, we are attempting to create embeddings that are more representative of psychiatric disorder and hate speech/misinformation communities than IUP embeddings could provide.
Embedding generation
A total of 1000 posts were pulled for each of the 54 communities selected from the “Subreddit comments/submissions 2005-06 to 2022-12” academic torrent [43]. Specifically, we pulled posts starting from September 15, 2022 and working backwards until the 1000 oldest posts had been collected. To minimize the introduction of biases, the age of a post was the only criteria determining which posts were collected; popularity or visibility of a post were not considered. The text data used for each post was comprised of the text used for the title, and the text used inside the post itself from the author.
Individual posts from a community were concatenated to derive distilled embeddings. The hate speech/misinformation communities received distilled embeddings, but not IUP embeddings, as these two embedding classes were only used later in the zero-shot classification analyses, where they served as testing data comprised entirely of distilled embeddings. For psychiatric disorder communities, we opted to obtain distilled embeddings of multiple communities as opposed to one community per disorder whenever possible. The rationale behind this was threefold: First, there is always the chance that a community claiming to be representative of a disorder is not actually representative of it. Second, a community could develop its own jargon and speech patterns that would be independent of the psychiatric disorder itself. Bringing in additional communities that may have differing community-specific characteristics could potentially mitigate this effect. While we created IUP embeddings of psychiatric disorder communities, we did not apply this multiple-community treatment to IUP embeddings of psychiatric disorders, as we considered them to already be inherently assigning extra weight to speech patterns that were unique to individuals (and therefore rendering other attempts to erase the influence of unique speech patterns to be rather ineffective).
Many transformer-based LLMs are available to generate text embeddings, including GPT, BERT, and LLAMA. We chose OpenAI’s GPT3 embedding model [44] for its simplicity and robustness. Specifically, the embedding engine named text-embedding-ada-002 [44] was used to generate embeddings for each text input as a vector of 1536 dimensions. Given the engine’s limitation of maximum allowed token length (<8192), if a community’s set of 1000 posts was too large for a single distilled embedding, it was split into several distilled embeddings instead. Efforts were taken to make sure the content of individual posts were not split across multiple distilled embeddings whenever this occurred. As different communities and posts have differing complexities in their text, and therefore different token counts, this led to communities having different numbers of distilled embeddings (Table 1). For IUP embeddings, a set of 50 posts each were collected from the selected communities, working backwards from September 15, 2022 like before.
Zero-shot classification
Zero-shot classification refers to tasking a machine learning model with accurately categorizing data into classes it has never encountered during training. This approach relies on leveraging semantic knowledge about the unseen classes, often through natural language descriptions or other contextual cues. Zero-shot classification is especially valuable in scenarios where obtaining labeled data for every class is impractical, allowing models to generalize beyond their initial training set.
Zero-shot classification is a form of classification task where the model is first trained on data comprising a set of labels (or in this case, embeddings), but not the label that is the target of the analysis. Instead, after the model is trained to identify these other labels, the model is then asked to classify the data comprising the target label, without it knowing about that label’s existence. This forces the model to classify the data from the excluded label as belonging to the trained labels that it is most similar to.
For the purposes of this study, a logistic regression model is trained to classify control and psychiatric disorder communities based on embeddings. The trained model was then applied on embeddings from hate speech and misinformation communities, which were then classified as belonging to the psychiatric disorder or control communities that were most similar in speech pattern.
Four zero-shot classification tasks were performed to explore the interplay of different embedding types and hate speech/misinformation with disorders (Table 2). The goal behind these zero-shot classification tasks is twofold: First, are embeddings and speech patterns from hate speech and misinformation communities more similar to those of control communities or those from psychiatric disorder communities. Second, not all psychiatric disorders are the same and therefore in these zero-shot classifications psychiatric disorder embeddings are separated into individual labels to better identify which ones are most similar to those of hate speech and misinformation.
Classification tasks 1 and 2 were performed with the intent of assessing the best combination of IUP and distilled embeddings to use when performing zero-shot classification of hate speech/misinformation. Specifically, classification task 1 was a classification of psychiatric disorder embeddings using IUP embeddings as training data and distilled embeddings as testing data, and vice versa for classification task 2.
Based on the results of classification tasks 1 & 2, classification tasks 3 & 4 were set as zero-shot classifications of the distilled embeddings of a total of 14 hate speech and misinformation communities. With zero-shot classification, the model is not given any embeddings from hate speech or misinformation communities as training data, and these embeddings comprise the entirety of the model’s testing data; as such, it is forced to classify hate speech and misinformation embeddings as being closer to either a psychiatric disorder distilled embedding or a control distilled embedding. Communities that appeared to be solely dedicated to misinformation instead of a combination of misinformation and hate speech were classified separately from hate speech communities, due to concerns that classifications for psychiatric disorders could differ between the two.
Topological data analysis
Text embeddings derived from LLMs are usually very high-dimensional (1536 dimensions in our case). While such high dimensionality is useful to capture complex hidden semantics of any given text input, it also renders traditional statistical methods (e.g., multivariant linear regression) and data visualization less effective. To further pinpoint the relation between hate speech/misinformation and psychiatric disorders in the general context of mental health, we performed a TDA to explore the global shape of derived embeddings. In general, TDA is an unsupervised machine learning method that excels in handling high-dimensional data (20,21,22). Unlike linear models such as principal component analysis, which have difficulty preserving the shape of high-dimensional datasets, TDA can capture nonlinearity by projecting a high-dimensional dataset onto a lower dimensional space in a systematical and sensible way. For example, in Mapper [45], a high-dimensional dataset is first projected into a lower dimension space by a filter function. In the lower dimension, the data is then segmented into several overlapping sections, called covers. Clustering is then performed in each cover, and the resultant clusters (referred to as nodes) that share data points are then connected by edges. By doing so, Mapper is capable of producing a visual representation of the global shape of the data without sacrificing non-linearity.
We used Mapper Interactive to generate a TDA mapping of the distilled embeddings [46]. First, a principal component analysis was carried out and the first two principal components were extracted to serve as a 2-dimensional filter function for Mapper. Both dimensions used 10 covers, each with 50% overlap. For our clustering algorithm, we used DBSCAN at 0.5 eps and 2 minimum samples.
Results
Classifications 1 and 2: Cross-comparison of individual user post and distilled embeddings as training and testing data
Here we sought to evaluate how a model trained on IUP embeddings would classify distilled embeddings, and vice versa. Evaluation of the psychiatric disorder embeddings showed excellent accuracy when IUP embeddings were used for training (86% accuracy), whereas distilled embeddings showed difficulty in adequately classifying individual posts, with an accuracy of 45% (Tables 3–4, Fig 1–2). Inspection showed that many of the inaccuracies seen in classification of IUP embeddings by a model trained on distilled embeddings were attributable to a frequent misclassification of embeddings as substance use disorders.
The number in each box indicates how many embeddings from a specific community were classified as belonging to a certain label.
The number in each box indicates how many embeddings from a specific community were classified as belonging to a certain label.
Several inaccuracies appear to be present in the results obtained from classifying distilled embeddings with a model trained by IUP embeddings. The three disorder community types with the greatest degree of misclassification were depression, CPTSD, and Schizoaffective Disorder/Schizophrenia. However, the existence of comorbidities suggests that several of these inaccuracies may very well not be inaccuracies at all, or are at the least very close to a correct classification. The depression distilled embedding misclassifications were in four classes: ADHD, Bipolar Disorder, Anxiety, and Suicidality, all of which are comorbid with depression, or in the case of Bipolar Disorder, contain depressive episodes within their diagnostic criteria [10,47,48].
CPTSD’s misclassifications were in PTSD and Borderline Personality Disorder. There is a high degree of feature overlap between BPD and CPTSD, and the debate over whether CPTSD is distinct enough from PTSD to warrant its own diagnosis in the DSM-5 is indicative of substantial similarities between these two disorders as well [49,50].
Finally, Schizophrenia and Schizoaffective Disorder were misclassified as two other disorders: Bipolar Disorder and Schizotypal Personality Disorder. Both of these disorders are capable of the psychotic features present in Schizophrenia and Schizoaffective Disorder, offering a reasonable explanation for the misclassification [10,51]. Overall, the correct classifications and the misclassifications being centered around disorders that are comorbid with the target disorder or downright include it suggest that a very small dataset of IUP embeddings was sufficient to train a model that could accurately classify a distilled embedding by the disorder they represented.
Classifications 3 and 4: Classification of misinformation/hate communities by disorder
Based on the results of the previous analyses, we concluded that the best design for our classification of hate speech and misinformation would be to utilize IUP embeddings as our training data, and distilled embeddings as our testing data. Zero-shot classification of the distilled embeddings for hate speech communities had a split between control classifications (20.18%) and psychiatric disorder classifications (79.82%) (Table 5, S18 Fig, S19 Fig). The four disorders that were most often classed for misinformation/hate communities were Schizoid Personality Disorder and the three Cluster B Personality Disorders included in the study: Narcissistic Personality Disorder, Borderline Personality Disorder, and Antisocial Personality Disorder. Of these disorders, Schizoid Personality Disorder accounted for 24.31% of all 218 hate speech embeddings and 30.46% of hate speech embeddings classified as belonging to a psychiatric disorder community, while the Cluster B Personality Disorders accounted for 49.54% of all hate speech embeddings and 62.07% of hate speech embeddings classified as pertaining to a psychiatric disorder community.
In communities that are more purely misinformation rather than a combination of misinformation and hate speech, the dominant psychiatric disorder classification was anxiety disorders (13.85%), though the vast majority of the 65 misinformation embedding classifications tended to lean towards control (73.85%), with only 26.15% of embeddings being classified as pertaining to psychiatric disorder communities.
Topological data analysis
The topological mapping of our collection of embeddings yielded several interesting observations. Firstly, all of the psychiatric disorder embeddings had either direct or indirect edges with each other, all occupying their own portion of space within the mapping (Fig 3). Secondly, this space comprising the psychiatric disorders was connected directly to a space comprising all of the hate speech embeddings (Fig 4) and almost all of the misinformation embeddings (Fig 5). What makes this notable is that almost all of the control embeddings had no connection to this portion of the space; this lack of connection indicates that this hate speech/misinformation/psychiatric disorder space shares certain characteristics within their speech patterns that are not generally seen within the control embeddings (Fig 6).
Yellow nodes within the space labelled psychiatric disorder nodes signify a node comprised of psychiatric disorder embeddings.
Yellow = No Hate Speech Embeddings, Red = Entirely Comprised of Hate Speech Embeddings.
Yellow = No Misinformation Embeddings, Red = Entirely Comprised of Misinformation Embeddings.
Yellow = No Control Embeddings, Red = Entirely Comprised of Control Embeddings.
While these observations are notable enough on their own, further inspection in conjunction with the results of our zero-shot classification analyses reveals that the patterns seen in the TDA mapping could be seen as predictive of the zero-shot classification results. The portion of space taken up by hate speech is closer and more immediately connected to the psychiatric disorders than the space occupied by misinformation. This corresponds to how there were far more hate speech embeddings classified as psychiatric disorders compared to misinformation embeddings; hate speech embeddings shared more similarities to psychiatric disorder embeddings. The hate speech/misinformation embeddings classified as control can be explained by the nodes in the hate speech and misinformation portions of space that contain a few control embeddings. Additionally, the portion of misinformation embeddings that occupied their own space lacked any connection with psychiatric disorders; it stands to reason that those embeddings would become classified as control.
Looking at the psychiatric disorder portion of the space in more granular detail illustrates further consistency with our zero-shot classification results. Each type of psychiatric disorder community occupied its own subsection of the space. If a psychiatric disorder community occupied a space closer to the connection point between the psychiatric disorder space and the hate speech space, then its speech patterns were more similar to those seen in hate speech. Inspection indicated that the psychiatric disorder communities occupying the connection point were the Cluster B personality disorder communities, and Schizoid Personality Disorder, with the communities for Narcissistic Personality Disorder and Schizoid Personality Disorder being most directly connected (Figs 7–10).
Yellow = No Narcissistic Personality Disorder Embeddings, Red = Entirely Comprised of Narcissistic Personality Disorder Embeddings.
Yellow = No Schizoid Personality Disorder Embeddings, Red = Entirely Comprised of Schizoid Personality Disorder Embeddings.
Yellow = No Antisocial Personality Disorder Embeddings, Red = Entirely Comprised of Antisocial Personality Disorder Embeddings.
Yellow = No Borderline Personality Disorder Embeddings, Red = Entirely Comprised of Borderline Personality Disorder Embeddings.
These two personality disorder communities were also the ones with the most hate speech embeddings attributed to them, lending credence to the idea that this TDA mapping is illustrating a trend where increased proximity of a psychiatric disorder community’s distilled embeddings to the hate speech portion of the space suggests it has an increased likelihood of having hate speech embeddings classified to it.
Additionally, the community embeddings for r/CPTSD were prominent in this region as well, corresponding with its mild amount of hate speech embeddings classified towards it (Fig 11).
Yellow = No Complex Post-Traumatic Stress Disorder Embeddings, Red = Entirely Comprised of Complex Post-Traumatic Stress Disorder Embeddings.
Communities for psychiatric disorders that received little to no hate speech classifications were further away from the connection point; figures for these disorders can be seen in the Supporting Information.
Based on the TDA mapping, we decided to examine what the zero-shot classification results would look like if the Schizoid Personality Disorder and Narcissistic Personality Disorder communities were excluded from the training data (S14 Fig, S15 Fig, S16 Fig). Doing so led to a redistribution of their assigned hate speech embeddings to CPTSD, BPD, and ASPD. An additional zero-shot classification excluding the aforementioned disorders in addition to CPTSD indicated that the embeddings assigned to CPTSD were transferred to BPD and ASPD (S17 Fig).
Discussion
Our classification model yielded an interesting classification pattern with regards to hate speech communities on Reddit: Their speech patterns are close enough to those seen in Reddit communities for Cluster B Personality Disorders that they can be classed as such under zero-shot classification. In a certain sense, this result is not surprising, as it is consistent with studies relating the Dark Triad to hate speech, and these disorders are strongly linked with a lack of empathy for others and violence against others, traits that can be seen as a product of hate speech [52–56]. Although our zero-shot classification results initially appeared to indicate that the only Cluster B personality disorder of import was Narcissistic Personality Disorder, our TDA mapping indicated that CPTSD and the other Cluster B personality disorders could be considered as close to the connection point with the hate speech region as well. This suggested that they could receive the hate speech classifications as well, and that Narcissistic Personality Disorder and Schizoid Personality Disorder were merely the disorders with speech patterns most similar to hate speech, not the only disorders with similarity. When we excluded Narcissistic Personality Disorder and Schizoid Personality Disorder from zero-shot classification (S14 Fig, S15 Fig, S16 Fig), this suspicion proved true; while some hate speech embeddings were reclassified as control, many went to the other Cluster B disorders and CPTSD. By making use of TDA, we were thus able to enhance the results of our zero-shot classification from depicting only two disorders as being substantially tied to hate speech to five disorders, those of Antisocial Personality Disorder, Narcissistic Personality Disorder, Borderline Personality Disorder, Complex Post-Traumatic Stress Disorder, and Schizoid Personality Disorder. While this shows that TDA is able to enhance zero-shot classification, the means by which we were able to use it to do so also indicates that the mapping of the psychiatric disorders could be viewed as a spectrum of similarity to hate speech; the closer a psychiatric disorder is to the hate speech region of space, the more similar it is, and vice versa.
The presence of Schizoid Personality Disorder as the second-largest classification for hate speech is surprising. Schizoid Personality Disorder’s restricted expression of emotions and detachment from social relationships does not appear conducive to hate speech, which would likely be generated from emotionality and from considering oneself as part of an in-group to compare against the target of hate speech [10,55]. While it is not impossible for this classification to be correct, we also note the possibility that this is a case where the weaknesses of using text embeddings from a psychiatric disorder community instead of individuals with confirmed diagnoses could be made apparent. In particular, individuals with Antisocial Personality Disorder may have misconstrued their lack of empathy for a lack of interest in social interaction. Comparison of the portions of space occupied by both communities on the TDA mapping shows a striking degree of overlap; Schizoid Personality Disorder is encompassing all of Antisocial Personality Disorder’s space, though it covers some additional space as well. This could also explain why Antisocial Personality Disorder had a hate speech classification count that could be considered to be astonishingly low; the Schizoid Personality Disorder and Narcissistic Personality Disorder embeddings overlapped with its space, and the model ultimately selected those two over Antisocial Personality Disorder, as can be seen by the trends in S14 Fig, S15 Fig, and S16 Fig. However, this is not guaranteed to be the case; it really could be that Schizoid Personality Disorder has a predilection for hate speech. Observation of another Cluster A personality disorder, Schizotypal Personality Disorder, showed that its region of space maintained a degree of continuity with that of Schizoid Personality Disorder (Fig 8, S11 Fig), offering an argument that the Schizoid Personality Disorder community on Reddit actually could represent their namesake. Determining if there is as strong a connection between Schizoid Personality Disorder and hate speech as depicted by our zero-shot classification results will likely require textual embeddings gathered from individuals with confirmed diagnoses of Schizoid Personality Disorder.
Complex Post-Traumatic Stress Disorder had notable results when juxtaposed to PTSD. Once other disorder communities occupying the connection point with hate speech were excluded from zero-shot classification, CPTSD was assigned a substantial amount of hate speech embeddings, but PTSD did not receive any (S16 Fig). Upon subsequent exclusion of CPTSD, the hate speech embeddings assigned to CPTSD did not transfer to PTSD, but instead to Borderline Personality Disorder (S17 Fig). This allows for several observations. First, speech patterns in Borderline Personality Disorder are more similar to those seen in hate speech embeddings assigned to CPTSD than PTSD is. Second, speech patterns in CPTSD are more similar to hate speech than those seen in PTSD. Third, the portion of space occupied by CPTSD has more overlap with the space occupied by Schizoid Personality Disorder and the Cluster B personality disorders (Fig 7–11). All of these seem to indicate that, for the purposes of this set of embeddings, the CPTSD community on Reddit exhibits patterns much more reminiscent of Cluster B than PTSD, to the point that not one embedding assigned to CPTSD was reassigned to PTSD. In terms of the debate surrounding whether CPTSD is unique as a construct, this would suggest that CPTSD is distinct from PTSD, but not from Cluster B personality disorders, especially Borderline Personality Disorder, the other construct that has been proposed as being the same as CPTSD [49,50]. However, we only had one community each for CPTSD and PTSD; a larger number of communities would be better for establishing a proper spatial relationship between CPTSD, PTSD, and Borderline Personality Disorder.
Zero-shot classification of misinformation communities indicated that their speech patterns could be similar to those of anxiety disorders in some instances. We believe this effect to have been at least partially influenced by our use of communities related to COVID, a topic prone to inciting anxiety [57]. Future studies into misinformation and psychiatric disorders may be best performed with embeddings from communities pertaining to a more diverse set of misinformation topics, and with a larger set of communities overall. Overall, our pure misinformation communities appeared to have far less psychiatric disorder classifications compared to communities dedicated to hate speech and their related misinformation, and they exhibited the characteristics of anxiety disorders instead of the characteristics of Schizoid Personality Disorder or Cluster B Personality Disorders that were seen in hate speech communities. Despite this, most of the misinformation embeddings maintained a connection with the hate speech region of space, and by extension the psychiatric disorder space. This suggests that there is a connection between psychiatric disorders and misinformation, though our dataset for this study is insufficient for properly comprehending it.
The greatest limitation of our study is that it must be noted that our results are not based on confirmed diagnoses; firstly, we performed classifications of embeddings that comprised an aggregate of users online, and psychiatric diagnoses are attributed to individuals, not groups of people. Secondly, our training data for psychiatric disorders is sourced from individuals claiming to have a particular psychiatric disorder; we have no evidence as to whether or not they speak the truth. However, several portions of our results indicate that these psychiatric disorder communities functioned as effective proxies. First, the high accuracy of our model when using distilled embeddings as testing data indicates that the model really does identify embeddings from psychiatric disorder communities on Reddit to be different from those of our control communities. Combined with the most prominent inaccuracies in Fig 1 being ones where a psychiatric disorder community was misclassified as a community for a psychiatric disorder with strong ties to it, this would appear to suggest that these embeddings of psychiatric communities served as substantially effective proxies for their respective psychiatric disorders. Second, observation of our TDA mapping shows that all of our psychiatric disorder communities shared the same region of space; that is to say, 866 embeddings comprising over 800,000 posts taken from 34 different psychiatric disorder communities were analyzed, and not one of them was considered as being so different from the others that they lacked an edge connecting them to the psychiatric disorder region of the mapping. Juxtaposing this to the control embeddings’ general pattern of occupying their own individualized regions of space shows that this set of psychiatric disorder embeddings is very unlikely to involve communities that do not represent at least some form of psychiatric disorder. Consider further how these disorder communities demonstrate overlap similarly to how one would expect their comorbidities would be (e.g., ADHD having a wide range over the space, signifying overlap with many disorders, Cluster B disorders sharing the same region of space, depression overlapping most of the suicidality space), and it becomes apparent that even if these communities are serving as proxies for these disorders, they are serving fairly well, with the potential exception of Schizoid Personality Disorder.
We had several other limitations in this study; time/cost limitations meant that our sample size of posts for our embeddings were somewhat smaller than ideal. Additionally, we do not have every psychiatric disorder represented here; notably, we do not have Histrionic Personality Disorder, one of the Cluster B personality disorders. Furthermore, Reddit was the only social media site we drew from; other sites may have different user demographics, and potentially different results.
There exists concerns about a “sink” problem in the use of classification for fake image detection, where the model trains itself to identify specific patterns to indicate an image is fake, which can then lead to the model failing to identify fake images that do not have those specific patterns [58]. It could be argued that said concerns could be applied to other classification models used for detection as well. However, the “sink” problem is known to occur in the setting of binary classifications (e.g., real vs. fake), and our classifiers do not make binary classifications, instead deciding to classify into one of a range of classes. This reduces the risk of this “sink” problem. Additionally, findings from our TDA mappings, which would not be affected by the “sink” problem, are rather consistent with those seen in our classifications (e.g., the disorders most closely related to hate speech on the mapping are Cluster B disorders, the same disorders that received the most classifications in the zero-shot tasks). This suggests further that our classification results are not being distorted by this “sink” problem.
Our use of what we defined as distilled embeddings as training data for classification tasks yielded unexpected results. While they proved superior to IUP embeddings when it came to classifying embeddings of their own kind, they were inferior in performance for zero-shot classification or when the testing data was comprised of IUP embeddings. It is possible that our method of aggregating multiple posts together caused some components of the speech patterns for a community to be effectively cancelled out inside the aggregated embedding, impairing their ability to properly classify. However, models trained on IUP embeddings much more accurately classified a distilled embedding testing dataset than one composed of IUP embeddings, suggesting that even with some speech patterns lost, distilled embeddings were better representatives of their communities (and therefore their proposed psychiatric disorder or some other construct) than IUP embeddings. We therefore believe that distilled embeddings have some value if used as representatives of a construct in a testing dataset whilst using a model trained on IUP embeddings.
It should be noted that the 50 posts used for the IUP embeddings of each community were also included in the set of 1000 posts used for the distilled embeddings. While this could be considered a limitation, the results of zero-shot classification tasks 1 and 2 are the opposite of what one would expect this inclusion to cause. Specifically, despite having 100% of the posts technically included in training data via being part of the distilled embeddings, the model trained on distilled embeddings had was less than half as accurate as the model trained on IUP posts that comprised only 5% of the posts used in the testing data. One would normally presume the opposite to occur with this overlap of 50 posts between the training and testing data, further strengthening the notion that these results indicate the process of combining posts in aggregate to create distilled embeddings has affected how speech patterns present in the distilled embeddings versus the IUP embeddings.
An additional explanation for the witnessed inaccuracies may lie in the overclassification of substance use disorders. The total number of distilled embeddings categorized as representing substance use disorder communities is 111, which is by far the highest number of embeddings assigned to any psychiatric disorder category. There is the possibility that this caused the model to become overly sensitive when it comes to detecting substance use disorders; future studies with more balanced numbers of distilled embeddings for each psychiatric disorder may be able to avoid this sensitivity issue.
Psychiatric disorders are often stigmatized, and there is a risk that improper or careless interpretation and analysis of data associated with both mental health and sensitive topics such as hate speech could lead to worsened stigma. Therefore, it should be made clear that the results in this study do not indicate that individuals with psychiatric disorders will engage in or are inclined to hate speech or misinformation. As a matter of fact, they most certainly do not indicate that they are the primary source or even a substantial source of hate speech. Care must be taken to minimize stigmatization and carefully examine the likely complex relations between hate speech, misinformation, and psychiatric disorders.
The matter of the directionality of the relationship between speech patterns in psychiatric disorders and hate speech is far from established: Is the relationship because those with psychiatric disorders are more vulnerable to hate speech, or is it that hate speech communities take on speech patterns and possibly other characteristics resembling psychiatric disorders as they develop? Further studies are needed to better answer these important questions.
Conclusion
In conclusion, our results indicate that hate speech communities on Reddit share speech patterns with Reddit communities for Schizoid Personality Disorder, three Cluster B personality disorders (Borderline, Narcissistic, and Antisocial), and Complex Post-Traumatic Stress Disorder. While the Cluster B disorders would be expected due to prior studies into the Dark Triad and hate speech, this nonetheless offers confirmation that they are acting similarly to their Dark Triad counterparts when it comes to hate speech. Furthermore, the two non-Cluster B disorders have not been discussed in hate speech or misinformation literature to the best of our knowledge, and offer new routes of investigation.
The association in speech patterns between certain psychiatric disorders and hate speech (and misinformation to a lesser extent) suggests that, despite hate speech and misinformation being a social phenomenon, framing and approaching hate speech/misinformation as though they were psychiatric disorders may prove beneficial in combating them. For example, designing counter-messaging against these issues could utilize elements of therapy used for the psychiatric disorders that are most similar to hate speech and misinformation. Although hate speech embeddings being classified most frequently as Cluster B personality disorders could be seen as problematic due to those disorders’ historical resistance to treatment, the successes in treating Borderline Personality Disorder through Dialectical Behavior Therapy indicates that treatment of these disorders is not impossible and could be applied here [59,60]. Furthermore, although hate speech shares similarities with Cluster B personality disorders, that does not mean that it actually is a Cluster B personality disorder; it is most likely less resistant to change than a personality disorder is.
Misinformation’s relations to psychiatric disorder communities proved to remain elusive; a clear connection can be seen in the TDA mapping, and zero-shot classification had over 25% of the misinformation embeddings assigned to psychiatric disorders, but it was difficult to elucidate a clear pattern with the dataset at hand. A more robust dataset may be able to identify clearer trends.
While Topological Data Analysis may run into computational challenges with a high number of IUP embeddings, the use of distilled embeddings allows for TDA to handle over a million posts in a single analysis. While their use is currently effectively limited to testing data, it may be possible to improve them to render them usable for training data as well.
Zero-shot classification proved to be an effective tool for identifying which psychiatric disorder communities were most similar to hate speech and misinformation communities. However, the addition of TDA allowed for a visualization of the space, and ultimately allowed for the observation that psychiatric disorders effectively had a spectrum of similarity when it came to hate speech. It also showed that all of our psychiatric disorder community embeddings were connected and similar to each other, and by extension, hate speech and misinformation, something that the vast majority of control embeddings failed to do. Up to this point, TDA has gone relatively unconsidered in psychiatric research; we suggest making use of its ability to visualize the space that one is using for classification tasks, as it can allow for the identification of routes of investigation that may have otherwise gone unnoticed.
Supporting information
S1 Fig. TDA Mapping of All Distilled Embeddings, Colored by Attention Deficit Hyperactivity Disorder Communities.
Yellow = No Attention Deficit Hyperactivity Disorder Embeddings, Red = Entirely Comprised of Attention Deficit Hyperactivity Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s001
(TIFF)
S2 Fig. TDA Mapping of All Distilled Embeddings, Colored by Anxiety Disorder Communities.
Yellow = No Anxiety Disorder Embeddings, Red = Entirely Comprised of Anxiety Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s002
(TIFF)
S3 Fig. TDA Mapping of All Distilled Embeddings, Colored by Autism Spectrum Disorder Communities.
Yellow = No Autism Spectrum Disorder Embeddings, Red = Entirely Comprised of Autism Spectrum Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s003
(TIFF)
S4 Fig. TDA Mapping of All Distilled Embeddings, Colored by Bipolar Disorder Communities.
Yellow = No Bipolar Disorder Embeddings, Red = Entirely Comprised of Bipolar Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s004
(TIFF)
S5 Fig. TDA Mapping of All Distilled Embeddings, Colored by Depression Communities.
Yellow = No Depression Embeddings, Red = Entirely Comprised of Depression Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s005
(TIFF)
S6 Fig. TDA Mapping of All Distilled Embeddings, Colored by Eating Disorder Communities.
Yellow = No Eating Disorder Embeddings, Red = Entirely Comprised of Eating Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s006
(TIFF)
S7 Fig. TDA Mapping of All Distilled Embeddings, Colored by Obsessive Compulsive Disorder Communities.
Yellow = No Obsessive Compulsive Disorder Embeddings, Red = Entirely Comprised of Obsessive Compulsive Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s007
(TIFF)
S8 Fig. TDA Mapping of All Distilled Embeddings, Colored by Post-Traumatic Stress Disorder Communities.
Yellow = No Post-Traumatic Stress Disorder Embeddings, Red = Entirely Comprised of Post-Traumatic Stress Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s008
(TIFF)
S9 Fig. TDA Mapping of All Distilled Embeddings, Colored by Schizoaffective Disorder Communities.
Yellow = No Schizoaffective Disorder Embeddings, Red = Entirely Comprised of Schizoaffective Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s009
(TIFF)
S10 Fig. TDA Mapping of All Distilled Embeddings, Colored by Schizophrenia Communities.
Yellow = No Schizophrenia Embeddings, Red = Entirely Comprised of Schizophrenia Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s010
(TIFF)
S11 Fig. Topological Mapping Colored by Schizotypal Personality Disorder Communities.
Yellow = No Schizotypal Personality Disorder Embeddings, Red = Entirely Comprised of Schizotypal Personality Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s011
(TIFF)
S12 Fig. Topological Mapping Colored by Substance Use Disorder Communities.
Yellow = No Substance Use Disorder Embeddings, Red = Entirely Comprised of Substance Use Disorder Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s012
(TIFF)
S13 Fig. TDA Mapping of Suicidality Communities Pertaining to Suicide.
Yellow = No Suicidality Embeddings, Red = Entirely Comprised of Suicidality Embeddings.
https://doi.org/10.1371/journal.pdig.0000935.s013
(TIFF)
S14 Fig. Zero-Shot Classification of Hate Speech, with Schizoid Personality Disorder excluded from analysis.
https://doi.org/10.1371/journal.pdig.0000935.s014
(TIFF)
S15 Fig. Zero-Shot Classification of Hate Speech, with Narcissistic Personality Disorder excluded from analysis.
https://doi.org/10.1371/journal.pdig.0000935.s015
(TIFF)
S16 Fig. Zero-Shot Classification of Hate Speech, with Schizoid Personality Disorder and Narcissistic Personality Disorder excluded from analysis.
https://doi.org/10.1371/journal.pdig.0000935.s016
(TIFF)
S17 Fig. Zero-Shot Classification of Hate Speech, with Schizoid Personality Disorder, Narcissistic Personality Disorder, and Complex Post-Traumatic Stress Disorder excluded from analysis.
https://doi.org/10.1371/journal.pdig.0000935.s017
(TIFF)
S18 Fig. Confusion Matrix of Zero-Shot Classification of Hate Speech Distilled Embeddings Using Individual User Post Embeddings as Training Data.
https://doi.org/10.1371/journal.pdig.0000935.s018
(TIF)
S19 Fig. Confusion Matrix of Zero-Shot Classification of Misinformation Distilled Embeddings Using Individual User Post Embeddings as Training Data.
https://doi.org/10.1371/journal.pdig.0000935.s019
(TIF)
Acknowledgments
We thank TAMU Academy of Physician Scientists for a Burroughs Wellcome Fund Scholarship to the first author.
References
- 1.
Mathew B, Dutt R, Goyal P, Mukherjee A. Spread of Hate Speech in Online Social Media. Proceedings of the 10th ACM Conference on Web Science; Boston, Massachusetts, USA: Association for Computing Machinery; 2019;173–82.
- 2. Castaño-Pulgarín SA, Suárez-Betancur N, Vega LMT, López HMH. Internet, social media and online hate speech. Systematic review. Aggression and Violent Behavior. 2021;58:101608.
- 3. Stecklow S. Why Facebook is losing the war on hate speech in Myanmar. Reuters. 2018. https://www.reuters.com/investigates/special-report/myanmar-facebook-hate
- 4. WR Nugent, TE Abrams, AA Joseph. The relationship between violent political rhetoric and mass shootings. J Soc Serv Res. 2022;48(2):246–58.
- 5. Ferrara E, Cresci S, Luceri L. Misinformation, manipulation, and abuse on social media in the era of COVID-19. J Comput Soc Sci. 2020;3(2):271–7. pmid:33251373
- 6. Suarez-Lledo V, Alvarez-Galvez J. Prevalence of Health Misinformation on Social Media: Systematic Review. J Med Internet Res. 2021;23(1):e17187. pmid:33470931
- 7.
Withers KL, Parrish JL, Terrell S, Ellis TJ. The relationship between the “dark triad” personality traits and deviant behavior on social networking sites. In: AMCIS 2017 Proceedings, 2017.
- 8. Ahadzadeh AS, Ong FS, Wu SL. Social media skepticism and belief in conspiracy theories about COVID-19: the moderating role of the dark triad. Curr Psychol. 2023;42(11):8874–86. pmid:34393466
- 9. J. Vossen T, L. Coolidge F, L. Segal D, J. Muehlenkamp J. Exploring the Dark Side: Relationships between the Dark Triad Traits and Cluster B Personality Disorder Features. J Psychiatry Psychiatric Disord. 2017;01(06):317–26.
- 10.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. 2013.
- 11.
Djuric N, Zhou J, Morris R, Grbovic M, Radosavljevic V, Bhamidipati N, editors. Hate speech detection with comment embeddings. Proceedings of the 24th international conference on world wide web; 2015.
- 12. Ilie V-I, Truica C-O, Apostol E-S, Paschke A. Context-Aware Misinformation Detection: A Benchmark of Deep Learning Architectures Using Word Embeddings. IEEE Access. 2021;9:162122–46.
- 13. Singh B, Sharma DK. Predicting image credibility in fake news over social media using multi-modal approach. Neural Comput Appl. 2022;34(24):21503–17. pmid:34054227
- 14. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems. 2020;33:1877–901.
- 15. Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P. Training language models to follow instructions with human feedback. Advances in neural information processing systems. 2022;35:27730–44.
- 16.
Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, et al. Gpt-4 technical report. 2023. https://arxiv.org/abs/2303.08774
- 17. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. Attention is all you need. Advances in neural information processing systems. 2017;30.
- 18. Devlin J, Chang M-W, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv. 2018. https://arxiv.org/abs/1810.04805
- 19. Wei J, Bosma M, Zhao VY, Guu K, Yu AW, Lester B. Finetuned language models are zero-shot learners. arXiv. 2021. https://arxiv.org/abs/2109.01652
- 20.
Rabadán R, Blumberg AJ. Topological data analysis for genomics and evolution: topology in biology. Cambridge University Press. 2019.
- 21. Chazal F, Michel B. An Introduction to Topological Data Analysis: Fundamental and Practical Aspects for Data Scientists. Front Artif Intell. 2021;4:667963. pmid:34661095
- 22. Carlsson G. Topology and data. Bull Amer Math Soc. 2009;46(2):255–308.
- 23.
Reddit. Reddit help. https://support.reddithelp.com/hc/en-us. 2023. 2023 June 20.
- 24. Gaudette T, Scrivens R, Davies G, Frank R. Upvoting extremism: Collective identity formation and the extreme right on Reddit. New Media & Society. 2021;23(12).
- 25. Lawlor A, Kirakowski J. Online support groups for mental health: A space for challenging self-stigma or a means of social avoidance?. Computers in Human Behavior. 2014;32:152–61.
- 26. Melton C, Olusanya OA, Shaban-Nejad A. Network Analysis of COVID-19 Vaccine Misinformation on Social Media. Stud Health Technol Inform. 2021;287:165–6. pmid:34795104
- 27.
Reddit .r/ivermectin. https://www.reddit.com/r/ivermectin. 2023. 2023 June 20.
- 28.
Reddit.r/vaccinelonghaulers. https://www.reddit.com/r/vaccinelonghaulers. 2023. 2023 June 20.
- 29. Klein C, Clutton P, Dunn AG. Pathways to conspiracy: The social and linguistic precursors of involvement in Reddit’s conspiracy theory forum. PLoS One. 2019;14(11):e0225098. pmid:31738787
- 30. Zadrozny B, Collins B. Reddit bans Qanon subreddits after months of violent threats. NBC News. 2018;:12.
- 31.
Lin JL. Antifeminism online. MGTOW (men going their own way): transcript. 2017.
- 32.
Reddit. r/MGTOW. https://www.reddit.com/r/MGTOW. 2023 June 20.
- 33.
Reddit. r/incels. https://www.reddit.com/r/incels. 2023. 2023 June 20.
- 34. Solon O. Incel”: Reddit bans misogynist men’s group blaming women for their celibacy. The Guardian. 2017. https://web.archive.org/web/20171108214200/ https://www.theguardian.com/technology/2017/nov/08/reddit-incel-involuntary-celibate-men-ban
- 35.
r/TruFemcels. https://www.reddit.com/r/TruFemcels. 2023. 2023 June 20.
- 36.
Reddit. r/GenderCritical. https://www.reddit.com/r/GenderCritical/.2023 June 20
- 37. Jhaver S, Chan L, Bruckman A. The view from the other side: The border between controversial speech and harassment on Kotaku in Action. 2017.
- 38. Peckford A. Right wing extremism in a video game community? A qualitative content analysis exploring the discourse of the Reddit GamerGate Community r/KotakuInAction. A Closer Look in Unusual Times: Criminological Perspectives from Crim. 2020;862:65–81.
- 39.
Cousineau LS. A positive identity for men?: Pathways to far-right participation through reddit’s/r/MensRights and/r/TheRedPill. Rise of the far right: Technologies of recruitment and mobilization. 2021;127–46.
- 40. Carlson CR, Cousineau LS. Are You Sure You Want to View This Community? Exploring the Ethics of Reddit’s Quarantine Practice. Journal of Media Ethics. 2020;35(4):202–13.
- 41.
Khan A, Golab L. Reddit mining to understand gendered movements. In: 2020.
- 42. Chow A. Reddit Moves to Control Hate Speech and Misinformation in Two Forums. Time. 2022. https://time.com/6160519/reddit-international-hate-speech-ban/
- 43.
Watchful 1. Subreddit comments/submissions 2005-06 to 2022-12. Academic Torrents. https://academictorrents.com/details/c398a571976c78d346c325bd75c47b82edf6124e
- 44.
OpenAI. Embeddings. https://platform.openai.com/docs/guides/embeddings. 2023 June 20.
- 45.
Singh G, Mémoli F, Carlsson GE. Topological methods for the analysis of high dimensional data sets and 3d object recognition. In: PBG@ Eurographics, 2007. 091–100.
- 46.
Zhou Y, Chalapathi N, Rathore A, Zhao Y, Wang B. Mapper Interactive: A scalable, extendable, and interactive toolbox for the visual exploration of high-dimensional data. 2021 IEEE 14th Pacific Visualization Symposium (PacificVis); 2021.
- 47. Thaipisuttikul P, Ittasakul P, Waleeprakhon P, Wisajun P, Jullagate S. Psychiatric comorbidities in patients with major depressive disorder. Neuropsychiatric disease and treatment. 2014;:2097–103.
- 48. Katzman MA, Bilkey TS, Chokka PR, Fallu A, Klassen LJ. Adult ADHD and comorbid disorders: clinical implications of a dimensional approach. BMC Psychiatry. 2017;17(1):302. pmid:28830387
- 49. Resick PA, Bovin MJ, Calloway AL, Dick AM, King MW, Mitchell KS, et al. A critical evaluation of the complex PTSD literature: implications for DSM-5. J Trauma Stress. 2012;25(3):241–51. pmid:22729974
- 50. Jowett S, Karatzias T, Shevlin M, Albert I. Differentiating symptom profiles of ICD-11 PTSD, complex PTSD, and borderline personality disorder: A latent class analysis in a multiply traumatized sample. Personal Disord. 2020;11(1):36–45. pmid:31259603
- 51. van Bergen AH, Verkooijen S, Vreeker A, Abramovic L, Hillegers MH, Spijker AT, et al. The characteristics of psychotic features in bipolar disorder. Psychol Med. 2019;49(12):2036–48. pmid:30303059
- 52. Kraus G, Reynolds DJ. The “A-B-C’s” of the cluster B’s: identifying, understanding, and treating cluster B personality disorders. Clin Psychol Rev. 2001;21(3):345–73. pmid:11288605
- 53. Ritter K, Dziobek I, Preissler S, Rüter A, Vater A, Fydrich T, et al. Lack of empathy in patients with narcissistic personality disorder. Psychiatry Res. 2011;187(1–2):241–7. pmid:21055831
- 54. Salgado RM, Pedrosa R, Bastos-Leite AJ. Dysfunction of Empathy and Related Processes in Borderline Personality Disorder: A Systematic Review. Harv Rev Psychiatry. 2020;28(4):238–54. pmid:32692088
- 55. Pluta A, Mazurek J, Wojciechowski J, Wolak T, Soral W, Bilewicz M. Exposure to hate speech deteriorates neurocognitive mechanisms of the ability to understand others’ pain. Sci Rep. 2023;13(1):4127. pmid:36914701
- 56. Bilewicz M, Soral W. Hate speech epidemic. The dynamic effects of derogatory language on intergroup relations and political radicalization. Political Psychology. 2020;41:3–33.
- 57. Leichsenring F, Leibing E, Kruse J, New AS, Leweke F. Borderline personality disorder. Lancet. 2011;377(9759):74–84. pmid:21195251
- 58. Ojha U, Li Y, Lee YJ. Towards Universal Fake Image Detectors that Generalize Across Generative Models. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 24480–9.
- 59. Choi-Kain LW, Finch EF, Masland SR, Jenkins JA, Unruh BT. What Works in the Treatment of Borderline Personality Disorder. Curr Behav Neurosci Rep. 2017;4(1):21–30. pmid:28331780
- 60. Low DM, Rumker L, Talkar T, Torous J, Cecchi G, Ghosh SS. Natural Language Processing Reveals Vulnerable Mental Health Support Groups and Heightened Health Anxiety on Reddit During COVID-19: Observational Study. J Med Internet Res. 2020;22(10):e22635. pmid:32936777