Offensive language detection techniques

Abusive language is an unwelcome online conduct based on using different remarks intended to be demeaning, humiliating, intimidating, mocking, ridicule, insulting, or belittling. These remarks may or may not be based on an individual’s protected status or protected activities such as race, color, religion, sex, national origin, sexual orientation, or gender identity of an individual [8]. By considering abusive language as an umbrella term, that covers different types of online abuse, extensive studies have been done to address hate speech [3, 8–10, 13, 15, 16], offensive language [1, 2, 12], cyberbullying [33, 34], aggression detection [11, 29, 34, 35], and toxicity detection [36].

Employing computational linguistics methods for identification of offensive language and hate speech in social media has been gaining attention in both machine learning [8, 9] and deep learning techniques [37–39]. The most predictive features such as bag of words, word and character n-grams, or word embeddings in combination with deep neural networks such as Long Short-Term Memory (LSTM) [37, 39] and Convolutional Neural Network (CNN) [24, 38] are used to address this problem. Furthermore, to train an accurate classifier, different supervised classification algorithms such as Support Vector Machines (SVM) [40], Logistic Regression [8, 9], Naïve Bayes, etc., have been employed.

Recently, transfer learning approach, in which prior knowledge gained from one domain will be applied to solve another problem from another domain, has been more accentuated to identify offensive and hateful content as evidenced in [13, 15, 35, 41]. Promising results yielded by applying different transformer-based language models, e.g., BERT [25] in different NLP tasks indicate the effectiveness of this transfer-learning based approach in different classification tasks as well as offensive language identification. Mozafari et al. [13] proposed different fine-tuning strategies with different neural network architectures as a classifier on top of the pre-trained BERT model to classify tweets as racist, sexist, hate, or offensive. Mnassri et al. [15] introduced an ensemble of multiple fine-tuned BERT models along with various deep neural networks, based on bootstrap aggregating and stacking in hate speech and offensive language detection tasks.

On the other hand, multilingual pre-trained transformer networks, e.g., multilingual BERT [25] and XLM [26] facilitate different downstream NLP tasks, especially in hate speech detection task with low-resource data [42]. A variety of ML and DL models have been proposed to address multilingual offensive language identification in Social Media in SemEval-2020 [1] including word statistics features [43] along with pre-trained transformer networks such as BERT [44], ALBERT and RoBERTa [28], mBERT [45], and XLM-RoBERTa [46]. Considering multilingual hate speech and offensive language detection, Aluru et al. [47] performed an exhaustive experiment on 9 languages from 16 publicly available hatred datasets on Facebook and Twitter. They used LASER and MUSE embeddings to extract sentences and word embeddings, respectively, and leveraged different neural networks models such as CNN-GRU, BERT, and mBERT to identify hateful content in both monolingual and multilingual scenario. Corazza et al. [48] investigated hate speech detection task in three different languages English, Italian, and German with a combination of neural network architectures (e.g. LSTM, BiLSTM, and GRU) and word-level and tweet-level features (e.g. n-grams, word embeddings, emotion lexica, social network-based features, etc.). Ousidhoum et al. [49] introduced a new multilingual dataset comprising English, French, and Arabic tweets from Twitter annotated with a variety of attributes related to Directness, Hostility, Target, Group, and Annotator. They experimented multilingual and multi-task learning approaches to address the problem of hate speech detection on the multilingual dataset. Moreover, Mozafari et al. [23] worked on the identification of hate speech and offensive language, using two diverse collections of publicly available datasets, one for hate speech in 8 languages, and the other for offensive in 6 languages using meta-learning.

Language-specific abusive language detection

Although many efforts have been dedicated to address the problem of hate speech and offensive language detection in high-resource languages such as English [8, 9, 50], recently concerns have been raised about other languages as well. Emerging recent shared tasks and academic events such as Kaggle’s Toxic Comment Classification Challenge in English, Automatic Misogyny Identification (AMI) at IberEval [17] and EVALITA [4] including Spanish and Italian languages respectively, identification of offensive language at GermEval [2, 51] in German language, identification of offensive language at SemEval-2019 [50] for English and SemEval-2020 [1] for Arabic, Danish, English, Greek, and Turkish languages, proceedings of the Workshop on Trolling, Aggression and Cyberbullying Workshops [34, 52], and proceedings of the Workshop on Abusive Language Online [5–7] shows the raising concerns towards hate speech and offensive language detection in different languages. These events and shared tasks mainly focused on different types of this phenomenon such as hate, offensive, misogyny, aggression, etc. in variety of languages. Table 1 summarizes the main concerns of the above events and the datasets and languages that are investigated in these tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Shared tasks in identification of abusive language in different types and languages.

https://doi.org/10.1371/journal.pone.0304166.t001

Our survey in Table 1 indicates that the attention towards languages with limited resources such as Bangla, Greek, Arabic, Danish, etc. are increasing, and providing annotated data for abusive content in this kind of languages is principal. Many previous studies have been dedicated to studying hate speech and offensive language detection tasks on some specific low-resource languages. In fact, Mubarak et al. [20] provided a list of obscene words and hashtags, which are common patterns in offensive and rude communications, from Twitter along with a large corpus of annotated user comments for obscene and offensive language detection in Arabic language. Guellil et al. [57] investigated the problem of hate speech against politicians in YouTube’s comments considering comments written with Arabic, Arabizi, Arabic words written with Latin letters, French, and English. Mubarak et al. [21] proposed a method to build an offensive dataset in Arabic language and analyzed the topics, dialects, and gender mostly associated with offensive content. Pitenis et al. [19] introduced the first Greek annotated dataset for offensive language detection on Twitter, named the Offensive Greek Tweet Dataset (OGTD). Experimenting different ML and DL models on Greek offensive language dataset indicated that LSTM and GRU with attention model results in the best performance. Furthermore, a large corpus from Twitter containing 36232 tweets in Turkish language was created by [22] to address the problem of offensive language in Turkish for the first time. In [18] authors proposed a BERT-based approach along with data augmentation techniques to identify aggressive from non-aggressive tweets written in Mexican Spanish. Considering the automatic detection of hate speech in a code-switching environment, where user writes in one language and then switches to another in the same sentence, authors in [58] proposed a pipeline to extract hate speech content in Hindi-English code-switched language (Hinglish) by leveraging profanity modeling, deep graph embeddings, and author profiling.

Low-resource South Asian languages such as Roman Urdu (scripts written in English language characters) and Urdu (scripts written in Urdu language characters) have gained rising attentions [59, 60]. Akhter et al. [59] introduced the first annotated corpus for offensive language detection task in Urdu language and provided profound experiments using ML and DL models to automatically detect abusive comments written in Urdu and Roman Urdu on YouTube’s videos. Khan et al. [60] collected and annotated tweets written in Roman Urdu, named Hate Speech Roman Urdu 2020 (HS-RU-20) corpus, in three levels: 1) Neutral or Hostile, 2) Simple or Complex, and 3) Offensive or Hate speech. They applied different ML and DL algorithms including Naive Bayes, Linear Regression, etc. to investigate the effectiveness of supervised learning techniques for hate speech detection in Roman Urdu.

There has been a significant focus on Persian language recently in the domain of offensive language detection. In fact, in 2021, Dehghani et al. [61] assembled a database of 33k Persian abusive language tweets. The database was tested using a list of 648 abusive Persian words. Implementing a deep neural network based on BERT, it gave good performance. Adding to that, Alavi et al. [62] provided a strategy to improve the BERT-based models’ performance on English and Persian Offensive Language Detection. They worked on generating more effective word embeddings, altering the ‘Attention Mask’ input to modify attention probabilities, which led to an improvement of 10% in Persian language. In 2022, we worked on this language along with many other low-resource ones in cross-lingual few-shot hate speech detection [23], we used meta-learning models based on optimization-based and metric-based (MAML and Proto-MAML) techniques. Moreover, Atei et al. [63] introduced Pars-OFF, a three-tiered annotated corpus prepared to identify offensive words in Persian. It is composed of 10K samples. In 2023, Kebriaei et al. [64] employed keyword-based data selection techniques in order to construct a 38K tweet corpus of Persian hateful and offensive language. They used crowdsourcing and an insulting Persian lexicon to collect the data, then they annotated the samples manually. Adding to that, Sheykhlan et al. [65] created the Pars-HAO, a 8k tweets dataset. They used a keyword-based procedure in order to extract samples with higher exposure to hate speech, which were then annotated by three individuals. As baselines, they integrated Support Vector Machine (SVM) and Logistic Regression (LR), with the Convolutional Neural Network (CNN) model. Then they used the ensemble Hard Voting to improve the classification task.

Based on those previous approaches, we discovered that offensive language detection in Persian has not been widely addressed in academic research due to the lack of publicly available annotated datasets in the domain, and we believe this study provides interesting insights for this task. The goal of our research is to gather tweet data from X and provide annotation practices in order to build an abusive language corpus in Persian. After that, various comprehensive experiments are carried out to assess how well classical ML, DL, and transformer-based models such as ParsBERT, ALBERT-Persian, mBERT, and XLM-RoBERTa perform in detecting offensive language in Persian. Our paper also presents a stacked ensemble methodology aimed at improving the overall performance of the models.

For a better understanding of the structure of our study, we present a diagram (Fig 2) proposing the flow of our paper. This structure shows the main steps we took during our study from literature review to data creation, models development and training, finishing with results and analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Paper structure diagram.

https://doi.org/10.1371/journal.pone.0304166.g002

Data collection

We focused on X because it is one of the most widely used microblogging systems and online platforms for sampling offensive and hateful content in different languages [1], and we retrieved Persian tweets from it using Twitter streaming API. We filtered the stream in Persian language by using both Twister’s language identification mechanism (by setting language parameter in the search query as “fa”) and some most frequent Persian conjunctions (by setting track parameter in the search query) to prevent crawling samples in other languages similar to Persian such as Urdu. The data was collected using a Python scraper for a two-month interval from June to August 2020. We used two main strategies: (1) random sampling and (2) lexicon-based sampling for data collection, which will be explained in the following.

One of the main difficulties in our data collection process is the fact that Twitter streaming API leads to receiving samples that cover just 1% of all tweets in near to real-time and a very small portion of resulted tweets include offensive or hatred content usually [66]. To investigate the ability of random sampling tweets to reflect offensive language in our data collection process, we selected 400 tweets randomly and inspected them by two experts who are native Persian speakers. Scrutinizing randomly sampled tweets by experts revealed that the actual offensive content constitutes a maximum of 2% selected tweets resulting in an unbalanced and inefficient sampling. Furthermore, the vast majority of offensive samples were related to Iranian political parties and governmental issues at that time or a Persian worldwide trending hashtag: (#Don’t_execute, # StopExecutionsInIran), which was launched in support of three young protesters in Iran. Therefore, to prevent a bias against some specific topics or targets during data collection, we used HurtLex seed lexicon [67], to filter more offensive tweets with diversity in topics and targets. HurtLex (https://github.com/valeriobasile/hurtlex) is a multilingual computational lexicon of offensive, aggressive, and hateful words organized in 17 categories in over 50 languages including Persian, with two main labels: conservative and inclusive referring to ‘offensive’ senses and ‘offensive’, ‘not literally pejorative’ and ‘negative connotation’ senses, respectively. We considered all conservative and inclusive words in 17 categories as keywords to filter tweets in our lexicon-based sampling strategy. Employing random sampling and lexicon-based sampling leaves us with 320K and 200K tweets, respectively. Finally, we selected 3000 tweets randomly from each sampling set (random and lexicon-based) for annotation step.

Data annotation schema

Abusive language is an umbrella term that encompasses different types of subtasks such as hate speech, cyberbullying, offensive language, etc. with common or different characteristics and there is a considerable overlap between these subtasks. To have some kind of uniform understanding of different subtasks related to abusive language and to prevent overlap of their definition and annotation, Waseem et al. [68] unified these subtasks by proposing a 2-fold typology to categorize abusive language into two majority incorporated groups: (1) the target of abuse (an individual or a group) and (2) the nature of the language (explicit or implicit). In addition, Zampieri et al. [12] considered the problem of abusive language definition as a whole and attempted to model the task hierarchically in which the type and the target of offensive content were identified. They proposed a three-layer hierarchical annotation scheme to label the Offensive Language Identification Dataset (OLID), a new English corpus from Twitter, as offensive or not-offensive, its type, and its target. Therefore, Following [12, 68], we developed an annotation protocol for our Persian corpus in three levels as follows:

• Offensive language detection: in the first step, tweets are distinguished as offensive or non-offensive. Similar to [12], tweets having any form of explicit or implicit insults, threats, incitement to hatred and violence, dehumanization, or profane language and swear words are considered as offensive. On the other hand, tweets without any form of offense, abuse, or profanity are considered as non-offensive.
• Categorization of offensive language: After discriminating offensive and non-offensive tweets, we categorize the type of offensive tweets as targeted or untargeted. Offensive tweets without any specific targeted profanity and swearing are considered untargeted. However, targeted insult refers to any offensive content addressed to an individual, a group, or others.
• Offensive language target identification: to make more distinct about the target of offensive contents, similar to [12], we use three target classes: individual, group, and other. If tweets include hateful messages purposely sent to a specific target (e.g. a famous person, a named or unnamed participant in a conversation, etc.), it will be labeled as individual. However, offensive tweets towards many potential receivers as a group of people with the same ethnicity, gender or sexual orientation, political affiliation, religious belief, or other common characteristics are defined as targeted group. Here, we do not consider any crowd of people as a group, but a crowd belongs to a specific unity or individual identity. Therefore, abuse and offense towards some individuals not belonging to our definition of group is considered as individual targeted. We consider another category for tweets in which the target of the offensive language does not belong to individual or group categories, and it is a kind of offense toward an organization, event or issue, situation, etc. as non-human entity target. Using different targets of offensive language in our annotation schema results in different concepts of abusive language. For example, offensive tweets targeted at an individual are known as cyberbullying whereas insults and threats targeted at a group are defined as hate speech.

Annotation process.

Since offensive language is a subjective and contextual-based concept that may differ from person to person, culture to culture, or society to society, and Persian is a low-resource language with fewer speakers all over the world in comparison with high-resource languages (e.g. English), employing Persian native speakers for annotating the corpora is crucial. Therefore, we use expert-level annotation approach as a common approach used for annotating low-resource data previously [19, 22, 59] to annotate the Persian corpus. Therefore, three highly educated volunteers from the author’s personal contacts, who were Persian native speakers, were enrolled to annotate the corpus. Two annotators were supposed to annotate all the selected tweets at three levels offensive language detection, categorization, and target identification, and in the case of agreement, the final label was set. Otherwise, the third annotator was asked to label the tweet again and then we took a majority vote. Owing to the subjectivity of offensive language identification in three levels of annotation schema and lack of context in tweets, as a short textual data, the annotating process is challenging with low inter-annotator agreement. Annotation consensus for two annotators on three levels of annotation schema was approximately 73%, in which the agreements in the first level of annotation schema (offensive vs non-offensive) was very high as 86%, in the second level 75%, and in the third level 60%. In the case of disagreement, the third annotator judged. The distribution of labeled data in the three levels of annotation is presented in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Distribution of annotated data in three levels of annotation schema.

A set of 6,000 out of 520,000 sampled data is randomly selected for annotation process.

https://doi.org/10.1371/journal.pone.0304166.t002

A few examples of annotated instances along with their categories for each level of the annotation schema are presented in Fig 3 in the ‘Supporting information’ section [22]. We include both Persian and its English-translated versions of tweets for ease of reading.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Tweet samples (original and translated) from the annotated data with their categories for each level of the annotation schema.

https://doi.org/10.1371/journal.pone.0304166.g003

Baselines

We conduct several experiments utilizing classical ML algorithms along with a different combination from a pool of features such as Term Frequency-Inverse Document Frequency (TF-IDF) on character and word n-grams, Bag of Word (BoW), word embeddings, Part Of Speech (POS) tagging, sentiment scores, etc. to train automatic offensive language detection models.

Classical machine learning (ML).

Initially, we start with a simple linear SVM classifier, as a well-performed classifier in this task according to the literature [40], trained with different tweet representations as feature embeddings. We use a set of three feature extraction methods TF-IDF on character n-grams and word n-grams, where n-grams are a contiguous sequence of n characters or words, and Bag of Words Vectors (BoWV) over fastText.

Features: to extract character n-grams, we consider n = 2 to n = 5. For word n-grams we consider n = 1 to n = 2 and extract word unigrams and bigrams in each tweet and eliminate words with more than 70% of frequency occurrence in all corpus. At the end, using TF-IDF all word and character n-grams are normalized. We use a logistic regression with L1 regularization, to reduce the dimensionality of the feature vectors of TF-IDF character and word n-grams. Considering the co-occurrences of each word in each document (tweet) in our annotated corpus, we create a document-term matrix and use the pre-trained word embeddings fastText with an embedding dimension of 300 to get initial vector representation of each word in tweet. The fastText is a static word embeddings representation of tweets that is pre-trained on Persian version of Common Crawl and Wikipedia (https://fasttext.cc/docs/en/crawl-vectors.html) using fastText model [69]. The average of fastText vector of words in each tweet is considered as tweet representation.

To investigate the impact of other text-mining features such as sentiment analysis scores, linguistic features, etc. on offensive language detection in Persian, we re-implement a state-of-the-art SVM-based classifier proposed by Davidson et al. [9] and map its feature extraction part in Persian language. Therefore, different features are extracted using Parsivar (https://github.com/ICTRC/Parsivar) Python package [70]. We normalize and tokenize each tweet and calculate: TF-IDF weighted word n-grams (unigram, bigram, and trigram); number of characters, words, and syllables in each tweet; number of user mentions, hashtags, retweets, URLs; TF-IDF weighted of POS tag n-grams (unigrams, bigrams, and trigrams of POS tags) in which we filter any candidates with a document frequency lower than 5. Using pertimental (https://github.com/pbarjoueian/pertimental) Python package, we also calculate sentiment polarity scores of each tweet as Negative, Positive, and Neutral. Furthermore, readability scores of each tweet are measured using two metrics Flesch-Kincaid Grade Level and Flesch Reading Ease, with common core measures (words and sentences’ length) and different weighting factors, to indicate how difficult a tweet in Persian is to understand. To calculate these scores, we consider the number of sentences in each tweet as fixed number one. After reducing the dimensionality of extracted feature vector using a logistic regression with L1 regularization, we apply a Logistic Regression with L2 regularization algorithm to train our classifier.

Thus, we define multiple classifiers named SVM_{Character n-grams}, SVM_{Word n-grams}, and SVM_BoWV accompanying Davidson algorithm as classical ML approaches. To summarize the baseline classical machine learning models, we added Table 3. It includes parameters, feature extraction techniques, and the specifications of each model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Baselines ML models.

https://doi.org/10.1371/journal.pone.0304166.t003

Deep learning (DL).

We employ a static word embeddings (e.g. fastText) representation of tweets to train different DL models combining convolutional neural networks (CNN), recurrent neural networks (GRU), and long short-term memory networks (LSTM).

Following previous studies on different publicly available datasets in this domain, we implement different DL models proposed by [24, 38, 39] on Persian annotated data. Authors in [24], proposed a CNN model trained on different features such as character n-grams, word vector embeddings, randomly generated word vectors, and a combination of character n-grams and word vectors to study the problem of hate speech identification. Here, we just use the fastText embeddings of words as word feature vectors, based on semantic information, to train a CNN model. The input of the model uses a 1D convolutional layer with 64 filters with a window size of 4, and it is converted into a fixed length vector using a pooling layer. Then, we add a max pooling layer with a pool size of 2 to capture the most important latent semantic features from the input tweets’ sequences. We use the Rectified Linear Unit (ReLU) activation function for CNN layers. To provide output in the form of probabilities for each of two classes in our binary classification task, we use a softmax activation function in the output layer. Finally, we compile the model by adjusting three parameters: loss, optimizer, and metrics. A binary-crossentropy loss function is used along with the Adam optimizer to adjust the learning rate throughout the training and the accuracy (as metric).

CNN together with Gated Recurrent Units (GRUs) [38] or LSTM [39] have also been explored as potential solutions in hate speech detection for other languages. Inspiring [38], we create a deep neural network combining convolutional and GRU neural networks. As the embedding layer, we use the pre-trained fastText embeddings to map each word in tweets’ sequences into fixed dimensional real vectors. To avoid overfitting, we add a drop-out layer with a rate of 0.2. Then, a 1D convolutional layer with 100 filters with a window size of 4 is added accompanying a ReLU activation function. To reduce the size of each feature map, the amount of parameters, and computations, we add a 1D max pooling layer with a pool size of 4. Then, the extracted features are fed into the GRU layer. Using a global max pooling layer, the highest values in each feature dimension is selected and the output vector is fed into an output layer with a softmax activation function. To train the model and predict probability distribution over two classes, we use the binary cross entropy loss function and the Adam optimizer.

To create a deep neural network combining convolutional and LSTM neural networks, we use the network architecture proposed in [39]. All the layers, structures, and parameters of this model is the same as CNN+GRU model except for GRU layer. Here, in CNN+LSTM model, we add a LSTM layer instead of GRU to the model.

In addition to the aforementioned models, we introduce a model by combining a bi-directional LSTM (BiLSTM) and CNN networks. As the embedding layer, we use the pre-trained fastText embeddings to map each word in tweets’ sequences into fixed dimensional real vectors. To avoid overfitting, we add a drop-out layer with a rate of 0.2. A bi-directional LSTM layer with 128 units followed by a 1D convolutional layer with 100 filters with a window size of 2 is added. The output of CNN layer is average-pooled and max-pooled globally and the results are concatenated. Then, Features encoded by CNN layer are fed into a dense layer with 64 units and the ReLU activation function. The dense layer is followed by the output layer with softmax activation function. The network is compiled with a binary cross entropy loss function and the Adam optimization algorithm.

Thus, we define multiple classifiers named CNN, CNN+GRU, and CNN+LSTM accompanying BiLSTM+CNN as DL approaches to identify offensive language. Our main intuition behind using these neural network architectures is to include both local and global contextual features in our offensive language detection problem. The convolution layer (CNN) will extract local and position-invariant features whereas the LSTM layer considers a long range of context dependencies, semantically, rather than local key-phrases. We have added all the deep learning models we used in Table 4, including feature extraction method, parameters and specifications of each model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Baselines DL models.

https://doi.org/10.1371/journal.pone.0304166.t004

Monolingual and multilingual transformer-based networks

Here we incorporate context into word embeddings (e.g. fastText) using different transformer-based language models (e.g. ParsBERT, ALBERT-Persian, mBERT, and XLM-RoBERTa) and fine-tune different pre-trained contextual representations by training them on our offensive language detection task data. Table 5 summarizes the information of different models used in this study, including their configuration, learning parameters, and training corpora.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Description of the transformer-based neural network models used in identification of offensive language in Persian.

https://doi.org/10.1371/journal.pone.0304166.t005

ParsBERT [30]: In this approach, we use a monolingual BERT model pre-trained on large corpora from numerous subjects (e.g., scientific, novels, news) with more than 2M documents, crawled from Internet’s web pages in Persian language called ParsBERT (https://github.com/hooshvare/parsbert). ParsBERT is a monolingual pre-trained language model based on BERT architecture with the same configurations as BERT_base [25] for Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) tasks. Before fine-tuning ParsBERT model in our downstream task, we first format the input sequences in such a way that each sequence is split into tokens, perpended with the classification token [CLS] to the start and appended the [SEP] token to the end. Then, the sequences are padded to the fixed maximum length of input sequences and attention masks are added to them. Here, we set the maximum sequence length to 128. After feeding input data to the pre-trained model, an additional untrained classification layer will be trained for downstream task. We consider the final hidden state corresponding to the classification token ([CLS]) as the aggregate sequence representation for our offensive language detection classification task. Therefore, we fine-tune ParsBERT on the input data split into 90% and 10% as training and validation sets, respectively, by just adding an output layer as a single linear classifier on top of the pre-trained BERT model.

ALBERT-Persian [31]: is a monolingual pre-trained language model with A Lite BERT (ALBERT) architecture [71] which is trained on a massive amount of Persian public corpora (Persian Wikidumps, MirasText) and six other manually crawled text data from a various type of Persian websites (BigBang Page scientific, Chetor lifestyle, Eligasht itinerary, Digikala digital magazine, Ted Talks general conversational, Books novels, storybooks, short stories from old to the contemporary era). ALBERT-Persian has significantly fewer parameters than a traditional BERT architecture. We fine-tune ALBERT-Persian by exactly the same way as ParsBERT.

mBERT [25]: is a multilingual task-agnostic language representation model with a 12-layer bidirectional transformer trained on Wikipedia pages of 104 languages with a shared word piece vocabulary. This model is pre-trained in two tasks masked language model and next sentence prediction and can be fine-tuned for text classification in any of 104 languages including Persian. Here we use mBERT to circumvent having to train a monolingual model for Persian language as a low-resource language and fine-tune it using a single linear classifier on top of the model.

XLM-RoBERTa [27]: is a transformer-based multilingual masked language model pre-trained on 100 languages, including Persian, using more than two terabytes of filtered CommonCrawl data. To fine-tune XLM-RoBERTa model on our target classification task, we add a linear layer on top of the pooled output, same as previous models.

As shown in Table 5, we use the Base version of ParsBERT, ALBERT-Persian, mBERT, and XLM-RoBERTa pre-trained models and more details regarding fine-tuning the models and hyperparameters used in this study are included in Section Training procedure.

Stacking ensemble model

Apart from single classifiers in our classical ML, DL, and monolingual and multilingual transformer-based neural network approaches, we use an ensemble learning technique to improve accuracy of the offensive language detection task with a combination of the aforementioned classifiers. Ensemble learning is a technique in which applying multiple learning algorithms and aggregating their decisions somehow results in better predictive performance than using any of constituent learning algorithms alone [72]. A variety of ensemble techniques have been applied in different applications and problems [73], specifically in offensive language detection [29], aggression identification [74], and hate speech detection [40] to achieve better performance to single classification methods.

According to the feature extraction and learning mechanisms, different aforementioned classifiers capture different aspects of offensive language detection task. For instance, classical ML approaches advantage of syntactical and hand-crafted features such as character and word n-grams, number of hashtags and mentions, number of exclamation marks, etc. to understand obfuscated and complex words, but they cannot capture contextual or semantical aspects of offensive language in social media content. On the other hand, in offensive language, context is very domain specific and a lack of vector embedding for some words in fastText pre-trained embeddings may affect ML performance while DL models may suffer from generalization due to the lack of enough training data. Furthermore, transformer-based pre-trained language models (e.g. BERT) have a pre-knowledge of a large corpus and can deal with context better even when there is not a large amount of annotated data [14]. Hence, different advantages and drawbacks of different classical ML, DL, and transformer-based pre-trained language models prompt us to make an ensemble classifier out of them to improve the performance of offensive language detection tasks.

Here we use a stacking ensemble technique in which, using a parallel architecture, all classifiers called base-level classifiers are performed independently and their predictions are fed into a meta learner called meta-level classifier to learn how to best combine the predictions from the classifiers. We consider different models in classical ML, DL, and transformer-based neural networks as base-level classifiers and an SVM as meta-level classifier. The stacking ensemble algorithm is summarized in Algorithm 1. First, we extract all features for different classical ML models (line 1) and prepare the input of DL and transformer-based neural networks according to previous subsections (line 2—line 13). Given the labeled dataset, we use a k-fold cross-validation approach on the entire data to learn base-level classifiers separately (line 5—line—line 8) and use their out-of-fold predictions as training features for meta-level classifiers (line 9—line 12).

Algorithm 1 Stacking with K-Fold Cross Validation

 Input: Training data , where x_i ∈ labeled data and y_i ∈ [0, 1], and T base-level classifiers

 Output: An ensemble meta-level classifier H

1 Step 1: Extract required features for classical ML and DL algorithms in base-level classifiers

2 Step 2: Adopt a cross-validation approach to prepare a training set for meta-level classifier

3 Randomly split into K equal-size subsets:

4 for k ← 1 to K do

5  Step 2.1: Learn base-level classifiers

6  for t ← 1 to T do

7   Learn a classifier h_kt from

8  end

9  Step 2.2: Construct a training set for meta-level classifier

10  for do

11   Get a record , where

12  end

13 end

14 Step 3: Select T/2 of base-level classifiers as T′

15 Select two least correlated base-level classifiers in each model category: Classical ML, DL, and Transformer-based DL

16 Step 4: Learn a meta-level classifier among with selected base-level classifiers

17 Learn a new classifier h′ based on the newly constructed dataset: , where is from T′

18 Step 5: Re-Learn base-level classifiers

19 for t ← 1 to T′ do

20  Learn a classifier based on

21 end

22 Return H(x) = h′(h₁(x), h₂(x), …., h_T′(x))

To give a more visual presentation of the stacking ensemble learning algorithm we used, we added the diagram in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Diagram of the stacking K-Fold cross validation.

https://doi.org/10.1371/journal.pone.0304166.g004

In order to facilitate further the comprehensibility of the k-fold stacked ensemble learning algorithm we used in our methodology, we provide a general overview of the mathematical presentation of this ensemble learning technique. In fact, as indicated in its algorithm, in stacking with k-fold cross-validation, the procedure involves numerous steps, including training the base-level models on the folds of the data, producing predictions for the validation fold, and then using these predictions as inputs to train the meta-level classifier. The mathematical presentation of this process is as follows, in which we highlighted the corresponding steps of the algorithm:

1) Base-level models training: (steps 1—2)

Split the dataset into K folds: , where each illustrates a fold.

For each fold :

2) Generate Predictions—Construct training set for the meta-level classifier: (step 2.2)

For each fold :

3) After selection of the least-correlated base-level classifiers, and getting T’ base-level classifiers (step 3), we go to:

4) Stacking: (Steps 4—5)

Create a new dataset from the predictions we got as input for the meta-level classifier where

Train a meta-level classifier H using the stacked features , then re-trained it using the main dataset D. We finally get H as trained meta-classifier.

Experiments

This section presents an extensive set of experiments in the identification of offensive language in our Persian corpus comprising classical Machine Learning (ML), Deep Learning (DL), transformer-based models, and the introduced ensemble model. Three binary classifiers are trained for different levels of annotation. The first classifier discriminates offensive tweets from non-offensive, the second one determines whether an offensive tweet is targeted or untargeted, and the third one predicts the target of offensive content as individual or group. We eliminated tweets labeled as other due to the sparsity of this class in our dataset (only 38 tweets), and considered only individual and group classes in the third classifier.

Pre-processing.

After collecting and annotating our Persian corpus, we perform several text pre-processing steps including: 1) using Parsivar NLP Toolkit https://github.com/ICTRC/Parsivar [70] for normalizing and tokenizing the text; 2) fixing Unicode; 3) removing line breaks, double spaces, emails, URLs, currency symbols, all tweet specific tokens (namely mentions, re-tweet tag, etc.), emoji, punctuation marks, numbers, and non-Persian characters; 4) removing hashtag sign and replacing the hashtag texts by their textual counterparts. In Persian, generally, a hashtag is concatenated with multiple words separated by ‘_’. Therefore, we split the strings after ‘#’ symbol into their constituent words by removing ‘_’; 5) correcting the spelling of words using SpellCheck module in Parsivar NLP Toolkit. Overall, Fig 5 provides a clearer visualization of these preprocessing steps.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Preprocessing steps of the dataset.

https://doi.org/10.1371/journal.pone.0304166.g005

We keep stop words in tweets to extract more contextual information from pre-trained language models such as ParsBERT. To fine-tune the transformer-based models, we used specific tokenizer and vocabulary provided by pre-trained models and we did not remove punctuation marks and numbers.

Training procedure

All classical ML models are performed using scikit-learn (https://github.com/scikit-learn/scikit-learn) python package. The word embeddings dimension in SVM_BoWV is fixed to 300. For DL models, we use Keras (https://github.com/keras-team/keras) python package. The initial embedding layer is seeded with a matrix of one embedding in size 300, derived from Persian pre-trained fastText, for each word in the training dataset. All input tweets are padded to sequences of 128 words and in case of a longer or shorter length, truncating or padding with zero values will be applied, respectively. Models are trained for 8 epochs with a batch size of 16. The learning rate of Adam optimizer is set to 1e-5 leading to obtaining more accurate results.

Transformer-based models are fine-tuned employing publicly available transformers (https://github.com/huggingface/transformers) library for Pytorch (namely pytorch-pretrained-bert). For all considered pre-trained language models (ParsBERT, ALBERT-Persian, mBERT, and XLM-RoBERTa), we utilize the pre-trained model, text tokenizer, and pre-trained WordPiece provided in each pre-trained model to prepare the input sequences for training. The maximum sequence length of the input sentences is set to 128 and in case the input length is shorter or longer, it will be padded with zero values or truncated to the maximum length, respectively. Models are fine-tuned with a batch size of 16 for 3 epochs. An Adam optimizer with a learning rate of 2e-5 is used to minimize the Cross-Entropy loss function. Furthermore, the dropout probability is set to 0.1 for all layers. As offensive language detection is a classification task, we directly modify and fine-tune classification classes of each model (BertForSequenceClassification in models with BERT architecture and XLMRobertaForSequenceClassification for XLM-RoBERTa model, in which a linear classification layer is added on top of the pooled output.

As the implementation and execution environment, we use Google Colaboratory tool (https://colab.research.google.com/notebooks/intro.ipynb) with an NVIDIA Tesla K80 GPU and 12G RAM (The code will be released publicly for further research in the camera-ready version of the paper).

Evaluation metrics:

We utilize a set of evaluation metrics in order to evaluate the performance of our models. In fact, considering the imbalanced nature of our datasets, we consider using Precision, Recall, F1 score per class, and the macro-averaged F1 score metrics.

Precision (P): measures the accuracy of positive predictions made by the model.

, where TP denotes true positives and FP denotes false positives.

Recall (R): or sensitivity, measures how much the model can identify all pertinent data samples.

, where FN denotes false negatives.

F1-score (F1): the mean of Precision and Recall, providing a balanced examination of the model’s performance.

.

Macro-averaged F1 score: averages the F1 score for each class.

, Where N is the number of classes, and F_1i is the F1-score for class i.

Results and analysis

To evaluate different models in a single or ensemble configuration, we use a k-fold cross-validation approach. Due to the imbalanced data that we have, we use Precision (P), Recall (R), and F1-score (F1) per class and macro-averaged F1 score as performance evaluation metrics. For classical ML models, we split data to train and test sets by 0.8 and 0.2. To train DL models and fine-tune other models based on monolingual or multilingual pre-trained language models, we consider 0.1 of the train set as a dev set for hyper-parameter tuning. The reported results are based on the test set.

Single models results.

Regarding the classical ML, DL, and transformer-based models described in Section Methodology, the first experiment aims to assess and compare the performance of different models along with different features in offensive language detection task in three different levels (offensive vs non-offensive, targeted vs untargeted, and individual vs group). The results of the experiments under k-fold cross-validation (k = 5) for three classifiers are demonstrated in Tables 6–8 in terms of P, R, and macro F1-score. In all tables, first column indicates the category of trained classifier using classical ML, DL, or transformer-based neural network algorithms. Second and third columns show performance of classifiers per each class in different annotation levels, respectively. Final column indicates the macro-averaged F1 score.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Results for offensive language identification (first level).

The bold and underlined numbers represent the first and second best scores, respectively, in each category: classical ML, DL, and transformer-based neural networks.

https://doi.org/10.1371/journal.pone.0304166.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Results for targeted offensive language identification (second level).

The bold and underlined numbers represent the first and second best scores, respectively, in each category: classical ML, DL, and transformer-based neural networks.

https://doi.org/10.1371/journal.pone.0304166.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Results for target type of offensive language identification (third level).

The bold and underlined numbers represent the first and second best scores, respectively, in each category: classical ML, DL, and transformer-based neural networks.

https://doi.org/10.1371/journal.pone.0304166.t008

Regarding Table 6, among all models, word n-grams are the most discriminator features for identification of offensive content in Persian where SVM classifier trained on word n-grams (n = 1 to n = 2), achieves the best performance 90.8% in terms of macro F1-score. The second best performing model is ParsBERT obtaining 87.8%, following SVM_Char and ALBERT-Persian with macro F1-score 87.0% and 84.9%, respectively. Among DL models, CNN+GRU outperforms other models with F1 score 82.7% which confirms the results of previous study [38] for English offensive language detection task. Although SVM_Word outperforms other models, a possible reason can be the problem of over-fitting in this traditional classification technique. Comparing the results of different DL and transformer-based models specifies that pre-trained language models such as ParsBERT and ALBERT-Persian that rely on their pre-knowledge existing in their embeddings layers perform better than DL models with fastText embeddings for each word. Furthermore, we can see that all models perform better at identifying non-offensive content compared to offensive where P, R, and F1 score of Non-offensive class are higher than offensive class.

Although the binary classification in the first level of annotation, offensive vs non-offensive, is an important task with a high performance, going deeper into the classification of targeted insult vs untargeted offensive content in the second level of annotation is more challenging. Given Table 7, it is obvious that different classifiers with different features have lower results in identifying whether an offensive tweet is a targeted insult towards an individual or group or it is an untargeted one with general abuse content.

The best macro F1-score, 89.6%, is achieved by training an SVM classifier on character n-grams (n = 2 to n = 5) features. The model trained using word n-grams (n = 1 to n = 2) follows this number by achieving 86.9%. Both ParsBERT and Davidson models provide nearly the same results whereas in DL models there is roughly a 14% reduction (or decrease) in the performance. Among transformer-based models, monolingual pre-trained language models ParsBERT and ALBERT-Persian outperform multilingual pre-trained language models mBERT and XLM-RoBERTa by achieving macro F1-score 69.1% and 57.2% in comparison with 53.1% and 46.2%, respectively. On the other hand, XLM-RoBERTa as a multilingual pre-trained language model performs the worst among all cases. Although the imbalanced data that we have in the second level of annotation gives rise to decreasing performance among DL and transformer-based models, mBERT is better than XLM-RoBERTa in capturing contextual information in Persian as a low resource language.

The results from Table 8 show that identifying target of offensive content as individual or group in the third level of annotation is more precise than the second level classification results, especially for DL and transformer-based models. SVM classifier trained on character n-grams (n = 2 to n = 5) performs the best and XLM-RoBERTa performs the worst among all models. Again pre-trained language model ParsBERT surpasses other transformer-based and DL models by achieving 77.2% F1-score, where ALBERT-Persian, CNN+GRU, and Davidson follow it by achieving 76.3%, 75.3%, and 75.1%.

Overall we observe that there is no single model outperforming others in the identification of offensive vs non-offensive content, targeted insult vs untargeted offense, and targeted offensive language as individual or group. However, SVM trained on character and word n-grams seems to be reliable in most cases where pre-trained language model ParsBERT is the second model with promising results in all three levels of classifications. On the other hand, we believe that the performance of DL models could be improved with a larger amount of labeled data in Persian offensive content, in company with better word embeddings such as fastText embeddings trained on a specific Persian textual content of social media. Generally, it can be conveyed that it is not easy to distinguish between targeted insult or thread and untargeted offensive language by applying the single models where the performance metrics of Untargeted class, in Table 7, are lower than Targeted class.

Ensemble model results.

In the second experiment, we investigate the stacked ensemble classifier using a combination of individual classical ML, DL, and transformer-based classifiers. Firstly, we divide input data into a 90:10 split as training and hold-out test sets. Then, we run k-fold cross-validation (k = 5) on training set to create out-of-fold predictions per each model as base-level classifier. These predictions will be selected and used as training features for meta-level classifier. To create test features for meta-level classifier, we make predictions for the test set (in each fold) and average all 5 predictions per model. The ensemble learning model utilized in this context is outlined in the ‘Stacking ensemble model’ subsection, where we offer a thorough explanation of our implementation approach for this ensemble algorithm.

As mentioned in Section Methodology, different models capture different characteristics of offensive language and they are skillful in this task in different ways. Therefore, obtaining an appropriate combination of base-level classifiers for ensemble learning is a challenge. As the training data for meta-level classifier is generated from the probability predictions of base-level classifiers’ outputs, we consider the correlation between predicted probabilities of each classifier as a linear discriminative metric for base-level model selection.

As depicted in Fig 6, we examine the pairwise Pearson Correlation Coefficient between the predicted probabilities of all base-level classifiers in three levels of annotation. The values range between −1.0 and 1.0 where a value of 1.0 indicates a total positive linear correlation, 0.0 shows a no linear correlation, and −1.0 shows a total negative linear correlation. Here, we consider positive linear correlation or no linear correlation scores for base-level model selection and do not include models with negative linear correlation in generating the ensemble model. From Fig 6a, it is observed that different classical ML, DL, and transformer-based models have different correlations. In classical ML and DL models, there are positive correlations between models’ predictions whereas in transformer-based models this value is low except for ParsBERT and ALBERT-Persian models. This is the same for classifiers in the second-level and third-level of annotation in Fig 6b and 6c except for SVM_Word and Davidson models in the third-level. CNN and BiLSTM+CNN models have the highest correlation contrary to SVM_BoWV and CNN+GRU models which have the lowest correlation among all first-level classifiers. In the second-level, SVM_Char and SVM_Word models have the highest correlation where for the third-level CNN and CNN+GRU models are the most correlated.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Pairwise Pearson Correlation Coefficient between the predicted probabilities of different single classifiers on out-of-fold test set.

First level (a) shows the correlation between the output predictions of classifiers trained on offensive vs non-offensive annotated data. Second level (b) shows the correlation between the output predictions of classifiers trained on targeted vs untargeted samples. Third level (c) shows the correlation between the output predictions of classifiers trained on targeted offensive towards individual or group.

https://doi.org/10.1371/journal.pone.0304166.g006

In this study, we presume that the low correlated or uncorrelated classifiers with high macro F1-score complement each other in an ensemble configuration. Hence, we select two least correlated models in each classical ML, DL, and transformer-based categories as the input of meta-level classifier in stacked ensemble model.

Fig 7 demonstrates the comparison of the ensemble classifier with the individual classifiers selected based on their correlations. It shows the distribution of macro F1 scores in k-fold cross-validation (k = 5). Different individual classifiers that are selected as the base-level classifiers for ensemble stacking among with their performance on out-of-fold test set in terms of averaged-macro F1-score are depicted in Fig 7(a)–7(c) for three level of annotation. For more emphasis, we include the average of macro F1 scores from k-fold cross-validation runs and compare the final performance based on that.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Offensive language identification performance among all models in three levels of annotation.

First level (a), Second level (b), and Third level (c) indicate performance of selected base-level classifiers accompanying stacking ensemble classifier in identification of offensive vs non-offensive, targeted vs untargeted offensive content, and the target of offensive language towards individual or group, respectively.

https://doi.org/10.1371/journal.pone.0304166.g007

For the classification task in the first level of annotation, as shown in Fig 7(a), we can see that the macro F1-score of offensive vs non-offensive language detection task has increased by 5% of its value where the best performing base-level classifier, SVM_Word, achieved 88.3% while stacking ensemble classifier achieves 93.1%. As shown in Fig 7(b), for the second level classifier, stacking ensemble model outperforms all single base-level classifiers by achieving 90.5% F1 score while on the contrary the best performing base-level classifier, SVM_Char, achieves 86.2%. As shown in Fig 7(c), ensemble stacking classifier outperforms the best performing single base-level classifier, SVM_Char with performance 81.7%, in identifying targeted offensive towards individual or group with 5% of improvement.

In summary it is noticeable that the stacking ensemble method that combines the least correlated classifiers in each category (classical ML, DL, and transformer-based models), with a variety of knowledge representation and different learning biases, has achieved the highest macro F1-score among the selected classifiers that performed as an individual classifier. Due to the lower noise included in aggregated results of multiple models in comparison with the results of single models, the stacked ensemble classifier has more stability and robustness in its predictions in the identification of offensive language task.

Future directions

In future work, we aim to deal with our imbalanced dataset by leveraging textual data augmentation techniques for Persian language, using various techniques like generative GPT3 PLM-based models [75] which were used on English data, but we believe we can employ it for low-resource languages like Persian. We aim to train more complex DL and Large Language models even in a multilingual configuration to deal with English-Persian code-mixing offensive language content on online social media. We are also eager to use our dataset to train on the recently released Large Language Models in order to learn more about their efficacy and generalizability, which will improve our comprehension and application of these innovative technologies. Furthermore, we believe that considering linguistic-based characteristics of offensive language in Persian as a low-resource language would give more precise results in detecting such content on social media.