2.1. Dataset development and preprocessing in Roman Urdu

Research has also been conducted to augment Roman Urdu hate speech datasets to overcome the scarcity and noise of the data. Azam et al. [1] used generative augmentation with MT5 and mBERT-MLM models to augment the RUSOLD and RUT datasets, achieving a maximum of 86% F1-score. Bilal et al. [2] introduced the RU-HSD-30K dataset, containing 30,000 posts, with normalization and phonetic mapping using UrduPhone, and achieved a maximum of 88.5% F1-score using the Bi-LSTM model with attention. Other datasets include RU-PHS [7] with 5,002 geo-tagged tweets, and BERT-RU [4], pretrained on 173K messages, achieving 97.25% F1. Other related work, such as that of Atif et al. in [3], further added more fine-grained labeling for hate types, while PURUTT in [15] and ORUD-Detect [16] provided larger corpora, using hybrid models or fine-tuned LLMs to surpass 90% F1.

2.2. Classical and deep learning approaches

Earlier methods used SVMs, Random Forests, and TF-IDF features but were not effective in handling out-of-vocabulary words and their ambiguity. GRUs with attention have also been used to achieve competitive F1-scores on smaller datasets in research papers like [17]. Hybrid models such as Passion-Net [5] and MBERT-SVM pipelines have also been used to achieve better results than transformer-based models on Roman Urdu datasets. BERT-based models have also been used to achieve good results, i.e., 94% F1-score.

2.3. Multilingual and transformer-based architectures

The performance of multilingual transformers is impressive in low-resource hate speech detection. XLM-RoBERTa and MuRIL, fine-tuned models, have shown more than 90% accuracy in Urdu and Indic hate speech detection tasks [6,18]. RomanBERT, a model that uses fastText subword-level embeddings, obtained an F1-score of 0.95 in the Roman Urdu hate speech detection task, which proves the effectiveness of semantic representation. QLoRA allows memory-efficient fine-tuning of large language models using low-rank adapters and quantization, which retains the performance of models in low-resource settings [19]. This is a low-rank adaptation of multilingual transformers, which has shown effectiveness in hate speech detection. In addition, lightweight transformer models, such as DistilBERT, have shown impressive performance with lower complexity [20]. This proves the effectiveness of efficient transformer models. Interpretability techniques, such as LIME, are used to increase transparency by explaining category-level predictions [21].

2.4. Cross-domain, multimodal and informal contexts

Studies on code-mixing, cross-domain, and multimodal settings emphasize the significance of preprocessing. Bi-LSTM with EfficientNet [22], ELMo and LASER embeddings [23], and CMSA-mBERT on code-mixed Roman Urdu–Punjabi text [24] emphasize that the model’s performance is largely dependent on the preprocessing phase. Other associated tasks such as sarcasm detection [25], insult detection [26], and hate speech/offensive text detection [27] emphasize the significance of the joint effect of lexical and contextual features.

2.5. Gaps and motivation for our approach

While prior studies show promising results, they reveal key gaps:

• Most models treat contextual and lexical signals independently, limiting nuanced understanding.
• Lexicon-based cues such as slurs and profanities are underutilized in transformer pipelines.
• Negation handling and fine-grained hate intensity modeling are often absent.
• Few works incorporate auxiliary supervision or explicitly guide models using hate lexicon gradients.

To address these gaps, we propose LEXF-ATT-XLM, a hybrid model that fuses multilingual contextual embeddings (XLM-R), sequential structure (BiGRU), and a novel lexicon-aware auxiliary objective. Our model incorporates hate lexicon intensity scores using an auxiliary supervision objective in a multi-task learning setup. Domain-specific preprocessing especially negation-aware normalization further enhances our model’s robustness to Roman Urdu’s noisy and unstructured nature. As illustrated in Table 1 Comparative Analysis of Existing Studies on Roman Urdu Hate Speech Detection (2022–2025), our approach significantly outperforms prior models by effectively addressing the linguistic challenges unique to Roman Urdu through a multi-pronged architectural and preprocessing strategy.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Summary of related studies on Roman Urdu hate speech detection, including datasets, models, and key results.

https://doi.org/10.1371/journal.pone.0354875.t001

3.1. Dataset description and statistics

The category-wise distribution is presented in Table 2. This study operates the RU-HSD-30K dataset, consisting of 30,000 Roman Urdu tweets annotated into Hate (H) and Normal (N) categories [2]. The category-wise distribution is presented in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Category-wise distribution of instances in the RU-HSD-30K dataset.

https://doi.org/10.1371/journal.pone.0354875.t002

Although the dataset is relatively balanced, a slight class skew is present. To ensure fair and reliable performance evaluation, weighted evaluation metrics are adopted throughout the experiments.

To obtain robust performance estimates and reduce variance due to data partitioning, we employ 3-fold stratified cross-validation over the entire dataset. Stratification preserves the original class distribution within each fold, ensuring consistent representation of both Hate and Normal classes across training and validation splits. In this setup, each instance is used once for validation and twice for training.

For each fold:

• Two folds are used for training.
• One fold is used for validation.

The process is repeated three times.

Final performance is reported as the average across all folds.

To mitigate optimization bias arising from class imbalance, imbalance handling strategies are applied only to the training partition of each fold. Specifically:

• Minority class samples are upsampled using random resampling with replacement.
• A class-weighted cross-entropy loss is employed.
• A weighted random sampler is used during mini-batch construction.
• Importantly, the validation partition remains unaltered to ensure unbiased and realistic performance evaluation.

3.2. Input representation and preprocessing

The model operates on Roman Urdu tweets that undergo specialized preprocessing to handle the linguistic idiosyncrasies of Roman Urdu text. Each preprocessed tweet is tokenized and fed into the XLM-RoBERTa transformer to obtain rich contextual embeddings. To reduce the noise like informal grammar, inconsistent spellings, and code-mixed elements and to preserve semantic cues, a specialized preprocessing pipeline is applied:

• Character Normalization: Repeated characters are compressed to a single or acceptable repeat (e.g., “bohottt” → “bohot”) to standardize expressive variations.
• Negation Preservation: Negation words (e.g., “nahi”, “mat”) are prefixed to subsequent tokens using a not_ marker (e.g., “nahi samjha” → “not_samjha”) to preserve polarity and syntactic dependencies.

To explicitly preserve negation semantics within contextual embeddings, tokens falling under the scope of a negation cue are transformed using the not_ prefix. This allows the transformer encoder to treat negated and non-negated forms as distinct lexical units (e.g., samjha vs. not_samjha), thereby reducing polarity ambiguity during representation learning. The transformation is applied only for contextual embedding purposes and does not eliminate the original lexical identity required for downstream lexicon-based processing.

• Removes special characters while preserving alphanumeric and whitespace
• All text is converted to lowercase to unify case-based variations.
• Tweets with missing or null content are excluded. Class labels are mapped to binary form for supervised classification: Hate = 1, Non-Hate = 0.

Each cleaned tweet is then passed to two parallel modules: a transformer-based encoder for contextual embedding and a lexicon scoring mechanism for hate intensity calculation.

3.3. Lexicon construction and scoring

Lexicon-guided scoring refers to using a predefined dictionary of offensive words (lexicon) where each word is assigned an intensity score. The model uses this lexicon to enhance its understanding of hate speech by considering these predefined scores. Each word in the lexicon was assigned a weight based on the severity of the word, which ranged from 0.75 to 2.475. These weights are calculated during the lexicon development process.

For a tweet, the lexicon-based hate intensity score is calculated as the sum of weights of all matched offensive terms present in the tweet. Thus, the final tweet-level score can exceed the maximum weight assigned to a term, which is 2.475, in cases where there are multiple hate expressions present in the same tweet. This calculates the additive hate intensity present in the tweet, which is used as an auxiliary regression target but not concatenated with the contextual embeddings.

3.3.1. Lexicon score annotation protocol.

A hate lexicon for Roman Urdu has been created using the training corpus RU-HSD-30K. Frequently occurring abusive words and hate expressions have been identified using frequency filtering. The hate lexicon comprises 1,248 unique hate words.

To capture varying levels of severity, each lexicon term was assigned a graded intensity score in the range:

The scale was defined as:

• 0.75–1.00: Mildly offensive expressions
• 1.01–1.75: Moderately abusive or targeted insults
• 1.76–2.50: Highly offensive or explicit hate speech

3.3.1.1. Annotation procedure: Three native Urdu speakers with expertise in linguistics and computational text analysis independently rated each lexicon term based on semantic severity and contextual usage in Roman Urdu social media discourse.

3.3.1.2. Final score computation: For a lexicon term annotated by annotators, the final intensity score is computed as:

where denotes the score assigned by annotator .

Final values reflect consensus-adjusted averaging after reconciliation of disagreements.

3.3.1.3. Inter-annotator agreement: Annotation reliability was measured using Krippendorff’s Alpha (α) for ordinal data:

where and represent observed and expected disagreement, respectively.

For ordinal scoring, disagreement is computed using squared distance:

The obtained agreement score:

indicates substantial inter-annotator reliability.

3.3.1.4. Lexicon feature integration: Given an input sequence , the lexicon feature is computed as:

where denotes the lexicon set and is the intensity score of a matched term. The aggregated score is incorporated as an auxiliary regression supervision signal within the LEXF-ATT-XLM model to guide hate-sensitive representation learning.

The lexicon score in our model is employed solely as an auxiliary regression signal to provide weak supervision for hate-sensitive representation learning, rather than as a standalone intensity metric. While the raw summation of term intensities reflects cumulative hate cues, explicit normalization (e.g., dividing by tweet length) is unnecessary because the XLM-RoBERTa embeddings and multi-head attention mechanism inherently capture positional and semantic dependencies, enabling the model to distinguish between sparse and dense occurrences. Moreover, the final hate classification is calculated by non-linear transformation and weights, which regulate the saturation effect and prevent the linear scaling of the calculated hate intensity with the number of lexicon hits. The proposed model ensures the effectiveness of the lexicon aggregation for representation learning, while the contextual and non-linear models handle the positional, dilution, and saturation effects.

3.3.2. Consistency between negation preprocessing and Lexicon lookup.

In Section 3.2, during preprocessing, negations are processed. A “not_” prefix is appended to words within the negated scope. For example, “samjha” becomes “not_samjha.” However, during lexicon matching, the original word is used before the prefix is appended.

Specifically, during lexicon feature computation, tokens containing the not_ prefix are decomposed as follows:

If the stripped base token exists in the lexicon , its corresponding intensity score is retrieved. However, since the token originated from a negated context, its contribution is attenuated.

Formally, the adjusted score is computed as:

where in our implementation.

The final lexicon feature is:

This design allows contextual embeddings to retain negation-aware semantics (via the not_ token transformation), while lexicon scoring operates on the canonical base form with controlled attenuation.

3.4. Proposed model architecture

The detailed model architecture integrates contextual language representations and lexicon-guided scoring within a unified deep learning model. As illustrated in Fig 2 the model comprises the following core components:

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Architecture of the LEXF-ATT-XLM model showing the integration of contextual embeddings, BiGRU, multi-head attention, and lexicon-based auxiliary supervision.

https://doi.org/10.1371/journal.pone.0354875.g002

3.4.5. Training and optimization.

To tackle class imbalance, both Weighted Random Sampling and a weighted cross-entropy loss were employed. Weighted Random Sampling ensures that there is a balanced batch of hate and non-hate samples during training. At the same time, the weighted cross-entropy loss function assigns greater weights to minority class samples, thus focusing on the correct classification of minority class data. Thus, the overall effect of these two techniques is stability during training, as there is balanced data in batches and at the same time, there is correction at the loss function level.

For the fine-tuning of the transformer-based XLM-RoBERTa model with stability and without overfitting on the relatively small RU-HSD-30K dataset, the learning rate for the AdamW optimizer was set to 1 × 10^(−5), and the model was trained for 4 epochs. Preliminary experiments have shown that increasing the learning rate and the number of training epochs results in overfitting, which is indicated by the deteriorated performance on the validation set. The model has not been subjected to early stopping; instead, 3-fold stratified cross-validation has been used for the model to generalize and prevent overfitting. The cross-validation strategy ensures that the model learns without overfitting the training set due to the gradient steps from the optimizer.

For the prediction of the lexicon score, the cross-entropy loss and the mean squared error loss are used together. The final loss is calculated as a weighted sum of the classification loss and the auxiliary loss:

where represents the binary classification loss and penalizes deviations between predicted and actual lexicon scores. The weight was determined empirically through preliminary validation experiments on a held-out subset of the training data, where values in the range [0.05, 0.2] were evaluated. The best balance between primary classification performance and alignment of the lexicon scores was achieved with a weight of 0.1, so that the auxiliary loss does not dominate the primary objective function while at the same time functioning as a regularizer. This prevents overfitting to the lexicon intensities while still utilizing the hate-sensitive representation learning.

The optimizer employed is AdamW with a learning rate of , weight decay, and a ReduceLROnPlateau scheduler. Gradient clipping with a max norm of 1.0 and dropout of 0.3 are incorporated for regularization. The model is trained using 3-fold stratified cross-validation to ensure generalization and mitigate overfitting. The training hyperparameters are summarized in Table 3.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Enhanced LEXF-ATT-XLM model Hyperparameters.

https://doi.org/10.1371/journal.pone.0354875.t003

The final fused feature vector, which incorporates contextual embeddings from the XLM-RoBERTa model that has gone through the BiGRU and multi-head attention mechanisms, as well as the auxiliary lexicon information, is used to train: (1) a binary classifier that differentiates Hate and Non-Hate text, and (2) a regression model that predicts the lexicon scores. This fusion effectively exploits the contextual and lexicon information together, with the auxiliary lexicon loss aiding in the attention to hate-sensitive tokens while maintaining the contextual information in the transformer model.

The learning rate of 1e-5 was selected after testing several values in preliminary experiments. This learning rate provided the best balance between convergence speed and performance, which is crucial for fine-tuning large models like XLM-RoBERTa. A dropout rate of 0.3 was chosen to prevent overfitting, especially given the relatively small dataset size. Higher dropout rates resulted in underfitting, while lower rates led to overfitting. This dropout rate offered a suitable balance for model generalization

3.5. Baseline and ablation study design

To rigorously evaluate and validate the effectiveness of the proposed Enhanced LEXF-ATT-XLM model and empirically verify the contribution of each component of the architecture, we will perform a set of comparisons with basic yet relevant baselines and ablations.

The evaluated configurations include:

• XLM-R Base: Fine-tuned XLM-RoBERTa with only a classification head, without the lexicon auxiliary task, BiGRU layers, or multi-head attention. This model serves as the primary baseline.
• BiGRU: Removes the sequential BiGRU layers, connecting XLM-R directly to the attention mechanism.
• ATT: Removes the multi-head attention mechanism, replacing it with mean pooling.
• LEXF: Removes the lexicon scoring auxiliary task and lexicon-based features.

These ablation variants are designed for testing the impact of sequential modeling, attention, and lexicon supervision separately. For testing these ablation models, all of them are trained under similar conditions and compared fairly.

The weighted F1-score is utilized as a primary evaluation metric for these experiments because of class imbalance in the RU-HSD-30K data set. Quantitative evaluation of these ablation experiments is presented in Section 4.

3.6. Evaluation metrics

The performance was measured with the help of standard metrics, Precision, Recall, F1-score, and Accuracy which was calculated over all folds. These metrics can provide a comprehensive understanding of the accuracy of the model in terms of the classification results, especially in dealing with the imbalance of the Roman Urdu hate speech data. Measures taken on population level, average metrics over folds were reported, such as Accuracy, Macro F1, Weighted F1, along with class-wise F1-scores, both of Hate and Non-Hate classes and revealed the overall effectiveness of the proposed model and its robustness.

4.1. Cross-validation performance

We evaluated the proposed LEXF-ATT-XLM model using 3-fold stratified cross-validation to ensure robustness and generalization across different data splits. For each fold, it was trained for four epochs under identical hyperparameter settings. To measure its performance, accuracy, precision, recall, macro F1-score, weighted F1-score, and class-wise F1-score were considered for Hate and Non-Hate categories. The best epoch for each fold is shown in Table 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. 3-fold cross-validation results (Accuracy and F1 scores) on RU-HSD-30K.

https://doi.org/10.1371/journal.pone.0354875.t004

The model attained an average weighted F1-score of 0.8883, indicating balanced performance across both hate and non-hate categories. The model attained an average weighted F1-score of 0.8883, indicating consistent performance across both hate and non-hate categories. However, the RU-HSD-30K dataset is nearly balanced, there is very little difference between the Hate (0.8894) and Non-Hate (0.8871) class F1-score, and it is statistically insignificant. This is a clear indication that the proposed model is able to generalize very well without any bias towards any label. However, it is worth noting that the major challenge with hate speech detection for Roman Urdu is not class imbalance.

Throughout the training process, we observed a consistent reduction in training loss across epochs, with the best performance typically achieved at epoch 4. This convergence pattern highlights the stability of our model architecture in learning from both contextual and lexicon-based cues.

4.2. Contributions of methodological components

Each of these components of the model architecture uniquely contributed to the final performance. We used XLM RoBERTa embeddings, which provided us with contextualized representations, especially helpful in handling noisy Roman Urdu text. We used the BiGRU encoder to model the sequential dependencies in the text, while the multi-head attention helped in focusing on the hate-indicative tokens in the text. We used the scores obtained using the lexicon as an auxiliary regression objective, thus emphasizing the hate patterns in the text.

Compared to simpler pooling-based approaches, BiGRU was useful in modeling user text as it can capture sequential dependencies. This was useful in capturing hate/toxic meanings, which are often spread throughout the sentence. Attention was also very useful in the final performance. It helped in focusing on the hate-indicative parts of the text. It also helped in making the model more interpretable since we can trace the decision-making process of the model. It is worth noting that we used the lexicon features as an additional objective. Lexicon scores were incorporated as an auxiliary regression target, providing an additional supervision signal without direct feature-level fusion. This dual usage of lexicon information reinforced hate-related patterns during training and helped align internal representations with the semantic notion of toxicity.

4.3. Lexicon-aided predictions and case analysis

To further understand the model’s behavior, we have also carried out qualitative analysis on a wide range of real Urdu text data. Table 5. presents sample predictions along with model confidence scores and lexicon hate scores.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Sample predictions with model output, probabilities, lexicon scores, and hate term analysis.

https://doi.org/10.1371/journal.pone.0354875.t005

Several important insights emerge from these examples. First, the model successfully flags explicit hate speech that aligns with lexicon cues (e.g., high lexicon scores). However, it also identifies hate speech in semantically hateful sentences with zero lexicon score (e.g., “main tumse nafrat karta hoon”), demonstrating its contextual generalization ability beyond keyword matching. This is a direct result of combining deep contextual modeling with lexicon-informed supervision.

On the other hand, a few borderline or sarcastic expressions like “Tumhare jaise logon se nafrat hai” and “Ye banda pagal hai, par dosti hai”, were misclassified or predicted as non-hate, reflecting the current model’s focus on overt hate and highlighting challenges in detecting implicit hate and nuanced intent.

4.4. Ablation study results

To empirically verify the contribution of each architectural component in the proposed LEXF-ATT-XLM model, we carried out systematic ablation tests on all variants of the model. These variants include XLM-R Base (a fine-tuned version of XLM-RoBERTa with a classifier on top), BiGRU (no sequential BiGRU layers), ATT (no multi-head attention mechanism, using mean pooling as alternative), and LEXF (no lexicon auxiliary supervision/lexicon features). All models were trained under the same conditions (i.e., using the same data splits, hyperparameters, and evaluation protocol). We use the averaged performance across 3-fold stratified cross-validation with weighted F1 as primary metric. Table 6 presents the ablation study results, demonstrating the weighted F1-scores achieved by the individual components of the proposed LEXF-ATT-XLM model.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Ablation study showing weighted F1 for components of LEXF-ATT-XLM.

https://doi.org/10.1371/journal.pone.0354875.t006

The XLM-R base model with fine-tuning yields a weighted F1 of 0.8463, which forms a solid transformer base. Adding BiGRU can enhance performance to 0.8703 (+2.84%), which shows the effectiveness of sequential modeling in modeling phrase-level and contextual interaction in Roman Urdu. Multi-head attention also enhances F1 to 0.8733 (+3.19) meaning that attention assists in refining on salient hate-indicative token. The removal of lexicon supervision (−LEXF) results in a 0.8603, a reduction of 3.15 percent of the full model, which highlights the relevance of explicit lexicon knowledge in the task of informal spelling and code-mixing typical of Roman Urdu. The complete model has a weighted F1 of 0.8883 that is a 4.96 percent increase over the XLM-R baseline. These findings prove that every added element makes a positive contribution to performance instead of merely making the situation more complex.

Fig 3 visually demonstrates the changes added by each component which have been incremental. The reliable increasing pattern in the XLM-R baseline to the entire LEXF-ATT-XLM model indicates the cumulative efficiency of sequential modeling, attention processes, and lexicon directed supervision.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Ablation study showing the effect of model components on weighted F1 (3-fold average) for RU-HSD-30K.

https://doi.org/10.1371/journal.pone.0354875.g003

4.5. Component analysis and trade-offs

The ablation results reveal three key insights:

1. Lexicon contributions are substantial. The largest single component drop is seen when lexicon supervision is removed, at −3.15% compared to the full model, which again underscores the benefit of using domain information in low-resource and noisy Roman Urdu.
2. Attention and sequential modeling are complementary. Attention (+3.19% over baseline) slightly outperforms BiGRU (+2.84% over baseline), suggesting that focusing on salient tokens is marginally more beneficial than adding sequential capacity alone. Both, however, provide meaningful gains.
3. Synergy and overlap. The sum of single-component gains (2.84% + 3.19% + 1.65% = 7.68%) is larger than the observed combined improvement (4.96%), indicating partial overlap in the information captured by different components. Nonetheless, the full model outperforms all partial variants, confirming that the components capture complementary information useful for detection.

Practical implication: For low-resource languages such as Roman Urdu, integration of lexicons is of utmost importance, followed by attention and sequential modeling as subsequent improvements.

Limitations. We report average cross-validation metrics. However, statistical testing (e.g., paired t-tests over folds, or confidence intervals using bootstrapping) of metrics for all pairwise comparisons is not carried out here and is left for subsequent work.

4.6. Comparison with state-of-the-art

We compare LEXF-ATT-XLM to prior representative work on Roman Urdu hate speech detection, evaluated on the same dataset RU-HSD-30K for fair evaluation. Table 7 presents a comparative evaluation of the proposed LEXF-ATT-XLM model against existing state-of-the-art approaches on the RU-HSD-30K dataset.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Comparison of LEXF-ATT-XLM with prior work on RU-HSD-30K.

https://doi.org/10.1371/journal.pone.0354875.t007

On RU-HSD-30K, our model improves over Jan et al. [2] by +0.33% F1 (0.8883 vs. 0.885). Although the differences in the datasets, annotation schemes, and evaluation protocols make it difficult to draw any hard and fast comparisons, the overall trend in performance improvements in all ablations and baseline comparisons suggests that lexicon-based multi-task learning, along with the sequential and attention modules, is useful in Roman Urdu hate speech detection.

4.7. Overall observations and future directions

The effectiveness of our hybrid model is justified by the fact that it has a strong overall performance and, in particular, has the capability of generalizing and is not limited to simple key word spotting. The interpretation of lexicon scores offered using either concatenation or regression forms offered interpretability and regularization to the model to make the distinction between high-intensity hate, mild-toxic language, and benign language.

Fig 4. Summarizes the model’s performance at Fold 1, Epoch 4. The LEXF-ATT-XLM model accurately classified most hate and non-hate instances, with relatively few misclassifications. It showed strong sensitivity to hate content, reflected by a low number of false negatives, and maintained solid performance on non-hate data despite slightly higher false positives. The accuracy, recall, and F1-scores were balanced well in both classes. Additional support of the model is confirmed by the macro and weighted averages to control the imbalance in classes. This was the most successful period of Fold 1, which denotes the stability of the model in the hate speech recognition in Roman Urdu.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Confusion matrix for LEXF-ATT-XLM predictions on RU-HSD-30K.

https://doi.org/10.1371/journal.pone.0354875.g004

The model however, sometimes fails in pragmatic understanding, especially in sarcastic, coded, or indirect hate cases. In future work, this could be addressed by integrating additional pragmatic features, sentiment cues, emoji/token context, or conversation structure. Attention weight visualization and interpretability analysis will also be explored in future work, which can provide deeper insights into the decision-making process of the model. Moreover, training the model on larger or more diverse corpora of Roman Urdu, possibly with the help of data augmentation or pseudo-labeling techniques, can be helpful for better generalization. To conclude, the proposed Enhanced LEXF-ATT-XLM model offers a linguistically well-informed and precise solution for the detection of hate speech in the Roman Urdu language, which can be extended to similar low-resource and code-mixed language scenarios.