3.1 Overall architecture

To address key challenges in few-shot intent detection—such as limited generalization capability and insufficient semantic representation learning caused by scarce training data—this study proposes a multi-task learning-based joint optimization framework named MBCS (Multi-task Learning with BERT for Classification and Semantic Similarity Comparison). As illustrated in Fig 1(a), MBCS adopts a hierarchical architecture consisting of: 1) An input layer responsible for preprocessing raw text and feature initialization; 2) An encoder layer that constructs deep semantic representations using a pre-trained language model; 3) A task branch layer that enables collaborative optimization through parallel execution of the main text classification task and an auxiliary semantic similarity comparison task. This architecture significantly enhances the representational capacity of the encoder via knowledge transfer across multiple tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. MBCS model architecture.

https://doi.org/10.1371/journal.pone.0335914.g001

Input Layer (Fig 1(a), Input Module): This layer converts raw text into numerical representations understandable by the model. For each input text, it performs tokenization and adds special tokens. The tokenized sequence is then mapped to token IDs from the vocabulary and converted into embedding vectors.

Encoder (Fig 1(a), Encoder Module): Comprising 12 layers, the encoder processes these embedding vectors to capture contextual relationships between tokens. Through multiple stacked encoder layers, the model captures increasingly complex and fine-grained features.

Task Branch Layer (Fig 1(a), Downstream Tasks Module): This layer consists of two sub-tasks—text classification and semantic similarity comparison—both sharing the same encoder. The semantic similarity comparison task enhances the encoder’s ability to learn discriminative textual representations by contrasting semantic vectors within the same class and differentiating those from different classes, thereby supporting the main text classification task. The projection head reduces vector dimensionality. The text classification task serves as the main objective, determining the intent category. It is followed by a fully connected layer that outputs a probability distribution over classes based on the encoder’s representations. This study leverages the Multiple Gradient Descent Algorithm (MGDA) to find a common gradient direction that stabilizes the training process for both tasks.

Additionally, the study incorporates semantic enhancement through a semantics-constrained synonym replacement strategy. By computing attention matrix scores, the most attended words in the sequence are replaced with synonyms of the same part of speech from a corpus, generating new sequences for data augmentation. As illustrated in Fig 1(b), the augmented samples are combined with the original data to retrain the model.

3.2 Fine-tune BERT

3.2.1 Pre-trained language model BERT.

The pre-trained language model BERT [29] learns rich linguistic representations from large-scale corpora by introducing the Masked Language Model (MLM) and Next Sentence Prediction (NSP) tasks. In the MLM task, randomly masked tokens in the input text are predicted by the model, enabling it to learn contextualized word representations. The NSP task, which involves determining whether two sentences are consecutive, helps the model grasp relationships between sentences. This pre-training strategy allows BERT to model both token-level and sentence-level semantic information, thereby capturing complex linguistic structures.

In intent detection tasks, BERT first tokenizes the input text into subword units using the WordPiece algorithm. This approach effectively handles out-of-vocabulary (OOV) words while reducing the size of the vocabulary. A special classification token [CLS] is added at the beginning of each input text to represent aggregated information of the entire sequence. For sentence-pair tasks, a separator token [SEP] is used to distinguish between the two sentences. In single-sentence tasks, the [SEP] token is appended at the end of the text.

The model retrieves token IDs corresponding to the text by looking up the vocabulary, where k denotes the sequence length after adding special tokens. If the input sequence exceeds the model’s maximum supported length n, it is truncated; if shorter, padding tokens [PAD] are appended to reach length n, resulting in the final textual representation . The input vector is obtained by summing three different types of vectors, as shown in Eq (1):

(1)

Among these, are derived from the subword units; indicate which sentence each token belongs to, distinguishing between different sentences; and are incorporated into each token to provide the model with an understanding of the sequential order of the input.

The encoder serves as the core component of the BERT model. Taking the BERT-base model as an example, its encoder consists of 12 Transformer encoder layers stacked in sequence. Each layer primarily comprises two sub-layers: a multi-head self-attention mechanism and a feed-forward neural network. The output of each layer undergoes residual connection and layer normalization to stabilize the training process and accelerate convergence. The input vector is processed by the multi-head self-attention mechanism, producing a context-aware representation matrix . The output Z incorporates interaction information between each token and all other tokens in the sequence, enabling it to capture global contextual relationships. After the multi-head self-attention mechanism, the input vector X is added to the output Z of the self-attention module, as shown in Eq (2). The result of this residual connection, denoted as , is then normalized via layer normalization to obtain .

(2)(3)

After residual connection and layer normalization, the output of the multi-head self-attention mechanism, denoted as , is fed into the feed-forward neural network. The feed-forward network processes the representation of each token independently to further extract features.

The feed-forward neural network consists of two fully connected layers with a ReLU activation function in between. The computational process is in Eq (4):

(4)

Here, W₁ and W₂ are learnable weight matrices, and b₁, b₂ are bias terms. The input and output of the feed-forward network are again processed through a residual connection and layer normalization, ultimately producing the output of the entire encoder. This output then serves as the input to the next encoder layer, iteratively refining the feature representation of the input text.

3.2.2 Fine-tuning.

Fine-tuning refers to the process of adapting a pre-trained model to a specific task by training it on a small amount of task-specific data. As a transfer learning strategy, it leverages knowledge acquired from large-scale pre-training to achieve strong performance on smaller datasets, thereby reducing training costs and enhancing the model’s generalization capability. In text classification tasks, assuming the parameters of the BERT model are denoted as θ, which are initialized using pre-trained parameters , the goal of fine-tuning is to minimize the loss function L on the target task to update the model parameters:

(5)

Here, x denotes the input sample, y the ground truth label, and D the dataset of the target task. represents the BERT model parameterized by θ. L is the cross-entropy loss function. The study employs full fine-tuning, optimizing all layers starting from . The update process is defined in Eq (6), where denotes the learning rate:

(6)

In the joint task of text classification and semantic similarity comparison, the study employs a multi-task fine-tuning approach where both tasks share underlying parameters. The optimization objective is defined in Eq (7), with T denoting the number of tasks and representing the weight assigned to each task. Details of the specific loss function will be introduced in Sect 3.4.

(7)

3.3 Semantics-constrained synonym substitution

Research on Data Augmentation via Synonym Substitution. Compared to other methods such as back-translation or generative adversarial networks, this approach better preserves semantic consistency and retains core meaning in data rich in domain-specific terminology. Unlike random word replacement, this method identifies the top-n words that receive the highest attention in a sequence by comparing weight scores in the model’s self-attention matrix—words that are likely critical to classification outcomes. It then substitutes these words with same-part-of-speech synonyms that have the highest similarity scores based on corpus statistics. Experimental results demonstrate that, compared to random substitution, this method enables models to more effectively learn discriminative features within sequences. Below is a detailed introduction to the method.

The multi-head self-attention mechanism is a core component of BERT, enabling the model to dynamically focus on different parts of the input sequence during text processing, thereby capturing rich semantic information. It projects the input vectors x into query, key, and value vectors using trainable weight matrices W_Q,W_K, and , respectively, as shown in Eqs (8) to (10).

(8)

(9)

(10)

For each word in the input sequence, the attention score with every other word (including itself) in the sequence is computed via a dot product operation, resulting in a vector of similarity scores. As shown in Eq (11), the query vector (a row in Q ) of each word is multiplied with the key vector (a row in K ) of every other word through dot product. The result is then scaled by a factor to prevent excessively large dot product values that may lead to gradient vanishing issues, where denotes the dimension of the key vectors. The resulting scores represent the attention degree of the word toward all other words.

(11)

Taking the BERT_base model as an example, it consists of 12 encoder layers, each containing 12 attention heads. Each head corresponds to a self-attention matrix and independently computes attention scores, which reflect the correlations between different positions in the input sequence. Different heads focus on distinct aspects: some pay more attention to syntactic structures, while others emphasize semantic relationships. The multi-head self-attention mechanism linearly concatenates the scores computed by the different attention heads, as shown in Eq (12):

(12)

where W_O is the output weight matrix and h is the number of attention heads. In the self-attention matrix, each row represents the attention scores that a word pays to all other words, while each column indicates the attention scores that a word receives from all other words. In this study, the BERT model was first fine-tuned on a specific corpus using multi-task learning, which enhanced the encoder’s ability to capture discriminative features of sentences for classification. Experiments revealed that attention heads in the middle layers are more adept at capturing such classification-related features in sequences. Based on the column-wise scores of the attention matrix (excluding [CLS] and [SEP] tokens), the nouns, adjectives, or verbs that received the highest attention were identified. These words were then replaced with same-part-of-speech synonyms selected from a manually constructed corpus, rather than through manual substitution.

During the synonym replacement process, a word often has multiple candidate synonyms. To select the most suitable replacement, the study calculates the cosine similarity between word vectors and filters the candidate most semantically similar to the target word for substitution. The specific procedure is as follows: each candidate word is substituted into the sequence and reprocessed through the BERT model to generate contextualized word embeddings. The cosine similarity is then computed between the two contextualized embeddings, and the candidate with the highest similarity score is selected to replace the original word, as shown in Eq (13):

(13)

where n is the dimensionality of the vectors, and A_i and B_i are the components of vectors A and B in the i-th dimension, respectively.

During actual testing, it was observed that this method has certain limitations: synonyms with different parts of speech from the target word could also achieve high similarity scores. To ensure the quality of data augmentation, a dual-validation mechanism was introduced: (1) Using a pre-trained language model to compute the semantic similarity between the original and substituted sentences, ensuring global semantic consistency; (2) Evaluating classifier confidence to guarantee that the modified samples can still be correctly classified.

When two or more keywords in the sequence were replaced, manual inspection revealed that even if the sequence passed the validation mechanism, the semantics of the sentence were somewhat compromised, or the structure became ungrammatical. This degradation increased with the number of substituted words. In our experiments, it was empirically observed that replacing 2 to 3 words per sample yielded the most significant improvement in the accuracy of the MBCS model, even when the resulting sentences were not always grammatically perfect.

3.4 Multi-task learning

Multi-Task Learning (MTL) is a machine learning paradigm designed to improve a model’s generalization and performance by learning multiple related tasks simultaneously. Unlike single-task learning, MTL enhances overall effectiveness by sharing representations or parameters to leverage correlations among tasks. This study proposes combining intent detection classification with semantic similarity contrastive learning, as the latter contributes to some extent in improving the model’s classification capability.

3.4.1 Text classification head.

The classification head is one of the final components of the MBCS model. The BERT model extracts contextual representations of the input text through its multi-layer Transformer encoder and outputs a hidden state vector. The classification head then maps this hidden state vector to specific class labels, selecting the category with the highest probability as the predicted user intent, as shown in Eqs (14) to (16). In this study, it consists of a fully connected layer (linear layer) followed by a softmax function.

(14)(15)(16)

For a multi-class classification problem, the cross-entropy loss function is given in Eq (17):

(17)

where y_i,c is the ground truth label of the i-th sample for class c and is the predicted probability of the i-th sample for class c. Here, C denotes the total number of classes, and N represents the total number of samples.

3.4.2 Supervised contrastive learning.

Contrastive Learning is commonly regarded as a self-supervised learning method. It treats augmented samples of the same instance as positive pairs and other samples within the same batch as negative pairs. By pulling similar samples closer and pushing dissimilar ones apart, it learns meaningful feature representations. Extensive research has demonstrated its effectiveness in significantly improving model performance.

However, a key challenge in self-supervised methods is the “false negative” problem, where a sample is incorrectly labeled as negative despite being semantically similar to the anchor. This introduces noise and can degrade model performance. Khosla et al. [30] extended the self-supervised contrastive loss to a supervised setting by incorporating label information. In their design, samples of the same class within a batch are treated as positive pairs, while those from different classes form negative pairs for contrastive learning. Experimental results showed that this method significantly outperformed the self-supervised contrastive model SimCLR in image classification accuracy.

(18)

where N is the number of samples in the batch; z_i, z_p, and z_n denote the embeddings of the anchor, positive sample, and negative sample, respectively; and τ is the temperature parameter used to control the sharpness of the similarity distribution. P(i) represents the set of indices of positive samples that belong to the same class as sample i , excluding i itself, while A(i) denotes the set of all samples in the current batch. In vector space, each sentence is represented as a point in a high-dimensional space. The dot product of sentence vectors can measure the semantic similarity between them, as it represents the sum of the extent of “pattern matching” across feature dimensions. Semantically similar sentences have similar distribution patterns in this high-dimensional space, resulting in a larger dot product.

The Eq (18) measures the similarity between samples by computing the dot product of their embeddings, scaled by the temperature parameter τ. When τ is small, differences in the similarity distribution are amplified, causing the model to focus more on sample pairs with high similarity. Conversely, when τ is large, the distribution becomes smoother, leading the model to treat all sample pairs more uniformly. By taking the negative logarithm, the objective of maximizing the similarity of positive pairs is transformed into minimizing the loss.

In the projection layer design of MBCS, a two-layer perceptron structure is adopted to optimize the feature space and enhance the effectiveness of supervised contrastive learning. The first fully connected layer (FC1) primarily serves to improve nonlinear expressive capability. This layer maps high-dimensional features from the backbone network to a lower-dimensional space while introducing the ReLU activation function to increase the flexibility of feature representation, enabling the model to better learn complex data distributions. The second fully connected layer (FC2) further transforms the reduced-dimensional features to make them more suitable for contrastive learning tasks. Its objective is to optimize feature projection such that samples of the same class are brought closer together in this space, while samples of different classes are pushed farther apart, thereby improving the performance of the supervised contrastive loss. Through this two-layer neural structure, not only is computational cost reduced, but discriminative power in the feature space is also enhanced, effectively boosting the overall performance of the MBCS model in intent detection and contrastive learning tasks.

3.4.3 Multiple gradient descent algorithm.

In this study, the classification task and the representation learning task work jointly, sharing inductive biases between them. Traditional multi-task learning methods typically minimize the sum of losses from multiple tasks. However, since the gradient directions of the two tasks may differ or even conflict with each other, it is uncertain whether each update iteration will mutually benefit both tasks, or whether some degree of mutual inhibition exists. Therefore, instead of simply combining the loss functions, the study aims to find a method that maximizes collaboration between the two tasks. To achieve this, multi-task learning is explicitly formulated as a multi-objective optimization problem, with the goal of identifying an optimal gradient direction g^* that balances the objectives of both tasks.

First, the parameters of the backbone network are denoted as θ, and the gradients of the loss functions of the two tasks with respect to the shared parameters θ are computed.

(19)

The direction of the gradient indicates the direction of the steepest ascent of the loss function, and its magnitude reflects the rate of change along that direction. The study aims to find a new gradient direction, referred to as the optimal gradient direction, in which both tasks are optimized as much as possible, as shown in Eq (20).

(20)

where the weights ,,  +  . This ensures that the gradients of the tasks are not summed in opposing directions. The original problem is thus transformed into an optimization problem of finding the minimum-norm solution , as shown in Eq (21). By minimizing the gradient norm, the shortest vector g^* is identified, whose value corresponds to a local or global minimum—i.e., the optimal solution under the current conditions.whose value corresponds to a local or global minimum—i.e., the optimal solution under the current conditions.

(21)

The above expression represents a typical convex optimization problem. The optimal weights are solved using a quadratic programming approach, with the objective function is given in Eq (22):

(22)

where is the quadratic gradient matrix,. From the above analysis, the objective function is subject to both equality and inequality constraints. For such a nonlinear programming problem, the KKT conditions are applied to solve it. The Lagrangian function is constructed in Eq (23):

(23)

where 1 is an all-ones vector, v is the Lagrange multiplier for the equality constraint, and is the Lagrange multiplier for the inequality constraint . If , then , indicating that the constraint is inactive; if , then , meaning the variable is constrained to zero. This relationship is shown in Eq (24):

(24)

where μ is a penalty factor. When , it indicates that the task contributes and will not be subject to additional constraints; means that the gradient of the task is sufficiently small at the current step and is temporarily excluded from optimization. Taking the partial derivative of Eq (24) with respect to λ and setting it to zero yields the Eq (25):

(25)

Eliminating v, the equation simplifies to:

(26)

Given that , a linear system is constructed by combining Eq (26) and then λ solved. The final loss function is is given in Eq (27):

(27)

4.1 Dataset

To comprehensively validate the effectiveness of the proposed MBCS framework, this study adopts a hybrid evaluation strategy combining “public benchmarks + domain-specific datasets”. Three intent detection datasets with distinctive characteristics were selected (detailed information in Table 1):

• SNIPS: Proposed by Coucke et al. [31], the SNIPS dataset contains 16,000 English utterances across 7 vertical domains (e.g., weather queries, music control, calendar management). It is widely used in intent detection and slot filling research due to its broad domain coverage, fine-grained annotations, and low noise level.
• ATIS: Originating from the DARPA speech understanding project, the ATIS dataset is a standard benchmark for intent detection and slot filling in the air travel domain (e.g., flight inquiries, ticket booking). Its high domain specificity aligns well with the few-shot professional scenario requirements of this study.
• Inspection Records Dataset: Inspection Records Dataset: Constructed by the research team through manual annotation of Chinese inspection records in the import-export cargo inspection domain, this dataset supports tax risk identification in China’s customs clearance context. It covers four high-risk intent types (e.g., under-declaration, false origin reporting, abnormal commodity classification, and key customs monitoring targets). The dataset exhibits strong domain specialization and high annotation complexity due to its professional nature.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Data collections.

https://doi.org/10.1371/journal.pone.0335914.t001

4.2 Experimental environment and parameter settings

The experiments were conducted on a system running Ubuntu 20.04.5 LTS and CUDA 11.2. The hardware configuration includes an Intel(R) Xeon(R) Gold 5218R CPU with a base frequency of 2.10 GHz, 80 cores, and 128 GB of RAM, along with an NVIDIA GeForce RTX A6000 GPU equipped with 48 GB of VRAM. The implementation was developed under the PyTorch (2.1.2 + cu118) framework.

The experimental parameters are shown in Table 2. The maximum sequence length (Max Length) is set to 128 for the two public datasets and to 512 for the private dataset. The “Epoch” field indicates the number of training iterations for the 5-shot, 10-shot, and full-training settings, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Experimental parameters.

https://doi.org/10.1371/journal.pone.0335914.t002

Common metrics for evaluating the effectiveness of classification models include accuracy and F1-score. Considering that the number of samples per class in the test set is not fully balanced, the study uses the weighted F1-score to assess model performance. The formulas are shown in Eqs (28) and (29).

(28)

(29)

where , ,C denotes the total number of classes, TP_i represents the number of true positives for the i-th class, FP_i refers to the number of false positives for the i-th class, and FN_i indicates the number of false negatives for the i-th class.

4.3 Comparative experiment

The study visualized the self-attention matrices of the same attention head across four sample sequences, as shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Heatmaps of self-attention matrix weights for different heads.

https://doi.org/10.1371/journal.pone.0335914.g002

Fig 3 illustrates the attention weight scores computed by different heads. Taking the BERTbase model as an example, it consists of 12 encoder layers, each containing 12 attention heads. Each head corresponds to a self-attention matrix and independently computes attention scores, which reflect the correlations between different positions in the input sequence. Different heads focus on distinct aspects: some pay more attention to syntactic structures, while others emphasize semantic relationships.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Heatmaps of self-attention matrix weights for different heads.

https://doi.org/10.1371/journal.pone.0335914.g003

Table 3 presents partial results of the synonym replacement strategy, using the SNIPS dataset as an example.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Synonym replacement examples (SNIPS, Top-1 attention word).

https://doi.org/10.1371/journal.pone.0335914.t003

Fig 4 visually compares the semantic similarity between partial synonyms and candidate words (including all parts of speech and contextual information).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Semantic similarity comparison of candidate words.

https://doi.org/10.1371/journal.pone.0335914.g004

The study conducted a comparative analysis of the semantic similarity between three sentences of the same category and sentences from other distinct categories, with the visual results presented in Fig 5. The figure clearly shows that the semantic similarity among sentences of the same category is significantly higher than that between different categories, intuitively reflecting the semantic consistency within categories and the distinctions between them.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Semantic similarity comparison of candidate words.

https://doi.org/10.1371/journal.pone.0335914.g005

To evaluate the performance of MBCS, the study conducted comparative experiments with baseline model selection. The results of MBCS were compared against those of the baseline model and several state-of-the-art models in few-shot intent detection. The findings demonstrate the superior performance of the MBCS model, with all experimental results reported on a percentage scale (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparative experiments with baseline models.

https://doi.org/10.1371/journal.pone.0335914.t004

The study evaluated the accuracy of three classic text classification models—FastText [32], T5 [33], and BERT—across three datasets. FastText is an efficient tool for text classification and word vector generation, proposed by the Facebook AI Research team in 2016. It combines a bag-of-words model with a shallow neural network, designed to handle large-scale text data efficiently while maintaining high classification accuracy. T5 (Text-to-Text Transfer Transformer), introduced by Google in 2019, is a pre-trained language model based on the Transformer architecture. It adopts the classic encoder-decoder structure and embodies the core idea of reformulating all natural language processing tasks into a unified “text-to-text” format, where both input and output are text sequences. This unified framework enables T5 to handle diverse NLP tasks—such as text classification, machine translation, question answering, and text summarization—in a generalizable manner (Table 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparative experiments.

https://doi.org/10.1371/journal.pone.0335914.t005

The study compared the proposed approach with baseline and state-of-the-art few-shot text classification models across three datasets. InductNet [34] innovatively employs a dynamic routing algorithm within a meta-learning framework to learn generalized class-level representations from limited samples, significantly improving text classification performance in few-shot scenarios. Specifically, InductNet consists of three modules: an encoder module, an induction module, and a relation module. The encoder module uses a bidirectional LSTM combined with a self-attention mechanism to encode input texts into fixed-size vector representations. The induction module, the core component of the model, utilizes a dynamic routing algorithm to map sample vectors from the support set to class vectors. By iteratively adjusting the connection strength between samples and classes, it automatically encapsulates class-level semantic information while reducing noise caused by sample diversity. The relation module employs a neural tensor layer to compare the query vector with class vectors and outputs a relation score indicating the degree of match between the query text and the class. Experimental results demonstrate that this model outperforms existing advanced methods on multiple text classification datasets.

In 5-shot settings, MBCS achieved accuracy improvements of approximately 3.19%,3.44% and 4.19% across the three datasets, with F1-score gains of 3.06%, 4.21% and 5.26%, respectively. In 10-shot settings, accuracy increased by 3.43%, 4.62%, and 4.91% with F1-score gains of 3.30%, 4.76%, and 5.33%, respectively. Compared to InductNet, the accuracy of the proposed method was boosted by up to 1.85% in 5-shot and 10.43% in 10-shot scenarios.

4.4 Ablation study

The study systematically conducted ablation experiments on the aforementioned three datasets to evaluate the individual contributions of the data augmentation strategy (DA), the multi-task learning framework (Multi-task). This comparative setup was designed to isolate and assess the independent impact of data augmentation techniques on model performance. The experimental design followed a rigorous ablation approach, focusing specifically on evaluating the effects of data augmentation technology and the representation learning auxiliary task on model performance (Table 6).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Ablation study (5-shot).

https://doi.org/10.1371/journal.pone.0335914.t006

The experimental results indicate that the performance of the synonym replacement strategy is significantly correlated with the quality of the training set. To ensure the reliability and stability of the experimental outcomes, the study randomly sampled multiple distinct subsets from the original training set and conducted multiple independent experiments on each subset. By performing statistical analysis on the results, excluding outliers, and averaging the outcomes, the study minimized the impact of training set sampling bias, thereby enhancing the representativeness and persuasiveness of the findings.

The results demonstrate that:

(1) This study systematically evaluated the proposed method on three datasets: SNIPS, ATIS, and Inspections Records, by testing the effect of replacing 1 to 5 synonyms per sentence. Experimental results show that the performance of the synonym replacement strategy is closely related to the original sequence length: longer sequences generally require the replacement of more synonyms to achieve optimal results. It is worth noting that, unlike English texts, Chinese sequences possess a natural advantage under this strategy—synonym replacement rarely leads to grammatical distortion or semantic ambiguity. This characteristic contributes to a more pronounced performance improvement of the strategy on the Chinese Inspections Records dataset.

The data augmentation strategy based on synonym replacement yielded an accuracy improvement of at least 1.85% and up to 3.39% in the 5-shot scenario. This suggests that the method performs better on out-of-domain data than on in-domain data. By introducing synonym variations, the study effectively expanded the lexical diversity of training samples and enhanced the model’s generalization capability for linguistic expressions.

(2) The incorporation of the contrastive learning auxiliary task led to more substantial performance gains, improving accuracy by up to 1.87% and at least 1.11% in the 5-shot setting. This indicates that cross-task knowledge transfer through a shared representation space can effectively alleviate the data sparsity issue in few-shot scenarios. The study argues that the data augmentation strategy primarily enhances the model’s ability to recognize synonymous expressions, while the representation learning auxiliary task significantly improves its discriminative capacity for semantics across different classes.