2.1 Aspect-level sentiment analysis

An essential subfield of sentiment analysis research is ABSA. It can anticipate the sentiment polarity of a sentence’s specific feature by processing fine-grained text [20,21]. ABSA finds extensive utility in areas such as evaluating product feedback, analyzing social media content, and conducting market studies. It can help companies understand customer feedback on specific product features or services in detail, thereby achieving more targeted improvements. ABSA’s core function is to accurately identify and analyze the sentiment orientations toward particular elements or entities in a text. ABSA requires that one focus on the aspect and sentiment-specific terms in the text and understand how they interact.

It is essential to take into account both the subtle nuances of sentiment expression and the semantic elements present in the context while doing sentiment analysis. Traditional sentiment analysis approaches assess the overall sentiment direction of the text in its entirety [22,23], whereas ABSA targets the sentiment associated with each aspect within a sentence. This fine-grained approach can evaluate opinions on specific entities. Previously, the primary methods for extracting sentence representations included recurrent neural networks and convolutional neural networks. Nevertheless, due to the architectural constraints of these models, they typically fail to effectively capture and encapsulate the syntactic and semantic features within sentences. The majority of the most recent aspect-level sentiment analysis techniques now in use makes use of graph neural network models. A specific type of deep learning called graph neural networks is made to handle graph data. It uses the syntactic structure of sentences to establish connections between aspect terms and the context in which they are used [15,18].

Meanwhile, in the field of ABSA, the application of capsule networks is gradually on the rise [52,53]. Compared with traditional methods, the capsule network precisely models the association between local and global text semantics through dynamic routing, accurately parsing the interaction between aspect terms and context in ABSA. Vectorized representation and multi-routing characteristics enhance the model’s capability to analyze sentiment semantics and polysemous words [39].

2.2 BERT model

Google’s research team proposed the BERT (Bidirectional Encoder Representations from Transformers) [28] model. BERT introduced a bidirectional context pre-training method, markedly enhancing the efficacy in numerous natural language processing challenges. Compared with the two text embedding methods, GloVe and Word2Vec, BERT has better results in context awareness, pre-training and fine-tuning, long-distance dependency processing, multi-task learning, and overall performance [29]. BERT preprocessing includes several essential steps to ensure the model can correctly process the input data. To handle rare words, the input text is first divided into words, and then the WordPiece technique is applied to break the text up into smaller word fragments. Then, BERT adds the “[CLS]” identifier at the beginning of the input to represent the classification information of the entire sequence and adds the “[SEP]” identifier at the end of each sentence to distinguish the boundaries between sentences. Then, the input tokens are converted into the corresponding ID through the vocabulary list, and the corresponding attention mask is generated to distinguish valid words from filler parts. At the same time, BERT generates segment IDs to distinguish different sentences and assist the model in comprehending the connections between phrases. Finally, the input data is padded to a uniform length to ensure input consistency during batch processing.

2.3 Graph neural networks

Within ABSA, graph neural networks have demonstrated significant promise. The Graph Convolutional Network (GCN) is a type of deep learning architecture specifically crafted for handling data structured in graph form [17]. The core of GCN is to aggregate a node’s features with those of its adjacent nodes using convolutional operations. This process encapsulates the local structure’s features within the graph, ensuring that a node’s final representation encompasses not only its own information but also incorporates the data from its surrounding nodes. This aggregation allows the model to effectively understand the graph’s overall structure. For example, Zhang et al. [30] pioneered using GCN for ABSA. They proposed implementing GCN on a syntactic dependency tree to effectively utilize aspect word context and dependency information in a sentence. Huang et al. [31] and Sun et al. [32] also proposed ABSA models based on graph neural networks to process features from multiple perspectives. In the ABSA task, graph neural networks are very effective in enhancing the role of context and aspect word dependencies.

The role of Graph Attention Networks (GAT) in ABSA is mainly reflected in its ability to effectively process unstructured data and capture complex dependencies, especially the relationship between words in text data [48]. Studying the sentiment tendency of particular textual elements (such product characteristics) is known as ABSA. This often entails figuring out how each word in the text is related to a specific phrase or component. Text analysis can more precisely capture the association between words and between words and aspect words because to GAT’s special attention mechanism, which allows it to give distinct weights to each edge in graph-structured data [49].

2.4 Multi-head self-attention mechanism

The multi-head self-attention mechanism plays a significant role in sentiment analysis [33]. It meticulously captures multi-level emotional and semantic information in text by processing multiple attention heads in parallel. To have an accurate understanding of the text’s emotion, each attention head can concentrate on distinct textual elements and different kinds of interactions, including grammatical structures, contextual impacts, and word emotional tendencies. This mechanism enables the model to identify subtle emotional differences in a complex text, such as sarcasm, puns, or implicit expressions, improving the accuracy and depth of sentiment analysis. In specific applications, multi-head self-attention can effectively process diverse texts such as user comments, social media posts, or customer feedback and provide enterprises with more accurate customer sentiment analysis and market trend predictions by capturing profound semantic and emotional clues [2]. This efficient information processing capability makes the multi-head self-attention mechanism an essential technology in sentiment analysis, significantly improving the overall performance of natural language processing systems.

2.5 Capsule networks

The capsule network was introduced by Hinton et al. [34], addressing the flaws and limitations of RNN and CNN, and has yielded promising outcomes in text categorization tasks. A collection of neurons that encodes a certain thing or item makes up a capsule. By sending vector-based data to the layer below, each capsule exchanges information with the capsules in the layer above. The term “dynamic routing" refers to this approach. In ABSA, the capsule network enhances the understanding of the context of the sentence and its internal structure by capturing the spatial relationship and syntactic structure between words. Compared with traditional convolutional neural networks, the capsule network can better preserve the hierarchical relationship and dependency between words and is thus more accurate in identifying aspect-related sentiment features. Its dynamic routing mechanism also improves the capture and classification of sentiment information in multi-aspect sentences, making it perform well in complex sentiment analysis tasks.

3.1 Feature extraction module

3.1.1 BERT pre-training.

To achieve a more comprehensive contextual encoding of sentences, we utilize a pre-optimized BERT architecture that converts input sentences into vector formats , where represents the length of the sentence and represents the embedding dimension. The representation of this model contains a unique tag [CLS] for classification and word tags, thus providing the global semantics of the sentence and the local representation of each word. To further capture the aspect-level and contextual sentiment analysis of sentences [35,36], we construct an input pair that combines sentences and specific aspects in the form of “[CLS] sentence [SEP] aspect word [SEP]”, which is used as the input of the BERT. This architectural approach not only preserves the semantic content of the sentence but also facilitates the model’s focus on the context surrounding individual aspect terms. Through the explicit representation of the linkage between sentences and aspect terms, the model’s proficiency in comprehending specific tasks is enhanced. After encoding, BERT generates an aspect-aware contextual representation , where is the global sentence representation and represents the embedding of each word in a specific context. This representation sequence is then input into subsequent modules, allowing our model to more effectively capture the deep semantic relationships of sentences and the fine-grained semantic differences of specific aspects, thereby improving performance in the following steps. The processing form is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. BERT preprocessing.

https://doi.org/10.1371/journal.pone.0328839.g003

3.1.2 Neighborhood enhancement module.

Inspired by the AOAN model [27], this paper builds multiple neighboring word combinations to highlight different ranges of neighboring word combinations related to a given aspect, thereby enhancing the model’s capture of the association between aspects and context. Considering that words surrounding a certain aspect word frequently include information useful for predicting emotion polarity [37,38], we construct several adjacent spans to represent the tokens surrounding the aspect term, encompassing the aspect term. The adjacent spans encompass diverse ranges around the aspect, each confined to the particular positional scope.

Specifically, we initially employ the moving mask mechanism to determine the relative location of each token relative to the aspect word referenced from [27]. Subsequently, we compute the relative distance d_j for each token in relation to the aspect word, defined as follows:

(1)

In the above formula, the variable M_j denotes the position of the number of token, with varying from to , where represents the input sentence’s length. M_a + 1 represents the aspect’s starting position, while m indicates its length. To represent the varied contexts of bordering words, then create multiple spans based on the threshold size , ranging from 0 to . This method enables us to generate neighboring spans. The mask vector for every surrounding term in the range is defined as follows:

(2)

represents the masking vector for items with segment length , where is a vector and is a null vector. The following equation provides the contiguous segment representation , where is the segment’s length:

(3)

In which the masking matrix is composed of masking vectors , each denoting the segment of length from the input sentence.

Concatenating the adjacent span representation with the original context representation H yields the adjacent span enhancement representation . A common linear transformation is then used to project these vectors into a single d-dimensional semantic space, which is represented as follows:

(4)

where stands for the cascade operation and and are the linear transformation layer’s weight and bias matrices.

3.2 Multi-scale feature processing module

Inspired by the dual-channel model [19,35], this model features a dual-channel processing architecture that includes a multi-head gated self-attention module and an advanced graph neural network module. These two channels process input data from different perspectives. Specifically, the multi-head gated attention mechanism is used to capture the global dependency of words in a sentence, ensuring that each word can obtain dynamic context information based on other words in the sentence. At the same time, the graph neural network channel utilizes graph convolution and attention mechanisms to examine adjacency relationships in the input text, extracting local structural features of neighboring words. In graph convolution, the adjacency matrix is used to aggregate text nodes to capture the local context information in the sentence, and the weights of different nodes are adaptively assigned through the graph attention mechanism to ensure that important neighbor information is better represented. Finally, the feature information of the previous two layers is integrated through the graph convolution layer.

The advantage of this architecture is its flexible dual-channel design, which can not only capture the sentence’s overall semantic structure but also consider the local relationship between adjacent words. By combining the attention mechanism and graph neural network, the model can improve its understanding and modeling capabilities of semantics at different levels of granularity.

3.2.1 Multi-head gated self-attention module.

(1) Multi-Head Self-Attention Mechanism (MHSA)

To improve the representational ability of distinct modality features, the encoding and decoding portions employ a multi-head self-attention mechanism. The query matrix Q, the key-value matrix K, and the value matrix V are first restructured into H matching sub-query matrices, sub-key-value matrices, and sub-value matrices, all of which have the same dimensions. The following formula describes the reconfiguration in detail:

(5)

Where X is the input vector, are trainable weight matrices.

The self-attention mechanism maps each vector (embedding of a word) of the input sequence into a query (Query, Q), a key (Key, K), and a value (Value, V) matrix. The formula is as follows:

(6)

Where is the output weight matrix and is the number of heads. Concat() represents the operation of connecting all heads. The multi-head attention mechanism enables the model to capture contextual dependencies across various subspaces.

The mechanism then conducts attention computations individually across each head. Afterward, the results from all heads are concatenated together to form the ultimate feature representation. The computational procedure is detailed below:

(7)

represents the projection matrix of the number of head, and represents the output features.

(2) Multi-head gated self-attention mechanism (MSAG)

By determining the reciprocal link between input sequences, MHSA is able to capture the sentences’ global semantic properties. However, there is a chance that MHSA might unintentionally draw attention to unimportant parts of the input because it can only calculate attention within a specific range. This can potentially lead to less accurate representations of the data, especially in tasks where it is crucial to focus on specific aspects or contexts within the input sequence. Inspired by the model of Xu et al. [41], this model introduces a multi-head gated self-attention mechanism (MSAG) to solve the abovementioned problems. The feature matrix X is introduced into the MSAG (Multi-Head Self-Attention with Gating) module, where the pivotal characteristics are discerned via the Self-Attention process. Following this, a gating mechanism is employed to extract a more decisive and nuanced set of textual features. The MSAG module is structured around three key sub-layers: it accepts X as the input feature set and models the intra-modal interactions among the words within the text via a multi-head attention mechanism and a subsequent dense feedforward neural network layer.

(8)

In the present scenario, weight coefficients are represented by and bias variables by . The gating mechanism layer is used to get the issue feature matrix once the feature matrix has been obtained. The following is the processing procedure:

(9)

In this context, stand for the weight matrices and for the bias terms. Once the feature matrix is obtained, the gated mechanism layer is utilized to produce the issue feature matrix . The process is described as such:

(10)(11)(12)

Where denotes the refined feature matrix ultimately produced by the MSAG unit, having suppressed the extraneous information yielded by layers of self-attention mechanisms. From Eq (9), the feedforward neural network’s output and the MHSA’s output can be combined. The integration unifies the semantic relationships revealed by the MHSA with the local textual characteristics harvested by the feedforward network; By utilizing Eqs (10) and (11), the sequential feature vector can be successfully confined, enabling the gated linear unit in Eq (12) to dynamically regulate the weight interplay within the feature vector; Following this process, both lower-level and higher-level semantic features are preserved in full, while noise and redundant information are concurrently minimized. The structure can be seen in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Multi-head gated self-attention mechanism.

https://doi.org/10.1371/journal.pone.0328839.g004

3.2.2 Graph neural network module.

(1) Graph Node Representation

Generate dependency graphs (adjacency matrices) for text data to obtain graph-structured relationships of features in aspect-level sentiment analysis tasks [40]. Parse the sentence through spaCy’s dependency parser, converting each word in the sentence and its grammatical dependencies into an adjacency matrix, where every row and column in the matrix represent a sentence word and a value of 1 represents the difference between the two words. There is a grammatical dependency between them, and zero means no dependency. The created dependency graph maintains the identification of words with themselves(the diagonal value is 1) while capturing the dependencies between words. The dependency graph can greatly enhance the model’s capacity to capture complex dependencies between words when processing emotional information related to aspect words. The Eq (13) referenced from [40] shows the construction method.

(13)

In this way, additional structured data for aspect sentiment analysis is provided by the dependency graph, which aids in the model’s comprehension of the text’s grammar and context and enhances its capacity to precisely represent sentiment polarity (such as positive, negative, and neutral) at the aspect level. Using the graph construction method, the initial feature matrix can contain rich features related to position information and text semantics. This method is particularly suitable for processing long or complex sentences and this can successfully raise the model’s sentiment analysis task performance.

(2) Graph Convolutional Layer Module

Graph Convolutional Layer: Graph Convolutional Layer: In a graph, G = (V, E), where V stands for the graph’s vertices (or nodes) and E for the edges that link them. In the context of GCN, the aggregation process involves applying convolutional operations to the feature data associated with the graph G. The characteristics of the neighbor nodes that are linked to any particular node in graph G are combined by this convolution process.By incorporating the structural information present in the network and maintaining feature invariance with regard to the relative locations of the nearby nodes, this approach makes it easier to update a node’s features. Consequently, each node in the graph can comprehensively learn not only its own high-level attributes but also the complex interdependencies with its higher-order neighboring nodes referenced from [40].

(14)

In the Eq (14), the node feature matrix for the number of layers is represented by , the node feature matrix for the number of layers by , and the weight matrix for the number of layers is recorded as . The node characteristics are then subjected to linear transformation; Sigmoid() represents a nonlinear activation function; represents the adjacency matrix A plus the identity matrix I (representing the self-loop of the node), the graph adjacency matrix is denoted by A, the identity matrix by I, and the symmetric matrix of by D.

(3) Graph Attention Layer Module

Graph Attention Layer: The graph attention layer incorporates an attention mechanism that assigns varying weights to neighboring nodes in the graph [13]. Initially, the node features are transformed into a new feature space via a linear mapping. Subsequently, attention coefficients for each node pair are computed. To stabilize the representation of nodes using self-attention, a multi-head attention mechanism [38] is employed to enhance the model’s representational capacity. In detail, the K-head independent attention processes are carried out according to the respective equations, and the outcomes of these attention mechanisms are averaged to obtain the final representation. For each node and its neighbor node , the attention coefficient between them is calculated, and the neighbor attention coefficient of each node is normalized using the softmax function. Finally, the neighbor node features are weighted and aggregated, and the normalized attention coefficient is used to weight the features of the neighbor nodes. The representation of the updated node hi is shown in the following equations : where C is the weight matrix in the number of D attention mechanisms, and A is the normalized attention value based on the number of B attention mechanisms.

(15)(16)(17)

where is the weight matrix in the amount of attention mechanisms, and is the normalized attention value based on the number of attention mechanisms.

(4) Graph Neural Layer Combination Module

Drawing inspiration from the GCNGAT model [26], we have developed an advanced graph neural network. To ensure a comprehensive information extraction method, this improved module combines the graph convolutional layer with the graph attention layer. As depicted in Fig 5, two graph convolutional layers and one graph attention layer make up the structure of the graph neural network module, which is tasked with identifying the relationships between the elements of the adjacency matrix of the input data. The module features a sequential relationship between layers, where the output from the first graph convolutional layer serves as the input to the first graph attention layer, and the output from this attention layer then feeds into the second graph convolutional layer. This progression is illustrated in Eqs (18), (19), and (20) referenced from [26]. The ultimate output graph constitutes the feature vector of the learning module, which is synthesized from the feature vectors acquired at each layer.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Graph neural network architecture.

https://doi.org/10.1371/journal.pone.0328839.g005

The global average pooling layer plays a critical role in the neural network architecture by consolidating the various sets of features extracted from the nodes and effectively reducing the dimensionality of the node features to a uniform size. As indicated in Eq (21), after passing through the global average pooling layer, the final feature representation is achieved.The graph attention mechanism is instrumental in dynamically adjusting the inter-node weights, allowing the model to focus on more relevant connections within the graph. Subsequently, the feature information from neighboring nodes is further disseminated and aggregated via the graph convolution layer. The model iteratively refines the node representations through two successive graph convolution operations, thereby enhancing the precision of feature extraction. The architecture is designed to learn from local feature representations by recursively stacking graph convolution and graph attention layers. This stacking process allows the model to delve deeper into the data, capturing nuanced variations in complex sentiment expressions and providing a comprehensive understanding of the emotional landscape within the text. The ultimate output from the graph representation learning component is a merge of feature vectors derived from each stratum in sequence. This amalgamated feature vector empowers the model to more adeptly detect nuanced variations in sentiment, thus providing a more profound and intricate understanding of the affective architecture inherent within the text.

(18)(19)(20)(21)

G and X serve as the connectivity matrix and attribute features of a particular subgraph within the text vector, respectively. H’ constitutes the ultimate feature depiction.

GCN can effectively capture the local structural information between nodes through graph convolution operations. It can learn the structural relationship between words and their context. GAT can dynamically adjust the weights of the edges between different nodes through its attention mechanism, which means that the influence of particular words can be automatically strengthened or weakened according to the importance in the context. In aspect-level sentiment analysis, combining these two networks means that the model can understand the fundamental structural relationship between words and dynamically adjust the importance of these relationships for specific aspects. For example, for the sentence “The service in this restaurant is good, but the price is on the high side", the model needs to distinguish which words are related to “service" and which are related to “price", and assign different weights according to the sentiment tendency in the context. This combination can improve the flexibility and accuracy of the model because it not only relies on fixed structural information but also adjusts the degree of attention to different words according to the needs in the actual context and better processes important information in the local area.

3.3 Feature extraction module

3.3.1 Adaptive feature fusion module.

To enhance the efficacy of feature integration for the ultimate sentiment analysis objective, it’s crucial to refine the text feature vector X and the graph neural network feature vector Y, as they may still retain extraneous noise. Consequently, we introduce an Adaptive Feature Fusion Module (AFFM) to address this challenge, as illustrated in Fig 6, influenced by the scholarly works of Dosovitskiy, Xu et al. [39–42]. The AFFM fine-tunes the output feature data by manipulating two distinct channels, thereby proficiently merging the two feature sets. This strategy enables a more thorough grasp and interpretation of the nuanced emotional and semantic variances within the text. Simultaneously, the integration of features across multiple scales enhances the model’s capacity for generalization, allowing it to absorb and interpret more sophisticated semantic fusion features. The AFFM calculates the dynamic integration of characteristics from the textual feature array X and the graph vertex feature array Y to produce the merged feature vector f. The algorithmic procedure is outlined as follows:

(22)(23)(24)(25)(26)(27)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Adaptive feature fusion module.

https://doi.org/10.1371/journal.pone.0328839.g006

In the calculation process described, the critical step involves executing a profound and detailed filtering of information on feature X from the multi-head gated self-attention mechanism and feature Y from the graph neural network, along with computing their weighted and transformed feature h via the previously mentioned equations. The equations can be used to dynamically adjust and , the global adaptive feature and the local adaptive feature , to adaptively control the inflow of feature information and realize the integration of global feature representation and local feature representation. The AFFM produces a combined feature depiction and subsequently conducts additional feature manipulation.

3.3.2 Capsule network module.

The capsule network can further capture the spatial relationship and hierarchical structure in the fused features of the two channels through multi-dimensional representation and dynamic routing, thereby generating a more refined feature representation. The capsule network’s dynamic routing mechanism enables it to process the features after dual-channel fusion selectively. It can adaptively decide which features are helpful for the final task and which features can be ignored. Through this mechanism, CapsNet can focus on extracting the most beneficial information for sentiment analysis tasks from the fused features, further improving the accuracy and robustness of the model. The illustrative plot of the internal mechanism of the capsule is shown in Fig 7 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Schematic diagram of the internal principle of the capsule.

https://doi.org/10.1371/journal.pone.0328839.g007

The capsule is located in the number of layer. The process of generating the prediction vector u_j|i according to the input vector is shown in Eq (28) referenced from [52,53], where is the weight matrix.

(28)

Dynamic Routing: The dynamic routing mechanism selects which upper-layer capsules the input capsule will pass information to, as shown in Fig 8. Through multiple iterations, the network can determine which upper-layer capsule is more suitable for receiving information from the lower-layer capsule. The network first assigns an initial routing coefficient to each input capsule i and output capsule j, and the initial value is usually set to 0. Routing coefficient update: In each iteration, the routing coefficient is updated so that the capsule associated with the correct pattern contributes more to the transmission of information. The formula referenced from [52,53] is as follows:

(29)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Dynamic routing mechanism.

https://doi.org/10.1371/journal.pone.0328839.g008

is the initial connection weight between input capsule i and output capsule j, and is the normalized routing coefficient, which indicates the probability of capsule i being transmitted to capsule j. Through this soft routing mechanism, the capsule network can dynamically adjust the information flow to ensure that important information is transmitted to relevant capsules rather than noise information.

The weighted sum of the primary capsule layer information calculates the output capsule layer referenced from [52,53]. Specifically, the output capsule is the weighted sum of all its input capsules, and finally, the output vector of the capsule layer is obtained after nonlinear transformation by the nonlinear activation function Squash:

(30)(31)

The image neural channel in the dual channel is responsible for capturing local contextual dependencies, and the other multi-head gated self-attention channel is responsible for capturing global features. The capsule network can further build a more hierarchical representation based on the fusion. The capsule network processes the fused features through its squash function and dynamic routing mechanism. CapsNet can filter out possible noise, especially in the scenario of multi-source information fusion, and retain the most compelling features, thereby enhancing the accuracy of sentiment classification. The capsule network can generate more accurate sentiment predictions for each aspect word when processing these fused features.

3.4 Sentiment classification

The combined representation can be used in the softmax function to predict the distribution of sentiment polarities:

(32)

where denotes the anticipated sentiment orientation, and are the trainable parameters. The following procedure explains the loss function in order to validate the whole network:

(33)

Where is the true sentiment classification, C is the count of classifications, represents the predicted classification, denotes each element of the parameter set that is subject to regularization, the parameter for the regularization term is denoted by , whereas the set of all parameters is represented by .

4.1 Experimental dataset

For empirical evaluation, we have performed experiments on three established and publicly accessible benchmark datasets. The datasets encompass reviews from the Lap14 (SemEval2014 [24]), Rest14 (SemEval2014 [24]). Furthermore, we include a dataset derived from Twitter(ACL14 Twitter [25]). The Twitter dataset exhibits a skewed class distribution. While we retain the original data distribution to reflect real-world scenarios, our design inherently mitigates imbalance through: Macro-F1 Focus: Prioritizing per-class equality over overall accuracy. Capsule Network Dynamics: Routing-by-agreement naturally suppresses majority-class dominance by requiring consensus among low-level features. Adaptive Attention: The gating mechanism downweights frequent neutral patterns unrelated to target aspects. Each of these datasets is comprised of entries that express one of three sentiment orientations: favorable, impartial, or unfavorable. Moreover, every sentence within these datasets is annotated with the pertinent aspect and its associated sentiment orientation. The details and distribution of the data across these three datasets are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Dataset distribution.

https://doi.org/10.1371/journal.pone.0328839.t001

4.2 Experimental parameters and evaluation indicators

The numerical experiments are carried out in the environment of Windows 10 (GPU/RTX 4060Ti, VRAM/16GB, RAM/32GB, Python 3.9). In the experiment, we refer to the recommendations of the literature [18,27] and set the hyperparameters of MSDC as follows: BERT-base-uncased, 768-dimensional word embedding is used as input; The hidden size is set at 768; The Adam optimizer with a learning rate of is used, which selected from the set value of the learning rate in the “4.6 Sensitivity analysis". This learning rate enables the model to converge stably. In the adaptive adjustment mechanism of Adam, it can make the gradient update stable. It can not only avoid slow training but also prevent the disruption of convergence. Moreover, it can reduce overfitting to enhance the generalization ability. Additionally, this learning rate may interact synergistically with other hyperparameters to achieve a good balance. According to the sensitivity of multi-module collaboration, we set the testing range of the dropout value to be between 0.2 and 0.5. When it is uniformly set to 0.3 after comprehensively comparing the performance of all modules, the comprehensive performance is the best. Meanwhile, the F1 score in verification is the highest, and the gap in overfitting is the smallest. To solve the overfitting problem, the model uses regularization, in light of the diverse and specific demands for harmonizing components like BERT, GNN, and capsule networks, and in order to construct a mutually reinforcing regularization approach in concert with Dropout and Adam, the regularization weight is set to , this value demonstrated the best performance during the experiment; The multi-head gated self-attention mechanism has a fixed number of attention heads of 12, the selection of its value has been specifically analyzed in the “4.6 Sensitivity analysis". The parameters of all models are initialized using a uniform distribution. In consideration of striking a balance between computational efficiency and memory constraints, curbing overfitting, boosting generalization, and suiting the characteristics of capsule networks and graph neural networks, the batch size is configured as 16 in our experiment. The network is trained across 20 epochs, and the maximum sequence length that may be used is 100 tokens.

This study uses accuracy A and macro F1 value as model assessment indicators to assess the effectiveness and performance of the MSDC model. The defining formulas are as follows:

(34)

(35)

(36)

(37)

4.3 Comparison of models and results analysis

The proposed MSDC model is contrasted with a number of baseline and advanced models to confirm its efficacy, particularly:

1) ATAE-LSTM [43] uses LSTM enhanced with an attention mechanism, which concentrates on crucial segments related to particular aspects within a sentence.

2) MGAN [46] applies GCN to the original framework of the dependency tree to integrate syntactic data.

3) RAM [45] utilizes a cyclical attention mechanism over sentence memory to highlight the significance of particular aspects.

4) ASGCN [47] employs GCN on the initial structure of the dependency tree to incorporate grammatical details.

5) BiGCN [44] uses a bidirectional propagation structure GCN detection model to perform in-depth analysis of features.

6) R-GAT [50] converts the dependency tree into a relational graph attention architecture in which the connections are delineated by the shortest path and the nature of the dependency, utilizing GAT for amalgamation through an attention-based process.

7) T-GCN [51] differentiates between relation types using an attention mechanism and incorporates attention layer integration to extract features from various GCN layers.

8) DualGCN [35] merges syntactic and semantic data using the Syntactic GCN and Semantic GCN modules simultaneously.

9) SSEGCN [36] captures semantic information via an attention mechanism and augments it with syntactic information by utilizing the minimum tree distance.

10) AOAN [27] captures sentiment information associated with specific aspects through the neighborhood expansion module and multi-view attention mechanism.

11) RDGCN [18] proposes a reinforced dependency GCN, which boosts ABSA through the refinement of grammatical dependency relations by employing diverse tactics.

4.4 Experimental results analysis

The information presented in the table is derived from the findings published in the original study. Table 2 reveals that the Accuracy (A) and F1 metrics of the introduced MSDC model have seen a noticeable uplift across the three datasets in comparison to other models. The comparison benchmark models in the experiment include non-BERT models (ATAE-LSTM, MGAN, RAM, ASGCN, BiGCN) and BERT models (AOAN, R-GAT, T-GAT, DualGCN, SSEGCN, RDGCN). This advancement is due to the model’s thorough accounting for the sentiment-related characteristics arising from the aspect interactions within sentences. By merging local and global feature information, the model effectively tackles the challenge of semantic matching inaccuracies, thereby facilitating the extraction of essential features that play a critical role in sentiment analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Performance comparison experimental results of MSDC model and SOTA methods(missing data is indicated by “-").

https://doi.org/10.1371/journal.pone.0328839.t002

Among the above SOTA methods, the primary factor contributing to the suboptimal performance of ASGCN is its exclusive focus on the interactional features between aspects and their contexts, neglecting the interactional features among aspects themselves. As for BiGCN’s underperformance, it might stem from its greater proficiency in capturing English collocations, which makes it less adept at handling expressions with weaker grammatical constructs. In addition, models that simultaneously utilize syntactic and semantic knowledge (DualGCN, SSEGCN) are generally better than models that rely on only one of the methods (R-GAT, T-GCN), which highlights the importance of jointly enhancing syntactic and semantic dependencies. Current standard models that incorporate distance grammars either uniformly decrease distance weights for equidistant elements or obfuscate the initial explicit weights via attention mechanisms and masking, both approaches being less effective than MSDC. In contrast to the DualGCN model, MSDC improves the accuracy of the three datasets by 0.7%, 0.61% and 1.81%; The F1 value increases by 0.60%, 1.28%, and 1.22%, respectively. The experimental results once again prove the effectiveness and generalization of our model. Meanwhile, the introduction of the BERT-based semantic encoding module has significantly improved our model, and the comparison between MSDC and MSDC+BERT can also reveal the role that BERT plays in the operation of the model, reflects that our model has achieved the most significant breakthrough in terms of BERT’s performance, proving that MSDC has acquired more valuable syntactic knowledge for ABSA.

The MSDC model is built upon the BERT-base architecture (110M parameters) and incorporates multi-scale feature fusion modules, including a Graph Neural Network with 1.2M parameters, a multi-head gated attention mechanism with 3.8M parameters, and a capsule network with 4.5M parameters. The total parameter count of the model is 119.5M. The model requires a total training time of 6.7 hours across three standard datasets, with CPU memory consumption ranging from 13.4G to 15.6G during the training process.

4.5 Ablation experiment

To explore the function of every modules on the performance of the MSDC.On three datasets, we conducted relevant ablation experiments.

(1) MSDC-NoSpan: The neighboring span enhancement module is removed and the result from the BERT encoding process is applied to the input of the two-channel module.

(2) MSDC-NoMSAG: The MSAG module is removed, and only the GNN retained as the sole channel for feature extraction.

(3) MSDC-NoGNN: The GNN module is removed, and only the MSAG is exploited as the sole channel for feature extraction.

(4) MSDC-NoFuse: Delete the adaptive feature fusion mechanism and only concatenate the features of the GNN module and the MSAG module.

(5) MSDC-NoCapsule: Delete the capsule network module and process the fused features directly.

(6) MSDC-Maxpool: Remove the global average pooling of MSDC and use the maximum pooling to analyse sentiment classification.

The results of the ablation studies are presented in Table 3, the results demonstrate each module’s contribution to the MSDC model’s overall effectiveness. Specifically, the effect of MSDC-NoSpan indicates that the neighborhood enhancement module effectively highlights the importance of neighboring words, thereby improving performance. The comparison of the results of MSDC-NoMHSA and MSDC-NoGNN shows that MHSA enhances the model’s processing of global features; GNN improves the model’s processing of local features. The combined use of the two modules enhances the model’s full utilization of multi-scale features. On the other hand, the effect of MSGA-NoFuse demonstrates the focus of the adaptive fusion module on key information during feature processing. The impact of MSGA-NoCapsule reflects the capsule network’s effective integration of fused features. Furthermore, the experimental performance of MSDC-Maxpool has notably declined, suggesting that equal consideration should be given to each multi-view representation for the final classification stage. In summary, the outcomes of the ablation studies validate the contribution of each MSDC module to enhancing the overall capabilities of the ABSA task.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Ablation experiment results.

https://doi.org/10.1371/journal.pone.0328839.t003

4.6 Sensitivity analysis

In the sensitivity analysis, this study aims to evaluate the model’s sensitivity to different hyperparameters and changes in input features to ensure the robustness and generalization ability of the model.

(1) Learning rate

The influence of learning rate on model performance is analyzed. By adjusting the learning rate within a specific range and recording the performance on the dataset, it is found that a learning rate that is too high may cause the loss of function value to oscillate during training, thus affecting the stability of the model. This paper conducts experimental analysis on the Restaurant and Twitter datasets and compares other learning rates. As shown in Figs 9 and 10, an optimal learning rate of is determined to balance the training speed and convergence quality.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Impact of learning rate on Twitter.

https://doi.org/10.1371/journal.pone.0328839.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Impact of learning rate on restaurant.

https://doi.org/10.1371/journal.pone.0328839.g010

(2) Number of attention heads

The model presented in this study employs a multi-head gated self-attention mechanism, which enables it to dynamically concentrate on various segments of the input text while handling multiple aspect terms. This mechanism encompasses several attention heads, each of which independently learns to direct its focus on distinct regions of the input to extract information pertinent to different aspects. Each head computes a set of weights that reflect the significance of the input’s various components with respect to a given aspect. Experimental evaluations on the Restaurant dataset were carried out with varying counts of attention heads 6, 12, 18 and 24 to assess their influence on the model’s performance. As shown in Fig 11, the model performs best when the number of multi-head attention heads is set to 12, and the model performs somewhat worse when the number of attention heads is adjusted to other values. The rationale is that a reduced head count restricts the model’s capacity to concentrate on various sizes and locations, raising the risk of data loss. Since the attention head itself is in matrix form, the result is discrete, and there is an error between it and the actual target. If there are more attention heads, the cumulative error may be larger, making the performance of the model worse.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Impact of attention head.

https://doi.org/10.1371/journal.pone.0328839.g011

(3) Number of GCN layers

The depth of layers in a graph convolutional network is a critical factor influencing model performance. This study performs a sensitivity analysis on the number of GCN layers. The analysis was carried out using the Laptop dataset, with the GCN layer count varied to 1, 2, 3, and 4. The model obtained an F1 value of 77.56% and an accuracy of 81.35% for a single GCN layer. The accuracy and F1 values were 82.41% and 79.38%, respectively, as the number of layers rose to two GCN layers. In comparison to the first layer, the model’s accuracy and F1 value rose by 1.06% and 1.81%, respectively. The model’s capacity to extract syntactic and semantic information was improved. However, it can be seen from Fig 12 that after the second layer was added, the model performance showed a downward trend. For the third layer and more layers, the accuracy and F1 value showed a significant decline. This may be because the model with too shallow layers failed to learn good semantic features, and the network structure was too deep, which caused gradient diffusion or overfitting. Therefore, this paper selected the model with 2 GCN layers for the best effect.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Impact of GCN layers.

https://doi.org/10.1371/journal.pone.0328839.g012

4.7 Dataset analysis

The training set of the Twitter dataset contains 6,218 samples, where the neutral class (0) accounts for 49.9%, while the positive (+1) and negative (-1) classes each represent approximately 25%. The test set contains 692 samples, with the neutral class reaching 50.7% and positive/negative classes each around 25%. In contrast, the Restaurant and Laptop datasets exhibit more severe class imbalance. For Restaurant: the training set (3,608 samples) shows a positive class prevalence of 59.8%, neutral class at 17.6%, and negative class at 22.6%, while the test set (1,120 samples) has a dominant positive class (65%) with neutral/negative classes each at 17.5%. For Laptop: the training set (2,328 samples) demonstrates a negative class proportion of 37.4%, neutral class at 19.9%, and positive class at 42.7%, with the test set (638 samples) showing a positive class majority at 53.4%, neutral class at 26.5%, and negative class at 20.1%.

Analysis reveals that Twitter’s neutral-dominated distribution might bias models toward neutral predictions, potentially degrading performance on minority classes and affecting Macro-F1. However, MSDC+BERT achieves a Macro-F1 of 77.24% on Twitter, outperforming counterparts like AOAN+BERT (76.54%) and RDGCN+BERT (77.14%). The minimal performance variance across datasets suggests that class imbalance has limited impact compared to SOTA model capabilities. MSDC’s superior Macro-F1 on Twitter highlights its ability to handle minority classes under neutral dominance, while consistent performance on Restaurant/Laptop demonstrates robust generalization. Experiments confirm that MSDC+BERT maintains stable Macro-F1 across all datasets, including Twitter (77.24%), Restaurant, and Laptop.

In the future, we plan to implement data augmentation strategies, such as AI-driven techniques (e.g., generative adversarial networks or context-aware text generation), to further mitigate the impact of class imbalance. By synthetically expanding underrepresented samples in the neutral, positive, and negative classes, we aim to enhance the model’s ability to generalize across diverse scenarios and improve its robustness in handling minority classes. These augmentations, combined with our MSDC model, are expected to bridge distribution gaps and elevate overall classification performance.

4.8 Case analysis

As seen in Table 4, we compare the MSDC with the other three baseline models using a case study on a number of sample words in order to more clearly demonstrate its advantages. The aspect words are included in brackets “[ ]" in each statement. Positive, negative, and neutral sentiments are represented by the capital letters P, N and O, respectively. Every model in the case study uses “BERT-base-uncased" for text encoding in order to provide a fair comparison between the MSDC model and other SOTA models. Next, use the suggested hyperparameters to train and assess all models, setting the batch size to 16 and the training epochs to 20.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparative case analysis of MSDC model and SOTA model.

https://doi.org/10.1371/journal.pone.0328839.t004

In the first example, the aspect word “beer selection" contains a clear positive sentiment word “Great", which allows all models to classify it correctly. However, since “beers" does not have an obvious sentiment word, the three baseline models misclassify it. The MSDC model processes the grammatical structure at multiple scales and analyzes the different sentiment states of “beer selection" and “beers". In the second example, although “finger swipes" has an obvious sentiment clue “fancy", the term “mouse" lacks a clear sentiment word. Therefore, SSEGCN incorrectly predicts “finger swipes" and “mouse" as positive. DualGCN links “finger swipes" with “mouse" but does not consider the relationship between the grammatical structures, resulting in misclassification. In contrast, our model correctly identifies the relationship between “finger swipes" and “mouse", and accurately corresponds sentiment words to aspects through the analysis of global features. In the third example, DualGCN only connects “XP" with “support" and ignores “Unfortunately" in the sentence, resulting in “XP" being misclassified. Meanwhile, SSEGCN only focuses on the second conjunction “and” and ignores the syntactic structure, resulting in incorrect predictions. R-GAT analyzes the sentence locally, resulting in incorrect sentiment analysis of “XP”. However, our model correctly associates aspect words and sentiment through the global and local syntactic dependencies of the sentence. In the fourth example, while “service” has an obvious sentiment clue “nice”, the aspect word “restaurant” lacks a clear sentiment word. Therefore, SSEGCN incorrectly predicts “restaurant” as neutral. DualGCN associates “restaurant” with “good” but ignores the importance of the word “only” in the syntactic structure, resulting in misclassification. In contrast, our model correctly identifies the relationship between “only” and “restaurant” while reducing the impact of other noise on the experiment and accurately classifies it. In summary, the predictions that match the true labels show that the MSDC model is more effective than the baseline model in capturing sentiment and syntactic knowledge in sentences.

However, the MSDC model also exhibits certain processing limitations. These limitations can be observed in adversarial scenarios where synonym substitution (“Great" “Good") in “Good beer selection, but the beers are average" may mislead the model to overlook the negative sentiment of “average" due to syntactic parsing errors. Complex syntactic structures like “Despite the fancy finger swipes, the mouse is not responsive" further challenge MSDC, as incorrect dissociation of “not" from “responsive" might lead to misinterpreting “mouse" as positive. Additionally, the model struggles with long-tail data such as slang terms like “YOLO" in “This restaurant’s vibe is YOLO, but the service is slow," where insufficient training on informal language could result in misclassifying “YOLO" as neutral/negative despite its inherent positivity.

Future research will address these challenges by developing adversarial training to enhance robustness, optimizing syntactic parsing for complex structures, and integrating few-shot learning to improve handling of slang and rare linguistic patterns.

5.1 Key research questions

Extract the key research questions from the Section “4.7 Case analysis" in the experiment:

RQ1: How does multi-scale syntactic analysis resolve ambiguous sentiment scenarios without explicit opinion words? (Corresponding to Example 1: The word “beers" has no sentiment, but the MSDC associates “Great" with “beers" through syntactic dependencies).

RQ2: Why does adaptive fusion outperform static dependency modeling in handling negation cues? (Corresponding to Example 4: Capturing the negative modification of “only" to “restaurant", which is ignored by DualGCN).

RQ3: Can global syntactic context mitigate local prediction conflicts? (Corresponding to Example 3: Combining the global sentiment of “Unfortunately" to correct the local analysis of “XP").

5.2 Impact of the MSDC

Our work bridges two critical gaps in ABSA: (1) The proposed dual-channel architecture formally unifies syntactic dependency modeling (via GNNs) and semantic relational learning (via gated attention), whereas prior studies treated them as separate components (e.g., ASGCN uses GNN alone). As evidenced by the error reduction on contrastive clauses, this synergy enables robust cross-domain generalization. (2) The capsule-based routing protocol introduces a novel paradigm for polarity disambiguation, fundamentally differing from traditional threshold-based methods. The capsule activation patterns reveal its capability to capture sentiment intensity gradients, which may inspire new evaluation metrics beyond categorical classification. Simultaneously, in the course of future development, it holds the potential to be extended to medical text analysis, such as the implicit sentiment within patients’ descriptions of symptoms. It proffers an entirely novel paradigm for aspect-based sentiment analysis, enabling multi-scale syntactic-semantic integration, thereby augmenting the precision of sentiment analysis.

5.3 Threats to validity and limitations

This study has several limitations related to the dataset. First, the dataset used in this research is relatively small, which may limit the generalizability of the findings. Additionally, the dataset is imbalanced, with certain categories underrepresented, potentially biasing the model’s performance. Second, the data were collected from a specific geographic region, which may not fully capture the diversity of other contexts. Third, the dataset contains missing values and noise, which were addressed through preprocessing, but residual errors may still exist. Finally, the dataset’s temporal scope is limited to a specific period, which may not reflect current trends. Future research could address these limitations by using larger, more diverse datasets and incorporating more robust data collection methods.