2.1 Sentiment classification

Sentiment classification based on the artificial neural network mainly endeavors to confirm the feature distance of sentiment in a short text by actively updating the weight relationship of the word vectors, and finally trains the optimal feature distance under the model [13]. Sentiment recognition is often accomplished through the use of Convolutional Neural Networks (CNN) [14, 15], but when the data dependence is relatively high, CNN often does not perform well enough due to the lack of internal dependence mechanism. Recurrent Neural Networks (RNN) [16, 17] can handle short-term dependence. However, vanishing and blowing up of gradients are often observed when dealing with long-term dependencies in text. Long Short-Term Memory (LSTM) networks and Bi-directional Long Short-Term Memory (BILSTM) networks [18, 19] has the ability to overcome the challenges of long short-term dependencies to a certain extent, but gradient disappearance still occurs when the text dependence length exceeds a certain limit. Attention mechanisms [20] can obtain the dependence between local and global, compared with CNN and RNN, have few parameters and low model complexity. However, they cannot effectively learn the location information in the text.

Using a single model alone has certain disadvantages. Currently, the mainstream research direction is to stack and modify models, which can help overcome some of these issues. For example, Yang et al. [21] implemented text classification by weighting a modified RNN and combining it with ATTENTION. Similarly, the design of a BILSTM structure incorporating attention is attributed by Wang et al. [22], that can give priority to various segments of the sentence. Dragoni et al. [23] used a neural word embedding approach combined with conventional network models to address the shortcomings of earlier techniques.

The combination of word embedding models and neural network models for sentiment recognition is a significant area of research [24]. For example, Liang et al. [25] employed a pre-trained word2vec structure for word embedding and used a CNN to continuously learn sentence Characteristics and classify sentiments with the aid of the attention mechanism. Meanwhile, Tang et al. [26] utilized a pre-trained Word2vec structure and employed a LSTM to acquire long short-term associations of the text, and then integrated this with an attention mechanism to classify sentiment.

2.2 Aspect-level sentiment classification

3.1 Short text and long text sentence recognition

To facilitate Natural Language Processing of short text T and summary equal-length text LT in the context of online education, a method is proposeed to convert them into word-level features. Specifically, for short text T, it is identified as a statement and converted into T_word, which consists of n words. For LT, which is typically longer and contains multiple sentences, compartments are created based on punctuation marks such as commas, periods, and each compartment is then converted into a sentence-level feature LT_sentence containing n words. Thus, LT_word is composed of m such sentence-level features. This method provides a useful tool for analyzing and processing natural language text in online education. (1) (2)

3.2 Acquisition of embedding vector Word2vec

Word2vec is a widely used pre-trained model for mapping words to vectors and measuring the distance and relationship between them. In this study, the skip-gram method of Word2vec is utilized to generate embedding vectors for the word-level features T_word and LT_word. Specifically, and are obtained by mapping each word in T_word and LT_word to a z-dimensional vector using the trained Word2vec model.

By leveraging the semantic information captured in the Word2vec vectors. An effective approach for analyzing and processing natural language text in the context of online education is provided by this method. A variety of applications can use this embedding vectors, such as measuring similarity between words and identifying relevant concepts in the text. (3) (4)

3.3 Aspect-level DDP-ASDN model construction

Sentence dependency refers to the dependency between words within a sentence. Label, part of speech, and location information of words can be calculated through dependency analysis. By syntactic analysis of the input text, pairs of associated words and long-distance dependent word equivalence can be obtained, making it one of the commonly used methods in sentiment analysis tasks. As an example, consider the following sentence, “I don’t like database, but the teacher is also cute.” a dependency analysis is shown in Fig 3. The relationship types between words are mostly related to nouns, and the evaluation object, “database,” and its corresponding sentimental polarity, “don’t like,” can be extracted based on the nature of words.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Sentence dependence analysis.

https://doi.org/10.1371/journal.pone.0295331.g003

The sentiment dictionary collects commonly used sentimental words and their corresponding label. The sentence dependency analysis model, which is added with the sentiment dictionary, can directly detect the correlation of sentimental words in text. In addition, the processing process of other unrelated words can be ignored. The framework consists of six layers. In the first layer, the incoming parameter V_word and LV_word are required to calculate the position p of the word and its corresponding label L. The second layer is the sentiment dictionary layer, where V_word and LV_word are compared with the pre-constructed sentimental dictionary. In the third layer, if V_word and LV_word belong to the sentimental dictionary, a new sentiment evaluation object and its corresponding sentiment evaluation label are built. In the fourth layer, two fully connected operations are performed to obtain h. In the fifth layer, a normalization operation is done to obtain using the softmax function. The sixth layer is the aspect layer, where a new aspect-level binary group is obtained by calculation. The architecture of the DDP-ASDN model is exhibited in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. DDP-ASDN model architecture diagram.

https://doi.org/10.1371/journal.pone.0295331.g004

The processing process of the DDP-ASDN model is outlined in Algorithm 1:

Algorithm 1: DDP-ASDN

Input: input parameters V_word, LV_word

Output: aspect-level binary group

1 some description;

2 for Until the last sentence do

3  Calculate the position p of the word and its corresponding label L;

4  P = {p^w1, p^w1, p^w1, ⋯, p^wn};

5  L = {l^w1, l^w1, l^w1, ⋯, l^wn};

6  Find a sentiment dictionary;

7  Dic = {W^k1, W^k2, W^k3, ⋯, W^kz};

8  Dic^l = {W^l1, W^l2, W^l3, ⋯, W^lz};

9  if V_word, LV_word in Dic then

10   Construct a new sentiment evaluation object;

11    ;

12  Construct the corresponding sentiment evaluation labels;

13   ;

14  Enter the hidden layer for calculation;

15   ;

16  Enter the softmax for the calculation;

17   ;

18  Combined into a new aspect-level binary group;

19   ;

20  end

21 end

3.4 CNN-BILSTM-ATTENTION model construction

The main features of the input vector are obtained through CNN, and are taken as the input for subsequent calculations to reduce the amount of computing resources required for the entire algorithm. Next, the input is passed through the BILSTM layer to collect the long short-term associations between words in the sentence. The following five steps can be used to describe the specific processing procedure.

Step 1: The eigenvalues and were computed by the CNN model’s convolution layers for the incoming V_word and LV_word, respectively. The computation process is defined as follows: (5) (6)

Relu is the activation function, refers to the weight, while corresponds to the bias term, and i is the index for the quantity of convolution kernels. Then, the convolution layer results and are passed into the maximum pooling layer Max() of the CNN model to calculate and . Finally, the results are passed through two fully connected layers, which are defined as follows: (7) (8)

Step 2: and are fed into the BILSTM layer separately to obtain and through forward propagation, followed by reverse propagation to obtain and . Finally, the forward and reverse propagation results of the short text are added to obtain , and the forward and reverse propagation results of the long text are added to obtain . which and represent hidden layer trainable parameters, and represent input layer trainable parameters, and represent bias, ⊕ represents the concatenation of vectors. (9) (10) (11) (12) (13) (14)

Position coding(Pe) is added after the BILSTM layer to effectively incorporate location information. First, the output of the CNN layer, and , and are fed into the position coding layer to obtain corresponding word position information. The location information is encoded using the transformer and reference in the position coding method. The corresponding position-coding is calculated as follows: (15)

The placement of each word in relation to others in the short text is denoted as , while the set of relative positions of each word in the long text is represented as . Similarly, the set of relative positions of each word in the aspect-level after the DDP-ASDN model is represented as , and z denotes the embedding dimension. The phase position coding of each word is combined into a new z-dimension matrix using the sine cosine function. The combined output matrix of short text, long text, and aspect-level is given as follows: (16)

Step 4: The results of position coding, , , and , are added to the results of the BILSTM layer, and , to obtain new encoded representations,short text and long text . (17)

Step 5: The encoded representations, , and sentiment evaluation label are used as input to the normalization layer, to obtain new short text encoding h^V, long text encoding h^LV and sentiment evaluation label enconding T^DDP. which performs the following steps: (18)

The impact factors are then computed based on the normalized results, which are defined as follows. The symbol a represents the ATTENTION mechanism, which calculates the attention between different words. (19)

The weight distribution of the attentions are obtained by calculating the influence factor, defined as follows: (20)

Then the weight factors are summed over, which can be defined as follows: (21)

Then the attention results of the two layers are summed to obtain the attention that contains the context and aspect-level relationships. This is defined as follows: (22)

3.5 Classification results

The attention vector undergoes normalization after being processed by a fully connected layer, and the classification label is obtained by taking the maximum value of the normalized result. Finally, the aspect-level Triples are combined for each aspect sentiment recognition ensemble. which W^Att represents the trainable parameters of the dual attention layer. (23) (24)

4.1 Datasets

Two publicly available datasets and one collected by our team were selected in this paper. The detailed descriptions of the datasets are provided below.

1. 1: The Fine-grained Automotive Comment Standard Dataset (dataset 1), comprises 56,920 comments collected from various automotive forums. User comments were annotated with labels such as manufacturer, brand, model, attribute, descriptive value, tendency, and more. This dataset can be download from: https://www.datatang.com/dataset/info/text/135
2. 2: The Tan Songbo-Hotel Review Corpus (dataset 2), consists of 10,000 articles automatically collected and collated from Ctrip. The corpus comprises 7,000 positive sentiment data and 3,000 negative sentiment data. This dataset can be download from: https://pan.baidu.com/s/1TrumHVMk-Kc4PJz8INMYbg
3. 3: The U+ Wisdom Teaching Platform Dataset (dataset 3), was extracted from the U+ New Engineering Wisdom Cloud Platform. The dataset includes course evaluation data and experimental summaries of 50 universities, comprising a total of 9,000 long texts and 12,000 short texts. The dataset contains two sentiment categories: positive and negative, which can be further decomposed into fine-grained sentiment, totaling 30,000. This dataset can be download from: https://www.eec-cn.com

All experiments were conducted on four Nvidia V100 GPUs, and the datasets were split into a ratio of 6:1:3 for training, validation, and testing sets.

4.2 Baseline methods

In order to assess the efficacy of the recommended techniques, a comparison is made with the ten most advanced methods as described below:

ATAE-LSTM [44]: The sentences are processed based on the LSTM model, weighted and aggregated by combining the association of context and aspect terms.

BiGCN [45]: A graph convolutional network with two levels of interaction is devised based on dependency tree and word co-occurrence relationships to fully learn the node representation.

TNet [46]: The sentence feature representation is encoded using BILSTM and undergoes continuous aspect-level context encoding and attention mechanism, and the final feature representation is extracted using CNN.

DualGCN [47]: The SemGCN two-channel structure integrates grammar knowledge and semantic information with SynGCN and attention mechanism based on the dependent analytical probability matrix. Orthogonal regularization and differential regularization in SemGCN help the framework to grasp meaning associations different from the grammatical structure.

MemNet [11]: Context sentences are considered as external memory and attention mechanism with multiple heads are applied to the word vector characterization of the context. The final aspect characterization is derived from the output of the last hop.

DM-GCN [48]: Syntactic graph and semantic graph are constructed based on dependent tree and multi-head self-attention mechanism respectively. Information is extracted through syntactic graph convolution (Syntax GCN) and semantic graph convolution (Semantic GCN) respectively. The Common GCN module is utilized with a parameter sharing strategy to acquire shared information from the two spaces. The information extracted from the three channels is fused and used for the classification task.

ASGCN [49]: Feature representations of sentences are acquired using BILSTM. Aspect-level context representations are learned through dependency tree-based GCN, and aggregated context representations are used for classification using the attention mechanism.

GTS [50]: A grid marking scheme is designed to solve the triplet separation problem in an end-to-end system. The adopted inference strategy makes full use of the indicating role between different opinion elements.

IMN+IOG [51]: The interactive network IMN is used to extract explicit target aspects in the sentence. Then, the IOG framework is used to derive the associated opinion words to generate the final triples.

IAN [52]: Two LSTM models are used to encode text environment and aspects. Additionally, the Interdependent attention mechanism is employed to model the relationships between them.

BERT [53]: The model is fine-tuned by simultaneously inputting modified new tokens as both [cls] and [seq].

RoBERTa [54]: RoBERTa is built on the basis of the language masking strategy in BERT, modifying key hyperparameters in BERT, including deleting the next sentence training objective in BERT, and using a larger batch size and learning rate for training. It is relatively friendly to Chinese text classification.

dotGCN [55]: This model introduced a discrete latent tree, a structure designed to serve as a substitute for the traditional dependency tree. This new structure is language-independent and specifically tailored to a particular aspect.

4.3 Evaluation criteria

The evaluation metrics employed in this article consist of Accuracy(Acc), Precision(P), Recall(R), and Macro-F1(F1). The definitions of the indicators are provided below [56]: (25)

TP stands for the true positive class, which is the case when an instance is a positive class and is predicted as such. The true negative class is denoted by TN, this occurs when an instance is classified as a negative class and is indeed negative. FP represents the false positive class, this happens when an instance is actually positive, but is classified as negative by the prediction model. The false negative class is denoted by FN, this is the scenario where an instance is falsely classified as negative despite being positive.

4.4 Experimental environment

In this paper, deep learning model is constructed based on Word2vec, CNN, BILSTM, POS (Position Coding), DDP-ASDN, ATTENTION, and Aspect-Level sentiment dictionaries. The parameter configurations are presented in Table 1:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. ASN-CSD model hyperparameter details.

https://doi.org/10.1371/journal.pone.0295331.t001

4.5 Experimental results

In this paper, the performance of the recommended ASN-CSD framework and ten baseline models are compared on three datasets to evaluate ABSA. The results are presented in Table 2 for the Fine-grained automotive comment standard dataset. Table 3 for the Tan Songbo-Hotel review corpus, and Table 4 for the U+ wisdom teaching platform dataset. It is experimentally demonstrated that the ASN-CSD model performed optimally on all three datasets, yielding the highest Accuracy performance, Precision, Recall, Macro-F1, and computational time. The suboptimal models are outperformed by the proposed model by approximately 0.1000 on all evaluation indices. The superior performance of the recommended framework can be come down to its dual attention mechanism of context and aspect-level sentiment dependence, which enables better identification of aspect-level sentiment. Furthermore, the proposed model exhibited excellent computational efficiency, with most models improving by approximately 100.0 seconds. This is due to the use of a sentiment dictionary to pre-identify the dependence relationship and the relationship between internal words, thus reducing computation time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Classification results on dataset fine-grained automotive comment standard dataset (Optimal results are bold).

https://doi.org/10.1371/journal.pone.0295331.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Classification results on Tan Songbo-Hotel review corpus (Optimal results are bold).

https://doi.org/10.1371/journal.pone.0295331.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Classification results on the U + Wisdom teaching platform dataset (Optimal results are bold).

https://doi.org/10.1371/journal.pone.0295331.t004

After conducting further analysis of the outcomes of the experiments, a noticeable observation is that most of the models demonstrated better achievement on the two public datasets than on the U + wisdom teaching platform dataset. This can be attributed to the highly specialized nature of the data in the domestic education field and the limited coverage of the sentimental dictionary. Moreover, the involvement of students in the sentimental evaluation summary can lead to ambiguous sentimental expressions that fail to express their true feelings. Additionally, it is noted that the performance of most baseline models are inferior on the Chinese datasets compared to their English counterparts. However, the ASN-CSD model effectively handles Chinese data and performs well in specialized fields, indicating its strong generalization ability and potential for future promotion. Furthermore, it is found that the use of sentimental dictionaries and dependency models often leads to superior performance compared to other models. This underscores the advantages of combining aspect-level sentimental dictionaries, dependency relationships, and contextual mechanisms for the ABSA task.

4.6 Ablation experiment

In this section, five sets of juxtaposition models are created to further verify the effectiveness of each module in the ASN-CSD model, which are specifically described as follows:

1. ASN-CSD∼CBA: The CNN-BILSTM-ATTENTION module is removed, leaving only the DDP-ASDN and position coding modules.
2. ASN-CSD∼POS: The position coding module is removed, retaining the CNN-BILSTM-ATTENTION and DDP-ASDN modules.
3. ASN-CSD∼ATT: The dual attention mechanism is removed, retaining the CNN-BILSTM module, position coding, and DDP-ASDN modules.
4. ASN-CSD∼ASD: The aspect sentiment dictionary is removed, retaining the CNN-BILSTM-ATTENTION module, position coding, and syntactic dependency.
5. ASN-CSD∼DDP: The syntax-dependent DDP is removed, retaining the CNN-BILSTM-ATTENTION module, position coding, and sentiment dictionary.
6. ASN-CSD: The complete model presented in this paper, which retains the CNN-BILSTM-ATTENTION module, position coding, and DDP-ASDN module. These models are designed to compare and evaluate the performance of each module, allowing for a better understanding of the contribution of each module to the overall performance of the model.

The achievement of the five juxtaposition models on the three datasets are presented in Figs 5–7. It is observed that the removal of syntactic dependence analysis and sentiment dictionary led to a significant decline in performance, with the ASN-CSD∼DDP showing the most significant decrease, and the average accuracy decreasing to below 0.2768. This highlights the crucial role of syntactic dependence analysis and sentiment dictionary in aspect-level sentiment recognition. Sentiment dictionary can identify sentimental words, while syntactic dependence can quickly identify the sentimental tendency of these words, thus demonstrating the effectiveness of the DDP-ASDN layer presented in this paper.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Classification results in fine-grained automotive comment standard dataset after removal of partial modules.

https://doi.org/10.1371/journal.pone.0295331.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Classification results In Tan Songbo-Hotel review corpus after removal of some modules.

https://doi.org/10.1371/journal.pone.0295331.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Classification results in U + Wisdom teaching platform dataset after removal of some modules.

https://doi.org/10.1371/journal.pone.0295331.g007

The performance of the ASN-CSD∼POS and ASN-CSD∼ATT models on the three datasets reveals a decrease in performance after removing the position coding or attention mechanism. In dataset 3, the minimum drop in precision is only 0.0064, which can be attributed to the BILSTM and syntactic dependence models capturing the relationship between words within the sentence and obtaining some of the characteristics of the internal relationship.

The ASN-CSD∼CBA model shows that aspect-level sentiment classification can still be performed by removing all contextual association features and retaining only the DDP-ASDN model. However, this approach results in a decrease in performance of around 0.1000, mainly due to the lack of context-level feature vectors.

When further examining the ASN-CSD∼CBA and ASN-CSD∼ASD models on datasets 1 and 2, it is observed that the ASN-CSD∼ASD performed better than the ASN-CSD∼CBA. This result highlights the importance of the sentiment dictionary for the sentiment of recognition at an aspect-level sentiment recognition, which is more crucial than the correlation characteristics between contexts. However, on dataset 3, the ASN-CSD∼ASD model performed less effectively than the ASN-CSD∼CBA model. The reason for this may be due to the fact that the data in dataset 3 uses more professional vocabulary, leading to lower coverage of sentiment dictionaries. This result further emphasizes the significance of the proposed sentiment dictionary in aspect-level sentiment recognition.

The worst performance for the three datasets is exhibited by the ASN-CSD∼DDP model, highlighting the importance of syntactic dependence analysis in aspect-level sentiment recognition. Without dependency analysis, it is impossible to obtain the direct sentimental relationship of sentences, leading to poorer aspect-level sentiment recognition. This underscores the necessity of this paper based on syntactic dependence.

Overall, the complete ASN-CSD framework achieved the best achievement for aspect-level sentiment recognition. The ablation experiments have strongly demonstrated the necessity and effectiveness of each module in the ASN-CSD.