https://orcid.org/0009-0002-8862-2439
Figures
Abstract
In the context of existing adversarial attack schemes based on unsupervised graph contrastive learning, a common issue arises due to the discreteness of graph structures, leading to reduced reliability of structural gradients and consequently resulting in the problem of attacks getting trapped in local optima. An adversarial attack method based on momentum gradient candidates is proposed in this research. Firstly, the gradients obtained by back-propagation are transformed into momentum gradients, and the gradient update is guided by overlaying the previous gradient information in a certain proportion to accelerate convergence speed and improve the accuracy of gradient update. Secondly, the exploratory process of candidate and evaluation is carried out by summing the momentum gradients of the two views and ranking them in descending order of saliency. In this process, selecting adversarial samples with stronger perturbation effects effectively improves the success rate of adversarial attacks. Finally, extensive experiments were conducted on three different datasets, and our generated adversarial samples were evaluated against contrastive learning models across two downstream tasks. The results demonstrate that the attack strategy proposed outperforms existing methods, significantly improving convergence speed. In the link prediction task, targeting the Cora dataset with perturbation rates of 0.05 and 0.1, the attack performance outperforms all baseline tasks, including the supervised baseline methods. The attack method is also transferred to other graph representation models, validating the method’s strong transferability.
Citation: Zhang Q, Qin Z, Zhang Y (2024) MCGCL:Adversarial attack on graph contrastive learning based on momentum gradient candidates. PLoS ONE 19(6): e0302327. https://doi.org/10.1371/journal.pone.0302327
Editor: Xiao Luo, University of California Los Angeles, UNITED STATES
Received: January 28, 2024; Accepted: April 2, 2024; Published: June 6, 2024
Copyright: © 2024 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The Cora dataset, CiteSeer dataset and PolBlogs dataset used in the current study are only used for research and can be available at https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz, https://linqs-data.soe.ucsc.edu/public/lbc/citeseer.tgz and https://netset.telecom-paris.fr/datasets/polblogs.tar.gz, respectively.
Funding: Funding name:Research on the Construction of Knowledge Graph in the Field of Telecom Fraud Funder: Senior Engineer,Zhenkai Qin Funding number: 2023GAZD018 Funders provide assistance in data collection, manuscript review, and revision.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Graph neural networks have gained widespread application across various domains due to their outstanding feature extraction mechanisms and excellent predictive performance. These domains include social networks [1, 2], recommendation systems [3, 4], and traffic prediction [5, 6]. By aggregating information from neighboring nodes, these networks enable models to effectively learn the underlying graph structure, leading to significant improvements in downstream tasks such as graph classification [7, 8], node classification [9], and link prediction [10, 11].
However, the application security of graph neural networks has raised concerns due to their weak interpretability and the discrete complexity of graph data. Recent research indicates that graph neural networks are susceptible to adversarial attacks [12], where attackers can deliberately insert imperceptible perturbations into the input, potentially causing prediction errors in the model and increasing the potential risks of graph neural network applications. For instance, attackers could forge transaction data in a banking system, leading to fraud detection models misclassifying fraudulent transactions as normal ones, resulting in unforeseen consequences.
By delving into the study of graph adversarial attacks, we can examine the issue from the attackers’ perspective, uncovering inherent security vulnerabilities in neural networks and, more effectively, preventing potential risks. This research enhances the robustness of graph neural networks, ensuring their security and reliability in practical applications. Adversarial attacks have been extensively studied in the field of deep learning [13, 14].
In graph research, numerous studies have explored the robustness of graph data. The generation of adversarial samples for graph data primarily relies on the saliency of structural gradients, i.e., the gradient information of the objective function concerning the adjacency matrix. Nettack [15] first proposed graph adversarial attacks, targeting specific nodes in the graph by introducing perturbations or noise through modifications to the graph structure (such as adding or removing edges) or node attribute features (such as adding or modifying attributes), resulting in misclassification of the targeted node by the model. Metattack [16] constructed an adversarial attack model using meta-learning methods, simultaneously considering the meta-learning process of graph neural networks. Xu et al. [17] introduced a topology attack method based on an optimization perspective. This method initially optimizes the graph, generating a series of topology attack samples, and subsequently utilizes these samples to attack graph neural networks. The paper also introduces projection gradient descent (PGD) and Min-Max strategies for optimizing topology attacks.
Gradients have been widely applied as a critical basis for generating perturbations. However, researchers have started to investigate the impact of the reliability and saliency of gradients on perturbation updates. Liu et al. [18], by studying the gradient errors caused by model optimization uncertainty and robustness discussed the role of gradients in graph adversarial attacks. They introduced features like momentum gradients and added Gaussian noise to reduce the impact of gradient uncertainty. EpoAtk [19] demonstrates that the discreteness of graph structures leads to the fact that the most significant gradients do not necessarily have the maximum impact on the model predictions. The paper proposes an exploratory method to alleviate local optima issues while generating perturbation graphs.
Currently, adversarial attacks on graph neural networks primarily rely on supervised methods [20, 21]. These methods heavily rely on accurate labels and are tailored for specific downstream tasks, while in reality, there is a substantial amount of unlabeled data. The costly nature of labeled data and graph structure complexity have become bottlenecks in developing supervised adversarial attacks.
Due to the high dependence on labeled samples, semi-supervised and unsupervised graph representation learning has become a widely studied focus [22–24]. Graph contrastive learning, as an unsupervised graph representation learning method, has been widely studied and applied in many practical scenarios due to its excellent feature extraction ability [25].
However, exploring the robustness of unsupervised graph representation learning remains challenging [26]. Adversarial attacks and defenses for unsupervised graph neural networks have become a hot topic in current research. Bojchevski et al. [27] conducted the initial study on unsupervised graph adversarial attacks, leveraging a property in generating embeddings using a random walk-based algorithm in spectral graphs. The study reduced the effectiveness of embeddings for newly generated graphs by modifying a limited number of edges. However, this method only applies to the DeepWalk model [28], and its generality and effectiveness on other graph neural networks and downstream tasks are limited.
Zhang et al. [29] employ ordinary gradients of the contrastive loss concerning the adjacency matrix of the data-augmented graph in adversarial attacks. After summing the gradients of the two views, they directly choose the edge corresponding to the most significant gradient value as the perturbed edge for the model attack, achieving specific adversarial effects. However, due to the discreteness of graph structures and the neglect of the reliability factor of gradients, this method is prone to local optima, resulting in suboptimal perturbation effects and slow convergence speed.
In order to solve the above challenges, an adversarial attack method on graph contrastive Learning based on momentum gradient candidates(MCGCL) is proposed in this paper. Specifically, after obtaining the node gradient matrix, a certain proportion of the previous gradient information is added to transform ordinary gradients into momentum gradients, thereby improving the convergence speed and accuracy of gradient updates. Subsequently, the gradients of two views are summed, and based on the candidate set size, gradient saliency, and perturbation rules, a candidate perturbation graph set is generated. All candidate perturbation graphs are assessed using an evaluation function, and the final perturbed edge is selected by comparing the evaluation results. This process aims to generate adversarial samples with more effective perturbation effects, enhancing the attack’s effectiveness. Experimental results demonstrate the method’s efficacy and transferability.
The main contributions based on this article are summarized as follows.
- Due to the discreteness of the graph data and the uncertainty of model parameter updates, the use of momentum gradients instead of ordinary gradients is proposed. The momentum gradients utilize previous gradient information to guide gradient updates, improving the stability and efficiency of the optimization process. Improve the accuracy of parameter updates and model convergence speed, improving attack effectiveness and reducing resource waste.
- Due to the non-Euclidean graph structure and the noise introduced by data augmentation, the most significant gradient values may have a lower impact on model predictions. This paper proposes an exploratory method based on a contrastive model for candidate selection and evaluation. This method mitigates the risk of adversarial attack results falling into local optima, generating more perturbed and impactful adversarial samples. The method enhances the accuracy of perturbations by reducing the risk of being trapped in local optima, thus improving the overall effectiveness of the generated adversarial samples.
- The proposed method is tested on three real datasets and two downstream tasks and is compared with various supervised and unsupervised attack methods. Extensive experiments demonstrate the effectiveness of our method. Furthermore, the attack method is extended to different graph contrastive learning models and other classical graph neural networks, confirming the strong transferability of our method. The effectiveness of the two proposed strategies is validated through ablation experiments.
Related works
Graph adversarial attack
Security concerns have become a focal point of attention with the rapid development and widespread application of neural networks. Designing efficient and broadly applicable adversarial attack and defense [30, 31] strategies has emerged as a current research hotspot. Existing attacks can be classified into several categories based on different criteria.
Based on different attack stages, attacks can be categorized into node poisoning attacks before model training [27] and evasion attacks during model training or testing stages where the attacker cannot modify model parameters and structures [32]. Depending on the attacker’s knowledge of the target model, attacks can be classified into black-box attacks [33], grey-box attacks [34], and white-box attacks [35]. According to different attack objectives, attacks can be further divided into targeted [15] and global attacks [16].
The attack scenario in this paper involves grey-box global poisoning attacks. Attackers leverage training data to train substitute models, allowing them to infer information from the victim model. Since altering node features has a limited impact on the perturbed graph structure and some graphs lack node features, such as the PolBlogs dataset, this paper focuses solely on perturbing the topological structure. The goal is to reduce the overall predictive performance of the model, aligning with the practical requirements of real-world applications.
Graph contrastive learning
Graph contrastive learning is a typical representation of unsupervised graph neural networks. DGI [36] is a pioneering study in graph contrastive learning, introducing an unsupervised learning method that maximizes the mutual information of graph data to learn node representations. GraphCL [37], during the data augmentation process, incorporates a comprehensive set of random augmentation strategies, considering both topological structure and node features. On the other hand, the contrastive learning model with a negative sample sampling strategy [38] effectively converted all nodes except the positive sample into negative samples by selecting nodes with labels different from the center node. In addition, it utilized GCN [39], SGC [40], and APPNP [41] as shared graph neural network models.
This paper adopts GCA [42] as the framework for contrastive learning. On the topological level, an enhancement scheme based on node centrality metrics highlights important connectivity structures. On the node attribute level, more noise is added to the features of less important nodes to disrupt node features, thereby forcing the model to recognize underlying semantic information. This paper’s data augmentation combines topological and feature levels, sharing a graph neural network with two layers and a Multilayer Perceptron(MLP). The graph embeddings generated through graph contrastive learning can be applied to downstream tasks such as node classification (predicting node categories) and link prediction tasks (predicting potential edges between nodes). The two-layer graph convolutional network(GCN) [39] structure is as follows:
(1)
Among them, represents the adjacency matrix with self-loops, where
.
denotes the degree matrix. The activation functions used are softmax and ReLU. The model parameters are denoted as W(0) and W(1).
(2)
(3)
The contrastive loss function is defined as shown in Eq (2). Here, the similarity measure utilizes cosine similarity, where β represents the cosine similarity function and τ is the temperature coefficient. The numerator computes the similarity of positive pairs of node embedding representations, while the denominator summarizes the similarity of positive and negative pairs. To reflect the symmetry and balance of the loss, this paper sum both l(ui, vi) and l(vi, ui).
Preliminaries
Notations
In the graph G(V, E), V represents the set of nodes with a total of n nodes, and E represents the set of edges. A ∈ RN×N denotes the adjacency matrix used to represent the graph’s topological structure, where 0 indicates no edge between nodes, and 1 indicates the presence of an edge between nodes. X ∈ RN×s represents the feature vectors of the nodes in the graph, where each node corresponds to an s-dimensional feature vector. The symbol definitions used are provided in Table 1.
Threat model
Attacker’s goal.
Before model training, perturbation is applied to the dataset by introducing noise, aiming to diminish the overall performance of the learned unsupervised graph contrastive learning model, consequently leading to misclassification in test results.
Attacker’s capability.
This paper investigates grey-box global poisoning attacks in graph adversarial scenarios. In real-world scenarios, attackers typically lack direct access to the target model. Instead, they leverage training data to train substitute models, enabling them to infer information from the compromised model. Additionally, this study adopts an unsupervised learning method, eliminating the need for access to data labels during the training process.
Problem definition
Graph adversarial attacks should be regarded as a bi-level optimization problem [43], with the inner loop involving training the graph contrastive learning algorithm and the outer loop focusing on generating perturbed graphs. The following equation defines the problem formulation for node-level graph adversarial attacks.
(4)
G′ can be G or . The attacker’s objective is to find a perturbed graph
that maximizes the loss value of the nodes, thereby reducing the overall prediction performance of the model.
The attack consists of two main steps: generating adversarial samples and attacking the model. The graph structure matrix is treated as a hyperparameter in generating adversarial samples. Forward propagation is employed to compute the embedding representations of graph nodes and the target loss. Backpropagation is then utilized to calculate the gradient information for all graph nodes under the current model. This gradient vector reflects the contribution of each node to the model’s prediction results. The gradient formula is represented as follows:
(5)
Arrange the obtained node gradients in descending order. The attacker determines the perturbation status of edges based on the saliency of node gradients and the actual existence of edges.The specific perturbation rules are as follows:
(6)
E ∪ (u, v),E\(u, v) represent adding and removing edges, respectively. Under the premise of perturbation rules, select the edge corresponding to the most significant gradient value as the perturbation edge for this iteration. Flipping edge Ei,j is a potential perturbation that negatively impacts the victim model. This perturbation is added to the perturbation vector determined in the previous iteration to generate the perturbation graph for this iteration. The attacker optimizes the perturbation vector to minimize the model’s predictive accuracy. The generation of perturbation graphs employs a greedy strategy, perturbing one edge at a time. The schematic diagram is as follows:
(7)
It is necessary to set a perturbation threshold to ensure the imperceptibility of adversarial samples while meeting the requirement of reducing the overall model performance. The perturbation includes both topological structure perturbation and node attribute feature perturbation. The formula is defined as follows:
(8)
Method
Overall
Fig 1 illustrates the framework structure for implementing contrastive learning adversarial attacks using momentum gradient candidates. The original graph undergoes adaptive data augmentation to generate two views. These generated views are input into a shared graph neural network to obtain node embedding representations for each view and compute the contrastive loss. Subsequently, during the backpropagation process, gradients of the loss concerning the adjacency matrices of the two views are calculated separately. Building upon the current gradients, momentum gradients are generated by incorporating a certain proportion of the previous gradients. Subsequently, the absolute values of the momentum gradients are calculated and arranged in descending order, entering the candidate phase. Candidate edges are selected based on the gradient saliency, the candidate set size and the perturbation rule defined in Eq (6), forming a candidate perturbation graph set. After evaluating each candidate graph with an evaluation function, the edge with the highest loss value is chosen as the perturbed edge for this iteration. This process is repeated until the number of perturbed edges reaches the threshold. The objective function is defined as:
(9)
represents the contrastive loss, θ and θ′ are the model parameters before and after perturbation, t1 and t2 are two randomly chosen augmentation functions. The constraint aims to balance the imperceptibility and effectiveness of the perturbation. The term 2σ indicates the same edge for symmetric positions about the main diagonal in the adjacency matrix. Additionally, this paper only perturbs the topological structure while keeping the node attributes unchanged to perturb the entire graph structure. Each view’s gradient is represented as:
(10)
and
respectively represent the gradients of the loss concerning two views in the k-th iteration.
Momentum gradients
The gradient is a critical factor in guiding the generation of perturbation vectors. The discreteness of graph data and the uncertainty of model parameter update will cause unreliable factors in the structural gradient. Meanwhile, the perturbation process of edges is iterative, and the perturbed edges generated in each iteration will affect the topological structure of the entire graph. The calculation of momentum gradient uses a weighted sum method, which guides the update of gradients by utilizing previous gradient information, reducing the oscillations phenomenon of gradients during the update process. Therefore, using momentum gradients instead of ordinary gradients can improve the speed of optimization convergence and the reliability of structural gradients, thereby increasing the success rate of attacks.
Specifically, this article proposes to guide gradient updates by accumulating previous gradient information in a particular proportion based on the two view gradient vectors generated by backpropagation. This accumulated gradient is called the momentum gradient, and the specified proportion is called the momentum coefficient, denoted as p. The formula for gradient update is as follows:
(11)
,
are two views of the k-th iteration of momentum gradients respectively,
,
are two views of the (k − 1)-th iteration of momentum gradients respectively,
,
two views of the k-th iteration of gradients respectively.
After data augmentation, differences arise between views and the original graph, impacting the saliency of gradients due to stochastic augmentation methods. The saliency of individual view momentum gradients cannot fully represent the saliency of the original graph gradients. To adequately capture the saliency of the original graph gradients and mitigate biases introduced by stochastic augmentation, this paper utilizes the combined saliency of gradients from two views to determine perturbed edges. The specific formula is expressed as follows:
(12)
Subsequently, the absolute values of the momentum gradients are calculated and arranged in descending order, entering the candidate phase.
Candidate and evaluation
Gradient saliency characterizes the importance of edges in model prediction. However, due to the discreteness of the graph structure and the presence of data augmentation noise, the most significant gradient may have a small impact on model prediction, leading to the risk of attack results falling into local optima. Therefore, this article proposes an exploratory method for candidate and evaluation.
Specifically,after obtaining the combined gradients from two views, this paper introduces an exploratory process of candidate and evaluation instead of simply choosing the edge with the most significant gradient as the perturbed edge for this iteration. Firstly, determine the size S of the candidate set. After sorting the momentum gradients, based on perturbation rules defined in Eq (6), S edges are sequentially selected to join the candidate set according to the edges’ actual existence and gradient saliency, forming a set of candidate perturbation edges. The candidate set H is represented as:
(13)
Secondly, each edge in the candidate set is individually added to the perturbation vector generated in the previous iteration, forming candidates for this iteration’s perturbation graph. All candidate perturbation graphs are then input into the evaluation function sequentially. The evaluation function used is as follows:
(14)
Z is the embedding representation obtained through forward propagation of the model trained in the previous iteration, is the contrastive loss value, and Z1 and Z2 represent the embedding representations of two views after passing through the shared graph network. The contrastive loss is computed, and the candidate edge with the worst loss is selected as the officially perturbed edge for this iteration. This selected edge is added to the previously generated perturbation vector, forming the perturbation graph for this iteration. The optimal perturbation graph is selected as follows:
(15)
represents the perturbed edge for this iteration, and
is the evaluation result for each perturbation candidate graph. The candidate evaluation process is akin to comparing the importance of candidate perturbed edges in the graph structure. The overall algorithm process is shown in Algorithm 1
Algorithm 1 MCGCL
Input: Original graph G = (A, X), differentiable encoder f, stochastic augmentation set T, number of perturbations σ, number of iterations K, momentum coefficient p,size of candidate set S.
Output: Poisoned graph G = (A′, X).
1: i = 0,A′ = A.
2: while i < σ do
3: Train f with A′ and X.
4: Δ′ = 0.
5: for k = 1 to K do
6: Sample two stochastic augmentations .
7: Obtain two views( (A′, X),
(A′, X).
8: Forward propagate(),(
) through f and compute contrastive loss
.
9: Obtain the gradients of and
w.r.t. the contrastive loss,
,
.
10: Compute momentum gradients of and
,
,
.
11: Sum the gradients of two views, +
.
12: end for
13: Sort in descending order of absolute gradient value.
14: for s = 1 to S do
15: Compute the [row, column] values of the edge corresponding to the gradient values in the adjacency matrix.
16: if A′[a, b] = 1, Δ′[a, b] < 0 or A′[a, b] = 0, Δ′[a, b] > 0 then
17: Form the perturbed candidate graph of the current iteration by adding [a,b] to the perturbed graph formed in the last iteration.
18: Calculate the loss of each candidate graph by evaluation function.
19: end if
20: end for
21: Compare and select the edge [m, n] with the highest loss as the perturbation edge of this iteration.
22: A′[m, n] = 1-A′[m, n].
23: Freeze the chosen edge and avoid being flipped again in next iterations.
24: i = i + 1.
25: end while
Complexity analysis
Time complexity.
Firstly, assuming the complexity of the forward propagation process in the GCA model is denoted as O(|E|). The time complexity for retraining the contrastive model once is denoted as O(K|E|), where K represents the number of iterations for training the model. The MCGCL method’s computational costs mainly involve generating momentum gradients, gradient sorting, and candidate evaluation. The process of generating momentum gradients can be considered recursive, and the complexity of weighted summation is constant, resulting in a time complexity denoted as O(σ). The time complexity for gradient sorting is O(|V|2), and for candidate evaluation, it is O(S|E|). Therefore, the total time complexity of MCGCL is denoted as O(K + S)σ|E| + O(σ) + O(σ|V|2).
Although the complexity of the gradient sorting process is denoted as O(|V|2), which mainly involves a straightforward sorting operation, the primary computational costs still lie in other parts. Disregarding the differences in surrogate model structure and complexity, leading to significant variations in computational efficiency due to retraining the model, the inherent computational efficiency of the MCGCL method is similar to that of the Metattack [16]. Compared with GLGA [29], the primary computational costs occur during the candidate evaluation stage. On the one hand, the additional costs have a negligible impact on improving the attack success rate. On the other hand, the trade-off between attack effectiveness and computational efficiency can be balanced by adjusting the size of the candidate set. For Min-Max [17], during poisoning attacks, it is necessary to retrain the model and utilize random sampling to generate each perturbation, resulting in additional computational overhead.
Space complexity.
Compared to CLGA [29], due to the need to calculate the adjacency matrix gradient, the memory requirement is O(N2), where N represents the number of nodes in the graph. Additionally, each iteration requires saving lists of candidate perturbation graphs and evaluation function results, with both sizes equal to the size of the candidate set S. However, MCGCL saves candidate perturbed edges rather than the entire candidate perturbation graph to reduce memory consumption. After each iteration of candidate perturbation is completed, the memory is immediately cleared and released, thereby reducing the memory cost. The Metattack [16] incurs significant storage space consumption while generating meta-gradients.
Taking into account the time and space costs of the MCGCL method, they are both within manageable limits. Simultaneously, adjusting the candidate set size allows for a balance between computational efficiency and attack effectiveness compared to other baseline methods. Using momentum gradients for parameter updates accelerates convergence, reduces the number of iterations required for model training, and improves attack effectiveness.
Experiments
Setup
Datasets.
This paper employs the Cora, CiteSeer, and PolBlogs datasets. Cora and CiteSeer datasets are citation networks where each node has corresponding attribute features, while the PolBlogs dataset is a social network graph with nodes lacking features. The basic details of the datasets are summarized in Table 2.
Baselines.
This paper focuses on global poisoning attacks. The supervised baseline methods employed include DICE [44], PGD [17], Min-Max [17], and Metattack [16]. CLGA [29] and Bojchevski [27] are the unsupervised baseline methods mentioned.
Experimental settings.
The baseline experimental data in this paper is sourced from CLGA [29]. For Metattack [16], PGD [17], and Min-Max [17] baseline methods, a two-layer GCN model is employed as the surrogate model. All attack methods involve initially generating perturbation graphs, which are then fed into the advanced graph-contrastive learning model GCA [42] for training and model accuracy testing. The GCA model uses a two-layer GCN as the encoder. To enhance the representativeness and reliability of the experimental results while reducing random errors, the paper conducts ten experiments separately for node classification and link prediction tasks, taking the average as the final result.
In this paper, the perturbation rates were set to 1%/5%/10% of the total number of edges in the original graph. We use grid search to determine the optimal value of hyperparameters, i.e. the candidate set sizes are chosen from {64, 96, 128}, and momentum coefficients are selected from {0.75, 0.8, 0.85, 0.9}. The optimal sizes for the candidate set and momentum coefficient are 128 and 0.85, respectively. For ease of comparing the effects of different attack strategies, the paper sets the hyperparameters of the GCA [42] model in the experiments, including temperature coefficient τ and random augmentation rate for data, to fixed values. Expressly, the temperature coefficient τ is set to 0.4, the topological augmentation rates for two views are set to 0.3 and 0.4, the feature augmentation rate is set to 0.1 and 0.0, and the optimizer used is Adam. The learning rate is set to 0.01. For the PolBlogs dataset, 32-dimensional vectors are experimentally generated randomly and used as node features as nodes lack feature vectors.
For the node classification task, the Cora and CiteSeer datasets partitioning follows the publicly available split provided by Yang et al. [45]. For the PolBlogs dataset, we divided the nodes into training, validation, and test sets with a ratio of 10%/10%/80%, respectively. The learned embedding representations are utilized as inputs to a logistic regression model, and the classification accuracy is calculated.
For the link prediction task, the three datasets partition the edges into train/test/val sets with a ratio of 70%/20%/10%. A 2-layer MLP is employed as the projection head to map the learned embeddings into a new latent space. The MLP is trained using negative sampling and margin loss for edge prediction, and uses AUC values to test the performance of model link prediction.
Experimental results and analysis of node classification
Table 3 presents the experimental results for node classification. Observing the table, except for the case on the Cora dataset, where the results are somewhat moderate when the perturbation rate is 0.01, our proposed method exhibits outstanding performance under other perturbation attacks. It significantly reduces the model’s accuracy in node classification tasks, generally surpassing other unsupervised methods.
Experimental results and analysis of link prediction
Table 4 presents the experimental results for link prediction. Observing the experimental data in the link prediction task, the Cora dataset exhibits performance comparable to or even better than supervised learning results when the perturbation rates are 0.05 and 0.1. Similarly, in the case of the CiteSeer dataset with a perturbation rate of 0.05, the method performs exceptionally well. For the PolBlogs dataset, except for the perturbation rate 0.01, our proposed method outperforms other unsupervised adversarial attack methods in link prediction. The experimental results demonstrate the effectiveness of our proposed attack method in link prediction tasks.
Ablation experiment
This paper conducted ablation experiments to compare the impact of the two proposed strategies on the attack effectiveness. The momentum gradient (MCGCL_M) and candidate and evaluation (MCGCL_C) methods represent ablation experiments conducted using only the momentum gradient and the candidate evaluation method, respectively.
Momentum gradients experiment.
The experimental results of MCGCL_M are presented in Table 5. Observing the experimental data, it can be noted that the generation of adversarial samples using the momentum gradient method alone, except for suboptimal performance in individual experiments, still demonstrates excellent overall attack effectiveness, highlighting the advantages of the momentum gradient.
Candidate and evaluation experiment.
The experimental results of MCGCL_C are presented in Table 6. Observing the experimental data, it can be noted that the overall perturbation effect is good, especially in link prediction tasks with perturbation rates of 0.05 and 0.1 on the Cora dataset and with a perturbation rate of 0.05 on the CiteSeer dataset. MCGCL_C demonstrates better performance than supervised learning in these scenarios. The candidate evaluation method evaluates multiple candidate graphs in the candidate set, avoiding information errors introduced by the most significant gradient values and reducing the risk of the attack results falling into local optima.
From the overall experimental results, on the one hand, the parameter settings (including momentum coefficient and candidate set size) leading to good performance are dataset and downstream task-specific. At the same time, when the downstream task is link prediction, our proposed method shows results comparable to or even better than supervised learning, especially on the Cora dataset. On the other hand, the results from ablation experiments demonstrate that certain individual methods may outperform joint testing results in specific scenarios (such as MCGCL_C in link prediction tasks on the Cora and CiteSeer datasets at a perturbation rate of 0.1). Overall, joint testing results still exhibit superior performance. The experiments also validate that changes affecting model prediction are more inclined towards adding edges rather than removing edges. This is because adding edges can alter the original graph’s topological structure while removing edges results in the loss of information from the original graph, a conclusion supported by the findings in literature [19].
Transferability analysis
On the Cora dataset, the MCGCL method uses the GCA [42] model to generate adversarial samples, which are then used to attack graph neural networks with different structures, such as GCN [39] and DeepWalk [28] models. A comparison is made with the overall best-performing supervised attack method, Metattack, and two other unsupervised attack methods to validate the transferability of the proposed method. Experimental results are presented in Tables 7 and 8.
For both node classification and link prediction tasks, evaluation is performed using classification accuracy and AUC. Other experimental settings remain consistent with the experiments above. Observing Tables 7 and 8, it is evident that even though the Bojchevski attack method is specifically designed for the DeepWalk model, the MCGCL method still achieves significant attack effectiveness against the DeepWalk model, surpassing Bojchevski, particularly in link prediction. In attacking the GCN model, the MCGCL method demonstrates superior attack effectiveness compared to other unsupervised methods. The experimental results substantiate that the proposed method, employing generated perturbation graphs to attack diverse graph neural network architectures, significantly reduces model prediction accuracy, thereby validating the method’s efficacy and robust transferability.
Visualization analysis
The results in Tables 3 and 4 show that the Metattack method exhibits the best overall performance. Therefore, in this paper, we conduct a comparative analysis of the embedding scatter plots between the Metattack and the proposed methods.
Fig 2 illustrates the visual scatter plot of the embeddings of the attacked model after generating perturbation graphs through different perturbation methods. On the other hand, Fig 3 displays the visual scatter plot of the node embeddings generated by the MCGCL method under different perturbation rates after passing through the GCA model. It can be observed from the figures that perturbation graphs with introduced noise reduce the model’s predictive capability, causing the boundaries between different categories to become blurred and pulling nodes towards the center. Due to the similarity in distance between central nodes and each category results in a decrease in the model’s predictive generalization ability.
From left to right are the embedded visualization scatter plots of clean graph, Metattack method, and MCGCL method, respectively.
From left to right are the embedded visualization scatter plots with perturbation rates of 0.01, 0.05, and 0.1, respectively.
Discussion
In the context of unsupervised attacks, the MCGCL method proposed in this paper uniquely addresses gradient reliability and saliency issues. By incorporating momentum gradients, the accuracy of gradient updates is enhanced. Simultaneously, leveraging candidate evaluation methods mitigates the problem of suboptimal attack outcomes resulting from reduced saliency of gradients. These effectively improve the attack accuracy.
Discussion of experimental results
Due to the utilization of labels as additional knowledge in the other four supervised baseline attacks, they are expected to perform better than unsupervised attacks. However, experimental results from Tables 3 and 4 demonstrate that MCGCL shows comparable performance, and in some cases, it even outperforms specific supervised baselines.
Simultaneously, for the PGD [17], Min-Max [17], and Bojchevski [27] methods, the classification accuracy does not continuously decrease with the increasing perturbation rates. This suggests that these three methods could have effectively targeted the edges crucial for the model predictions, indicating the robustness of graph contrastive learning against these attacks. In contrast, other baseline methods improve attack effectiveness with increasing perturbation rates. However, Metattack [16] and DICE [44] methods heavily rely on labels, whereas this paper, building upon the CLGA [29], addresses issues arising from reduced gradient reliability and saliency. Consequently, our method achieves overall superior attack effectiveness.
Limitations and mitigation methods
Due to the message passing mechanism of Graph Neural Networks (GNNs), the derivation process of structural gradients involves node features. This results in the propagation of noise from node features to structural gradients. Concurrently, graph contrastive learning effectively harnesses the advantages of topological and feature-level data augmentation, skillfully mitigating the structural gradient noise arising from node feature perturbations.
In addition to the saliency of gradients, factors influencing the attack results falling into local optima may also involve the greedy strategy in sequentially selecting perturbed edges while generating perturbation graphs, where the generation processes of perturbed edges may mutually interfere. Therefore, building upon candidate evaluation, this paper contemplates the potential effectiveness of mitigating the risk of attack results settling into local optima by introducing a more extensive candidate domain. An additional recombination phase is introduced to expand the candidate domain, aiming to reduce the interference in the generation process of perturbed edges. The next step involves rigorous experiments to validate this proposition.
Conclusion
In this paper, within the context of unsupervised graph contrastive learning adversarial attacks based on gradients, we propose an exploratory method using momentum gradient candidates to address the local optima problem arising from the unreliability and saliency of gradients. The method’s effectiveness is validated across different datasets, downstream tasks, and perturbation rates. Experimental results demonstrate that the proposed method outperforms other unsupervised methods regarding attack effectiveness. It even surpasses supervised baseline methods in some link prediction experiments, exhibiting superior performance. This research accelerates convergence speed, enhances the accuracy of parameter updates, and consequently improves attack effectiveness. Additionally, the strong transferability of the proposed method is empirically demonstrated through experiments.
In future work, in terms of adversarial attacks, we plan to alleviate the problem of local optima caused by the sequential selection of perturbed edges proposed in the second section of the “Discussion” by expanding the candidate range to obtain a wider range of choices, and demonstrate the effectiveness of this method through experiments. More importantly, we plan to investigate defense strategies against attacks in graph contrastive learning. Currently, most defense strategies are primarily designed for supervised attacks, utilizing additional label information for model retraining and applying metrics such as degree centrality to prune malicious edges. Our future research direction will concentrate on proposing defense strategies tailored explicitly for countering unsupervised attacks. This aims to reduce dependence on labels, thereby enhancing the model’s robustness by acquiring minimal information.
References
- 1. Choi J, Ko T, Choi Y, Byun H, Kim CK. Dynamic graph convolutional networks with attention mechanism for rumor detection on social media. PLOS ONE. 2021. pmid:34407111
- 2.
Li X, Chen L, Wu D. Adversary for Social Good: Leveraging Attribute-Obfuscating Attack to Protect User Privacy on Social Networks. In: Security and Privacy in Communication Networks: 18th EAI International Conference, SecureComm 2022, Virtual Event, October 2022, Proceedings. Springer; 2023. p. 710–728.
- 3. Dawn S, Das M, Bandyopadhyay S. SoURA: a user-reliability-aware social recommendation system based on graph neural network. Neural Comput Appl. 2023;35(25):18533–18551.
- 4.
Wang Y, Song Y, Li S, Cheng C, Ju W, Zhang M, et al. DisenCite: Graph-Based Disentangled Representation Learning for Context-Specific Citation Generation. In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22—March 1, 2022. AAAI Press; 2022. p. 11449–11458. Available from: https://doi.org/10.1609/aaai.v36i10.21397.
- 5. Furno A, Faouzi NEE, Sharma R, Zimeo E. Graph-based ahead monitoring of vulnerabilities in large dynamic transportation networks. PLOS ONE. 2021. pmid:33760878
- 6. Jiang W, Luo J. Graph neural network for traffic forecasting: A survey. Expert Syst Appl. 2022;207:117921.
- 7. Xie Y, Liang Y, Gong M, Qin AK, Ong Y, He T. Semisupervised Graph Neural Networks for Graph Classification. IEEE Trans Cybern. 2023;53(10):6222–6235. pmid:35476555
- 8. Wang H, Wu J, Zhu X, Chen Y, Zhang C. Time-Variant Graph Classification. IEEE Trans Syst Man Cybern Syst. 2020;50(8):2883–2896.
- 9. Li B, Pi D. Learning deep neural networks for node classification. Expert Syst Appl. 2019;137:324–334.
- 10. Jin W, Jung J, Kang U. Supervised and extended restart in random walks for ranking and link prediction in networks. PLOS ONE. 2019. pmid:30893375
- 11. Duan L, Ma S, Aggarwal CC, Ma T, Huai J. An Ensemble Approach to Link Prediction. IEEE Trans Knowl Data Eng. 2017;29(11):2402–2416.
- 12. Yang J, Xu H, Mirzoyan S, Chen T, Liu Z, Ju W, et al. Poisoning scientific knowledge using large language models. bioRxiv—Bioinformatics. 2023.
- 13. Xu H, Ma Y, Liu HC, Deb D, Liu H, Tang JL, et al. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing. 2020;17:151–178.
- 14. Kotyan S, Matsuki M, Vargas DV. Transferability of features for neural networks links to adversarial attacks and defences. PLOS ONE. 2022. pmid:35476838
- 15.
Zügner D, Akbarnejad A, Günnemann S. Adversarial Attacks on Neural Networks for Graph Data. arXiv—CS—Machine Learning. 2018. doi:arxiv-1805.07984.
- 16.
Zügner D, Günnemann S. Adversarial Attacks on Graph Neural Networks via Meta Learning. arXiv—CS—Cryptography and Security. 2019. doi:arxiv-1902.08412.
- 17.
Xu K, Chen H, Liu S, Chen PY, Weng TW, Hong M, et al. Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective. arXiv—CS—Social and Information Networks. 2019. doi:arxiv-1906.04214.
- 18.
Liu Z, Luo Y, Wu L, Li S, Liu Z, Li SZ. Are Gradients on Graph Structure Reliable in Gray-box Attacks? arXiv—CS—Machine Learning. 2022. doi:arxiv-2208.05514.
- 19. Lin X, Zhou C, Wu J, Yang H, Wang H, Cao Y, et al. Exploratory Adversarial Attacks on Graph Neural Networks for Semi-Supervised Node Classification. Pattern Recognition. 2022.
- 20.
Zügner D, Akbarnejad A, Günnemann S. Adversarial Attacks on Graph Neural Networks. In: David K, Geihs K, Lange M, Stumme G, editors. 49. Jahrestagung der Gesellschaft für Informatik, 50 Jahre Gesellschaft für Informatik—Informatik für Gesellschaft, INFORMATIK 2019, Kassel, Germany, September 23-26, 2019. vol. P-294 of LNI. GI; 2019. p. 251–252. Available from: https://doi.org/10.18420/inf2019_29.
- 21. Sun L, Dou Y, Yang CJ, Zhang K, Wang J, Yu PS, et al. Adversarial Attack and Defense on Graph Data: A Survey. IEEE Trans Knowl Data Eng. 2023;35(8):7693–7711.
- 22.
Ju W, Yang J, Qu M, Song W, Shen J, Zhang M. KGNN: Harnessing Kernel-based Networks for Semi-supervised Graph Classification. In: Candan KS, Liu H, Akoglu L, Dong XL, Tang J, editors. WSDM’22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21–25, 2022. ACM; 2022. p. 421–429. Available from: https://doi.org/10.1145/3488560.3498429.
- 23.
Song Y, Ju W, Tian Z, Liu L, Zhang M, Xie Z. Building Conversational Diagnosis Systems for Fine-Grained Diseases Using Few Annotated Data. In: Tanveer M, Agarwal S, Ozawa S, Ekbal A, Jatowt A, editors. Neural Information Processing—29th International Conference, ICONIP 2022, Virtual Event, November 22-26, 2022, Proceedings, Part III. vol. 13625 of Lecture Notes in Computer Science. Springer; 2022. p. 591–603. Available from: https://doi.org/10.1007/978-3-031-30111-7_50.
- 24.
Ju W, Yi S, Wang Y, Long Q, Luo J, Xiao Z, et al. A Survey of Data-Efficient Graph Learning. CoRR. 2024;abs/2402.00447. https://doi.org/10.48550/arXiv.2402.00447
- 25.
Qin Y, Wang Y, Sun F, Ju W, Hou X, Wang Z, et al. DisenPOI: Disentangling Sequential and Geographical Influence for Point-of-Interest Recommendation. In: Chua T, Lauw HW, Si L, Terzi E, Tsaparas P, editors. Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM 2023, Singapore, 27 February 2023–3 March 2023. ACM; 2023. p. 508–516. Available from: https://doi.org/10.1145/3539597.3570408.
- 26.
Wang J, Luo M, Li J, Liu Z, Zhou J, Zheng Q. Robust Unsupervised Graph Representation Learning via Mutual Information Maximization. CoRR. 2022;abs/2201.08557.
- 27.
Bojchevski A, Günnemann S. Adversarial Attacks on Node Embeddings via Graph Poisoning. arXiv—CS—Social and Information Networks. 2018. doi:arxiv-1809.01093.
- 28.
Perozzi B, Al-Rfou R, Skiena S. Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining; 2014. p. 701–710.
- 29.
Zhang S, Chen H, Sun X, Li Y, Xu G. Unsupervised graph poisoning attack via contrastive loss back-propagation. In: Proceedings of the ACM Web Conference 2022; 2022. p. 1322–1330.
- 30. Zhai Z, Li P, Feng S. State of the art on adversarial attacks and defenses in graphs. Neural Comput Appl. 2023;35(26):18851–18872.
- 31. Hsieh I, Li C. NetFense: Adversarial Defenses Against Privacy Attacks on Neural Networks for Graph Data. IEEE Trans Knowl Data Eng. 2023;35(1):796–809.
- 32.
Chang H, Rong Y, Xu T, Huang W, Zhang H, Cui P, et al. A restricted black-box adversarial framework towards attacking graph embedding models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34; 2020. p. 3389–3396.
- 33.
Wang B, Li Y, Zhou P. Bandits for Structure Perturbation-based Black-box Attacks to Graph Neural Networks with Theoretical Guarantees. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2022. p. 13379–13387.
- 34.
Liu Z, Luo Y, Zang Z, Li SZ. Surrogate Representation Learning with Isometric Mapping for Gray-box Graph Adversarial Attacks. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining; 2022. p. 591–598.
- 35.
Lin L, Blaser E, Wang H. Graph structural attack by perturbing spectral distance. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2022. p. 989–998.
- 36.
Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD. Deep Graph Infomax. arXiv—CS—Social and Information Networks. 2018. doi:arxiv-1809.10341.
- 37.
You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y. Graph Contrastive Learning with Augmentations. arXiv—CS—Machine Learning. 2020. doi:arxiv-2010.13902.
- 38. Miao R, Yang Y, Ma Y, Juan X, Xue H, Tang J, et al. Negative samples selecting strategy for graph contrastive learning. Information Sciences. 2022;613:667–681.
- 39.
Kipf TN, Welling M. Semi-Supervised Classification with Graph Convolutional Networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings; 2017. Available from: https://openreview.net/forum?id=SJU4ayYgl.
- 40.
Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K. Simplifying graph convolutional networks. In: International conference on machine learning. PMLR; 2019. p. 6861–6871.
- 41.
Klicpera J, Bojchevski A, Günnemann S. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019; 2019. Available from: https://openreview.net/forum?id=H1gL-2A9Ym.
- 42.
Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L. Graph Contrastive Learning with Adaptive Augmentation. arXiv—CS—Machine Learning. 2020. doi:arxiv-2010.14945.
- 43.
Ma J, Deng J, Mei Q. Adversarial Attack on Graph Neural Networks as An Influence Maximization Problem. In: Candan KS, Liu H, Akoglu L, Dong XL, Tang J, editors. WSDM’22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21–25, 2022. ACM; 2022. p. 675–685. Available from: https://doi.org/10.1145/3488560.3498497.
- 44. Waniek M, Michalak TP, Wooldridge MJ, Rahwan T. Hiding individuals and communities in a social network. Nature Human Behaviour. 2018.
- 45.
Yang Z, Cohen WW, Salakhutdinov R. Revisiting Semi-Supervised Learning with Graph Embeddings. In: Balcan M, Weinberger KQ, editors. Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016. vol. 48 of JMLR Workshop and Conference Proceedings. JMLR.org; 2016. p. 40–48. Available from: http://proceedings.mlr.press/v48/yanga16.html.