2.2.1 DeepWalk.

To capture the network topology information, [8] proposes a DeepWalk approach, which learns features that describe the graph structure. The optimization techniques, originally designed for language modeling, are used for learning social representations of a graph’s vertices. DeepWalk takes an graph as input and produces its latent structure representation as an output. The learned structural features are used in many applications such as network classification and anomaly detection with outperform results.

DeepWalk algorithm, borrowing the concept in language modeling, considers a set of short truncated random walks as its own corpus, and the graph vertices as its own vocabulary. There are two main components, a random walk generator and an update procedure. Given graph G as input, the random walk generator samples uniformly a random vertex v as the root of the random walk W_v. For each vertex v, being the start of γ random walks, |W_v| vertices are selected in order randomly. Aiming at updating the vector representation of v, Skip-Gram [25] is utilzed to maximize the probability of its neighbors in the walk W_v. Inspired by DeepWalk, we propose a DirectedWalk algorithm to learn network vectors of accounts in the transaction network.

2.2.2 Learning the network vector.

We generate a corpus and a vocabulary from the directed and dynamic network, which is the only required input for learning the network vectors. Give G = (V, E, W, T), the vertex set V is considered as its own vocabulary, and the directed sequential transaction paths are seen as its own corpus. It is well known that, in DeepWalk, the relationship strength of two vertices is determined by the frequency they occur in an adjacent position in random walks. Here, as defined in Definition 2, the strength of the relationship between any two vertices v_i and v_j is determined by weight w_ij of e_ij and w_ji of e_ji. Moreover, in G, the order of vertices that passed by a transaction path are not random but time-related. Therefore, we propose a DirectedWalk algorithm for learning the network vector of the transaction vertices. Each vertex can be seen as the start vertex of a directed walk with a maximum length of l. For the last visited vertex v, the walk will pass over all of its directed neighbors, only if v has trading with them within a time interval τ. The walk grows up iteratively until not satisfies the constraints.

Algorithm 1 DirectedWalk(G, l, τ, w, d)

Require:

 network G = (V, E, W, T)

 maximum walk length l, timestamp interval τ

 window size w, embedding size d

Ensure:

 matrix of vertex rpresentations Φ ∈ R^|V|×d

1: the directed corpus NC is initialized to be empty;

2: for v_i ∈ V do

3:  p_i = [v_i], p_i ⊆ P_i

4:  while ∃p_i.state = True do

5:   NodeSentence p = P_i.pop() (pop a p in its active state)

6:   v = p.lastvertex, t = v.timestamp

7:   retrieve v’s directed neighbor vertices ((v₁, t₁), …, (v_m, t_m)) from G

8:   for j ∈ m do

9:    if (∃t_k ∈ t_j, t < t_k < t + τ) or (len(t_j) = 1 and ) then

10:     p adds a new vertex (v_j, t_j)

11:     if v_j has no directed neighbor vertex or len(p) = l then

12:      p.state = False

13:     end if

14:     P_i = P_i.push(p) (push p into P_i)

15:    end if

16:   end for

17:  end while

18: end for

19: NC = NC ⋃ P_i

20: for c ∈ NC do

21:  SkipGram(Φ, c, w)

22: end for

Line 2-18 in Algorithm 1 shows the core of our algorithm. The outer loop specifies the paths that start with each vertex and generates a time series ordering of the directed neighbor vertices. The state factor of p_i is initialized to True and will be reset to False, in condition that its last vertex has no directed neighbor vertex or the length attains l. On condition that all of the paths reach their False states, the generation procedure of P_i starting at v_i is completed. As depicted in line 8-16, for the last passed vertex v, the timestamp information is used to determine whether a vertex will be appended to v. In a summary, fixing a start vertex, the longer the weight vectors of the passed directed edges are the more walks will be produced by our DirectedWalk approach. Therefore, the frequent traders are more likely to exist in walks and appear in a window with high probability.

It is shown in line 21, the Skip-Gram model [25] is employed to maximize the co-occurrence probability Φ among the vertices in the time ordered walks. Skip-Gram, using the independence assumption, iterates over all possible collocations in directed corpus NC within the window w.

We finally obtain the optimal network vector representations of the vertices by using the same learning process of DeepWalk. DirectedWalk maps the directed and dynamic transaction network into a d-dimensional vector space. The vertices that contain similar local directed topological structures are mapped into adjacent vectors.

2.2.3 Constructing the local topological feature matrix.

For ∀v ∈ V, given in G, the local topological structure of vertex v is the subgraph G′ of vertex set . In Sect. 2.2.2, the structure of v in G′ is embeded into a d-dimensional network vector v^t. Having calculated the Euclidean distance between v^t and , we store the top k nearest in ascending order. This yields the local topological feature matrix T_v as follows, (1) where, (1 ≤ i ≤ k) is the network vector of , while denotes the network vector of v itself.

3.2.1 The first route in the TRHD-CNN architecture.

The first route (i.e., Route 1 in Fig 6) extracts the local topological structure features of an involving account from its input matrix T_v. As described in Sect. 2.2.3, the row vector of T_v, is the minimum unit in describing its local topological structure. Thus, the convolution structure of this route is similar to that of the MHD-CNN model, with the following process steps.

The first route of the TRHD-CNN model fed with T_v employs the same architecture as the MHD-CNN model, except from containing the softmax output layer (seen in Fig 5). Then, vector , part of the output of the convolutional component of TRHD-CNN, will be used in the subsequent classification procedure. The structure of Process 1 is expalined as following:

1. Input layer. For ∀v ∈ V, the input data is the matrix T_v.
2. Convolutional layer. The input matrix T_v is used as the feature map in the first layer. The convolution kernels are same with those used in MHD-CNN. For the lth convolutional layer, the processing function on each feature map is shown in formula 5, (5) where, f(•) represents an activation function, the operator “*” expresses the convolution operation, X^l−1 denotes an input feature map, denotes the ith convolution kernel as mentioned in Sect. 3.1, and , means the bias vector. Therefore, the kind of output feature maps is ∑_i K_i times that of the input.
3. Pooling layer. The processing, on the feature map of the lth pooling layer, is shown in formula 6, (6) where, operator “⋅” represents a pooling function, which is used for downsizing the ith input matrix . The notations and denote the weight matrix and bias vector respectively. The number of feature maps is not changed by the pooling procedures.
4. Connection layer. In this layer, the input feature maps, each in size of 1 × 1 and is obtained from the last pooling layer, are concatenated into a one-dimensional vector .
5. Fully connected layer. This layer compressed the vector into a shorter vector X^re in length of K₁ by using a fully connected structure.

3.2.2 The second route in the TRHD-CNN architecture.

The second route (i.e., Route 2 in Fig 6) is primarily used to extract discriminative classification features from the time series feature matrix of a vertex v, denoting as W_v. The difference between W_v and T_v is that the minimum feature representation unit is a row vector for matrix T_v, while for matrix W_v each data element does. Therefore, a new kind of convolution structure, different from the first route, is needed for extracting more useful information. The second route of TRHD-CNN is designed as following, shown in Fig 7.

1. Input layer. For any vertex v, the input data is its time series feature matrix W_v. The length of the row vector of W_v is set as the longest transaction sequence in network G. W_v is a sparse matrix, for the reason that the trading frequencies of accouts are quit different.
2. Convolutional layer and pooling layer. As shown in Fig 7, the structure, two convolutional layers with each followed by a pooling layer, is built referring to the following details. Moreover, the trading behaviors of different accounts are independent, which are hidden in their time series transaction amounts. Therefore, the convolution kernels of the first convolutional layer, c₁, is created in form of 1 × n₁, n₁ ∈ N₁. Taking the matrix W_v as input, this first layer generates |N₁| × k₁ feature maps by using k₁ convolution kernels on each input feature map. In order to extract the trading characteristics of a single vertex, the pooling window is created in 1 × p₁ format. That means a pooling window does not slide over adjacent vertices. Similar to the first one, the second convolutional layer is also built in the multi-kernel structure. c₂ kinds of convolution kernels are utilized, in which each in 1 × n₂, n₂ ∈ N₂ format. Aim to describe the local region features of a feature map, we employ pooling windows in shape of m × n on the second pooling layer. Here, m and n equal the row and column values of the corresponding input feature map respectively. The processing of the convolutional layer and the pooling layer correspond to formula 5 and formula 6 in Sec. 3.1, respectively.
3. Fully connected layer. There are two fully connected layers in this route. The first one connects the |N₁| × k₁ × |N₂| × k₂ feature maps, generated by Sec: Conv. + Pool., into a one-dimensional vector . Subsequently, another fully connected layer is constructed to compress vector and balance the effects of each route. is reduced into of length K₂, and then is used as a part of the input for the succeeding classification process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. The second route structure in the TRHD-CNN model.

Input. is the input layer. Fir: Conv. + Pool. is the first convolutional layer and its following pooling layer, which contains c₁ kinds of convolution kernels and generates c₁ × k₁ output feature maps. The window size of this pooling layer, win, is 1 × p₁. In the Sec: Conv. + Pool. part, each kind of input feature map, corresponding to a convolution kernel of c₁, generates c₂ × k₂ output feature maps by using c₂ kinds of convolution kernels. And, the pooling window, in size of m × n, down-samples each input feature map into an element. Cona. denotes the concatenation layer that splice the feature maps into a column vector. FullyC. represents the fully connected layer. The last FullyC. is utilized to compress the output into a lower dimensional vector.

https://doi.org/10.1371/journal.pone.0220631.g007

3.2.3 Concatenating the two routes of TRHD-CNN.

This part describes the classification component of TRHD-CNN, including three portions: concatenation portion, compression portion and classification portion.

1. The concatenation portion. The vectors and , come from the first route and the second route, are concatenated into vecto .
2. The compression portion. A fully connected layer is used to compress X_mer into .
3. The classification portion. The portion acts as the output layer of TRHD-CNN, outputs the category of a vertices v with .

4.3.1 Parameters for learning network vector.

The length d of vector appeared in Sect. 2.2, is set as 200, for the reason that the classification effect is not improved significantly with d increasing from 200 to 500. The top 20 nearest neighbors in of vertex v is selected to make up the matrix T_v. Here, h equals 3.

4.3.2 Common parameters of MHD-CNN and TRHD-CNN.

Experimental results show that the two CNN models converge under 100 epochs, therefore we set the training termination condition to a fixed epoch value of 100. In our experiments, during each iteration, the learning rate is dynamically adjusted to accelerate the convergence, i.e. the learning rate is selected from the set {0.01, 0.05, 0.09, 0.13, 0.17, 0.21, 0.25} in descending order according to the training errors. In these two models, regularization term L₂ and dropout rate are introduced to avoid over-fitting. These two parameters are assigned referring to the default values used in the relevant study [28], i.e. the correlation coefficient of L₂ and dropout rate are set to 10e-4 and 0.5, respectively.

4.3.3 Parameters of MHD-CNN.

The maximum length of the directed edge weight vectors (i.e. d_h) is derived from our transaction network based on real dataset. The maximum length value equals to 1359. To achieve optimal classification performance, the MHD-CNN model adopts the ReLU activation function in convolutional layers and max pooling function in pooling layers.

4.3.4 Parameters of TRHD-CNN.

In this part, we list the parameters of each component of TRHD-CNN in turn.

A. Parameters of the convolutional component: In the first-route of the TRHD-CNN model, the values of all parameters and the selected activation functions are consistent with those of the MHD-CNN model. In the second-route, the used parameter settings are listed in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Values of the second route parameters.

https://doi.org/10.1371/journal.pone.0220631.t003

In its first convolutional layer, the number of convolution kernels c₁ and the number k₁ of each kind of kernel are set as 4 and 100, respectively. And then the shapes of c₁ convolution kernels are set as {1 × n₁, n₁ = 2, 3, 4, 5}. The width of pooling window of the first pooling layer p₁ is set as 3. Similarly, in the second convolutional layer, c₂ and k₂ are set to 3 and 50, respectively, and these c₂ convolution kernels shaped like {1 × n₂, n₂ = 2, 3, 4}. The following pooling layer adopts a pooling window in shape of m × n, where m and n are the number of rows and columns of the corresponding input feature map, respectively.

B. Parameters of the classification component: The experimental results show that the concatenated vector, when K₁ and K₂ are set as 100 and 50 respectively, achieves the best results in classifying bank accounts.

4.4.1 Evaluating different architecture parameters of TRHD-CNN.

We test the influence of the value of parameters K₁ and K₂, and the number of convolutional layers on the classification effects of TRHD-CNN, respectively.

(1) Tuning the values of K₁ and K₂. In this section, we tune the parameters K₁ and K₂ to receive the best classification results for TRHD-CNN. The details are shown in Fig 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. The ROC curves of TRHD-CNN with different output vector lengths of the two routes.

https://doi.org/10.1371/journal.pone.0220631.g008

In the legend, the first-route means the parameter K₁, and the second-route corresponds to K₂. The value of AUC means the area under the curve. As well known, the greater the AUC value is, the better the classification performance of the model will be. As shown in Fig 8, TRHD-CNN model achieves the best results with K₁ = 100 and K₂ = 50. Therefore, we set K₁ to 100 and K₂ to 50 in TRHD-CNN. The result indicates that the impact of the typological network data is greater than that of the time series data on classification effects, which further illustrates the advantage of our improved two-route framework.

(2) Adding extra convolutional layers. It is found that the TRHD-CNN model receives the best results when the first-route contains one convolutional layer followed by a pooling layer, meanwhile the second-route contains two convolutional layers and each layer is connected to a pooling layer. The reason why no significant improvement in classification performance is made when increases the convolutional layers, may be that the features is already well described by TRHD-CNN model by shallow network structure.

4.4.2 Comparing the classification performances of different models.

In this section, three groups of experiments are carried out on the three models, the LOF method [29], the MHD-CNN model and the TRHD-CNN model, for testing their performance in classifying bank accounts.

Note that we test several traditional abnormal account detection methods based on statistical features in our prior research [29]. [29] focuses on detecting bank accounts by using traditional classifiers with novel devised three kinds of behavioral features. In [29], the classification problem on an imbalance dataset is seen as an abnormal detection problem. The utilized classification features, being extracted from their financial time series transaction records, are grouped into three categories: transaction statistical features, network behavioral features and periodic behavioral features. A comparative analysis of several classical classifiers, one-class SVM, isolation forest (IForest), local outlier factor (LOF) and robust covariance (RC), is made in [29]. Given a predefined outlier fraction, the experiment results show that IForest achieves the best F1-score, i.e. 58%. It is obviously that the traditional abnormal account detection methods are uncompetitive with the following ones.

Therefore, in the following paper, the performance of IForest method is regarded as a benchmark, and the other two models are tested with different parameter values as mentioned in Sect. 4.3. In particular, for IForest method, three values of a necessary threshold parameter, i.e. 0.01, 0.08, 0.1, meaning the outlier factor are selected in our experiments. The optimum outlier factor, which achieves the best result on the training set is selected for IForest. The classification results of the three models on the dataset are shown in Table 4, where the optimal results of MHD-CNN and TRHD-CNN are exhibited.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. The classification results of the three models.

The numbers in parenthesses refer to the corresponding average results on training set.

https://doi.org/10.1371/journal.pone.0220631.t004

As shown in Table 4, both CNN models achieve better results than the traditional statistically methods (i.e. IForest), proving the superiority of deep learning methods in mining classification features. The reason may be the statistical features are manual extracted, which are static during the classification process. However, the classification procedure of CNN model contains a feedback process, which modifies the adopted classification features dynamically. Moreover, the MBGD algorithm, a back-propagation method used for update the parameters, ensures the CNN models converges to the minimum loss state. Therefore, CNN always chooses the most effective features according to specific classification tasks, and learning the optimal classification parameters, automatically. The improvement of CNN is not intuitively obvious, referring to the generally high accuracies of the three methods. It should be noted that the recall is a more important evaluating indicator in a fraud identification scenario. Moreover, the high accuracy is caused by the imbalance between the number of illegal samples and normal samples. Table 5 gives details of the methods’ classification results on test set, where the indicators true positive, false negative, false positive, true negative are respectively denoted as TP, FP, FN, TN. Each time, one fifth dataset is randomly selected as test set, containing 175 MLM accounts and 1854 normal accounts. As presented in Table 5, 144 MLM accounts are identified by TRHD-CNN, in contrast with the amount 100 IForest recognized. And yet, TRHD-CNN makes significant improvement over MHD-CNN in TP.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. The confusion matrix of the three methods.

https://doi.org/10.1371/journal.pone.0220631.t005

On the whole, the TRHD-CNN is higher than the MHD-CNN in precision, recall and F1-score, indicating different types of convolution kernels have advantage of extracting more useful features.

The results in Table 4 show that TRHD-CNN better than the other methods on accuracy and recall obviously. To further investigate the superiority of TRHD-CNN, we perform the Paired Samples T Test (SPSS) for statistical significance. Four times five-fold validation was performed on the data, and the accuracy results on the testing set are used for SPSS. Here, IForest and MHD-CNN are compared with TRHD-CNN respectively. The results are shown in Table 6. According the definition of Probability Value (p-value), the smaller the p-value, the more significant the difference is. Moreover, we can draw a conclusion that the difference is statistically highly significant when p-value is smaller than 0.01. Based on the p-values in Table 6, we reject the null hypothesis, indicating a definite improvement provided by TRHD-CNN.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Results of SPSS on accuracy.

https://doi.org/10.1371/journal.pone.0220631.t006

In the process of testing the proposed CNN models, we adopt the method of dynamically selecting learning steps to accelerate the convergence of the learning model. Fig 9 shows the changes of training and testing errors with increasing iteration times for CNN models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Training and testing errors of TRHD-CNN and MHD-CNN with various iteration times.

https://doi.org/10.1371/journal.pone.0220631.g009

The x-axis and the y-axis represent the number of iteration times and the cost values of the objective function, respectively. As shown in Fig 9, along with the iteration number increases, all curves diminish sharply at the beginning, then decline in decreased speeds, and finally converge to stable values. Obviously, the training errors and testing errors of the TRHD-CNN model are lower than those of the MHD-CNN model in general, and the reason is that the TRHD-CNN model is helpful in extracting more useful classification features. In addition, using the strategy of dynamic selection of step size, the fluctuations of the cost curves almost disappear after a certain number of iterations (beginning at the 80th iteration). Hence, reducing the step size after a specific iteration number is helpful for the objective function to converge to an optimal value correctly and rapidly.

In summary, the experiment results indicate that the existing methods cannot take advantage of the information, embedded in the time series transaction data, efficiently. While, the TRHD-CNN model, which employs a two-route convolution structure for extracting the specific classification features, gains a significant advantage over other algorithms. Its superiority in the recall indicator proves the complementary of local topological structure features and time series transaction features in describing the category of an account. And yet, to say that TRHD-CNN is time consuming in learning a new coming account’s classification features.