Low-rank matrix embedding.

Many existing approaches for embedding graphs attempt to fit the full graph representation into a latent space of a small, pre-defined number of dimensions dim. Their motivation is that a low dimensional projection leads to improved generalization and that naturally continuous features as produced by spectral (e.g., Modularity Maximization as in [14], Laplacian Eigenmaps as in [16], RWModMax [17]) or neural representation learning (e.g., DeepWalk [18], LINE [19]) approaches better capture the affiliation degree of a vertex to latent social dimensions. However, this may lead to only a “rough” representation that would lack a lot of structural information given that the dimensionality of the feature matrix is selected a-priori. The alternative is to use expensive cross-validation and fine-tune the dimensionality to a specific labeling. Furthermore, spectral methods calculate a number of eigenvectors equal to the dimensionality dim, something that can be very costly. It must also be noted that when the graph size is very large, the space required to store the features (O(dim ⋅ |V|)) becomes a non-trivial problem [20].

We run Laplacian Eigenmaps [21] on our toy graph, shown in section A of Table 1. We show the first three non-trivial eigenvectors (dim_1–3) of the graph Laplacian matrix, which are sufficient to capture the community structure of this simple graph. We note that vertices 9 and 10 are embedded at the same point in X due to structural equivalence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Examples of low-rank and community-based representations.

https://doi.org/10.1371/journal.pone.0173347.t001

Community-based embedding.

Attempting to address some of the limitations of low-rank embedding methods, a limited number of approaches using community detection [22] have been tried for user embedding. These include the overlapping EdgeCluster method [20] and the hierarchical Multi-Resolution Overlapping Communities (MROC) method [23]. We first explain the means by which the community-based embedding is made and then discuss the theoretical limitations of typical community detection paradigms used for its formation. Suppose we execute any community detection method. The number of communities detected is |C|, where C is the full set of communities. The number of vertices in a community c is |c|. Graph vertices are then represented by the community indicator matrix X, defined in Eq 1.

(1)

We call this operation community-based embedding, denoted by X = embed(C). Another way to view such an embedding is a binary representation such as those usually found in text classification where the communities are terms, the vertices are documents and the non-zero values of X denote a document containing a term. By aggregating a number of vertices into a community, one implicitly encodes a kind of similarity between the vertices of the same community, even though they may not be directly linked. The community-based embedding examples in sections B-D of Table 1 correspond to three main types of community: disjoint, overlapping and hierarchical.

Disjoint community detection: Given Eq 1, each vertex embedding will have exactly one non-zero element in the dimension corresponding to the community it belongs. More importantly, all the vertices belonging to the same community are collocated at the same point on X. For example, suppose that a hypothetical disjoint method correctly groups together vertices 0–3 as community A_d, vertices 4–7 as B_d and vertices 8–11 as C_d (see section B of Table 1). Now suppose that vertex 12 gets allocated to community C_d. That would mean it would be on the same latent point as all vertices from label C, whereas it is highly likely that this vertex is also associated with label B.

Overlapping community detection: This type of community leads to higher distinctiveness in vertex embeddings on X by possibly associating a vertex with multiple latent dimensions. Suppose now that a hypothetical overlapping method correctly groups together vertices 0–3 as community A_o, vertices 4–7 as B_o and vertices 8–11 as C_o (see section C of Table 1). It might be possible for such a method to have vertex 12 also be assigned to B_o and C_o.

Hierarchical community detection: In such cases, vertices are embedded in X using their assignment to communities at different resolutions. If we consider vertex 4 to be a boundary vertex associated both with label A and label B, then a hypothetical hierarchical scheme that considers a hyper-community comprising vertices both of label A and B (AB—see section D of Table 1) after merging communities A and B, will capture the multiple affiliation successfully. Still, all the other vertices from label B will also share a common feature with all the label A ones, something that may be undesired in case there is no true label hierarchy.

Vertex-centric community embedding.

We posit that since each vertex has its own local view of the graph, considering vertex-centric communities results in a vertex representation that is much more granular and detailed compared to the previously presented community representations. We now describe two types of vertex-centric community, the first capturing the local connectivity structure in high resolution, and the second additionally capturing missing links between vertices.

Base vertex-centric communities: We define the very high resolution base community [23]b_v of a vertex v as the set of adjacent vertices plus the ego vertex, b_v = N(v) ∪ v. Their value as high resolution features that encode user preferences can be understood by considering vertex 13. Many techniques that aim for a mesoscopic community representation of a graph might overlook the possible triple affiliation implied for vertex 13 and there is no guarantee that they will capture it. Furthermore, hierarchical community detection might not be of use in this case, as the affiliation of this vertex to a hyper-community containing numerous (or all) vertices is not very informative. In contrast, the base community of a vertex implies similarity among its member vertices, as it captures vertex-centric preferences given that the connections were made voluntarily (see homophily). The same applies to vertices 12 and 4. Consider the feature matrix X_base = embed(B), where B is the set of all base communities. Using base communities for vertex embeddings results in points in the high-dimensional space ℜ^|V|, which are by definition proximal to the ones of adjacent vertices. However, in sparsely annotated graphs many vertices and especially the ones with small degree, will not have annotated neighbors and thus will be difficult to classify using just base communities.

Extended vertex-centric communities: In order to capture missing links for each vertex in the graph, we propose the inclusion of an additional type of community that is based on local searches around each vertex. We consider extended vertex-centric communities e_v that contain the neighbors of a vertex plus any not directly connected, yet similar vertices. To this end, we will use a type of random walk based similarity called regularized commute-times, to be described in sub-sub-section Fast similarity vector Calculation. See, for example, in section B of Table 2 that the local community around vertex 13 (e₁₃) contains vertex 15, in addition to its neighbors. Consider further that our method (see sub-section Unsupervised community-based embedding), with an appropriate parameter selection, will identify two user-centric communities after searching around vertex 11. The base community (b₁₁), comprised of its neighbors, and another one (e₁₁) that also incorporates vertex 12. This means that vertex 12 is characterized through its association with four vertices (including itself): b₅, b₈, b₁₂ and e₁₁. Association with e₁₁ reinforces the assignment of label C to vertex b₁₂.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Vertex-centric communities lead to more personalized embeddings.

https://doi.org/10.1371/journal.pone.0173347.t002

Relational classification.

The first related line of work refers to approaches in which the data do not inhabit a Euclidean space, but are described through their relations. We note that these methods are completely label-dependent. Our own approach is different in that the community-based social relation representation is done in a completely unsupervised way and may be reused in multiple experiments.

Collective classification: For classification on graphs, collective classification has been used extensively [13]. Such methods, however, have been outperformed by approaches using low-rank matrix representation. Specifically, LapEig used in a social network context [16] outperformed Link Based Classification (LBC) with relaxation labeling for collective classification [24] and the weighted vote relational neighbor classifier (wvRN) [25] with iterative classification [24, 26] on the ASU-Flickr dataset. ARCTE achieves even higher accuracy than LapEig on that dataset, so we elected not to include collective classification approaches in our study in order to leave room for more recent and competitive methods.

Random walk similarity matrix methods: Semi-supervised random walk methods for estimating vertex-to-vertex similarities also take into account the number of paths between vertices instead of the less distinctive measure of geodesic distance. Various graph-kernels or similarity matrices have been developed [12, 27]. However, they require the inversion of a matrix (O(n³)) as large as the input graph. Certain approaches [28] exploit advances towards near-linear system solutions [29] for diagonally dominant matrices or the small-world property of real-world graphs [30] to expedite the similarity matrix calculation, though they still require space quadratic to the number of samples. Finally, we note that the semi-supervised label diffusion method [28] has been outperformed by the low-rank matrix embedding approach—specifically by the RWModMax [17] method we included in our comparisons.

Local community detection.

When a graph becomes too large, a sensible hypothesis is that one does not need access to the full graph structure in order to extract a community near a specific vertex. This concept has sparked the interest for local community detection methods, such as the random walk based Nibble algorithm [31] and its improvement PageRank-Nibble [32] that were used in constructing near-linear time spectral graph sparsifiers [33], solvers of symmetric, diagonally dominant systems [29] and conventional community detection [34]. The method by [32] relies on the calculation of a personalized, vertex-centric array of similarities for a seed vertex and a conductance sweep [31] of the sorted values to search for a good conductance cut.

We consider user-centric community detection as a step in our social relation latent representation algorithm that is distanced from the aforementioned approaches by the fact that we a) perform it in an unsupervised way for all users in a graph and it does not need to be seeded with a set of labeled users [34, 35], b) we use a novel heuristic for thresholding the similarity vector and capturing missing links instead of using costly sweeps, and c) we make computational improvements on the calculation of similarity vectors.

User profiling.

Whereas we focus on topic-oriented user classification, other efforts attempt to use graph-based techniques to reveal other aspects of users’ profiles. For example, a study involved the identification of authoritative users in Twitter based on text content and links/interactions [6]. According to it, the latter type is more informative although feature fusion yielded the best results. Furthermore, graph-regularized non-negative matrix factorization of text features was successfully used to identify spam accounts in Twitter [36]. A PageRank variation was proposed by [37] to combat spam account link farms. Also noteworthy are methods that try to extract role similarity [38] and types [39] from structural complexity in user graphs.

Finally, it should be noted that some recent works have pointed to some limitations of graph-based approaches. In a recent study [40] a number of graph-theoretic features such as various centrality measures, counts and ratios of followers, friends, replies, etc. did not offer significant discrimination. Another study [41] leveraged user connectivity in the second of a two-step classification process. First, users were classified using text- and behavior-based features and then a neighbor voting scheme was applied for updating the labels, something that did not lead to improved accuracy. Yet, both of the aforementioned approaches did not fully leverage the potential of graph-based representations, since the first did not utilize user connectivity for calculating similarities, while the latter relied on neighbor voting which has been found to be inferior to low-rank graph representations [14].

Fast similarity vector calculation.

Similarity vectors for all seed users must be calculated and then sorted. To this end, working only on the non-zero elements is necessary for a scalable solution. Avoiding to propagate trivial values by assuming that similarities with distant vertices are almost zero leads to faster methods and storing only non-zero values results in sparse similarity vectors. In this section, we propose an improved variation of fast, sparse, vertex-centric similarity vector calculation [32] by deriving a fast algorithm that approximates cumulative PageRank differences, which results in the discovery of more relevant vertices with fewer iterations.

Denote by the probability of an agent randomly walking on a graph being on vertex v at time step t and by the row vector the probability distribution for all vertices. The distribution after a random step is , where W = D⁻¹ A is the Markov chain transition probability matrix and D is the diagonal degree matrix. By altering the random walk process to restart at each step to an initial distribution s with a given restart probability ρ ∈ [0, 1], converges to a stationary distribution called the PageRank vector k_pr. When s is anything but the uniform distribution, we call the resulting stationary distribution personalized PageRank [42]. We will consider s = e_v, where e_v is a distribution with all probability concentrated on the seed position v, thus leading to user-centric similarity vectors. The user-centric PageRank is shown in Eq 2.

(2)

We base our approach on the concept of updating a single element u ∈ V per iteration, which has been adopted by different approaches [32, 43, 44]. The method described by [43] is for simultaneous calculation of multiple PageRank vectors by maintaining a heap, the relevant part of the work by [44] describes a method for calculating vertex-centric Katz score vectors also by maintaining a heap, whereas a lazy random walk based PageRank method is introduced by [32] where a queue replaces the heap, leading to lower complexity. A lazy random walk implies that there is a non-zero probability that the agent will perform a self-loop at any given time step t. We show in Eq 3 the single element update for a single PageRank vector by means of a non-lazy random walk.

(3)

where is the residual probability vector at time step t and I_u is a zero matrix with a single unit element at the u-th place of the main diagonal. In order to avoid propagating trivial values, only a significant element u from the residual is used per update; this means that only one transition probability row W_u: = I_u W is needed to update the residual. Eq 4 is the incremental rule for updating the residual.

(4)

One has to initialize the approximate solution as a zero vector and the residual and update alternatively Eqs 3 and 4 until convergence. Since we multiplied r⁽⁰⁾ by ¹/_ρ, we need to also multiply the update in Eq 3 by ρ in order to calculate a probability distribution (i.e. summing to 1), if needed. However, we can avoid that by multiplying the resulting vector by ρ after the final step, T. Following previous approaches [32, 44] we select an element u if it has a value ^r(u)/_d(u) > ε, where ε is a predefined threshold.

The reason why it is preferable to use a non-lazy random walk over a lazy one in user-centric PageRank calculation is described in S2 Appendix.

For our second improvement, consider the vector ck_δpr of cumulative differences k_δpr between the PageRank distribution at step t in Eq 5.

(5)

The incremental rule for calculating ck_δpr is shown in Eq 6. We propose the alternative update of the rules in Eqs 6 and 4 for the calculation of cumulative PageRank differences. We initialize . A derivation and further discussion can be found in S3 Appendix.

(6)

The vector can be calculated as follows: one maintains a queue of all elements in the degree-normalized distribution r with probability exceeding the threshold ε. We perform exactly one iteration for each element (see S2 Appendix), by removing lazy steps since they result in some probability remaining in r. Any element in r that has now surpassed the threshold is appended to the queue. We repeat until , ∀u ∈ V. We formalize the above generalized technique in Alg 4. We denote by ./ the elementwise division operation and by deg the array of vertex degrees. The reason we perform this element-wise division is to get a similarity measure that is more focused on the locality of a target vertex to the seed than its high degree and is explained in S3 Appendix. One can substitute the Alg 5 for the lazy random walk PageRank [32]. In order to utilize our improved versions without a laziness factor one must substitute Algs 6 and 7 for PageRank and cumulative PageRank differences respectively. We also denote by w_vu the transition probability from vertex v to u.

Algorithm 4 Vertex-centric Similarity Vector Calculation

INPUT: W, v_seed, ρ, ε

OUTPUT: k^(t+1), r^(t+1)

1: initialize k⁽⁰⁾ ← 0,

2: initialize , r⁽⁰⁾(v_seed)←1

3: loop[while ]

4:  loop[∀v ∈ V_r]

5:   k^(t+1), r^(t+1) ← update(k^(t), r^(t), v, ρ)            ▷ Algs 5, 6 or 7

6:  end loop

7: end loop

8: k^(t+1) ← k^(t+1)./d           ▷ Calculate regularized commute-times

Algorithm 5 PageRank Lazy Update

INPUT: W, , r^(t), v, ρ, λ

OUTPUT: , r^(t+1)

1: loop[while ]

2:  

3:  loop[∀u ∈ N(v)]

4:   r^(t+1)(u)←r^(t)(u) + (1 − ρ)(1 − λ)r^(t)(v)w_vu

5:  end loop

6:  r^(t+1) ← (1 − ρ)λr^(t)

7: end loop

Algorithm 6 PageRank Limit Update

INPUT: W, , r^(t), v, ρ_eff

OUTPUT: , r^(t+1)

1:

2: loop[∀u ∈ N(v)]

3:  r^(t+1)(u)←r^(t)(u) + (1 − ρ_eff)r^(t)(v)w_vu

4: end loop

5: r^(t+1)(v)←0

Algorithm 7 Cumulative PageRank Differences Update

INPUT: W, , r^(t), v, ρ_eff

OUTPUT: , r^(t+1)

1: loop[∀u ∈ N(v)]

2:  

3:  r^(t+1)(u)←r^(t)(u) + (1 − ρ_eff)r^(t)(v)w_vu

4: end loop

5: r^(t+1)(v)←0

Finally, we perform a local search only for seed vertices with degree above 1. Given that the similarity vector k is an approximation of a degree-normalized random walk with restart, it is impossible for any vertex to have a larger similarity score than thxze unit degree seed or the adjacent. Naturally, we extract only the base community for such seed vertices, although their representation can also be improved with ARCTE. As shown in Fig 3, these one-degree vertices may still participate in the extended user-centric community of some other vertex, as the adjacent vertices may be penalized due to high degree.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Vertex-centric communities on degree-1 vertices.

We do not calculate extended vertex-centric communities for seed vertices of degree equal to 1. However, they may be included in extended communities centered on other vertices.

https://doi.org/10.1371/journal.pone.0173347.g003

Threshold parameter selection.

The threshold parameter ϵ regulates the approximation to the true PageRank vector. No formal methodology has been proposed regarding its selection in vertex-centric methods. In a recent empirical discussion [34] the authors claim that ϵ ∈ [10⁻⁵, 10⁻⁶] offers good local exploration without increasing the computational complexity prohibitively. We frame this rule by further observations of our own. We denote by d_ave(v), d_max(v), d_min(v) the average, maximum and minimum degrees of the neighborhood N(v). For example, we give the definition of the maximum neighborhood degree: d_max(N(v)) = max(d(u)), ∀u ∈ N(v).

1. a). We calculate an effective threshold value ϵ_eff that takes into account the one-hop structure around each seed vertex v_seed. Specifically, we calculate to encourage a more strict approximation in case the degrees of the adjacent vertices are relatively larger than the degree of the seed. This accommodates for the greater complexity in the local graph structure.
2. b). There is an implicit, hard upper bound ϵ_max. Consider the residual distribution r after one iteration. An amount of probability averaging to ^{(1 − ρ)}/_d(vseed) will be on the adjacent vertex positions in r. At least one such value must be higher than the corresponding vertex’s degree normalized threshold ^ϵ_eff/_d(vadj) in order for a second iteration to take place. As such, ϵ_max = ^d_max(v)/_d(vseed).
3. c). We also apply a soft lower bound. Since we are interested in local graph exploration, we do not want to overly spend computational resources by propagating values needlessly further away from each seed vertex. As such, we penalize the ϵ_eff if it is larger than the value that guarantees a probability push on all adjacent to the seed vertices. Specifically, we calculate ϵ_eff = ^{ϵ_eff+d_min(v)}/₂

Parallelization.

We note that each user-centric similarity vector and subsequent community detection is independently calculated. Furthermore, the transition probability matrix W is read-only for the purpose of ARCTE and does not require the acquisition of a lock in order to read from it. Under a shared physical memory model, ARCTE is an embarassingly parallel algorithm, which means that each vertex seed may be mapped to a separate processor (such exploitation of independent processes in graphs for parallelism has been successfully used before, e.g., in the calculation of shortest paths [45]). The reduction of the parallel outputs to a single feature matrix is identical to the serial result. For instance, if the machine used for the experiments had 8 cores we would theoretically expect a speed-up of up to 8x for 8 parallel tasks (threads, processes), which is never observed in practice given the overhead required to prepare and initiate the tasks as well as to reduce the results into a single matrix. Fig 4 depicts the execution time versus number of tasks. Given the fact that multiprocessors are capable of handling an additional small number of asynchronous tasks, we observe that there is no further noticeable degradation of performance as one creates tasks beyond 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Parallelized feature extraction time.

ARCTE execution time versus number of parallel tasks on ASU-Flickr dataset.

https://doi.org/10.1371/journal.pone.0173347.g004

We adopt the following simple parallelization scheme. Let t_num be the number of tasks we want to initiate. We sort the degree array in increasing order. We then map to the task t the vertices corresponding to the degree values found in the sorted degree array positions t + i × t_num where i ∈ [0, ^|V|/_tnum − 1]. This way, vertices are distributed in a degree-wise balanced way among tasks.

SNOW 2014 Graph (SNOW2014G).

We extracted mention and retweet social interactions to form the graph edges from the tweet collection introduced in the SNOW 2014 Data Challenge [49]. The labels we gathered belong to various types of user attribute, as depicted in Fig 5. The procedure for extracting and annotating the graph is described in S1 Appendix. We opted for a connected and undirected graph of users in order to make the method comparisons fair. We required the former quality since disconnected graphs are problematic for the application of spectral methods (see ASU-YouTube) and the latter because not all competing methods are applicable to directed graphs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. User labels in SNOW2014G.

Starting from upper-left word cloud and going clockwise: A: Attributes (e.g., occupation). B: Geographical association. C: Religious or political stance. D: Associated themes. E: News story relevance.

https://doi.org/10.1371/journal.pone.0173347.g005

ASU-Flickr (ASU-FR) [14].

Graph vertices represent users in the Flickr (https://www.flickr.com/) image and video hosting platform. Flickr users may follow each other and also subscribe to specific interest groups.

ASU-YouTube (ASU-YT) [50].

Graph vertices represent users in the YouTube (https://www.youtube.com/) video sharing website. Apart from uploading videos, users form a subscription graph among them and also subscribe to various interest groups. It was introduced in [50] and has been used to evaluate scalable algorithms [20] by keeping the labels with more than 500 vertices as ground truth. Since the graph used in [20] was disconnected, we perform one more post-processing step and keep only the largest connected component, since the spectral methods in our comparative study could not converge for a disconnected graph even after 4 days of continuous execution.

Insight Resource Multiview (IRMV) [51].

These are five multi-view datasets with manual annotation of user stances (e.g., political or sports). In order to perform similar experiments as with our own SNOW2014G dataset, we extracted an undirected graph that integrates two social interaction graph views, namely a mention and a retweet graph (see S1 Appendix). Since they are all small and of similar size, we will report only on results from the IRMV-PoliticsUK dataset.

Laplacian Eigenmaps (LapEig) [21].

This technique has been used for embedding OSNs [16]. We calculate the dim eigenvectors corresponding to the dim smallest eigenvalues of the symmetric normalized Laplacian , where L = D − A is the Laplacian; excluding the one corresponding to the zero-valued eigenvalue.

Replicator Eigenmaps (RepEig) [52].

This is the name we give to the computation of the eigenmaps of the Replicator matrix R = λ_max I − A, where λ_max is the largest eigenvalue of A. While the Laplacian is related to probability preserving random walks, the Replicator describes a diffusion process of an agent that transitions simultaneously to all adjacent vertices.

Random Walk Modularity Maximization (RWModMax) [17].

Circumventing the modularity measure’s resolution limit [53], the random walk modularity measure assesses statistical significance of communities based on random walk paths instead of edges. The top dim eigenvectors of the random walk modularity matrix are computed. We used the implementation provided by the authors (https://github.com/rdevooght/RWModMax).

Deepwalk [18].

This method treats random walk paths as documents and then applies deep representation learning to embed the users in a low-dimensional space. We used the implementation provided by the authors (https://github.com/phanein/deepwalk).

LINE [19].

This is an efficient method that attempts to preserve both the first and the second order connectivity of the vertices in a low-rank matrix embedding. We used the implementation provided by the authors (https://github.com/tangjianpku/LINE).

Louvain [54].

A hierarchical disjoint community detection method considered state-of-the-art in terms of both speed and community quality [22]. We used a Python implementation available online (https://bitbucket.org/taynaud/python-louvain).

Edge Clustering (EdgeCluster) [20].

This is an overlapping community detection method, specifically designed for extracting features from OSN graphs. It is an edge-centric k-means variant that exploits the adjacency matrix sparsity to calculate only relevant similarities. We used the implementation provided by the authors (http://leitang.net/social_dimension.html).

Multiple Resolution Overlapping Communities (MROC) [23].

A hierarchical community detection algorithm also designed for OSN graphs. It iteratively merges communities from highest to lowest resolution based on their similarity. In order to avoid the calculation of all possible pairs, the merging is based on heuristics such that a binary tree community hierarchy is produced.

Cluster Affiliation Model For Big Networks (BigClam) [55].

A fast, overlapping community detection method based on a generative matrix factorization model. We used the implementation from the Stanford Network Analysis Project (SNAP) website (http://snap.stanford.edu/).

Order Statistics Local Optimization Method (OSLOM) [56].

A hierarchical, overlapping community detection method. A number of clean-ups can be ran for each level to better assess community significance, although this increases runtime. We used the authors’ reference implementation (http://www.oslom.org/).

Base Communities (BaseComm).

These are defined as: X = A_sup + I, where A_sup is the binary support matrix of A.

For OSLOM, Louvain and MROC we used the hierarchy of communities for the formation of X. For the eigenvector calculations performed in the spectral methods LapEig and RepEig we used the ARPACK [57] package implementation of the Implicitly Restarted Lanczos method. For sparse matrices, the method complexity may reach O(i|E|), otherwise it scales as O(i|V|²), where i is the number of iterations. Finally, some discussion on further alternative competing methods and the reasons for not including them in our study were discussed in the related work sub-sub-section Relational classification.

Similarity vector calculation for user classification.

We compare the performance of our fast Cumulative PageRank differences (Fast-CPRD) method with the fast PageRank (Fast-PR) method, which we derive by substituting the similarity vector calculation step with Algs 7 and 6 respectively. Fig 6 depicts the results for the SNOW2014G dataset and we use for both methods the parameters from Table 4. We note that the Fast-CPRD method pushes probability values to the positions of the similarity vector k that correspond to the neighbors of the seed vertex from the first iteration. This guarantees that the neighbors will have non-zero values when the truncation operation is attempted. In our ARCTE variation that utilizes the Fast-PR method, we only try to extract an extended vertex-centric community around a seed vertex only when its neighbors have non-zero values; otherwise only the base vertex-centric community is extracted.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Similarity vector method comparison for user classification on SNOW2014G.

https://doi.org/10.1371/journal.pone.0173347.g006

Similarity vector calculation efficiency.

We also report the efficiency of vertex-centric similarity vector calculations. We made comparisons for the ARCTE parameters given in Table 4 for the three datasets. In the case of PageRank calculation with a restart probability ρ_eff and a laziness factor, we assume λ = ¹/₂, which implies ρ = ^{ρ_eff(1 − λ)}/_{1 − ρeff λ}. We do this so that both the Lazy-PR [32] and the Fast-PR method calculate an approximation to the same PageRank vector. We notice both a significant speed-up in execution time and a smaller number of operations for the PageRank comparison. Furthermore, we also compare the times with our Cumulative PageRank differences adaptation, which requires the exact same number of limit push operations as the fast PageRank method. We expect it to be marginally slower than the fast PageRank because the Cumulative PageRank differences push operation is a little heavier in computations. Evidently, the slightly slower execution of Fast-CPRD is justified by its increased predictive performance over Fast-PR (see previous paragraph). The execution times reported are averaged across 5 runs for each dataset and the measurements were performed using the Python profiler (https://docs.python.org/3.4/library/profile.html). The results are given in Table 5. The speed-up reported is with respect to the Lazy-PR method [32].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Similarity vector calculation speed-up.

https://doi.org/10.1371/journal.pone.0173347.t005

ARCTE parameter selection.

The calculation of our vertex-centric similarity vector is dependent on two variables: a) the restart probability ρ, and b) the threshold ϵ. A smaller restart probability encourages exploration such that vertices further from the seed have the chance to be ranked higher. As ρ → 0, the random walker performs a random walk without restart. A smaller threshold parameter leads to a better approximation to the true similarity vector as defined in Eq 2. Fig 7 depicts the performance of ARCTE for ϵ = 10⁻⁵ and variable ρ and Fig 8 depicts the performance of ARCTE for ρ = 0.1 and variable ϵ for a 4% training set in the SNOW2014G dataset. As a general rule, smaller ρ and ϵ values lead to better results. The only caveat here is that this also leads to larger execution times. Specifically, for our implementation, we note a 26.61% decrease in feature extraction time for ARCTE with ρ = 0.1, ϵ = 10⁻⁴ and a 30.83% increase for ρ = 0.1, ϵ = 10⁻⁶ compared to the parameters we report in Table 4. Similarly, we get a 39.37% decrease for ρ = 0.2, ϵ = 10⁻⁵ and a 183.06% increase for ρ = 0.01, ϵ = 10⁻⁵. In order to balance a performance showcasing the strength of ARCTE and manageable execution times, we opted for more moderate parameter values (ρ = 0.1, ϵ = 10⁻⁵) for the series of comparative experiments in sub-sub-section User classification performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. ARCTE performance on SNOW2014G: restart probability perturbation.

https://doi.org/10.1371/journal.pone.0173347.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. ARCTE performance on SNOW2014G: approximation threshold perturbation.

https://doi.org/10.1371/journal.pone.0173347.g008

Classifier parameter perturbation.

The SVM hardness C is also an important parameter in this series of experiments, as shown also in the RWModMax paper [17]. We show the effects of varying this parameter in Fig 9 for the ASU-Flickr dataset. On the x-axis, we perturb the linear SVM C parameter and we show the Macro-F1 measure on the y-axis. The results are similar for all training sets, but for reasons of space consumption we elected to report results only for the 4% training set. Generally, the perturbation of C does not lead to extreme variations, although we see that ARCTE is somewhat less dependent on this parameter when compared with the most competitive low-rank representation methods for the ASU-Flickr dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. SVM C parameter perturbation on ASU-Flickr.

https://doi.org/10.1371/journal.pone.0173347.g009

User classification performance.

The results for the SNOW2014G, ASU-Flickr, ASU-YouTube datasets and IRMV-PoliticsUK are depicted in Figs 11, 12, 13 and 14 respectively. In Table 6 we show for all cases depicted in Figs 11–14 the winner and runner-up methods. We further report the training set percentages for which the improvement of the highest ranked method over the second highest is statistically significant (p < 0.01) as calculated via a paired t-test. A fair comparison was ensured by using common training-test set partitions for each training set percentage among the different methods. Finally, for each training set percentage we calculate the maximum absolute and relative F1 score improvement percentages of the winner method over the runner-up and we report the largest for each case.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Performance on SNOW2014G (best viewed in color).

https://doi.org/10.1371/journal.pone.0173347.g011

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Performance on ASU-Flickr (best viewed in color).

https://doi.org/10.1371/journal.pone.0173347.g012

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Performance on ASU-YouTube (best viewed in color).

https://doi.org/10.1371/journal.pone.0173347.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Performance on IRMV-PoliticsUK (best viewed in color).

https://doi.org/10.1371/journal.pone.0173347.g014

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Top two performing methods comparison.

https://doi.org/10.1371/journal.pone.0173347.t006

Firstly, we note that we used our improved cumulative PageRank differences method (Alg 7) for the similarity calculation in the ARCTE algorithm. Furthermore, we applied supervised community weighting as described in sub-section Supervised community weighting to all community detection methods in order to improve the results.

According to the results, ARCTE outperforms all the competing methods, with the exception of the F1-Macro measure for ASU-Flickr, where it is still competitive with respect to the winners (MROC and BaseComm) as the score difference is not significant for all training set percentages. Specifically, BaseComm surpasses ARCTE with p < 0.01 for training set percentages in {4–10} and ARCTE is actually the leading method with p < 0.01 for a 1% training set. Furthermore, ARCTE clearly surpasses BaseComm and MROC in F1-Micro. We note that for the three other datasets, ARCTE dominates by reaching even 4.29% above the runner-up for SNOW2014G, 3.44% for ASU-YouTube and 14.24% for IRMV-PoliticsUK in terms of Macro-F1 score. The maximum relative improvements reach 22.49%, 11.92% and 26.72% for the three datasets respectively. Finally, the winning performance of ARCTE is also significant with p < 0.01 for the majority of cases (except for the F1-Macro on ASU-Flickr) as can be seen in Table 6. Another observation of note is that ARCTE performs comparatively well for small training set percentages, as is mostly evident on ASU-Flickr, ASU-YouTube and IRMV-PoliticsUK and less so on SNOW2014G since the improvement of ARCTE gets larger for bigger percentages. As for ASU-Flickr, ARCTE surpasses MROC and BaseComm even in the Macro-F1 measure for 1% training set.

We also note that while ARCTE is consistently near the top of the competition, no other competing method can boast a similar behavior. Indeed, while Deepwalk, LINE and LapEig score quite well on SNOW2014G, albeit quite lower than ARCTE, their performance is not competitive in the other datasets. Similarly, the performance of MROC on SNOW2014G is not comparable to the rest of the competition. ARCTE also outperforms two methods based on neural word representation learning, Deepwalk and LINE, in all datasets. We chose as representatives of state-of-the-art community detection techniques the following: Louvain, OSLOM and BigClam. Generally, they are shown not to perform competitively to either low-rank matrix representations or specifically crafted community detection techniques for user classification. Indeed, the methods that are distinguished by their results are ARCTE, MROC and to a lesser extent the BaseComm approach. The latter technique requires far lower computational cost compared to the rest and is easy to implement but its performance deteriorates significantly in sparsely labeled datasets, such as SNOW2014G. Finally, although ARCTE wins the competition, MROC is a notable competitor, achieving high F1 scores for all datasets, although its behavior seems very similar to BaseComm and its quadratic complexity makes it impractical for very large graphs.

We also performed the same experiments on these datasets for a subset of the better performing methods for training set percentages up to 90%. The figures and discussion can be found in S1 Fig.

Feature extraction method execution times.

As an empirical benchmark, we report execution times in seconds for all the competing methods in Table 7. There is an implicit caveat though: as mentioned before, we used some optimized implementations available online (e.g., BigClam) or utilized linear algebra packages (e.g., LapEig and RepEig). Our own Python implementations of ARCTE and MROC are without any effort at any noteworthy software optimization techniques, with the exception of simple coarse-grained parallelism in ARCTE. Finally, the OSLOM times reported correspond to all the clean-up runs (we noticed improved accuracy for more runs). The results from Table 7 indicate that apart from accomplishing a good performance in terms of accuracy for user classification, ARCTE is also one of the fastest methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Feature extraction method execution times (in seconds).

https://doi.org/10.1371/journal.pone.0173347.t007

ARCTE parameter selection.

With ARCTE, we see that as one decreases the restart probability ρ and the threshold parameter ϵ, the performance further improves with the caveat of increased execution time. By increasing them, the performance drops as it becomes more difficult to capture extended vertex-centric communities. However, even if no extended vertex-centric communities are captured, ARCTE behaves similar to the Base Communities algorithm, which performs reasonably well across datasets. This behavior, in addition to ARCTE’s robustness with respect to SVM hardness C perturbation attest that ARCTE is a reliable approach to user classification.

Cumulative PageRank differences.

The promising results reported in sub-section Similarity vector comparison led us to an interesting observation. User-centric community detection via thresholding the PageRank vector has been used extensively in problems such as local community detection methods [34], matrix sparsification [33] and nearly linear time solution of symmetric, diagonally dominant systems [29]. Our improvement may seamlessly substitute the PageRank calculation in any of these applications, leading to faster and improved results.

Feature weighting for low-rank matrix representations.

During the experimental design, we attempted to use feature weighting/selection also for the low-rank matrix representation methods. However, the continuous valued nature of the features produced by such methods requires additional computational steps. For example, one might try to first discretize the features and then apply feature weighting as normal. We tried discretization via bidimensional histograms along with χ² weighting in order to weigh or remove weak features, but we did not notice any improvement; in fact there was a decrease in performance in some cases. We did not try advanced approaches, such as Parzen window discretization or numerical integration, as this would increase execution time significantly and introduce more tunable parameters, thus defeating the purpose of using a light-weight method to boost the importance of potent features.

Community detection for graph embedding.

As was shown in section Results, ARCTE is a very appealing feature extraction approach for user classification. In most cases it outperforms all the competing methods and is consistently at least of comparable performance. This is especially apparent in the larger datasets (i.e. SNOW2014G, ASU-YouTube) where it significantly exceeds the performance of the main competitors, i.e. MROC, Deepwalk, LINE and LapEig. The comparison with LapEig on the ASU-YouTube dataset was missing from many recent studies [16, 18, 19, 23] that utilized the dataset. It further exhibits robustness to parameter perturbation and as a community detection method it lends itself to simple, but very effective supervised community feature weighting. The latter enables us to avoid placing confidence in features/communities for which we do not have significant evidence of their predictive power. Another way to view this is as a kind of automatic matrix dimensionality determination, which has been mentioned as an open problem for user classification before [16]. Since features that are independent of a label’s presence are weighted by 0, they are practically discarded for the specific experiment.

Insights on user classification comparative study.

Among the community detection methods, ARCTE, MROC and BaseComm were significantly better performing than EdgeCluster, BigClam and OSLOM. We attribute this gap in performance to the fact that the former three also focus on high-resolution structure in the graph, whereas the latter three are designed to produce a mesoscopic representation of a graph. We further believe that the leading performance of ARCTE compared to the community-based methods in the majority of cases is due to the fact that it focuses on utilizing multiple resolution user-centric communities, thus providing more informative representations for each user. The better performance of ARCTE compared to all the low-rank matrix representation methods can also be attributed to the aforementioned idea, as well as to the improvement brought by the supervised community weighting. This combination allows for the exploitation of both the graph structure and the known label similarities within communities, for all community-based methods. The recent LINE and DeepWalk exhibited similar performances in all the datasets. By adopting the parameter selection of the original articles we found that DeepWalk outperforms LINE on the ASU-YouTube dataset, although LINE performs better on our own SNOW2014G dataset. LINE is the most recent method included in the comparison. The idea behind LINE is to preserve graph similarities, based on both first and second order connections in the graph. We believe that this concept is not as sound as ARCTE’s ability to capture extended user-centric communities, which may extend beyond or even exclude second order connected vertices in a principled way, i.e. based on the regularized commute-times similarity measure.