Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

A network community structure similarity index for weighted networks

Abstract

Identification of communities in complex systems is an essential part of network analysis. Accordingly, measuring similarities between communities is a fundamental part of analysing community structure in different, yet related, networks. Commonly used methods for quantifying network community similarity fail to consider the effects of edge weights. Existing methods remain limited when the two networks being compared have different numbers of nodes. In this study, we address these issues by proposing a novel network community structure similarity index (NCSSI) based on the edit distance concept. NCSSI is proposed as a similarity index for comparing network communities. The NCSSI incorporates both community labels and edge weights. The NCSSI can also be employed to assess the similarity between two communities with varying numbers of nodes. We test the proposed method using simulated data and case-study analysis of New York Yellow Taxi flows and compare the results with that of other commonly used methods (i.e., mutual information and the Jaccard index). Our results highlight how NCSSI effectively captures the impact of both label and edge weight changes and their impacts on community structure, which are not captured in existing approaches. In conclusion, NCSSI offers a new approach that incorporates both label and weight variations for community similarity measurement in complex networks.

Citation: Malekzadeh M, Long JA (2023) A network community structure similarity index for weighted networks. PLoS ONE 18(11): e0292018. https://doi.org/10.1371/journal.pone.0292018

Editor: Sudipta Chowdhury, Marshall University, UNITED STATES

Received: July 18, 2023; Accepted: September 10, 2023; Published: November 29, 2023

Copyright: © 2023 Malekzadeh, Long. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data can be accessed publicly via its original source, and a statement containing the link to the source website is included in the manuscript.

Funding: This project was funded by a Western Catalyst Grant to JL. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Due to their complex structure, the identification of communities continues to be an essential part of the analysis of networks [1]. Accordingly, measuring similarities between communities is a fundamental part of analysing community structure in different (usually related) networks [2–11]. Methods for assessing network community similarity can be employed to assess network models (e.g., comparing to known communities) [12, 13], comparing different states of the system (e.g., before and after an intervention), or time series analysis in networks [14].

Although several methods exist for quantifying network community similarity, such as the widely-used Jaccard index [2] and mutual information [15], they predominantly focus on community labels and neglect the intricate relationships encoded within edge weights. A critical drawback arises when a uniform metric treats dissimilarities in edges and their corresponding weights equivalently. For example, in a subway network, the cost of incorrectly clustering the central station (with a high number of edges and traffic flows due to its functionality) is the same as the cost of incorrectly clustering an intermediate station (with a low number of edges and traffic flows). This gap in the existing literature underscores the need for a novel approach that effectively incorporates edge weights into the community similarity calculation. Additionally, the application of current methods is not possible when comparing networks with different number of nodes.

We propose a novel network community structure similarity index (NCSSI) as a similarity index for comparing network communities. The formulation of the NCSSI is based on the concept of edit distance, whereby the cost of each node changing its community is computed based on their edges and corresponding weights. By taking this approach, the NCSSI provides an advantage over existing methods in that it is able to incorporate community labels, edges and their weights into a single measurement of community similarity. Employing simulated data and real-life New York Yellow Taxi flows data, we highlight how NCSSI compares to existing approaches and provides an alternative view of how similarity structure can be measured between communities.

The structure of the article is as follows: in section 2 we provide a general background on network structure, community detection, and similarity of communities. Then we introduce NCSSI and its implementation. In section 4 we experiment with NCSSI and simulated data in two different scenarios: on benchmarks to compare with existing similarity methods, and on benchmarks where we change the weights of random nodes to highlight the limitation of existing similarity methods. Then in section 5 we implement NCSSI on a real-life mobility dataset, New York Yellow Taxi flows data. In the discussion and conclusion, we discuss the advantages of the proposed method and the results, and we conclude the study.

2. Background/literature review

Many complex systems can be modeled by networks (also commonly termed graphs) [1]. A network consists of a set of nodes (or vertices) and a set of edges [16]. Edges can include varying weights or direction and nodes can have multiple attributes. Different systems can be modeled by different structures of graphs such as directed [17], weighted [18], overlapping and clustered [19], or hierarchical graphs [20].

Unlike random graphs, in which the probability of having an edge between two nodes among all pairs of nodes is equal and the distribution of edges is homogenous [21], real-life networks show a high level of heterogeneity in edge distribution [1]. These heterogeneities are often represented as clusters within which intra-cluster edge density is much higher than inter-cluster edges. These clusters are typically referred to as communities in networks [22] and these two terms are usually used interchangeably. In real-life networks, nodes of communities usually share similar characteristics and/or similar interests (e.g., social networks; [22]) and/or have higher levels of interactions (e.g., transportation networks; [23]). Finding communities in networks can provide new insights into the structure of networks and enable us to better understand the underlying complex system.

Different algorithms have been developed to detect communities from networks. Graph partitioning methods divide the graph into a specific number of communities with a specified number of nodes to minimize the number of inter-community edges [24, 25]. Hierarchical algorithms comprise two types [26]: bottom-up (agglomerative) in which all similar communities are merged until reaching a specified threshold and top-down (divisive) which adhere to the opposite direction–dividing dissimilar communities until reaching a specified threshold. A very well-known divisive community detection method was proposed by Girvan and Newman [22, 27]. In Girvan and Newman’s method, edges are iteratively removed based on their value of betweenness which expresses the frequency of shortest paths between all pairs of nodes that pass along the edge. An alternative set of methods are modularity-based algorithms [28, 29]. Modularity, as a quality measurement of community detection, is computed based on the comparison between the existing structure of edges in the subgraph and the density of edges in the subgraph if the nodes were attached irrespective of the community structure [27, 30]. Spectral algorithms [31], methods based on statistical inferences [32], and algorithms that use deep learning methods [33] are the other alternatives. For a more comprehensive review of clustering and community detection algorithms, refer to [1, 34].

Methods for measuring the similarity (or differences) between two sets of communities have been employed in methodological work to test the performance of a community detection method on a benchmark [35], or to cross-check the results of community detection using different methods [13]. In applied settings, network community similarity is used across various domains such as studying similarity of sets of communities over time [36], and comparing different types of networks [37]. While there is extensive literature on community detection methods, there is a lack of attention to similarity measurements for network communities [1].

There are four primary types of measures for assessing network community similarity: pair counting, community matching, information theory, and distance. Pair counting methods identify corresponding communities in two different sets. If a node is identified in the same community (two communities with the highest overlapping) in two different sets, it is considered correctly detected. The Rand index [4] and its adjusted version [2], the Jaccard index and its adjusted version [2], and the Mirkin metric [3] are all pair counting methods. Community matching measures are based on finding the largest overlap between two sets of communities. However, these methods may only consider the largest similar portion of communities, potentially disregarding certain parts of the community set as a whole. Classification error, defined by Meilă & Heckerman [38], and the normalized Van Dongen metric [5] are examples of community matching measures. A third type of similarity measure is based on information theory [39]. Using information theory, if two sets are alike, there is little information needed to derive one set from the other. Less similarity between the two sets indicates that more information is needed to infer one from the other. A commonly used measure of this type is the normalized mutual information [15]. Lastly, distance-based measures in which the number of movements and divisions are considered to calculate the distance between two sets. Movement is defined as the minimum number of nodes that are needed to be moved from one community to the meet set of two communities to match, and divisions are defined as the number of divisions needed for one community to match with the meet set of two communities [6, 7].

3. Network community structure similarity index (NCSSI)

We first define notation that will be used throughout the paper. Let:

V be a set of n nodes in network N

E be a set of edges that connect network nodes

W be a set of weights associated with each edge

A community set is defined as a set of >1 communities derived from a given network. The goal of NCSSI is to quantify the similarity between two community sets. NCSSI is derived from the edit distance concept in which the distance is computed based on the total cost to transform one community set into another. The critical feature of NCSSI is the cost function which is associated with each of the necessary edits (inserts and removes) to perform this transformation. We should emphasize that in NCSSI, we are not trying to transform one network to another but rather to transform one community set into another. This is an important distinction as transforming the whole structure of the graphs is unnecessary when the principal focus is only on the community structures. Hence, NCSSI focuses on nodes whose labels differ between the two sets. Then, for each of these nodes, we calculate the cost associated with the minimum edits needed to change the community labels. Calculation of NCSSI follows a three-step method outlined below.

3.1 Step 1 –Pairing communities

To find the similarity between two community sets, we must find the most similar community pairs in each of these two sets. Since both community sets have different community labels, to find the most similar community pairs, we must calculate the most overlapping communities in each set in terms of both nodes and weighted edges. To calculate this, assume we have two communities x and y, from the community sets X and Y, respectively, where the intersection set of their nodes is ∩xy and the union set is ∪xy. We focus on incorporating edge weight information into the measurement of NCSSI. Therefore, we sum all edge weights (W) between the nodes in ∩xy and ∪xy and adjust them by dividing it by the total weights of edges in the network. Since we have two community sets with different configurations regarding their edges and associated weights, we need to calculate the sum of all edge weights for both Networks. Finally, the overlap score (OS) of communities x and y is defined as the ratio of the sum of all adjusted edge weights in ∩xy to ∪xy: (1)

Where OSxy is the overlap score between communities x and y from the community sets of X and Y; ∩xy and ∪xy are the intersection and union sets of nodes from communities x and y; and and are the edge weights between i and j in the community sets X and Y, respectively.

To find the paired communities, we need to calculate all the possible OS values between communities of two community sets, resulting in an OS matrix. The unordered bijective pairs are chosen by the maximum OS, meaning each community in a set is paired only with one community from the other set with the highest overlap. It is possible to have different numbers of communities between the two community sets. If, due to the difference in the number of communities in sets, a community (let us assume x) is not paired with a community in the other set, the pair will be denoted as (x, ϕ).

3.2 Step 2 –Calculating edit costs of nodes

Based on the definition of communities, the stronger the intra-community links, and the weaker the inter-community links, the stronger the structure of communities. Hence, in calculating NCSSI, we concentrate on the nodes’ labels and the edge weights associated with the nodes.

In this step, we need to calculate the minimum cost associated with the edits needed to transform the node i’s membership (Eqs 2 and 3). In Eqs 2 and 3, we assume that we have community set X, which contains communities x1 and x2, and Y, which contains communities y1 and y2. Additionally, x1 and y1, and, x2 and y2, are paired with each other.

(2)(3)

Where and is the adjusted edit cost of node i in the community sets X and Y, respectively; Within Eq 2, x1 is the community in which i is a member; and x2 is the community which is the paired community of the changed community y2; similarly within Eq 3, y2 is the community in which i is a member; and y1 is the community which is the paired community of the changed community x1; j is a node member of the communities x1, x2, y1, and y2; and Wij is the weight associated with the edge between nodes i and j.

These costs reflect the necessary modifications required to transform the membership of a node from one community to another while considering the associated edge weights. However, it is important to note that these costs may not be on the same scale for both community sets X and Y, as they depend on the specific characteristics of the community sets and the corresponding edge weights. To calculate edit costs on a common scale, it is necessary to adjust them by the sum of intra-community edge weights associated with node i and the sum of weights between i and paired community nodes. To adjust edit costs, we need to compute the adjustment factors for both community sets X and Y (Eqs 4 and 5). It should be noted that the adjustment factors include weightings from all nodes irrespective of whether the node’s community membership remains unchanged or changed.

(4)(5)

Where AFX and AFY are the adjustment factor associated with community X and Y, respectively. The notation for i, j, x1, x2, y1, y2, and Wij remains the same as previously defined.

By dividing the edit costs by the adjustment factors, we obtain the adjusted edit costs on a common scale for both community sets (Eqs 6 and 7). These adjusted edit costs are bounded between 0 and 1 for each community set, ensuring that both community sets are on the same scale and comparable. It is important to note that the sum of all adjusted edit costs for each community set cannot exceed 1, as they represent the relative transformation costs within the community sets.

(6)(7)

Where and are the adjusted edit cost of node i in the community sets X and Y, respectively. The notation for , AFX and AFY remains the same as previously defined.

As an example, with the same assumption as before, we have community set X, which contains communities x1 and x2, and Y, which contains communities y1 and y2. Additionally, x1 and y1, and, x2 and y2, are paired with each other. If node i is a member of x1 in community set X and a member of y1 in community set Y, as the two communities are paired with each other, the node i has not changed its community membership—NECi is zero. However, if node i is a member of x1 in community set X and a member of y2 in community set Y (Fig 1), as these two communities are not paired with each other, it means that the node has changed its community membership and we need to consider it in our calculation. In this case, in each community set (let us first consider community set X and then we must repeat this procedure for community set Y) we have two communities that we need to concentrate on; x1 in which i is a member, and x2 which is the paired community of the changed community y2 (Fig 1).

thumbnail
Download:
Fig 1. An example of edit cost calculation when node i has changed its community membership.

Blue lines and red lines represent the edges that are considered in the calculation of the edit cost related to node i, as intra-community edges and edges to the paired community. Dashed black lines represent the inter-community edges and black lines represent the intra-community edges that were not considered in the calculation of the edit cost related to node i. The paired communities in different community sets are colored the same (dark green and light green). All the other communities that are not considered in the edit cost calculation for node i, are colored gray.

https://doi.org/10.1371/journal.pone.0292018.g001

3.3 Step 3 –NCSSI calculation

NCSSI is the total cost associated with transforming a node’s membership and is equal to the sum of the adjusted edit costs of the node in each of the two sets. Consequently, to get the total cost of transforming one community set into another, we need to sum up all nodes’ NECs from each community set. Then the total cost is obtained by averaging the total adjusted edit costs of both community sets (Eq 8). The selection of the average as the method for combining the adjusted edit costs is justified by several factors. Firstly, taking the average ensures a balanced consideration of both community sets, allowing for equal evaluation of their respective edit costs. Secondly, it preserves the information from both sets by incorporating the collective impact of changes, rather than favoring one set over the other (unlike selecting the minimum or maximum). Thirdly, the average maintains sensitivity to changes in both community sets, capturing the nuanced transformation between the structures. Finally, since the adjustment factors and edit costs are already on the common scale, the average does not introduce bias towards a specific range.

(8)

Where NCSSIXY is the Network Community Structure Similarity Index of community sets X and Y; i is a node from the set of nodes that have changed their community memberships. This results in a value between 0 to 1, which highlights the distance between two community sets. To convert distance to similarity, it is then subtracted from 1. The final value is then defined as the NCSSI of the two sets of communities ranging from 0 to 1 where 0 represents low similarity and 1 identical sets of communities (Fig 2).

thumbnail
Download:
Fig 2.

A) An example of identical sets of communities in which community pairs are (red, blue; OS = 1) and (green, yellow; OS = 1) and NCSSI = 1. B). An example of the most dissimilar sets of communities in which community pairs are (red, blue; OS = 0) and (green, yellow; OS = 0) and NCSSI = 0. In these examples, all weights assigned to the edges were 1 for simplicity.

https://doi.org/10.1371/journal.pone.0292018.g002

4. Simulation tests

4.1 Benchmark

A benchmark network is a network in which all nodes’ community labels are known. These benchmarks were originally introduced to enable researchers to test the results of community detection methods. Here, we are not interested in testing the clustering and community detection methods, rather, we want to test the community similarity measures using these benchmarks. We use computer-generated benchmarks as they enable us to adjust the network and its attributes with specifically chosen parameters. The most popular class of computer-generated benchmarks works based on pin as the probability of having intra-community links for nodes and pout as the probability of having inter-community links while these two probabilities are independent [40]. A specific case of this type was proposed by Girvan and Newman [22] in which a graph with 4 communities and 32 nodes in each community is considered. This benchmark is not an ideal representative of real-life networks as the number of communities and the number of nodes in each community are constant and nodes’ degrees are almost similar in each community.

The Lancichinetti-Fortunato-Radicchi (LFR) benchmark introduced by Lancichinetti et al., [41] is a modified version of this model in which the heterogeneity of nodes’ degrees, the community sizes, and the number of communities is taken into account (Fig 3). In the LFR the distribution of nodes’ degrees and the size of communities are determined by power law functions with two parameters of τ1 and τ2, respectively. The intra-community degrees are determined by the fraction of 1 – μ of nodes’ degree. Thus, the inter-community degrees are computed based on the fraction of μ of nodes’ degrees. When μ is 0 we have highly separated communities but when μ is 1 we have highly interconnected communities. Here, we use the LFR benchmark in our experiments.

thumbnail
Download:
Fig 3.

A) LFR benchmark [41] when μ equals 0.05, and B) LFR benchmark when μ equals 0.95. Each color is representative of the nodes’ communities.

https://doi.org/10.1371/journal.pone.0292018.g003

4.2 Different community detection methods

We generated simulated community graphs for comparison using the LFR benchmark. We varied μ from 0.1 to 0.95 and for each μ we ran the process 100 times as suggested by [41] to avoid random bias. We used n = 1000 nodes in the graph and τ1 = 1.5 and τ2 = 3, as the parameters of the LFR benchmarks. We deployed the Louvain community detection algorithm [42] to derive communities from each benchmark. We computed the similarity index using the Jaccard index, mutual information, and NCSSI by comparing the communities detected by the algorithm to the known benchmark communities. We repeated this process deploying Clauset’s algorithm [28], asynchronous label propagation algorithm [43], and semi-synchronous label propagation algorithm [44] (S1 Appendix).

The Jaccard index and mutual information resulted in low similarity between two sets of communities when μ > 0.5 (Fig 4). However, considering the network’s structure in the similarity measurement, we observe that increasing μ decreases the similarity between known and computed community sets. The relationship between μ and NCSSI is more linear and monotonic than with the Jaccard index and mutual information measures. The reason that we still observe a low-level of similarity when μ > 0.5, is that even in networks with low community structure, there is an inevitable random degree of similarity across the community sets. Based on LFR benchmark algorithm, when μ equals 0.5, the number of inter-links and cumulative intra-links (links with all the other communities) will be equal. Although community structure is commonly perceived to weaken after μ = 0.5, it is important to note that in networks with a substantial number of nodes and communities, community structure can still be discernible even when μ > 0.5. This is because the presence of a large number of nodes and communities ensures that the intra-links within each community outnumber the inter-links to individual communities. Thus, despite the higher fraction of inter-community edges, the overall network can still exhibit noticeable community structure.

thumbnail
Download:
Fig 4. Similarity scores deploying Louvain similarity measure on LFR benchmarks using two existing similarity measurement methods (the Jaccard index and mutual information) and the proposed method NCSSI.

https://doi.org/10.1371/journal.pone.0292018.g004

4.3 Manually changing nodes’ community memberships and their weights

In this example, we used an LFR benchmark (n = 1000, τ1 = 1.5, τ2 = 3, and μ = 0.2), in which all edges had the same weight of 1. We randomly selected a subset of 10 nodes, which constituted 1% of all nodes. To assess the impact of weights, we increased the weights of all the edges connected to these nodes incrementally from 10 to 100 (with an interval of 10). Then we changed these nodes’ communities as an intervention (intentional misclustering). We aimed to examine how different similarity measures can capture the difference between the communities before and after the intervention. We repeated this process for each weight 100 times and took the mean similarity score of the 100 repetitions.

The Jaccard index and mutual information measures were not sensitive to the changes in the weights of nodes as increasing the weights associated with these nodes did not affect similarity scores (Fig 5). In contrast, NCSSI captured the effect of increasing weights associated with the misclassified nodes, leading to lower similarity between community sets. These results clearly demonstrate that NCSSI is sensitive to changes in edge weights and the subsequent effect on network community similarity, while Jaccard index and mutual information measures are not.

thumbnail
Download:
Fig 5. Similarity values for an LFR benchmark while changing the weights associated with 10 randomly chosen nodes using two existing similarity measurement methods (the Jaccard index and mutual information) and the proposed method NCSSI.

https://doi.org/10.1371/journal.pone.0292018.g005

5. Case study: NYC Taxi flow dataset

As a real-life example of how NCSSI can be implemented, we compared how community structures computed based on human mobility flows differ from 2011 to 2021 using Yellow Taxi Pick-up/Drop-off data provided by the New York City Taxi and Limousine Commission, USA [45]. This dataset contains more than 1.3 billion trips for 13,587 taxis in New York City, USA. The locational information in the dataset is aggregated into 263 taxi zones. Each data entry includes a pick-up zone and time, and drop-off zone and time, along with other attributes. We aggregated the data seasonally and created a flow (Origin-Destination) matrix between taxi zones for each season which is a form of a spatial network where zones represent nodes and taxi trip frequencies between zones represent edge weights. We then detected the communities employing Louvain’s algorithm for each season. To avoid seasonal effects, we compared each season’s community structure to the previous year’s. There is no true community structure to compare to, however we can make some hypotheses. First, we do not expect to see community structures change drastically over time, and year to year communities should be relatively stable. However, we do expect to see strong differences associated with the COVID-19 pandemic and associated changes in human mobility patterns in 2019–2020. Further, the advent of ridesharing services (e.g., Uber, Lyft) which originated in 2011 may have an impact over time as they increase in popularity, especially in the less dense areas of NYC. Seven taxi zones appeared to have no trips for at least one season throughout the study period. This causes an inconsistency in the number of nodes in the networks. As mutual information and Jaccard index require the same number of nodes in both networks, this limitation forced us to eliminate inconsistent zones in the networks to compare the results of different methods.

We observe a relatively low similarity level between all pairs of consecutive years throughout 2011 to 2021 using the Jaccard index (Fig 6), but this is perhaps not surprising as Jaccard index always presented the lowest similarity between benchmarks. However, the similarity scores of the Jaccard index showed the greatest variation (i.e., lack of consistency in similarity) between consecutive pairs of years (Fig 6). In general, mutual information found higher similarity values relative to Jaccard index (Fig 6), but again showed highly variable results from consecutive pairs of years. NCSSI exhibited the highest similarity values in each pair of years, and more interestingly, NCSSI demonstrated more stable similarity values in all pairs of years leading up to 2019. We see the effect of the COVID-19 pandemic on mobility result in a drop in similarity values in 2019–2020 spring for all measures. In the winter, fall, and spring we see that Jaccard index and mutual information measures show that this similarity continues to be low in the 2020–21 comparisons, but this pattern is not observed using NCSSI.

thumbnail
Download:
Fig 6. The seasonal level of similarity in community structures for pairs of consecutive years using different indices from 2011 to 2021.

The vertical axis demonstrates the similarity score. The dotted red line highlights the three periods of time we discuss in this section (spring 2016–2017; summer 2016–2017; and summer 2020–2021).

https://doi.org/10.1371/journal.pone.0292018.g006

We chose three periods of time to compare different scenarios and mapped them to visually evaluate the results (Fig 7). The first period is the spring of 2017 compared to the spring of 2018, where the Jaccard index and mutual information resulted in a low level of similarity, but NCSSI showed a high level of similarity. Next is the summer of 2017 compared to the summer of 2018, where all measures demonstrated a high level of similarity. Finally, as an interesting period related to the Covid-19 pandemic, we compared the summer of 2020 to the summer of 2021 where all measures showed a relatively low level of similarity. We observe when the number of zones that changed their communities are higher, as in 2017–2018 Spring and 2020–2021 Summer, the Jaccard index and mutual information resulted in a low level of similarity. However, while the number of communities that changed their community was high, NCSSI did not result in a low level of similarity as the weights of the intra-community flows and flows to the paired community were not substantial in this case, resulting in low edit costs (Fig 8). On the other hand, when the flows substantially changed (as in the summer of 2020–2021), this results in high edit costs, and NCSSI captures a relatively low similarity level (Fig 8).

thumbnail
Download:
Fig 7. Maps of New York communities computed based on NY Yellow Taxis flow data.

Taxi zones colored red represent the areas that changed their communities, and gray zones, represent the areas that did not. Base map and its related data are from OpenStreetMap and OpenStreetMap Foundation [46].

https://doi.org/10.1371/journal.pone.0292018.g007

thumbnail
Download:
Fig 8. Intra community adjusted flows and adjusted flows to the paired community for taxi zone number 222 for the spring of 2016 and 2017, and the summer of 2020 and 2021.

This zone is chosen since it changed its community in all the chosen periods. Intra-community adjusted flows are presented with blue and adjusted flows to the paired community are presented with orange. The width of the flows is based on their intensity; the thicker, the higher rate of adjusted flows.

https://doi.org/10.1371/journal.pone.0292018.g008

We plotted intra-community adjusted flows and adjusted flows to paired communities for the spring of 2016 and 2017, and the summer of 2020 and 2021, choosing taxi zone #222, which had a community change in all three periods of interest (Fig 8). It is important to note that the calculated adjusted flows presented in the plots are obtained by dividing the flows by the corresponding adjustment factors associated with each season and year. It is crucial to emphasize that these adjusted flows should be distinguished from actual flows. Despite the fact that the actual flows during the 2020–2021 period experienced a significant decrease, the higher values of adjusted flows compared to the 2016–2017 period is attributed to the flow adjustment process. This process is exclusively employed for the purpose of facilitating visualization, enabling us to visually compare the adjusted flows and providing clarification on how NCSSI incorporates edge weights in its calculations.

We observe that the adjusted flows in the summer of 2020 and 2021 have a higher values relative to the spring of 2016 and 2017. While the number of taxi zones that changed their communities is almost the same in both cases (2016–2017 Spring = 190, and 2020–2021 Summer = 193), the adjusted flow levels differed greatly. For instance, the intra-community and paired community adjusted flows of the zone taxi #222 for the spring of 2016 are 0.003, and 0.141, for the spring of 2017 are 0.017, and 0, for the summer of 2020 are 0.066, and 0.26, and for the summer of 2021 are 0.043, and 0.18 respectively. Since the Jaccard index and mutual information only consider the difference in community memberships and not the flows, they resulted in low values of similarity in both scenarios.

6. Discussion and conclusion

In the simulated examples, we changed the fraction of inter-community edges (μ) of the benchmarks from 0.1 to 0.95. The Jaccard index and mutual information methods exhibited low similarity after a certain point (μ > 0.5) [13], while NCSSI had a more consistent decrease across the range of the fraction of inter-community edges (from 0.1 to 0.95). Therefore, NCSSI seems to be more closely associated with the fraction of inter-community edges than both the Jaccard index and mutual information methods. This observation underscores the intrinsic sensitivity of these methods to changes in network configurations. While the Jaccard index and mutual information methods exhibited diminishing sensitivity when μ > 0.5, NCSSI consistently responded across the full μ range, capturing variations in edge structures. This behavior highlights NCSSI’s relationship with μ, distinguishing it from the Jaccard index and mutual information methods.

In another experiment, we intentionally changed the labels of several nodes and their associated weights to create an intentional difference in the community sets. We observed that Jaccard index and mutual information could capture the differences created by mislabeled nodes, but changing weights did not affect their similarity values as they are primarily sensitive only to changes in the node labels rather than changes in the weights. In contrast, NCSSI demonstrated a distinct advantage in capturing the difference when the weights of intervened nodes change. NCSSI effectively considered both the label and weight information in the communities, enabling it to capture the impact of changes in weights in measuring the similarity between community sets. While there has been considerable focus on integrating weights into community detection methods [42, 47, 48], little attention has been given to incorporating edge weights into similarity measurement methods for community sets. This experiment highlights the distinct advantage of NCSSI over mutual information and the Jaccard index methods, where both label and weight variations are related factors in measuring similarity between community sets.

We employed NCSSI on the New York Yellow Taxi flows dataset as a real-life case study. We computed similarity scores for each season between each pair of consecutive years. Different factors such as technological advancements [49], economic development [50], political and environmental factors [51–53] can influence human communities. The pace of these changes is insufficient [54] to expect major variations in human community structures within two consecutive years. However, we observed a relatively high rate of change in the Jaccard index and mutual information results. On the other hand, NCSSI demonstrated a more stable similarity degree between the communities, except after the outbreak of COVID-19 in early spring 2020.

It is common that entities in complex systems change (established or annihilated) [55]. For example, people might die or be born in social networks, or through policy changes, the number of census tracts might differ over time. While in most cases, community detection is conducted for the same set of nodes [1], there are cases where the number of nodes changes (e.g., social networks, friendship, or professional networks). The NCSSI can handle this without requiring any modifications. To elaborate, when a node is added or removed from one community set to another, the corresponding edge weights connected to that node will also change (added or removed). As NCSSI effectively captures the transformation of community sets by pairing similar communities and calculating the edit costs associated with node membership and their edge weight changes, node additions or removals and consequently edge weights additions and removals will directly impact the NCSSI result. Hence, NCSSI is well-equipped to handle cases where the networks being compared have different numbers of nodes.

In cases when the NCSSI yields high similarity scores for two sets of communities, it is important to recognize that the underlying community set topologies could still differ. It is crucial to emphasize that the NCSSI does not assess the overall similarity of network (graph) structures. Instead, its focus is specifically on quantifying the similarity between the structures of the community sets themselves.

In conclusion, NCSSI offers a new approach to better measure and assess community similarities. By considering both node labels and edge weight variations in communities, NCSSI enhances the accuracy of the similarity measurement between community sets. It also allows measuring similarity between communities with different number of nodes. This capability is particularly valuable where the community sets are of varying sizes and compositions. It is worth noting that NCSSI incorporates the principles of edge importance, weight awareness, and edge submodularity, introduced in [56] primarily for graph similarity measurements, since it effectively captures the impact of edge changes only on the edges associated with nodes that change their communities, acknowledges the significance of edge weights in weighted graphs, and accounts for the varying importance of edge changes in dense and sparse networks.

References

  1. 1. Fortunato S. Community detection in graphs. Phys Rep. 2010;486(3–5):75–174.
  2. 2. Meilă M. Comparing clusterings—an information based distance. J Multivar Anal. 2007;98(5):873–95.
  3. 3. Mirkin B. Mathematical classification and clustering. Vol. 11. Springer Science & Business Media; 1996.
  4. 4. Rand WM. Objective criteria for the evaluation of clustering methods. J Am Stat Assoc. 1971;66(336):846–50.
  5. 5. Dongen S. Performance criteria for graph clustering and Markov cluster experiments. CWI (Centre for Mathematics and Computer Science); 2000.
  6. 6. Gustafsson M, Hörnquist M, Lombardi A. Comparison and validation of community structures in complex networks. Physica A: Statistical Mechanics and its Applications. 2006;367:559–76.
  7. 7. Gusfield D. Partition-distance: A problem and class of perfect graphs arising in clustering. Inf Process Lett. 2002;82(3):159–64.
  8. 8. Wallace DL. A method for comparing two hierarchical clusterings: comment. J Am Stat Assoc. 1983;78(383):569–76.
  9. 9. Fan Y, Li M, Zhang P, Wu J, Di Z. Accuracy and precision of methods for community identification in weighted networks. Physica A: Statistical Mechanics and its Applications. 2007;377(1):363–72.
  10. 10. Zhang P, Li M, Wu J, Di Z, Fan Y. The analysis and dissimilarity comparison of community structure. Physica A: Statistical Mechanics and its Applications. 2006;367:577–85.
  11. 11. Traud AL, Kelsic ED, Mucha PJ, Porter MA. Community structure in online collegiate social networks. arXiv preprint arXiv:. 2008.
  12. 12. Hu Y, Chen H, Zhang P, Li M, Di Z, Fan Y. Comparative definition of community and corresponding identifying algorithm. Phys Rev E. 2008;78(2):026121. pmid:18850911
  13. 13. Lancichinetti A, Fortunato S. Community detection algorithms: a comparative analysis. Phys Rev E. 2009;80(5):056117. pmid:20365053
  14. 14. Duan D, Li Y, Jin Y, Lu Z. Community mining on dynamic weighted directed graphs. In: Proceedings of the 1st ACM international workshop on Complex networks meet information & knowledge management. 2009. p. 11–8.
  15. 15. Danon L, Diaz-Guilera A, Duch J, Arenas A. Comparing community structure identification. Journal of statistical mechanics: Theory and experiment. 2005;2005(09):P09008.
  16. 16. Saoub KR. Graph theory: an introduction to proofs, algorithms, and applications. CRC Press; 2021.
  17. 17. Barabasi AL, Oltvai ZN. Network biology: understanding the cell’s functional organization. Nat Rev Genet. 2004;5(2):101–13. pmid:14735121
  18. 18. Bast H, Delling D, Goldberg A, Müller-Hannemann M, Pajor T, Sanders P, et al. Route planning in transportation networks. Algorithm engineering: Selected results and surveys. 2016;19–80.
  19. 19. Palla G, Derényi I, Farkas I, Vicsek T. Uncovering the overlapping community structure of complex networks in nature and society. Nature. 2005;435(7043):814–8. pmid:15944704
  20. 20. Kemp C, Tenenbaum JB. The discovery of structural form. Proceedings of the National Academy of Sciences. 2008;105(31):10687–92. pmid:18669663
  21. 21. Erdős P, Rényi A. On the evolution of random graphs. Publ Math Inst Hung Acad Sci. 1960;5(1):17–60.
  22. 22. Girvan M, Newman MEJ. Community structure in social and biological networks. Proceedings of the national academy of sciences. 2002;99(12):7821–6. pmid:12060727
  23. 23. Zheng Y, Capra L, Wolfson O, Yang H. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST). 2014;5(3):1–55.
  24. 24. Kernighan BW, Lin S. An efficient heuristic procedure for partitioning graphs. The Bell system technical journal. 1970;49(2):291–307.
  25. 25. Barnes ER. An algorithm for partitioning the nodes of a graph. SIAM Journal on Algebraic Discrete Methods. 1982;3(4):541–50.
  26. 26. Hastie T, Tibshirani R, Friedman JH, Friedman JH. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. Springer; 2009.
    • 27. Newman MEJ, Girvan M. Finding and evaluating community structure in networks. Phys Rev E. 2004;69(2):026113. pmid:14995526
    • 28. Clauset A, Newman MEJ, Moore C. Finding community structure in very large networks. Phys Rev E. 2004;70(6):066111. pmid:15697438
    • 29. Newman MEJ. Fast algorithm for detecting community structure in networks. Phys Rev E. 2004;69(6):066133. pmid:15244693
    • 30. Newman MEJ. Analysis of weighted networks. Phys Rev E. 2004;70(5):056131. pmid:15600716
    • 31. Donetti L, Munoz MA. Detecting network communities: a new systematic and efficient algorithm. Journal of Statistical Mechanics: Theory and Experiment. 2004;2004(10):P10012.
    • 32. Newman MEJ, Leicht EA. Mixture models and exploratory analysis in networks. Proceedings of the National Academy of Sciences. 2007;104(23):9564–9. pmid:17525150
    • 33. Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:160902907. 2016.
    • 34. Jin D, Yu Z, Jiao P, Pan S, He D, Wu J, et al. A survey of community detection approaches: From statistical modeling to deep learning. IEEE Trans Knowl Data Eng. 2021.
    • 35. Huang J, Sun H, Han J, Deng H, Sun Y, Liu Y. Shrink: a structural clustering algorithm for detecting hierarchical communities in networks. In: Proceedings of the 19th ACM international conference on Information and knowledge management. 2010. p. 219–28.
    • 36. Yang T, Chi Y, Zhu S, Gong Y, Jin R. Detecting communities and their evolutions in dynamic social networks—a Bayesian approach. Mach Learn. 2011;82:157–89.
    • 37. Taya F, de Souza J, Thakor N V, Bezerianos A. Comparison method for community detection on brain networks from neuroimaging data. Appl Netw Sci. 2016;1(1):1–20.
    • 38. Meilă M, Heckerman D. An experimental comparison of model-based clustering methods. Mach Learn. 2001;42(1):9–29.
    • 39. MacKay DJC. Information theory, inference and learning algorithms. Cambridge university press; 2003.
      • 40. Condon A, Karp RM. Algorithms for graph partitioning on the planted partition model. Random Struct Algorithms. 2001;18(2):116–40.
      • 41. Lancichinetti A, Fortunato S, Radicchi F. Benchmark graphs for testing community detection algorithms. Phys Rev E. 2008;78(4):046110. pmid:18999496
      • 42. Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E. Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment. 2008;2008(10):P10008.
      • 43. Raghavan UN, Albert R, Kumara S. Near linear time algorithm to detect community structures in large-scale networks. Phys Rev E. 2007;76(3):036106. pmid:17930305
      • 44. Cordasco G, Gargano L. Community detection via semi-synchronous label propagation algorithms. In: 2010 IEEE international workshop on: business applications of social network analysis (BASNA). IEEE; 2010. p. 1–8.
      • 45. New York Taxi & Limousine Commission. New York Taxi & Limousine Commission. 2023 [cited 2023 Mar 9]. TLC Trip Record Data. Available from: https://www.nyc.gov/site/tlc/about/data.page.
        • 46. OpenStreetMap Wiki. Basemap [Internet]. 2022 [cited 2023 Aug 29]. Available from: https://wiki.openstreetmap.org/w/index.php?title=Basemap&oldid=2371479.
          • 47. Meo P De, Ferrara E, Fiumara G, Provetti A. Generalized Louvain method for community detection in large networks. In: 2011 11th International Conference on Intelligent Systems Design and Applications. 2011. p. 88–93.
            • 48. Steinhaeuser K, Chawla N V. Community detection in a large real-world social network. In: Social computing, behavioral modeling, and prediction. Springer; 2008. p. 168–75.
              • 49. Surry DW, Baker FW III. The co‐dependent relationship of technology and communities. British Journal of Educational Technology. 2016;47(1):13–28.
              • 50. Summers GF, Branch K. Economic development and community social change. Annu Rev Sociol. 1984;10(1):141–66.
              • 51. Fischer AP, Paveglio T, Carroll M, Murphy D, Brenkert-Smith H. Assessing social vulnerability to climate change in human communities near public forests and grasslands: A framework for resource managers and planners. J For. 2013;111(5):357–65.
              • 52. Duerden F. Translating climate change impacts at the community level. Arctic. 2004;204–12.
              • 53. Schoennagel T, Balch JK, Brenkert-Smith H, Dennison PE, Harvey BJ, Krawchuk MA, et al. Adapt to more wildfire in western North American forests as climate changes. Proceedings of the National Academy of Sciences. 2017;114(18):4582–90. pmid:28416662
              • 54. Marsden P V, Smith TW, Hout M. Tracking US social change over a half-century: The general social survey at fifty. Annu Rev Sociol. 2020;46:109–34.
              • 55. Negre E, Missaoui R, Vaillancourt J. Predicting a social network structure once a node is deleted. In: 2011 International Conference on Advances in Social Networks Analysis and Mining. IEEE; 2011. p. 297–304.
              • 56. Koutra D, Shah N, Vogelstein JT, Gallagher B, Faloutsos C. Deltacon: Principled massive-graph similarity function with attribution. ACM Transactions on Knowledge Discovery from Data (TKDD). 2016;10(3):1–43.