Preliminary analysis

For this work, we considered data from two subjects from the study described in [2]. In this study patients were being showed a sentence followed by an arrangement of symbols two symbols (symbol1 and symbol2). The symbols that were used were“+”, “$”, and “*”. The sentences were of the form “it is true that symbol1 is above/below symbol2” (case 2) or of the form “it is not true that symbol1 is above/below symbol2” (case 3) followed by an arrangement of the symbols relative to the sentence (for instance ). Eventually, the subjects had to answer the question presented in the first stimulus. The first stimulus was presented to the subjects for 4 seconds, followed by a blank screen for another 4 seconds. The second stimulus was presented for a maximum time of 4 seconds; this part ended when the subject responded to whether the sentence was appropriately describing the arrangements. A relaxation period of 15 seconds was the last part of the trial before the next trial began. Thus, the total duration of the trial was approximately 27 seconds. fMRI data were collected every 0.5 seconds. 6 subjects were used to conduct this study where each subject went under 54 trials as described above (with 8 fixations periods interpolated–case 1). The Carnegie Mellon Institutional Review Board (IRB) approved this research and collection of data for this study. Patient records/information was anonymized and de-identified prior to analysis.

Data collected from this study were processed and stored as.mat files that are ready to being processed by MATLAB (the MathWorks, Inc., Natick, MA, US.). No additional smoothing took place. For each subject, the data consisted of time series corresponding to the fMRI activity of each voxel. Therefore for each subject, the dimension of the data was number_of_trials × number_of_voxels × duration_each_trial.

In order to assess the functional connectivity of the brain we used correlation between voxels. In specific, we constructed the affinity matrix of the voxels as follows: For each pair of voxels X and Y we calculated the correlation between them as where T is the total number of observations and and are the averages of the signals. The adjacency matrix M was constructed as follows: where the threshold was chosen adaptively in order to keep the 10% of the most significant connections. Building adjacency matrices based on the biomedical data using different interdependence measures is a well-examined technique [15, 16, 17]. The threshold for fathoming weights was set under the perspective to try to keep the most meaningful edges based on their respective histogram [18]. Correlation though suffers from flaws just as the linearity assumption [19] but analysis of these and how they can be avoided goes beyond the scope of this paper.

There were few analyses that took place prior to clustering in order to assess the validity of the study.

1. We constructed three correlation matrices for each case and each subject (a total of 18 matrices) by calculating the correlation coefficients for each trial and averaging over the number of trials for each state. An initial difficulty that was raised when trying to conduct multi-subject analysis was the fact that the brains of the subjects were different. Despite this we followed the philosophy of [3] by averaging the activity in each ROI. In turn, we were interested if connectivity could be related to the different states and subjects. We performed the Network-based statistics with t-statistics threshold 3 and level of significance 0.05. Our design matrix for the Generalized Linear Model [6] was used in an effort to discriminate a) between the different cases and b) between different subjects. In a total of 5000 evaluations of the algorithm there were no significant results and therefore we concluded that there is no statistical difference between the matrices in terms of subjects and cases. Therefore we restricted our analysis by choosing data for each subject and for an average of trials in each subject instead of conducting a multi-subject and multi-trial analysis. In order to quantify differences, each correlation matrix was chosen using data from 3 time intervals: a) the first 4 for seconds where each subject was presented with a question b) the consequent 4 seconds where each subject was given a blank screen, and c) the last four seconds was exposed to an arrangement of symbols. Therefore we ended up with 6 matrices for each of these three segments.
2. We applied state-of-art algorithms in order to evaluate the modularity of our networks (defined later in the text). In specific, we executed the Louvain algorithm [20] for the correlation matrices that we considered for analysis and we found [mean 0.2963, standard deviation 0.0380]. These results raised questions as to whether our connectivity networks were hiding any form of organization. In fact this point discouraged us from investigating the network organization under co-affinity matrices where partitions with maximum modularity are being used in order to debunk hierarchical organization of networks. For additional affirmation of our results, we compared statistically our graphs with hierarchically nested random graphs by using the statistical z-score for average modularity [8]. Our graphs failed the test with a threshold of z_t = 2.3627 and confidence level of 1%. However, as will shall see later in the paper, this become the incentive to apply a “complementary” approach (using our algorithm) where instead of finding partitions with high modularity, we seek partitions with low modularity and we take the probability of two nodes being in the same cluster over all such partitions.

Regularity partitioning algorithm

We continue by building the theoretical framework of the regularity partitioning algorithm. We begin by introducing the definition of a pseudo-random or ϵ–regular pair (or simply regular). We will use |X| to denote the cardinality of an arbitrary finite set |X|. We will use the notation e(X,Y) to denote the number of edges between sets X and Y. A bipartite graph with vertex clusters X and Y respectively we is said to be ϵ–regular [21, 22] if for every A ⊂ X and B ⊂ Y, with sizes |A| > ϵ |X| and |B| > ϵ |Y|, it holds that where d(X,Y) (and d(A,B) respectively) stands for the fraction of the edges between the sets divided by the product of their cardinalities; of the corresponding sets, i.e., the quantity .

This definition is correlated with distribution of the number of edges between the pair.

Using Fact 1 (see appendix), by choosing a small ϵ and having in mind that density cannot be greater than 1, we can quantify the probability of edges connecting clusters of bipartite graphs. Therefore, if we can separate a graph into such randomly behaving pairs, we can have a probabilistic framework of the distribution of the edges and therefore of the existence of subgraphs in the graph. Formally, a partition of graph vertex set V of an arbitrary graph in k is clusters is called ϵ–regular (or simply regular) if it is equitable (every cluster has the same size) except perhaps a “junk” cluster P₀ and:

1. |P₀| < ϵ|V|
2. All but most ϵ k² of the pairs (P_I,P_j),1 ≤ i < j ≤ k are ϵ–regular.

The existence of such a partition for any graph has been proved in [9]. We mentioned at the introduction the impediments regarding the computationally efficiency of a potential algorithm that could decide if a partition of the vertex set of the graph can be ϵ–regular. Without diving into many technical details (that are beyond the scope of this paper), an efficient algorithm for obtaining such a partition can be found in the next lines. Note that the algorithm takes as input a) the graph under consideration, b) the ϵ parameter, c) the l parameter that defines the initial number of clusters, and d) the h parameter that restricts the clusters that are keep getting refined from having cardinality less than h.

The regularity partitioning algorithm for the graph G(V,E) and inputs ϵ, l, h is described as follows:

1. Obtain an initial partition by dividing the vertex set V into l + 1 clusters P₀,P₁,P₂,…,P_l. The cluster P₀ is used in order to certify that the clusters P_i, 1 ≤ i ≤ l will have equal sizes. Set current refinement number k₁ = l.
2. If the size of the clusters becomes less than h then halt. Otherwise use the black-box algorithm to argue about whether pairs (P_i,P_j) are ϵ–regular If they are not, then return certificates.
3. If there are at most ϵ–regular pairs, then halt and the current partition is an ϵ–regular partition and output that partition.
4. Otherwise, use the black box algorithm for refinement the partition further. This algorithm will result in a new partition with at most 1 + l k_i classes.
5. Set k_i+1 = l k_i as the new refinement number, Q as the new partition under consideration, and i = i + 1 and repeat from step 2.

As the reader might observe, the algorithm keeps refining the obtained partition until the resulting partition is ϵ–regular.

The black-box algorithm for proving if a pair is ϵ–regular is defined in [14]. The outline of the algorithm is the following. For a bipartite graph, if there exists a set with significant deviation then three possibilities can take place: 1) the average degree of the graph will be small 2) there exists a set such that a small number of vertices have degree that deviates from the average degree in a specific way and 3) there are subsets in A and B such that their relative density exceeds a specific number.

The black-box algorithm for refining further a partition is also described in [14]. The outline of the algorithm is as follows: For the pairs that are not ϵ–regular return the certificates as above. Then partition the initial set V as follows: each certificate A should contain sets of vertices where m is and l is the initial refinement number k₁.

When partitioning the collection of vertices in the initial partition and in the consequent partitions during the black-box refinement algorithm, we partition sequentially, i.e., having the vertices labeled as 1,…,|V| we start from 1, we select a set of size indicated by the current step of the algorithm, then we select the next set and so on. This way enables us to satisfy the disjoint property in the refinement algorithm and speeds up the computational time. In addition, a technical point worth mentioning is that each partition is associated with a “junk” cluster (usually indexed with 0) where the algorithm puts vertices that are not included in the partition. This can be viewed as a technical manipulation of the algorithm in order to keep the size of the clusters equal.

As we mentioned before, the aforementioned fact enables us to quantify the number of edges between ϵ–regular pairs. An example of this is a relevant result regarding the lower bound of the number of triangles. Following the spirit of [23], for any triple of subsets of the vertex set V, with (X,Y), (Y,Z), and (Z,X) are ϵ–regular (with densities α,β, and γ respectively), we calculate the number of triangles as:

The previous formula tells us that in case of regular pairs, the number can be approximated from below the number of triangles in totally random tripartite graph. In the following experiment we count the number of triangles by randomly choosing 3 clusters (with uniform probability) from the regularity partition and calculating the lowest bound on the number of triangles out of 10 such repetitions.

Finally, the index of a partition P = V₀ ∪ V₁ ∪ … ∪ V_k can be defined as

The index of a partition can be viewed in a probabilistic way as the expectation of edges between the clusters of the partitions [24]. This quantity can be characterized as the energy of the partition in terms of how many edges exist in a pseudorandom graph and can be used for comparison between different partitions. An interesting fact is that when refining an ϵ–regular partition the index increases by a quantity that is a function of ϵ [21]. Therefore, the partition obtained in each step is a partition that has low modularity but increased index from the partition of the previous step. Interestingly enough, since the index is bounded above by 1, this idea leads to the proof of convergence of the algorithm. In addition, the algorithm allows us naturally to collect a family of partitions that have decreased modularity.

Modularity and organization

An ϵ–regular partition can be seen as a collection of almost random bipartite graphs that approximate a network. It is therefore expected that the resulting partitioning of the algorithm will suffer from decreased modularity. In the work of Sales-Pardo et al. [8], the authors define a partition space of all partitions with maximum (local) modularity. In specific they consider the co-affinity between node i and j as the probability of these nodes ending up in the same cluster of a partition that has a local maximum modularity. We remind the reader that the modularity of a network partition P is defined as where L is the total number of edges, l_i is the total number of nodes within cluster i, d_i is the sum of degrees of all nodes inside cluster i, and the sum is over all m clusters of the partition.

Our approach complements the aforementioned approach as we define the probability of node i and j to belong to the same cluster over partitions with local minimum modularity. These partitions with decreased modularity stem from running our regularity partitioning algorithm. We construct the modified co-affinity matrix and we seek to develop a diagonal-block structure that will provide us with information of how the network is constructed in blocks of clusters that have high edge density between them and low edge density within. After constructing the modified co-affinity matrix, we use the reverse Cuthill-McKee algorithm [25] in order to move larger values to the diagonal and obtain a better perspective of the organization of the nodes.

One major application that stems from obtaining a regularity partition is the reduction of the number of nodes. In particular, suppose that we obtain a partition of the graph G(V,E) with k clusters and an exceptional set; i.e., V = V₀ ∪ V₁ ∪ … ∪ V_k where (V_i,V_j), i,j ∈ 1,…,k are regular pairs. Then the reduced graph of G is the graph with nodes that correspond to each cluster and edges take place only between regular pairs with density above d. It has been proved (we refer the reader to the Appendix for the corresponding theorem) that every graph properties of the original graph are mitigated to the reduced graph. Therefore, by reducing the data using regularity clustering allows us, up to a level of approximation, to contain all the information of the original data. In practice, when it is the case that ϵ is significantly small, we consider edges in the reduced graph between all the pairs and not only the pairs with density above d [14].

The final step after reducing the dimensionality of the network data is to apply well-known algorithms to the reduced data. An immediate advantage of this is from the computationally load efficiency perspective. In clustering techniques such as the Ncut algorithm [26] the main steps in obtaining the partition lie in computing the affinity matrix and, in turn, computing the eigenvalue decomposition. This can be extremely intensive for large datasets and therefore the advantage of our algorithm from that aspect is eminent. Using this approach, our philosophy was the following: We reduced the data after applying a regularity-partitioning algorithm by building the corresponding reduced graph. In turn, we applied the Ncut algorithm in order to obtain a clustering of the graph. We mapped this clustering to the original data and, eventually, we distributed the remaining points of the junk cluster to the closest clusters in order to obtain a final partition of all the vertices.

As we mentioned in the first section, we wish to compare this approach to the direct approach that applies the Ncut algorithm to the original data. As far as measuring the accuracy between the different approaches, a common measure used in the bibliography is the one that compares the ground truth labels of the data and the resulting labels obtained from the partitioning algorithm. Specifically, the accuracy is defined as where n is the number of data points considered, y_i represents the truth label for point i, and c_i is the obtained cluster label for point i. The function σ equals is if its arguments are equal and 0 otherwise. The function map is a function that permutes the cluster labels to the points such that an optimal match is attained. This is usually executed via the Hungarian method [27].

A major problem in our case is the fact there is no ground truth for evaluating the accuracy of the clustering. Reducing the data and applying clustering techniques has been proved to work with better or similar accuracy than the application of the clustering techniques to the original data [14]. In our work we considered as ground truth labels the regions of interest identified in the experiment [2].

Regularity

We prove the following fact regarding the probability of a vertex having a neighbor in a regular pair. We will use the notation e(a,Y) to denote the number of edges between as single vertex a and a set Y.

Reduced graph

Regarding reducing the data, consider a graph R(V,E) and an integer t. Let R(t) be the graph obtained from R by replacing each vertex in V by a set V_x of x independent vertices, and joining u ∈ V_x and v ∈ V_y if and only if (x,y) ∈ E.

We used the following theorem that shows us how to use the reduced graph of an arbitrary graph.

Index of a partition

We will use X × Y to denote the Cartesian product between two arbitrary sets X and Y. The characteristic function χ(A) of a set takes the value 1 when the element under consideration belongs to A and 0 otherwise.

As we mentioned in Fact 1, the density d(X,Y) between two clusters X,Y can be viewed as the union of the probabilistic events of the edges between the two clusters:

Similarly, if we partition the sets X and Y into X₁, X₂ and Y₁, Y₂ then we can write the Cartesian product as (X₁ × Y₁) ∪ (X₁ × Y₂) ∪ (X₂ × Y₁) ∪ (X₂ × Y₂) and therefore we can define densities on each part. For example for X₁ × Y₁, we can define

This can be extended to a partition of a arbitrary size trivially. Consequently, we can see that each density corresponds to a pair of clusters can be viewed as a probability measure of the edges between the pair. Now we can define the function f: X × Y → R as e(X,Y). Analogously, we define this function restricted on refinement P with k clusters where χ is the characteristic function of the set and refers to the X_i and Y_j. Now a crucial point is to consider the 2-norm of this function in the space of the discrete square integrable functions. The square norm of this function restricted in the partition P is where ρ is defined as before. This quantity is exactly the expectation of the edges in this space of functions and corresponds to the index of the partition.