2.1 Bayesian model averaging for clustering

Consider a quantity of interest Δ which is present in every model across a set of candidate models for a given analysis. Given data Y with dimension D, consider a set of posterior estimates Δ_m, m = 1, …, M, each obtained from a corresponding model . The BMA framework provides a weighted average of these estimates, given by (1) where is the posterior probability of model , given by (2) Here is the prior probability for each model, and is the marginal likelihood for each model (also called the model evidence) [7]. As is common in BMA applications, here we assign priors to give equal weight to each model, with . Alternative approaches could be used for assigning prior probability for each model, as addressed in the Discussion.

Following Russell et al. [8], we consider the pairwise similarity matrix as a common property Δ which is present across all clustering algorithms of interest. The similarity matrix S^m of pairwise co-assignment probabilities for any clustering model will have dimensions N × N. Since clustering solutions are combined at the level of pairwise co-allocation probabilities via similarity matrices, this has the benefit of avoiding any issues regarding alignment of cluster labels across the different models. Each element s_ij of the similarity matrix represents the probability that data points i and j belong to the same cluster g_k∀k = 1, …, K_m where K_m is the total number of clusters in model : (3) where g_k is the kth cluster, and indicates the probability that point i is a member of cluster g_k in model [11]. Here Δ_m = S^m = {s_ij}, i, j = 1, …, N. For ‘hard’ clustering methods such as k-means or agglomerative hierarchical clustering, these pairwise probabilities will be 0 or 1, while for ‘soft’ clustering methods such as a Gaussian mixture model, these pairwise probabilities can take any value between 0 and 1. To represent each clustering solution as a pairwise similarity matrix S^m, we can use the N × K_m matrix A^m of cluster allocation probabilities where each element contains the probability that point i is allocated to cluster k under model . We calculate the similarity matrix for each model by multiplying A^m by its transpose, and setting the diagonal to 1 [8]: (4)

2.2 Approximating posterior model probability with clustering internal validation indices

As introduced above, BMA has previously been implemented within the model class of Gaussian mixture models, by calculating an element-wise weighted average across the similarity matrices representing each set of clustering results [8]. These authors weighted results from each mixture model according to an approximation of posterior model probability based on the BIC. Assuming equal prior probability for each candidate model, this is equivalent to weighting each model by its adjusted marginal likelihood as a proportion of the sum of the adjusted marginal likelihoods across all candidate models: (5) where (6) Here is the likelihood of the data given the model, κ_m is the number of estimated model parameters for the model, and N is the number of observations [12]. This is the negative of the usual construction of the BIC, and a larger number of model parameters κ_m will result in a smaller estimate for the approximated posterior model probability of model . The BIC has a theoretically established use for estimating marginal likelihood and posterior model probability in the context of Gaussian mixture models [9]. It can be seen from Eqs 5 and 6 that the weighting method used by Russell et al. is constructed to recover an estimate for the likelihood [8]. From Eq 6, the likelihood is assumed to be a multivariate Gaussian mixture, calculated as: (7) where μ is a D-dimensional vector of means, π₁, …, π_K are the mixing probabilities used to weight each component distribution, K is the selected number of mixture components, Σ is a D × D covariance matrix, |Σ| represents the determinant of Σ, and is a multivariate Gaussian density given by (8)

While ideally we would like to use a measure such as the BIC with strong theoretical support for approximating the marginal likelihood to weight each model, the BIC is not viable for our application of weighting solutions generated from multiple classes of clustering algorithm. The BIC is theoretically supported for estimating the posterior model probability for GMMs, but in practice the BIC has a number of shortcomings for the purpose of estimating posterior model probability in the context of Bayesian model averaging for clustering solutions generated by multiple algorithms. For example, three considerations are as follows. First, BIC scores are not able to be directly compared across multiple classes of clustering algorithm, and are not able to be generated at all for some classes of clustering algorithm without a likelihood term, such as hierarchical clustering. Second, the exponentiation step in Eq 5 required to estimate the marginal likelihood from the BIC tends to result in a large majority of the overall weight being assigned to one model. Some evidence suggests that in this way the BIC works well for model selection (assigning all weight to a single model), but not as well for model averaging [13, 14]. Third, there are known instances where the BIC is not a good reflection of clustering analytic objectives. For example, the BIC has well documented difficulties for model selection in high dimensional settings [15], including a tendency towards underfitting and selecting overly parsimonious mixture models with too few mixture components [16, 17].

Instead of the BIC, we can consider cluster internal validation indices as a set of measures which offer methods for assessing model quality and approximating posterior model probability across clustering algorithms with different constructions and objective functions [18]. CIVIs are typically developed to reflect common traits of clustering analytic objectives shared across algorithms including compactness, separation or inter-cluster density for a particular clustering of a dataset. Compactness describes how closely related the data points are within a cluster, and is typically measured by within-cluster variance or sum of squared distances of all points from their respective cluster centres. Separation describes how distinct clusters are from each other, and is often measured by the distances between cluster centres or minimum pairwise distances between points across clusters [19]. Similar to cluster separation, the goal for inter-cluster density is that the density of points in the area between clusters is low in comparison with the density within the considered clusters [20]. The BIC can be applied as a CIVI, for purposes including choosing a suitable number of clusters for a finite mixture model. From Eqs 6–8 it can be seen that in the context of Gaussian mixture models, the BIC is driven by a ratio of within-cluster variance (compactness) to overall variance.

Internal validation indices are commonly interpreted in a way that is analogous to the marginal likelihood, being used to make some judgement about model quality or goodness of fit in order to decide between multiple candidate models with differing numbers of cluster K_m. There are established parallels between different CIVIs and objective functions, loss functions and likelihoods for clustering algorithms. Some CIVIs have similar structures to the objective functions of algorithms for which they were developed and will tend to preference results generated by those algorithms [21]. For instance, the Xie-Beni index has a clear link to the objective function for the Fuzzy C-Means algorithm [22]. Other indices are developed to reflect more general analytical objectives that are common across the likelihoods or objective functions for many clustering algorithms, such as the Calinski-Harabasz index [23], or the S_Dbw [20], among others [18]. CIVIs have been used as loss functions for clustering with neural networks [24], and for measuring and comparing model quality across different clustering algorithms [25, 26].

While it would be preferable to start from an estimation of the marginal likelihood for each model to approximate posterior model probability, in this application this is not viable. Instead we take the approach of starting from an internal validation index to substitute for the marginal likelihood term, and building an approximation for the posterior model probability to weight each candidate clustering solution.

We acknowledge that the process of selecting an appropriate CIVI for model weighting opens the door to myriad candidate validation indices, from which the analyst must choose an appropriate measure to suit the goals of their clustering analysis. We view this as a strength and a designed feature of our method as it does not automate the choice of an appropriate measure to weight models across all scenarios, and instead requires the analyst to consider and choose a weighting measure that is appropriate for the clustering analytic objectives of their application. Just as the analyst must make reasoned and considered decisions about preparing data, choosing appropriate clustering algorithms, and choosing appropriate numbers of clusters, a CIVI should be chosen that is appropriate for weighting each model in a given analysis. In this paper we make recommendations regarding two indices which are likely to perform well for approximating clustering posterior model probability in many common applications—the Calinski-Harabasz (CH) and S_Dbw indices [20, 23]. These two indices are well supported with evidence regarding their utility for comparing model quality across solutions generated from different clustering algorithms [25], and their robustness to different challenging features of clustering data [27]. Both of these were developed as algorithm-independent indices, reflecting general clustering analytic objectives such as cluster compactness, cluster separation, and inter-cluster density, and reducing bias towards any one class of algorithm. We address some caveats and limitations of these indices in the Discussion.

Similarly to the BIC which is driven by a ratio of cluster compactness to overall variance for GMMs, the CH index is an internal validity measure representing a ratio of cluster separation to compactness, calculated with a ratio of between-cluster sums of squares to within-cluster sums of squares, penalised by the number of clusters in the model [23]. This index is calculated as: (9) where n_k is the number of observations allocated to cluster g_k, d(x, y) is the distance between x and y, c_k is the centroid of g_k, and x ∈ g_k are the data points allocated to cluster g_k. Higher CH scores indicate better internal clustering validity, with more separated and compact clusters.

The S_Dbw index is calculated as the sum of an intra-cluster variance term Scat(K) that measures cluster compactness, and a density term Dens_bw(K) that measures inter-cluster density: (10)

The intra-cluster variance term Scat(K) is defined as: (11) where σ(C_k) is the variance of cluster C_k and σ(D) is the variance of the dataset. The inter-cluster density term Dens_bw(K) is defined as: (12) where u_kj is the mid-point between c_k and c_j. Dens_bw represents a ratio of inter-cluster density to within cluster density, with lower values indicating better separation between clusters. Lower S_Dbw scores indicate better internal clustering validity, with more compact and well-separated clusters.

Having chosen a CIVI to act as a weighting variable for each model, we propose the following normalised weight as an approximation for the marginal likelihood for each model: (13) where ≔ indicates ‘is defined as’. Using this approximation of the marginal likelihood in Eq 13 and setting equal prior probability for all input models , we arrive at the following approximation for posterior model probability, substituting in to Eq 2: (14)

While these are the two indices that we recommend due to evidence of their strong performance across a range of algorithms and settings, users of the clusterBMA package can select any cluster internal validation index implemented in the clusterCrit R package. Details on available cluster internal validation indices and their interpretation (e.g. whether to be maximised or minimised) are provided in a previous publication, and in documentation accompanying the package [28].

2.3 Symmetric simplex matrix factorisation for probabilistic cluster allocation

Having represented each candidate set of clustering results as a similarity matrix as in Eq 4 and having calculated normalised weights as in Eq 13, we can generate a consensus matrix C which is a posterior similarity matrix of co-assignment probabilities, using a weighted average of similarity matrices from input models S^m, m = 1, …, M. We calculate the N × N consensus matrix C as the element-wise weighted average of the similarity matrices from each candidate model, weighted by the normalised weights : (15)

We then generate final probabilistic cluster allocations based on this consensus matrix using symmetric simplex matrix factorisation (SSMF), a method developed in the context of an approximate Bayesian method for clustering [29], and applied for Bayesian distance clustering [30]. Having specified a final number of clusters K_BMA, this method can be used to factorise an N × N posterior similarity matrix, in this case the consensus matrix C, into an N × K_BMA matrix of cluster allocation probabilities resulting from this BMA pipeline, A^BMA.

For each input model, we suggest choosing the optimal number of clusters K_m based on a variety of cluster internal validation indices. To select the number of clusters for the final BMA clustering solution K_BMA, one possible heuristic is to select the largest K_m across the input models. SSMF as implemented by Duan includes an L2 regularisation step [29], which is useful for emptying redundant clusters in the final clustering results represented in A^BMA. This L2 regularisation step can result in fewer final clusters than selected for K_BMA. For instance, where a model with fewer clusters K_f is heavily weighted by relative to other models, the clusterBMA solution may contain K_f combined clusters after L2 regularisation even where a larger K_BMA is selected. Given the reduction of redundant clusters with L2 regularisation, another possible heuristic for choosing K_BMA would be to choose a larger number of clusters than the largest K_m across the input models, accommodating the possibility of different sets of sub-clusters appearing across different input models.

This method enables quantification of uncertainty for probabilistic cluster allocations. Following previous work in Bayesian clustering, we can measure uncertainty in this allocation as the probability that the estimated cluster allocation g_i for point i is not equal to the ‘true’ cluster allocation for that point, [30]. This uncertainty measure incorporates both within-model and across-model uncertainty for cluster allocation from the input candidate models.

2.4 clusterBMA overview

Here we present a high level overview of the methodological steps involved in clusterBMA. The intention is to provide a reference structure for the reader, making the detailed explanations of each individual step easier to understand in the broader context of this framework. The method implemented in clusterBMA consists of the following steps:

1. Calculate results from multiple clustering algorithms on the same dataset. These clustering solutions can be produced by any ‘hard’ (binary allocation, e.g. k-means or hierarchical clustering) or ‘soft’ (probabilistic allocation, e.g. Gaussian mixture model) clustering algorithm [19], and can each contain a varying number of clusters. Results from each model should be in the form of a N × K_m allocation matrix A^m, where N is the number of data points, k = 1, …, K indexes the clusters in the model, and m = 1, …, M indexes the input models.
2. Represent the clustering solution A^m as a N × N pairwise similarity matrix S^m.
3. Compute an approximate posterior model probability to weight each input solution, calculated as a normalised weight based on a CIVI such as the CH or S_Dbw indices [20, 23].
4. Calculate the consensus matrix C as an element-wise weighted average across the similarity matrices S^m, m = 1, …, M from (2), weighted by the approximation for posterior model probability from (3).
5. Generate a final set of averaged probabilistic cluster allocations using symmetric simplex matrix factorisation of the consensus matrix in (4), a method proposed in an approximate Bayesian clustering context [29].

An R package clusterBMA implementing this method has been developed and made available on Github [31].

3.1 Simulation study 1–methods

We designed this principal simulation study to compare the performance of clusterBMA with several other ensemble clustering algorithms, using simulated datasets with low (2), medium (10) and high (50) numbers of dimensions, and varying conditions of low, medium and high levels of separation between simulated clusters.

We generated 10 replicates of simulated datasets under 9 combinations of each level of cluster separation and number of dimensions. Simulated datasets were generated using the R package clusterGenerate, resulting in a total of 90 simulated datasets [32]. This package allows the user to simulate data from multivariate normal clusters, and easily control the degree of separation between the clusters. Each simulated dataset contained 1500 data points, with 500 points in each of 3 clusters. For each number of dimensions (2, 10 and 50) we generated 10 high separation datasets (separation value = 0.1), 10 medium separation datasets (separation value = -0.05), and 10 low separation datasets (separation value = -0.15). These separation values were chosen heuristically through trial and error, based on visual inspection of the plotted values in 2 dimensions.

For each simulated dataset, we calculated clustering solutions with k = 3 clusters using 9 clustering algorithms: Hierarchical Clustering with average linkage, using the R base package stats [33, 34]; Divisive Analysis Clustering (DIANA), using the R package cluster [35, 36]; k-means, using the R base package stats [34, 37]; Partitioning Around Medoids (PAM), using the R package cluster [35, 36]; Affinity Propagation, using the R package apcluster [38, 39]; Spectral Clustering, using the R package kernlab [40, 41]; Gaussian Mixture Model (GMM), using the R package mclust [42, 43]; Self-Organising Maps (SOM), using the R package kohonen [44]; and Fuzzy C-Means, using the R package e1071 [45, 46].

For each set of 9 clustering solutions, we combined results across these algorithms using clusterBMA, and compared its performance to four other cluster ensemble methods. We used the Calinski-Harabasz index as the CIVI for clusterBMA weighting in Eq 13, as the data were generated from multivariate normal clusters and clusters were approximately spherical. The other cluster ensemble methods included for comparison against clusterBMA were the Cluster-based Similarity Partioning Algorithm (CSPA) [47], Linkage-Based Cluster Ensembles (LCE) [48], K-modes [49], and Majority Voting [50]. These ensemble methods were applied using the R package diceR [10]. Performance for each ensemble method was assessed using the Adjusted Rand Index (ARI) to assess the degree of agreement between the combined clustering solution from each ensemble method and the true cluster labels for the simulated data [51]. The ARI was calculated using the R package pdfCluster [52].

For each dataset, we also calculated the ARI for clusterBMA using a subset of the data points which had a high probability (p > 0.8) of allocation to the final averaged clusters for each dataset. This allowed us to demonstrate a unique feature of clusterBMA relative to these other methods, where it enables probabilistic allocation to final ensemble clusters and measures model-based uncertainty arising from ambiguity or disagreement between different clustering solutions. This feature can be used to confine cluster-based inference to be made only for those points which have low model-based uncertainty, and refrain from making clustering inferences for points with a high degree of ambiguity in their cluster allocation across different input solutions.

3.2 Simulation study 1–results

Table 2 presents the mean and standard deviation for ARI scores between each ensemble solution and the true cluster labels, for 10 simulated datasets in each combination of cluster separation level (high, medium, low) and number of dimensions for the simulated dataset (2, 10, 50). Examples of 2-dimensional datasets at each level of cluster separation are visualised in Fig 1. S1 Table in S1 File presents the mean model weights assigned to each of the 9 clustering algorithms, across the combinations of separation levels and numbers of dimensions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Example scatter plots of 2-dimensional simulated datasets at each level of cluster separation.

https://doi.org/10.1371/journal.pone.0288000.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Simulation study results—ARI mean (standard deviation) across 10 simulated datasets, comparing clusterBMA with four other ensemble clustering methods.

https://doi.org/10.1371/journal.pone.0288000.t002

These results shows that clusterBMA had similar or better performance relative to the best performing alternative ensemble methods under all conditions, and substantially outperformed all competing ensemble methods for 50-dimensional simulated data with medium or low separation between clusters. For 50-dimensional simulated data with low cluster separation, performance measured by mean ARI for clusterBMA (ARI = 0.57) was 1.16 (CSPA ARI = 0.49) to 7.12 (Majority voting ARI = 0.08) times better than competing ensemble methods.

Further, when considering only points with high probability (p > 0.8) of allocation to final clusters, clusterBMA offered much higher accuracy across all datasets when confining inference to points with low model-based uncertainty in the model averaged solution. The proportion of points with (p > 0.8) of allocation to final clusters varied from 0.97 (High separation, 2 dimensions) to 0.67 (Low separation, 50 dimensions).

3.3 Simulation study 2

In the second simulation study we aimed to demonstrate the strength of clusterBMA for incorporating and accounting for the uncertainty that arises across multiple candidate models, when trying to identify cluster structure in data that may be representing overlapping or poorly separated groups. We generated three simulated datasets using the R package clusterGenerate. Each simulated dataset contained 500 data points, with 100 points in each of 5 clusters, at three levels of separation between clusters (S1 Fig in S1 File). For each simulated dataset we applied k-means, hierarchical clustering using Ward’s method, and Gaussian mixture model, selecting the number of clusters K_m = 5 for each. We combined each set of three clustering solutions using clusterBMA with K_BMA set to 5, and the Calinski-Harabasz index as the CIVI for weighting.

S5 Fig in S1 File presents the final cluster allocations for each simulated dataset generated by clusterBMA, with point size scaled according to uncertainty of cluster allocation, where larger points representing higher uncertainty of allocation to a final cluster. It is evident that as the degree of separation present between clusters in the data becomes lower, the degree of uncertainty in cluster allocations rises due to ambiguity and disagreement in clustering results across the multiple input algorithms. As real world data will typically not have clearly separated clusters, this demonstrates that in these common scenarios with messy data overlapping among possible clusters, it is valuable to use this Bayesian model averaging approach to take model-based uncertainty into account. Our method incorporates and quantifies this uncertainty, enabling cluster-based inferences that are better calibrated for this model-based source of uncertainty that is often ignored when using results from one chosen clustering algorithm.

3.4 Simulation study 3

The third simulation study demonstrates the ability of clusterBMA to average across models with differing numbers of clusters. We generated a simulated dataset contained 300 data points, with 100 points in each of 3 clusters. We calculated two clustering solutions: k-means with K = 3, and hierarchical clustering with K = 2. As above, we applied the clusterBMA pipeline to combine results from these two models, with K_BMA set to 3.

S6 Fig in S1 File displays the clustering solutions generated by each algorithm, and the combined solution from clusterBMA. From panel (c) in S6 Fig in S1 File, there is low model-based uncertainty for cluster 1, moderate model-based uncertainty for clusters 2 and 3 where the algorithms disagree on the number of clusters for these points, and high model-based uncertainty at the border of cluster 2 with cluster 1 where the algorithms disagree on the allocation of marginal points. These results demonstrate that our approach can combine clustering solutions across models with with differing numbers of clusters K_m.

4.1 Case study methods

In this section we present a case study applying clusterBMA in the scenario of clustering young people based on resting state electroencephalography data, and highlight the utility of probabilistic allocations with quantified model-based uncertainty in an applied health research setting.

4.2 Statistical analyses

This case study involved a multi-stage analysis pipeline. The first stage for clustering based on EEG frequency characteristics included automated EEG pre-processing [53], frequency decomposition with multitaper analysis [56, 57], and selection and calculation of 8 summary features in the frequency domain. The second stage included dimensionality reduction using principal component analysis, and applying three popular unsupervised clustering algorithms to this dimension-reduced data: k-means [37], hierarchical clustering using Ward’s method [33], and a Gaussian Mixture Model (GMM) [42].

We calculated results for k-means using the kmeans() function with default settings from the base package ‘stats’ in R [34]. We calculated results for hierarchical clustering using the hclust() function with method ‘ward.D2’ from the base package ‘stats’ in R. We calculated results for GMM with default settings including Euclidean distance and a diagonal covariance matrix using the R package ‘ClusterR’ [58]. For the calculation of internal validation indices for GMM results in this work, we use the crisp projection with allocation of each data point to the mixture component in which it has the highest allocation probability.

For each clustering algorithm, the optimal number of clusters K_m was selected on the basis of a number of internal validation indices which could be calculated for all three methods. Internal validation indices can be used to identify the number of clusters that creates the most compact and well-separated set of subgroups in the data [19]. Each index is calculated based on a slightly different construction, so using a selection of multiple indices can be more robust than relying on a single index. Indices used included the Dunn index [59], silhouette coefficient [60], Davies-Bouldin index [61], Calinski-Harabasz index [23], and Xie-Beni index [22]. Clustering internal validation indices were calculated using the R package clusterCrit [28].

To probabilistically combine clustering results across these three algorithms, we implemented clusterBMA using the Calinski-Harabasz index to generate weights for each model, as all of the algorithms appeared to generate clusters with approximately spherical variance in 3 dimensions, and we did not have any a priori reasons to expect strongly non-spherical clusters. Each set of cluster allocations was represented as a similarity matrix, and a normalised weight was calculated using Eqs 11 and 12. Subsequently we calculated an element-wise weighted average across the similarity matrices using these weights, producing a consensus matrix. From this consensus matrix we applied symmetric simplex matrix factorisation to generate final probabilistic cluster allocations with associated uncertainty.

4.3 Case study results

From the principal component analysis, the first three principal components were retained which together explained 80.6% of the overall variance. On the basis of the internal validation criteria introduced above, we chose to implement a 5-cluster solution in each of the three individual clustering methods. Further details on selecting the number of clusters K_m for each method are provided in a previous publication [53]. Table 2 presents the number of individuals assigned to each cluster for the three clustering algorithms, and to the final clusters generated from clusterBMA. For each algorithm, cluster labels (1–5) have been assigned by decreasing cluster size except for HC clusters 3 and 4, for which labels were switched for the sake of clearer visual comparison across plots in Fig 2. This relabeling step was applied only for the sake of visual clarity, as clusterBMA does not require cluster labels to be aligned across the candidate models. Fig 2 presents the clustering results from each algorithm, plotted according to each two-dimensional combination of the three retained principal components. This figure indicates that there is broad agreement between the 3 methods on cluster structure and allocations, with some differences particularly for allocation of individuals at the edges between larger clusters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. 2D scatter plots of individuals by principal component scores, coloured by cluster membership, K_m = 5.

Top row = k-means; middle row = HC; bottom row = GMM. Left column = PC1 v PC2; middle column = PC1 v PC3; right column = PC2 v PC3.

https://doi.org/10.1371/journal.pone.0288000.g002

Fig 3 displays heatmaps of similarity matrices representing results from each of these algorithms, and also indicates the corresponding approximate posterior model probability, acting as a normalised weight for each model calculated from Eq 13 using the CH index. These normalised weights were: 0.35 for k-means; 0.30 for hierarchical clustering; and 0.35 for GMM. Taking an element-wise weighted average of these matrices, we calculated a consensus matrix C for which a heatmap is also presented in Fig 3. Finally, we applied SSMF to the consensus matrix C with K_BMA = 5 to generate a matrix A^BMA of final cluster allocation probabilities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Heatmaps of similarity matrices for each clustering algorithm and BMA consensus matrix.

https://doi.org/10.1371/journal.pone.0288000.g003

Fig 4 presents the cluster allocations generated by clusterBMA, plotted according to each two-dimensional combination of the three retained principal components. In this plot the points are scaled according to uncertainty of cluster allocation, , with larger points representing higher uncertainty of allocation to a final cluster. These points with high uncertainty are largely at the boundaries between clusters, indicating the model-based uncertainty relating to clustering structure that would be ignored if choosing only one ‘best’ model out of the candidate algorithms. clusterBMA enables further analysis or prediction that can take this model-based uncertainty into account, which is not possible with other ensemble clustering methods. These outputs, including probabilistic allocation to averaged clusters and incorporation of model-based uncertainty, are useful for interpretation and statistical communication in the setting of applied health research and clinical practice. For instance, in scenarios where clusters might represent health phenotypes or clinical biomarkers, it is valuable for applied practitioners to understand the strength and uncertainty of allocations to clusters for the purpose of developing subsequent inferences and making assessments regarding clinical implications.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. 2D scatter plots of individuals by principal component scores, coloured by final cluster membership.

Point size is proportional to the uncertainty of cluster allocation, . Larger points have greater uncertainty.

https://doi.org/10.1371/journal.pone.0288000.g004