3.1 K-Means algorithm

K-Means is one of the partition-based clustering algorithms. For the dataset containing n data objects, the K-Means algorithm clusters these objects into k clusters. Data objects are assigned to the cluster centres, which are chosen randomly, and at the end of each iteration, the cluster centres are updated until there is no change in the centres.

The pseudocode of the K-Means algorithm is shown in Algorithm 1:

Algorithm 1 K-Means Clustering

1: Input: dataset D, the number of clusters K, maximum number of iterations itr

2: Output: K clusters and K cluster centroids

3: randomly choose K objects from D as the initial centroids C

4: while t > itr do

5:  for each object x_i in D do

6:   for each centroid C_j in C do

7:    calculate the distance from x_i to C_j

8:   end for

9:   assign x_i to the nearest centroid

10:  end for

11:  calculate new cluster centres

12: end while

3.2 Random sampling

Random sampling selects a random sample of size n from a set of size N (where n ≤ N) with the goal of ensuring that all data can be selected. Although this method is simple, it may not be suitable for uncertain datasets that require numbering during sampling. However, in data mining, where the overall data is known, the uncertainty of the entire dataset does not affect sampling, leading to better results. Random sampling is particularly suitable for large-scale data processing as a larger sample size can better represent the entire dataset.

3.3 Reservoir sampling

Reservoir sampling is a simple and unbiased random sampling technique proposed in the literature [24], which selects without replacement a random sample of size n from a set of size N. The term “reservoir” defines a storage area for storing potential candidates for the sample. The first step of the reservoir algorithm is to put the first m records of the file into a “reservoir”. Then, the record is iterated in sequence to evaluate whether the record replaces the existing record in the reservoir. If it passes the evaluation, a record is randomly selected from the reservoir for replacement. When all records are traversed, a random sample of size n can be generated. Reservoir sampling can ensure that each record is drawn with equal probability.

4.1 Concept of the algorithm

The classical K-Means algorithm begins by randomly selecting initial centroid points for clustering. However, this approach is sensitive, as the clustering results are influenced by the initial centroids, and random selection can lead to local optima [1]. Therefore, it is crucial to choose suitable initial centroids for the K-Means algorithm. Sampling, a commonly used statistical method, has been introduced into clustering. Arthur et al. [8] selects one sample at a time based on probabilities, with objects that are farther away having a higher probability of being chosen. Other references select multiple samples at once, considering the probability and incorporating the mutual distance between data objects. However, this method’s limitation is that the selection of the next sample depends on the previously selected sample, and only distance is considered for selecting outliers.

In a method that considers overall features and determines the appropriate number of samples, initial centroids are selected through pre-clustering. Yin et al. [15] shows that increasing the number of samples can significantly improve the clustering results’ quality. If the number of pre-clusters k′ ≫ k, it increases the likelihood of obtaining the global optimum.

In this paper, the SOSK-Means algorithm utilizes the weighted jump reservoir method to perform random sampling i times on the dataset, obtaining n′ samples for pre-clustering, and forming k′ clusters and k′ initial cluster centroids at each sampling. This approach considers that intra-cluster data distributions, cluster sizes, and inter-cluster distances are different. The algorithm calculates the weighted intra-cluster variance by evaluating the radius and variance of each cluster, where variance reflects the degree of dispersion of data objects from the centroid in the cluster. The SOSK-Means algorithm employs the weighted max-min distance with variance to select k clusters, reducing the chance of selecting sparse clusters with outliers. The selected k clusters have dense intra-cluster data and well-separated inter-cluster data. By calculating the centroid values of the k clusters, ik centroid values can be obtained. The best initial centroid is then selected based on the mean square error, representing the original data’s clustering centroids. During the iteration, a novel distance comparison method is used to reduce computation time.

4.2 Weighted jump researvoir sampling

From the previous section, reservoir sampling can ensure the randomness of sampling when dealing with large data of unknown size. Since each data point has the same probability of being extracted, this method can be regarded as uniform random sampling. However, if the distribution of the dataset is extremely uneven, the sample taken may not be sufficient to represent the entire dataset. If the sample is selected by iterating all the data, it increases the computational cost.

Therefore, based on the literature [25], this paper improves reservoir sampling and proposes weighted jump reservoir sampling. v_i is assumed to be a data point, and the weight of v_i is w_i (w_i > 0). The random function random(L, H) generates a uniform random number in (L, H). Given a random variable X, F_X(x) is its distribution function.

Definition 1: Assume U₁ and U₂ are random variables that obey a uniform distribution in [0, 1], , , w₁, w₂ > 0, then: (1) Therefore, the probability that the data point is selected for sampling is proportional to its weight w_i, and the probability of each data point being selected is not known before sampling. If the weight of a data point is greater than that of other data points, the closer the value of X_i is to 1, the more likely the data are to be saved. In the following, X_i represents the feature value k_i.

Definition 2: It can be seen from the mathematical induction that for α ∈ [0, 1], w_i > 0, , U_i ∈ U(0, 1), then: (2) The smallest key in R is the current threshold T. The following random variable X_w can be used to generate the appropriate exponential “jumps”: (3) Let W_i,j = w_i + w_i+1 + … + w_j(i ≤ j); when X_w satisfies the following formula, the data for v_i is obtained: (4)

The probability of jumping to read v_i from Eq (2) is as follows: (5) The size of v_i’s eigenvalues, k_i and T are compared to decide whether to put it in the pool for replacement. Then, the probability of entering the pool by Eq (2) is as follows: (6)

Definition 3: Assume that the algorithm has processed c − 1 items (c > m). T is the current threshold in the reservoir. The next item to be examined is V_c. According to Eqs (2) and (4), for each i = c, c + 1, …, n, the probability P_next(V_i) that V_i is the next item that enters the reservoir is: (7)

The two main operations of weighted jump reservoir sampling are generating weights for data and reading data by jumping and using a reservoir of size m to store the final sample candidates. The comparison of feature values can ensure that the data with greater weight are more likely to be retained. Jumping reduces the number of generated random variables. All operations can be completed by one iteration of the data set. Since the computational cost of generating a large number of random variables may be high, weighted jumping reservoir sampling improves the complexity of classical reservoir sampling.

The pseudocode of weighted jump reservoir sampling is as Algorithmic 2.

Algorithm 2 Weighted Jump Reservoir Sampling

1: Input: A population V of n weighted data

2: Output: A reservoir R of size m

3: Insert the first m items of V into R

4: for each item x_i ∈ R do

5:  Calculate a key , where u_i = random(0, 1)

6: end for

7: The smallest key k_i in R is the current threshold T

8: repeat

9:  Generate X_w according to Eq (3)

10:  Jump from the current data V_c to V_i and satisfy inequality (4); then, make V_i become V_c

11:  Let , r = random(t_w, 1), calculate

12: until the data set is processed

13: if k_i > T then

14:  Replace the data item with the minimum key T in R with item v

15:  Calculate the new T in R

16: end if

17: Calculate the new T in R

4.3 Sample-based initial cluster centroid selection scheme

The original dataset is assumed to be {x₁, x₂, …, x_n}, where n is the size of the dataset, and each data point has w-dimensional features, i.e., x_i = (x_i1, x_i2, …, x_iw). The dataset is divided into k clusters, and the centres of the data clusters are c₁, c₂, …, c_k.

After sampling the weighted jump reservoir to obtain the sample dataset, pre-clustering the dataset is equivalent to performing a cluster. The pre-clustering number k′ is much larger than the real clustering number k because the K-Means algorithm is a stochastic hill-climbing algorithm on the logarithm likelihood function space, making k′ ≫ k similar to developing more climbing paths, which increases the possibility of reaching the global optimum.

The literature [16] recommends that the value of k′ is , where μ_min is the amount of data contained in the smallest cluster, and n is the overall amount of data. However, sometimes it is difficult to know the value of n for a large amount of data, so the algorithm in this paper automatically counts n values and then uses the value of n to calculate k′.

Definition 4: The selection of the pre-clustering number k′ is: (8) where n/k is replaced by μ_min because if the value of n cannot be determined, the value cannot be accurately obtained.

At the same time, to ensure that k′ centroids can be clustered in the algorithm, the number of samples should be greater than k′; otherwise, clustering cannot be performed. Therefore, when the number of samples is larger than max(α ⋅ k, n^5/8), it is obvious that max(α ⋅ k, n^5/8) ≥ min(α ⋅ k, n^5/8).

4.4 Selecting initial cluster centroids

k’ clusters are formed by pre-clustering the sample, k clusters are selected by the maximum and minimum distances with intra-cluster variance, and then the initial centre is obtained.

(1) Cluster Radius

The cluster radius is the maximum distance between all the points and the centroid, and its value is: (9) where x_i is the object in cluster k, and d(x_i, c_k) is the Euclidean distance between the data point x_i and the centroid c_k. (2) Intra-cluster Variance

The variance is used to measure the discrete degree between data and expectation values (i.e., the mean). The intra-cluster variance is calculated as follows: (10) where M_k is the average distance of object x_i to centroid c_k, |C_k| is the number of data points in cluster k, and the intra-cluster variance reflects the degree of deviation of clusters x_i and M_k. If the intra-cluster variance is smaller, the intra-cluster objects in the cluster are denser.

After obtaining the radius and variance of each cluster, the variance within the cluster is: (11) where R is the set of all cluster radii, and the fraction in Eq (11) represents the normalization of r_k to a number in [0, 1].

(3) Weighted Max-Min Distance with Variance

After pre-clustering into k′ clusters, the cluster centres constitute the candidate set T = {t₁, t₂, …, t_k′} of the initial cluster centres, and the final set of initial cluster centroids is C = {c₁, c₂, …, c_k}. For the candidate t_k in T, the max-min distance with variance is: (12) where min{d(c_i, t_k)} is the minimum distance between the candidate t_k and the set C of selected initial centroids.

The larger min{d(c_i, t_k)} is, the greater the distance between the candidate and the set of selected initial centroids, the smaller the weighted intra-cluster variance of cluster k, and the denser the data distribution. When the value of the weighted max-min distance is larger, it means that intra-cluster data are dense and inter-cluster data are far apart, and the cluster centroid is used as the next initial cluster centroid.

Therefore, the steps to select the initial cluster centre after pre-clustering are as follows:

1. Randomly choose a candidate t_a from T as the first initial clustering centroid c₁, c₁ = t_a, and delete t_a from T.
2. Choose a candidate t_b as the next initial clustering centroid c₂, c₂ = t_b, t_b satisfies , t_k ∈ T, and delete t_b from T.
3. Repeat Step 2 until a total of k centres are chosen from the remaining candidates in T.

In the K-Means algorithm, the Euclidean distance is used to calculate the distance between data, and the data are divided into clusters. In the execution of the algorithm, it is only necessary to calculate which centre is closest to a certain point, and it is not necessary to calculate the exact Euclidean distance value from each data point to each centroid, so a quick comparison method to calculate distance is used in this paper.

Definition 5: A distance formula newDist is as follows: (13)

The Euclidean distance euclDist of the K-Means algorithm can be calculated by the following equation: (14) where w is the number of features. It is assumed that the centre point c is (c₁₁, c₁₂, …, c_1w) and the data point x is (x₁₁, x₁₂, …, x_1w). Comparing the two distance formulas, it is easy to show that newDist ≤ euclDist.

Comparing each data point and the centre point distance, the minimum distance obtained previously by bestDist and newDist can be obtained by the L2 norm of m and x. If newDist > bestDist, then euclDist > bestDist, then there is no need to calculate euclDist, which saves much calculation work. If newDist < bestDist, then calculate the Euclidean distance euclDist, compare its size with bestDist, and then directly use the L2 norm of m and x obtained when calculating newDist.

4.5 Algorithm distributed strategy

It is found in the literature that traditional clustering algorithms become ineffective in clustering large datasets [26]. MapReduce can solve this problem, but MapReduce is sensitive to iterative calculations. Each round of jobs needs to read data from the disk and write the results back to the magnetic disc file system, which greatly increases system overhead such as I/O, and iterative calculations often occur. Therefore, the K-Means algorithm needs to be optimized for distributed computing. Spark is a new generation of large-scale data processing and computing frameworks. In terms of computing efficiency, its memory-based execution is much faster than MapReduce, which reads and writes to disk multiple times. Lydia et al. [27] pointed out that experiments prove that when the data are in memory, the execution speed of Spark is 100 times faster than MapReduce, and the speed of accessing data from disk is also 10 times faster.

According to the sample dataset obtained by sampling in Section 4.2, the subsequent processing of the SOSK-Means algorithm only processes the sample dataset instead of all the data.

(a) pre-clustering

Put the RDD samples in, whitch number is k’. The progress will output an array of pre-clustering results preResults. The step is as folows:

1. Step 1: Sort the RDD samples in descending order, select the first k’ data points as the initial cluster centroids of the pre-clustering, and then broadcast the k’ centroids to all worker nodes.
2. Step 2: Define the internal function merge, which is used to accumulate the value of two key-value sequences with the same key.
3. Step 3: Perform the mapPartitions operation on the RDD example, calculate which cluster the data in each partition belongs to, and generate a key-value sequence.
4. Step 4: Perform a reduceByKey operation on each key-value sequence to summarize, use the merge function to merge the values of the same key, and calculate the global data attributes of each cluster.
5. Step 5: Calculate the sum of squares of the difference between the data in each cluster and the mean sqSums(j) and obtain key-value pairs of the form (j, sqSums(j)).
6. Step 6: Combine the same key j to obtain the global key-value array tempList1.
7. Step 7: According to tempList and tempList1, the key-value array preResults is generated to obtain k’ clusters.

(b) selecting cluster centroids

Input the RDD sample and Pre-clustering results preResults, then the array of initial cluster centroids will be output.

1. Step 1: Add the cluster centroid of the cluster with the smallest variance in preResults to centersList as the first initial centroid and delete the centre from preCenters.
2. Step 2: Calculate the weighted variance of each cluster in preResults.
3. Step 3: Do follow steps until the number of elements in centersList is more than k:
   1. (1) Perform a foreach operation on preResults and calculate the weighted max-min distance from the centre of each cluster to the centre of centersList.
   2. (2) Select the cluster centre corresponding to the largest distance to add to centersList and delete it from preResults.
4. Step 4: Calculate the error sum squares of the centroid of the group to the RDD sample.

(c) parallel K-Means clustering

This progress need input the dataset RDD, maximum number of iterations iter, threshold and the initial centroid array centerList. The updated centre array centerList will be output.

1. Step 1: Broadcast centerList to Worker node.
2. Step 2: Do follow step until the number of iterations is less than or equal to iter or the cluster centre change value is less than threshold:
   1. (1) Perform the mapPartitions operation on the RDD input, compare the distance from the centroid to the data in each partition, record the index of the centroid to which it belongs, and form a key-value sequence.
   2. (2) Perform the reduceByKey operation on the key-value sequence, merge the key-value sequence, remove the farthest and nearest data in each key, calculate the new centre point and update the centerList.

4.6 Analysis of the processing time

Assuming that the number of rows of the original dataset is n₀, the number of sampling times is m, n_s rows of data are extracted each time, and the feature number of each row is C. F₁ shuffle operations are generated for n₀, and F₂ shuffle operations are generated for n_s. The number of cluster nodes is S, the communication bandwidth between the machines in the cluster is B, the i-th machine is S_i, the total time for processing data is T(S_i), and other time is t.

Assuming that the data of each machine are shuffled to other nodes and the amount of data remains unchanged, the total time spent T is: (15)

From the above Eq (15), the total time is mainly determined by the amount of data, the number of samples, the number of shuffles, and the execution efficiency of each machine, where n_s is determined by n₀. When the amount of data is constant and m is determined, reducing the number of shuffles and optimizing the time overhead of shuffles are the keys to solving the algorithm performance bottleneck.

4.7 Algorithm performance optimization

According to the theoretical analysis in the previous section, shuffling is the most performance-consuming part of the algorithm in Spark because this part contains a large number of disk IOs, serialization, network transmission, and other operations. If it is not properly tuned, the execution speed of the algorithm is very slow. Therefore, it is optimized according to the logical DAG diagram of the SOSK-Means algorithm. As shown in Fig 1, the application of the SOSK-Means algorithm generates at least four jobs:

• job_0: Triggered by the count operator, there is only one stage, used to read in the statistics of the data set.
• job_1: Triggered by the take operator and divided into two stages: use the accumulator variable to summarize the calculation results and use the shuffle operation to extract sample data from the original data set.
• job_i: Triggered by the pre-clustering collectAsMap operator, where i ≥ 1, and the value of i depends on the number of samples. Two stages are generated: the broadcast operation of the pre-clustering centre and the shuffle operation. Used for pre-clustering to select the initial centre.
• job_j: Triggered by the collectAsMap operator of iterative clustering, where j ≥ 1, and the value of j depends on the number of iterations. Contains two stages: the broadcast operation of the optimal initial cluster centre and the shuffle operation. According to the algorithm’s DAG graph, it is optimized in the following aspects:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Logic DAG of the soak-Means.

https://doi.org/10.1371/journal.pone.0308993.g001

(1) Shuffle operator is avoided

No shuffle is generated in job_0. This job reads the original data set to form an RDD. Since this RDD is used multiple times, it is cached and persists in the memory to avoid cumbersome calculation from the source every time the operator is executed. Because the RDD data are directly extracted from the memory and then the shuffle operation is performed, the cost of reading and writing to the disk is reduced, and it is suitable for the iterative calculation of the algorithm.

(2) Shuffle operator is used to process partition

In job_1, the foreachPartition operator calculates the sum of all samples in each partition with one call. Compared with the one-time call of the foreach operator to process a piece of data, it is of great help to the performance improvement. Similarly, the mapPartition operator that performs local clustering in each partition in job_i improves the performance similarly to foreachPartition, instead of the normal map operator, calling to process one partition at a time.

(3) Shuffle operators are used for pre-aggregation

Pre-aggregation means that when the task of the current stage executes shuffle write, each node performs an aggregation operation on the same key locally. Since multiple identical keys are merged, each node has only one key locally, and then the next one. When the stage task executes shuffle read, when other nodes pull the same key on all nodes, it greatly reduces the amount of data that needs to be pulled, thereby reducing disk IO and network transmission overhead.

The reduceByKey operator is used in both job_i and job_j to replace the groupByKey operator. Because the reduceByKey operator pre-aggregates the same key locally according to a user-defined function, the groupByKey operator does not pre-aggregate, and the performance is relatively poor compared to that of reduceByKey.

(4) Broadcast external variables

Both the pre-clustering and iterative clustering of the algorithm use the external variables that save the initial centre. By default, Spark replicates [replicate] multiple copies of this variable and transfer them to each task in Executor via the network. Especially in pre-clustering, the number of initial centres is much larger than in iterative clustering. If this variable is large, a large number of copies is transmitted in the network, occupying the memory of Executor in each node and affecting performance.

Therefore, for the above situation, the variable is broadcast. The broadcast variable ensures that only one copy of the variable resides in the memory of each Executor, and the task shares the copy, which not only reduces the number of copies and Executor memory overhead but also avoids variables. The load generated by multiple transmissions between nodes improves execution efficiency.

(5) Merge temporary files

In the shuffle write process, by default, the task of the current stage creates a temporary disk file for each task of the next stage. When it is the turn of the next batch of tasks to execute, a new temporary file is created, so many temporary files are generated, which affects performance.

The consolidation mechanism is turned on for this. When Executor executes a batch of tasks and then executes the next batch of tasks, the next batch of tasks does not create new files but reuse the files created by the previous batch of tasks and write the data into an existing temporary file. In this way, files created by multiple tasks can be merged to a certain extent, thereby greatly reducing the number of temporary files and improving shuffle write performance.

(6) Dual aggregation

In clustering, if the amount of data of individual clusters is larger than that of other clusters, that is, the amount of data corresponding to individual keys in the program is too large, when shuffle operators such as reduceByKey are executed, data skew occurs and task execution is particularly slow. During shuffle reading, individual tasks are allocated a larger amount of data, the execution time is slower than other tasks, and the running time of the entire spark job is determined by the slowest task.

In this regard, a dual aggregation solution is adopted to solve this problem. The first time is partial aggregation. After finding the key that caused the data skew, a random number prefix is added to each key. At this time, the same key becomes multiple different keys. Then, aggregation operations are performed on the prefixed data, and partial aggregation is performed so that a large amount of data originally processed by one task can be distributed to multiple tasks for processing. As a result of partial aggregation, multiple different keys become a small number of different keys, but the corresponding data have been aggregated once. Finally, the prefix of each key are removed and another aggregation is performed to obtain the global aggregation result, thereby solving the problem of excessive data processing by a single task.

(7) Kryo is used for serialization

When shuffling and caching the RDD above and using broadcast variables, the data need to be serialized before they can be stored. Spark’s default serialization speed is slow, and the serialized data occupy a large amount of memory. Therefore, the Kryo serialization library is set in the configuration file for final optimization to ensure high serialization efficiency.

5.1 Experimental environment

The Spark cluster consists of 1 master node and 8 slave nodes in the Linux system environment. Each node of the Spark cluster runs the CentOS 6.5 operating system, equipped with dual Intel Core (TM) CPUs, 4GB of RAM, and a 500GB hard disk. The cluster is configured with Hadoop 2.6.4, Spark 3.2.3, Scala 2.10.5, and JDK 1.8.5.2.

5.2 Description of the datasets

To validate the performance of the proposed algorithm, this paper selects several commonly used synthetic datasets for clustering analysis as well as some real-world large datasets, as detailed below:

The Iris flower (Iris), banknote (Bank), seeds, Wifi_Localization (Wifi), Planning Relax (Plrx), and wine datasets were selected from the UCI Machine Learning Repository. Additionally, the R15 and D31 datasets were obtained from the University of Eastern Finland.

Regarding the real-world datasets, we selected the credit card fraud detection dataset (CCFD) [28] and the KDD Cup 1999 Data (KDDc99) [29]. The CCFD dataset contains 284,807 instances with 30 features, including Time, Amount, and 28 anonymized PCA-transformed features (V1 to V28), and is highly imbalanced with only 0.172% fraudulent transactions. The KDDc99 dataset includes 4,898,431 instances with 41 features categorized into basic, content, and traffic features, suitable for clustering and classification tasks in network intrusion detection, with multiple classes representing different types of intrusions and normal connections.

All the datasets along with their details are shown in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Database details.

https://doi.org/10.1371/journal.pone.0308993.t001

5.3 Experimental design

We conducted a comparative experiments and an performance experiment. We compared SOSK-Means with the following baseline methods:

• K-means: Spark MLlib implements a scalable and efficient K-means clustering algorithm, utilizing distributed memory computing to handle large-scale datasets
• Bisecting K-means: Spark MLlib provides a Bisecting K-means clustering algorithm, leveraging hierarchical clustering and distributed memory computing for large-scale data processing.
• EOAK-means [3]: EOAK-means enhances the traditional K-means clustering algorithm by integrating the Equilibrium Optimization Algorithm (EOA) to select the optimal number of clusters dynamically and improve clustering quality.
• K-Plus Anticlustering [30]: This method extends k-means by considering distribution moments (means, variance, higher-order moments) to maximize between-group similarity. It partitions elements into disjoint groups to maximize between-group similarity and within-group heterogeneity, reversing the logic of cluster analysis.
• LBKC [31]: This method utilizes the lower bound of Euclidean distance to safely avoid a large number of unnecessary distance calculations, thereby accelerating the k-means process.

The comparative experiment‘s performance was evaluated using 5 standard metrics: accuracy, recall, jaccard index, rand index, and F-score.

5.4 Experiment of the clustering effect

For the accuracy, recall, Jaccard index, Rand index, F1-score and time values of each algorithms on each dataset, the results are shown in Tables 2 to 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of the ACCURACY of the datasets between each algorithm.

https://doi.org/10.1371/journal.pone.0308993.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of the RECALL of the datasets between each algorithm.

https://doi.org/10.1371/journal.pone.0308993.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of the JACCARD INDEX of the datasets between each algorithm.

https://doi.org/10.1371/journal.pone.0308993.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of the RAND INDEX of the datasets between each algorithm.

https://doi.org/10.1371/journal.pone.0308993.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison of the F1-score of the datasets between each algorithm.

https://doi.org/10.1371/journal.pone.0308993.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Comparison of the TIME of the datasets between each algorithm(unit:s).

https://doi.org/10.1371/journal.pone.0308993.t007

According to the experimental results, the clustering indices of the SOSK-Means algorithm are similar to those of the baseline algorithm, indicating that the algorithm achieves high computational accuracy while maintaining high computational speed. Overall, the SOSK-Means algorithm performs well on most datasets. However, due to the random generation of multiple sets of initial centers, the indeterminable randomness of the SOSK-Means algorithm results in average clustering performance.

5.5 Performance experiment

To measure the calculation speed of the algorithm, the SOSK-Mean algorithm uses 5 million rows (689 M), 10 million rows (1.34 G), 15 million rows (2.01 G), 20 million rows (2.69 G), 25 million rows (3.36 G), and 30 million rows (4.03 G) of artificially synthesized data sets. The results are compared with those of the K-Means algorithm.

The acceleration ratio is the ratio of the running time consumed by the same task in a uniprocessor system and a parallel processor system. It measures the effect of algorithm parallelization. Figs 2–5 shows the accelerating ratio changes of the four algorithm under different numbers of cores and different data volumes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Different algorithms’ performance across 4 cores.

https://doi.org/10.1371/journal.pone.0308993.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Different algorithms’ performance across 8 cores.

https://doi.org/10.1371/journal.pone.0308993.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Different algorithms’ performance across 12 cores.

https://doi.org/10.1371/journal.pone.0308993.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Different algorithms’ performance across 16 cores.

https://doi.org/10.1371/journal.pone.0308993.g005

On the whole, when the number of cores is constant, as the amount of data increases, the acceleration ratio of the SOSK-Means algorithm shows an increasing trend, and when the number of cores is 4, the acceleration ratio of the algorithm increases slowly. When the data volume is less than 15 million rows, the SOSK-Means accelerating ratio grows slowly because the cluster startup and communication between nodes take considerable time and overhead. When the data volume is greater than 15 million rows, the accelerating ratio approximates a linear increase, and the time to process the data is greater than the cluster deployment time. When the amount of data is constant, the acceleration ratio of the algorithm increases with the increase in the number of cores because an increase in the number of cores means an increase in nodes, which gives play to the time advantage of parallel computing. In general, as the amount of data and the number of cores increase, the acceleration ratio of the SOSK-Means algorithm is larger.

5.6 Ablation experiment design

5.6.1 Components.

As outlined above, the proposed method improves upon the basic k-means algorithm through five distinct modules. The codes and descriptions of these modules are detailed in Table 8. We evaluated the F1 score and execution time for each selected module and conducted experiments on the CCFD and KDD99 datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Components details.

https://doi.org/10.1371/journal.pone.0308993.t008

5.6.2 Combination ablation experiments.

The results are shown in Tables 9 and 10, which indicate that when the A, B, or C components are not used, clustering accuracy decreases. The clustering accuracy gradually improves with the sequential addition of these components. When the D or E components are not used, the efficiency in handling large-scale data significantly decreases. When only the A and B modules are added, clustering accuracy improves noticeably but does not reach optimal levels. When the A and E modules are not added, the efficiency in handling large-scale data significantly decreases.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Combination ablation experiments on CCFD.

https://doi.org/10.1371/journal.pone.0308993.t009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Combination ablation experiments on KDDc99.

https://doi.org/10.1371/journal.pone.0308993.t010