1 Introduction

The prime goal of clustering analysis is the organization of a dataset into a finite number of segments according to the similarities within objects. Ideally, the set of objects in the same segment should be comparably similar to one another than to the objects belonging to different partitions [1]. Each individual partition is known as a cluster, whereas the objects belonging to the same cluster are called its members [2, 3]. Its applications covers many real-world applications, ranging from business and economics, marketing, pattern recognition, medical sciences, image processing to big data analysis [4]. For example, in the field of market segmentation, better marketing strategies can be adopted by clustering the customers with similar demographic or buying characteristics [5]. In a similar passion, clustering might be helpful in better understanding the disease and targeting appropriate treatment by sub-grouping the patients into homogeneous sets based on psychological inventory scores [6]. Since the notion of clustering is not precisely described, consequently, several algorithms/models have been proposed in the literature, and all of them may result in well different clustering solutions [7, 8].

It is already true that in recent years, a considerable amount of research work conducted is based on investigating the spatio-temporal properties of clustering. In these applications, clusters are no longer considered static objects, as they are affected by changes occurring in the underlying population [9, 15]. The inclusion of new data records to the original population over time may affect the cluster’s memberships, and entirely different clustering solutions may be generated at later time-points. This transition in clustering solutions may include the disappearing of a specific cluster(s), migration of some elements from one cluster to another, splitting of a cluster into several, several clusters splicing together to form one, survival of a cluster and emerging of new ones. The survived clusters can experience internal transition, including changes in location, size, and density [10, 11]. Various topics such as spatio-temporal, evolutionary, stream, and incremental clustering address this issue by adopting the dataset that changes over time. Tracing and understanding the phenomena behind this transition is of practical importance for effective decision-making. This can be helpful in various fields like marketing, fraud detection, networking, scientific publication, health, etc. [12].

In many real-world applications, clustering of the data stream is performed all time to identify the changes occurring in the pattern of the underlying phenomena [13]. Since in a stream, new data items are continually generated, which join the underlying population at a regular interval. Therefore, in order to control part of the data that contributes to the pattern in data mining, the stream needs to be discretized into subsets based on some attributes that have an order. This data discretization into subsets is called the windowing approach and is mainly done based on time. Some of the most commonly used examples are landmark, sliding, and damped window models [14]. These models are discussed in the next section.

[15] introduces the notion of evolutionary clustering to process the times-tamped dataset by producing a sequence of clustering solutions. That is, a clustering solution for each time-step of the temporal data. The algorithm optimizes two competing criteria i.e. each clustering in the sequence should be similar to the clustering at the previous time-step, while at the same time should accurately reflect the data arriving during that time-step. This framework is further extended to spectral clustering [16], density-based clustering [17], and Hierarchical Dirichlet Process with the Hidden Markov model [18].

Using a totally online method, Hyde et al. [19] offer an algorithm that clusters the evolving data streams into arbitrary shaped clusters. The approach consists of two stages: the first stage finds micro-clusters in the datasets, and the second step merges these micro-clusters into macro-clusters. In a similar vein, Fahy et al. [20] describe an Ant Colony Stream Clustering technique built on a density-based methodology that recognises clusters as a collection of micro-clusters. To read a stream and create micro-clusters in the window pane, the method uses a tumbling window model. By combining the related clusters based on a similarity index, these clusters are then further refined. Fahy and Yang [21] further enhance this technique to address the multi-density issue in the density-based clustering strategy. This method uses the local radius of each cluster to identify clusters, and it then tracks changes in the solutions. For the first time, multiple view clustering challenges are addressed by Huang et al. [22] in MVStream clustering method. In order to assign cluster labels to the data items that include summary statistics, this technique creates support vectors from various views of the data objects. Similarly, some studies have been conducted for measuring the similarities between the trajectory in the dynamic environment [23–25].

2 Window models

In a landmark window model, all items that arrive after some specific time-point (landmark time) are maintained and cannot be discarded irrespective of window size. The window size is uncontrolled and keeps increasing as time progresses [26, 27]. The data records that arrives in the interval (t_i−1, t_i) are accumulated according to the equation given by: (1) where n is the number of time-points and t is the current time-point. Implementation of the landmark window model will generate n window panes, where each pane contain data items evolving from starting time-point t₁ to the current time-point t_i.

The sliding window model, on the other hand, is based on a fixed size of window w that contains only those objects falling in the interval [t_i − w + 1, t_i], while older cases are discarded. In such type of model, as time progress, the window slides forward while keeping its size w by including new data records and discarding the older ones [27, 28]. The scenario of the sliding window model can be described in the equation below: (2) (3) (4) (5) where m is the number of window panes and is equal to n − w + 2, n is the number of time-points, and w is the sliding window size.

3 The change detection algorithm

In order to monitor and trace the evaluation of clusters extracted from re-clustering of cumulative datasets [29] introduced a framework known as ‘MONIC’ algorithm. This algorithm is based on clustering cumulative datasets arriving at discrete time-points t₁, t₂, …, t_n. Initially, the data is collected at time-point t₁, and as time progresses new data records join the data set at regular interval of time. The initial datasets d₁, d₂, …., d_n, are accumulated and re-clustered at each time-point t₁, t₂, …, t_n to monitor and detect the cluster evolution over time.

The algorithm is mainly based on the idea of a non-symmetric overlap matrix between two clustering extracted from cumulative datasets at two different time-points. Let be a set of clusters extracted from dataset D_i at time point t_i and is referred to as first clustering. Similarly, let be a set of clusters extracted from dataset D_j at time point t_j (i<j) and is referred to as second clustering. Then the overlap matrix can be defined as: (6) where k₁ is the number of clusters from the first clustering ξ_i, and k₂ is the number of clusters from second clustering ξ_j. This will generate a matrix of order k₁*k₂, where rows and columns describe first and second clustering respectively. The value on the corresponding element of the matrix represents the similarity index between cluster X_i and Y_j. The MONIC framework assumes hard clustering where each observation belongs to one and only one cluster [30].

In the context of this algorithm, the transition is the change experienced by a cluster X_i ϵ ξ_i, when it has been perceived at second clustering ξ_j. This change in the clustering solution is referred to as an external or internal transition. External transition concern the relationship of cluster found at clustering ξ_i to the clusters found at clustering ξ_j, whereas internal transition is regarded as changes that occurred in the structure of the survived clusters.

The external transition is categorized into five categories i.e. Survive, Merge, Split, Disappear, and Emerge candidates. The cluster X_l ϵ ξ_i may survive into Y_m ϵ ξ_j, clusters may merge to form Y_m ϵ ξ_j, or cluster X_l ϵ ξ_i may split into various daughter clusters . If a cluster X_l ϵ ξ_i does not experience any of the above transitions, then it disappears. Similarly, if a cluster Y_m ϵ ξ_j is not a result of any external transition from its ancestors, then it is a newly emerged candidate. The overlap between X_l ϵ ξ_i and Y_m ϵ ξ_j serve as an indicator of identifying the external transition experienced by clusters at clustering ξ_i. This value is compared with a minimum threshold value say τϵ[0.5, 1] to identify match of X ϵ ξ_i in Y ϵ ξ_j. A cluster X_l ϵ ξ_i is said to survive in Y_m ϵ ξ_j if this is the only cluster that has an overlap of greater than τ_survive. If at least two clusters from X ϵ ξ_i (such as have an overlap of greater than τ_survive with Y_m ϵ ξ_j), then it is a case of merge i.e. X₁ and X₂ merge to form Y_m. Furthermore, a cluster is said to split in daughter clusters, if the overlap of X_l with and is greater than τ_split and collectively their overlap is greater than τ_survive, i.e. for split the following two conditions are required. (7) (8)

where M is the number of daughter clusters from second clustering.

The overlap can not be used as an indicator for monitoring the changes in the form of survived clusters. The shift in the location of the survived cluster (X_l → Y_m) can be traced by calculating Euclidean distance between their centroids normalized by the minimum radius. This information can be summarized in the following formula: (9) where and are the centroids of clusters X_l and Y_m respectively, and is the Euclidean distance between them. The r denotes radius of the corresponding clusters and is computed as the maximum distance of an object from its cluster centroid. If the absolute value of location.difference is greater than τ_location, then the algorithm will detect a shift in location of the survived cluster.

For density transition, the average distance of objects from cluster centroid can be computed. The formula for the density of cluster is given by: (10)

The difference in density of cluster X_l survived in Y_m is normalized by the minimum radius i.e. (11)

If the absolute value density.difference is less than τ_density then there is no change in density of the survived cluster. On the other hand, if the absolute value is greater than τ_density then a change in density would be detected. If density.difference is positive then the cluster is more compact than its ancestors, otherwise, it becomes more diffuse.

4 Package description

The state-of-the-art “MONIC” algorithm is implemented in the R-software via package clusTransition. The package can be used for tracing and monitoring the evolution of clustering solutions in cumulative datasets over time. In this section, we briefly describe the functions and methods exported by the package in detail. Fig 1 below demonstrates the workflow of the package.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Workflow diagram of the package.

The Transition function exported by the package offers three different options for importing datasets. The function then trace changes in clustering solutions.

https://doi.org/10.1371/journal.pone.0278146.g001

Table 1 below summarizes the functions, methods, and classes exported by the package along with its corresponding arguments and slots.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Functions, methods and classes exported by the package clusTransition.

https://doi.org/10.1371/journal.pone.0278146.t001

More details about these functions and classes are described below.

5 Simulation example

Let us assumes that a data stream consist of datasets d₁, d₂, …, d_n arriving at corresponding time-points t₁, t₂, …, t_n respectively. For the generation of initial dataset d₁, we use a generator that takes into account the number of clusters (k), size of each cluster, and separation value between theme [31]. While the generator for generating other streams like d₂, d₃, …, d_n consider the center of each cluster, size of each cluster, and the co-variance structure between them as input [32, 33].

As a working example, we generate a data stream sprouting at four consecutive time points.Fig 2 below demonstrates the scenario for generating datasets d_i, i = 1, 2, 3, 4 at four time points. The new objects joining the underlying population are shown by red color whereas older records are displayed by black color.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Data stream generated at four discrete time stamps.

The new data items at each time stamp is shown by the red color, whereas the older data items are shown by black color.

https://doi.org/10.1371/journal.pone.0278146.g002

6 Pre-processing

Prior to the implementation of the change detection algorithm in cluster solutions over time, the user needs to pre-specify some relevant parameters. First of all, the user needs to decide a suitable windowing approach for the accumulation of datasets evolving at successive time points. For this purpose, we offered two types of windowing approaches in the package i.e. landmark and sliding window models. Implementation of the windowing approach will accumulate the datasets at corresponding time points according to the chosen model and will generate window panes at successive time points. In the second phase, the optimal number of clusters in each window pane D_i at the corresponding time point must be determined using an appropriate technique. For illustration purposes, we use worked examples based on the datasets simulated in section IV. The datasets are accumulated according to the landmark and sliding windowing approaches, and then the optimal number of clusters was estimated in each window pane D_i.

The implementation of the landmark window model will produce four window panes. Each pane will contain the datasets generated between [t₁, t_i], where t_i represent the current time point. Table 2 below demonstrates the number of objects and optimal number of clusters in each window pane D_i estimated from Gap statistics at corresponding time points t_i.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Optimal number of clusters in landmark window model datasets.

https://doi.org/10.1371/journal.pone.0278146.t002

Similarly, the implementation of a sliding window of size 3 will generate 3 window panes. Table 3 below demonstrates the number of objects and optimal number of clusters in each window pane D_i.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Optimal number of clusters in sliding window model datasets.

https://doi.org/10.1371/journal.pone.0278146.t003

7 Implementation of function Transition()

In this section implementation of the primary function, Transition() is presented using working examples. The data stream simulated in section 5 is used for monitoring the cluster evolution over time. The function provides three different options for importing the datasets, which are explained in subsections below.

7.4 moplot() function

Fig 3 displays the graphical summary of an object of class Monic generated by Transition() function as output. The stack bar-plot in the top left corner displays the survival and absorption ratio at successive time points. The Figure illustrates that all clusters survived at time point t₁, and hence the survival ratio is 1. However, at time point t₂ 3 out of 4 clusters survived resulting in a 0.75 survival ratio. Similarly at time point t₃ 3 out of 5 clusters survive, while 2 merged. This resulted in 0.60 survival and 0.40 absorption ratios respectively. Consequently, no cluster disappears and no newly emerged candidate were detected at any of the time points. This can be seen from pass-forward ratio, which is unity at all time points except t₂ where one cluster splits into daughter candidates.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Data stream generated at four discrete time stamps.

The new data items at each time stamp is shown by the red color, whereas the older data items are shown by black color.

https://doi.org/10.1371/journal.pone.0278146.g003

8 Real data example

To demonstrate the practicality of the package and deeply understand applications of cluster evolution, we investigate three real-life datasets. To comprehend the notion of transformation in social, political, and moral attitudes of European nations; the Human Values datasets were extracted from European Social Surveys [35]. The changes in electricity consumption of inhabitants were traced using Individual Household Electricity Consumption dataset. Similarly, the Intel Lab sensors streaming dataset was used to show the applications of the framework. Both these data streams were extracted from the home page of “UCL Machine Learning Repository”.

8.1 Application to human values scale

As a case study, we extract eight datasets each corresponds to a single round of European Social Surveys (ESS) conducted in years 2002, 2004, 2006, 2008, 2010, 2012, 2014, and 2016 respectively. The dataset consist of 25024 individuals who respond to the Schwartz Value Survey (SVS) for computing basic human values and can be downloaded from the URL https://ess-search.nsd.no/CDW/ConceptVariables. The ten basic values are Benevolence, Universalism, Self-direction, security, Confirmatory, Hedonism, Achievements, Traditions, Stimulation, and Power [35]. The k-means clustering algorithm was implemented to sliding window-modeled datasets at each time point. Whereas, the number of clusters in the respective datasets was estimated from the well-known GAP statistic. Fig 4 below describe the evolution of clusters at time point t_i, i = 1, 2, 3, 4, 5, 6, 7 in Human Value scale datasets. which demonstrates that two clusters C₁₁ and C₁₂ survived over time. The first imperative cluster was C₁₁(C₁₁→C₂₂→C₃₂→C₄₂) that emerged at t₁(2002) and survived until t₄(2006, 2010). However, the cluster survived till 2010, but experienced internal transition and became more diffused eventually disappeared at time-point t₅. The second vibrant cluster was C₁₂(C₁₂→C₂₄→C₃₃→C₄₁→C₅₂→C₆₃→C₇₁) which survive through the entire time span. This was the most important cluster because not only it survives over time but also turns out to be denser. Mostly the new respondents of SVS surveys over the years joins this cluster. The shift in location was observed for this cluster at time-point t₂ and t₃, and afterward, remain stable. The first external transition was experienced in the cluster C₁₄ which split into two clusters and ultimately disappeared. The algorithm also detects a cluster C₆₁ that emerged at t₆(2010, 2014) and pass-forward while absorbing elements of the cluster C₆₂.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Transition of clusters in basic human values datasets.

https://doi.org/10.1371/journal.pone.0278146.g004

8.2 Application to Individual Household Electric Power Consumption

As a second example, the Individual Household Electric Power Consumption dataset for the years [2006, 2010] was used. This dataset comprises of 2075259 households characterized by seven numerical attributes. The dataset is available at machine learning repository [36] and can be downloaded from https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption. A sliding window model of size 2 was used for accumulation of the stream at successive time points. In this section, we use the CLARA algorithm to extract clusters from the datasets at successive time points. Whereas the average silhouette method was used to estimate the optimal value of k in each window pane. Fig 5 below demonstrates the evolution of clusters at time point t_i, i = 1, 2, 3, 4, 5 in individual household electric power consumption datasets. The algorithm detect that all of the four clusters survive (C₁₁→C₂₁, C₁₂→C₂₁, C₁₃→C₂₃, and C₁₄→C₂₄) experiencing internal transition and became diffuse during [2006, 2007]. A shift in location for only one cluster C₁₃ was detected, whereas other clusters were stable to change in location. Similarly, three clusters survive (C₂₁→C₃₁, C₂₂→C₃₃, and C₂₄→C₃₄), one cluster disappear (C₂₃→ ⊙), and one cluster emerged (⊙→C₃₂) during [2007, 2008]. Two of the survive clusters became diffuse, while one cluster became compact than its predecessors. Likewise, one cluster survive (C₃₃→C₄₃), three disappears (C₃₁→ ⊙, C₃₂→ ⊙, and C₃₄→ ⊙), and three newly emerged clusters (⊙→C₄₁, ⊙→C₄₂, and ⊙→C₄₄) were detected during [2008, 2009]. Afterwards all four clusters disappears (C₄₁→ ⊙, C₄₂→ ⊙, C₄₃→ ⊙, and C₄₄→ ⊙), and three new clusters emerged (⊙→C₅₁, ⊙→C₅₂, and ⊙→C₅₃) during [2009, 2010].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Transition of clusters in Individual Household Electric Power Consumption datasets.

https://doi.org/10.1371/journal.pone.0278146.g005

8.3 Intel Lab dataset

In this section, we used the publically accessible dataset recorded from 54 sensors deployed at Intel research laboratory during February 28^th and April 5^th, 2004. Each sensor record information on temperature, humidity, voltage, and light every thirty-one seconds. The dataset comprises of 2.3 million readings collected from 54 sensors. The sensors were designed to make it energy-efficient and consume power only in sensing environment and transmitting data. We select only a subset of measurements from this dataset and include readings from sensor-1 only. This subset of the data consists of 43,047 readings from sensor-1 and can be downloaded from the URL https://www.kaggle.com/datasets/divyansh22/intel-berkeley-research-lab-sensor-data.

We accumulate the dataset according to the landmark window model, and as the flow is uniform, so we consider 9000 records per time period. This implementation generates 5 window panes of cumulative datasets. The shadow statistic decided the optimal number of clusters in cumulative datasets at the corresponding time point. The Partitioned Around Medoids (PAM) algorithm was used for extracting clusters from datasets.

Fig 6 below demonstrates the transitions of clusters at time points t_i, i = 1, 2, 3, 4, 5 in Intel Lab dataset. The algorithm detect that all six clusters survive (C₁₁→C₂₁, C₁₂→C₂₂, C₁₃→C₂₄, C₁₄→C₂₅, C₁₅→C₂₆, and C₁₆→C₂₃) while one new cluster emerge (⊙→C₂₇) at time point t₂. All survived clusters experience internal transition and became more diffuse. Also six clusters survive (C₂₁→C₃₁, C₂₂→C₃₂, C₂₄→C₃₃, C₂₅→C₃₄, C₂₆→C₃₅, and C₂₇→C₃₆) and one cluster disappears (C₂₃→ ⊙) at time point t₃. Cluster C₂₄ experience double internal transition i.e. shift in location and change in density, while other clusters only became diffuse. Likewise, five clusters survive (C₃₁→C₄₃, C₃₂→C₄₅, C₃₄→C₄₄, C₃₅→C₄₂, and C₃₆→C₄₇), one cluster disappears (C₃₃→ ⊙), and two clusters emerged (⊙→C₄₁ and ⊙→C₄₆) at time point t₄. Similarly, five clusters survive (C₄₂→C₅₄, C₄₃→C₅₆, C₄₄→C₅₇, C₄₅ →C₅₃, and C₄₇→C₅₅), two clusters merge ({C₄₁, C₄₆}→C₅₁), whereas one cluster emerge (⊙→C₅₂) at time point t₅.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Evolution of clusters in Intel Lab dataset.

https://doi.org/10.1371/journal.pone.0278146.g006

For further details and understanding the significance and practical applications of monitoring changes in clustering solutions of streaming datasets see Atif et al [37].

9 Concluding remarks

In this paper, we introduce an R package clusTransition dedicated to trace the evolution of cluster solutions in cumulative datasets. The package implements state-of-the-art algorithm MONIC for modeling and tracing the transition of cluster solutions in dynamic datasets. This algorithm is based on re-clustering of cumulative datasets D₁, D₂, …, D_n arriving at corresponding time-points t₁, t₂, …, t_n and monitor the changes occurring in these cluster solutions. The changes comprise of clusters that still exist, split into various, absorbed by others, disappeared and newly emerged. The clusters that survived in external transition may experience a change in location and density called internal transition. We have applied clusTransition package on synthetic as well as on real-life datasets to look insight into change detection framework.

10 Limitations of the package

The clusTransition package takes into account batch processing, where the stream is discretized and the gathered data is put into the windowing model. The datasets are not clustered upon arrival immediately in real time. Similarly, the use of sliding and landmark models either contain the data items or entirely ignore them at subsequent time-points. A damped window model, on the other hand, assigns each object, depending on its arrival time, exponentially decreasing weights. Future plans call for adding support for the damped window model to the R package for change detection.

The paradigm for cluster transition monitoring presupposes hard clustering, which requires that each item be assigned to one and only one cluster. This assumption implies that the strategy cannot be used to density-based or model-based clustering approaches, leaving the problem open for further investigation.