Related works

When confronted with datasets too large for a single machine or spread across diverse locations, distributed clustering presents a notably effective strategy. The paper [4] comprehensively reviews various implementations of k-means clustering algorithms for handling large-scale datasets, providing a detailed comparative analysis of these implementations, highlighting their optimizations for enhanced performance on extensive datasets. Additionally, the article compiles an inclusive list of datasets used for benchmarking to ensure a rigorous evaluation, enabling a thorough assessment of each algorithm’s effectiveness under diverse data conditions.

The paper [5] presents innovative algorithms for distributed clustering tailored to k-means and k-median objectives. These methods address the challenge of communication complexity in distributed systems by leveraging coreset construction being a technique that compresses the data into a more miniature representation, although maintaining clustering accuracy enabling efficient computation across diverse networks. Experimental evaluations on large datasets reveal that these approaches outperform existing coreset-based distributed clustering methods. The authors suggest both theoretical proofs and experimental findings based on synthetic and real-world data to substantiate the efficacy of the methods. While reducing data transmission, this method incurs high computational overhead. Moreover, it lacks built-in privacy mechanisms for secure distributed clustering.

The study proposed in [6] introduces a unified framework for gradient-based distributed data clustering, solving optimization problems with several objectives presenting a family of algorithms tailored for clustering in distributed environments. This framework is designed to handle large-scale datasets distributed across multiple nodes, emphasizing improved communication efficiency and clustering performance in decentralized networks. The experimental results demonstrate the effectiveness of the approach across various settings, particularly in scenarios where a small number of agents manage large datasets. This approach is designed for scenarios with small agent groups and large local datasets, making it inefficient for systems with a large number of agents and limited local data per node.

In [7], a distributed k-means algorithm is introduced to mitigate node communication problems. The standout feature is quantized messages for more efficient data transmission to neighboring nodes. Note that, unlike the method described here, the algorithm presented further in the present article exploits compressed sensing. While this approach in [7] minimizes data transmission, it lacks robustness in highly dynamic environments with rapid cluster structure evolution. The scenario examined in [8], specifically the analysis of small datasets across a large number of nodes bears a partial resemblance to the context investigated in this paper. The method employs consensus algorithms. Max consensus is used for cluster initialization through leader elections. Finite-time average consensus enables agents to compute cluster centroids using only neighbor communications in a distributed manner. The method has high latency in achieving consensus and does not efficiently adapt to time-sensitive scenarios. The article [9] presents a refined penalized clustering model to analyze large-scale fused data using a nature-inspired optimization algorithm to improve the clustering performance. The study does not explicitly address real-time clustering in decentralized systems and lacks attention to privacy.

The cluster consensus problem consists of two key components: clustering algorithms and control mechanisms. As demonstrated in [10–13], early research primarily focused on centralized clustering methods such as genetic algorithms, bipartite matching, and k-means clustering. These approaches rely heavily on global information, which can pose challenges when access to comprehensive data is limited or unreliable.

Conversely, research in [14–17] has emphasized the development of distributed control laws to achieve cluster consensus in multi-agent systems across diverse scenarios, including swarm robotics, higher-order systems, and networks with antagonistic interactions. However, these approaches often face challenges regarding the controller complexity and computational overhead, as exemplified by the Riccati equation-based solutions in [15,16]. These studies often overlook the integration of dynamic clustering methods, relying instead on fixed or time-varying communication topologies [14–16] . This can lead to reduced adaptability and outdated clustering results, as these methods fail to account for real-time changes in system states or datasets. Limited coupling strengths or network connectivity further restrict their applicability. Specifically, the method in [14] relies on complex controllers, leading to high communication overhead and limiting scalability in large networks. The approach [16] introduces a significant computational burden and lacks inherent privacy preservation mechanisms and is unsuited for rapidly changing topologies and suffers from high communication operating costs. The performance of the approach discussed in [17] is contingent upon predefined clustering, and it does not prioritize fast consensus building. While more straightforward distributed control strategies, such as those presented in [18], offer more straightforward implementation, they are frequently modified to specific scenarios like containment control with fixed communication topologies that can constrict their flexibility in real-world applications.

Another issue connected to the problematics discussed is network systems entailing many interacting nodes communally functioning in an unknown environment. These types of systems are prevalent in numerous practical applications, garnering a significant interest within the research community. Examples include swarm robotics, distributed sensor networks, and electrical power grids, as illustrated in [19–21].

However, the algorithms mentioned are computationally expensive for high-dimensional data. In practice, data often have a sparse structure due to compact clustering of node states, and in such cases, a new paradigm like Compressed Sensing (CS) can be useful. CS [22,23] is an innovative and rapidly growing research area that has attracted significant interest across multiple disciplines, including electrical engineering, applied mathematics, statistics, and computer science. CS provides a framework for simultaneously sensing and compressing finite-dimensional vectors through linear dimensionality reduction. Its critical insight of CS is that high-dimensional sparse signals can be accurately reconstructed from a limited number of measurements using computationally efficient algorithms. However, the authors do not mention published works explicitly addressing the application of the CS theory combined with deep learning methods for distributed clustering, which have been used extensively in various domains, integrating these methodologies for distributed clustering remains less explored in the literature.

Recent advancements in CS have yielded cutting-edge architectures and methodologies that address challenges such as overfitting, computational efficiency, and scalability. CombNet [24] exemplifies this innovation by employing deeper neural architectures paired with smaller convolutional kernels, effectively minimizing overfitting while improving performance. DeepInverse [25] introduces a transformative approach by replacing fully convolutional layers with a transposed measurement matrix for initial signal reconstruction. This adaptation reduces the model’s complexity, allowing for efficient processing of long signals without block division. The method achieves faster computations and enhanced reconstruction quality, particularly at low compression ratios. Expanding upon this, DeepCodec [26] incorporates a learned sampling mechanism customized to the training dataset, improving reconstruction accuracy and reducing computational overhead. Similarly, ConvCSNet [27] adopts a learnable measurement matrix but utilizes linear layers for compression. Its dual-branch CNN-based architecture enables it to process full-sized signals effectively, like DeepInverse. Residual Networks (ResNet), which mitigate vanishing gradient issues in deep learning, have also been integrated into CS. For instance, the Deep Residual Reconstruction Network [28] combines adaptive sampling techniques with ResNet blocks to optimize reconstruction performance. Generative Adversarial Networks (GANs) have also significantly evolved in CS applications. For instance, [29] augments the ReconNet framework by incorporating a discriminator that distinguishes between real and generated data. This setup enables the generator to map compressed measurements directly to reconstructed outputs, achieving efficient and accurate signal recovery. In MRI reconstruction, [30] leverages GANs with ResNet-based generators and CNN-based discriminators. Despite the inherent challenges in GAN stability, [31] simplifies this process by focusing on detail completion, leading to faster convergence and streamlined models.

Main advantages and contributions

The proposed method addresses several key limitations of previously published works by focusing on communication efficiency, privacy preservation, and fast consensus in decentralized clustering.

Advantages:

• Communication Efficiency: By leveraging compressed sensing, data exchange is minimized while maintaining clustering accuracy, enhancing scalability for large distributed systems.
• Privacy Preservation: Conventional methods often involve sharing raw or partially processed data. In contrast, the proposed approach shares only aggregated data, applying randomized transformations to ensure data security.
• Fast Consensus: Utilizing an accelerated consensus algorithm enables quicker stabilization of clustering assignments, making the method well-suited for dynamic environments.
• Scalability: The approach is optimized for networks with numerous agents and limited local data per node, overcoming the constraints of existing methods that assume access to extensive datasets.

Contributions:

• This work introduces a novel decentralized clustering framework that integrates compressed sensing with an accelerated consensus protocol to gather and process compressed states across a network efficiently. By reducing dimensionality and employing consensus mechanisms for distributed data aggregation, the system enables nodes to estimate the global cluster structure while minimizing data transmission collaboratively. Each node independently generates compact representations that encapsulate clustering structures, ensuring consistency across the network. A pre-trained neural network reconstructs the overall cluster organization, enabling efficient large-scale data processing. Unlike traditional distributed clustering methods focusing solely on fixed or time-varying network topologies, the approach dynamically adapts to evolving states in real-time, requiring minimal computational resources.
• The integration of compressed sensing with a consensus mechanism facilitates decentralized grouping. The offline phase involves training a neural network on an artificial dataset to define expected cluster configurations. During the online phase, nodes generate compact representations of the clustering structure, which the trained model processes to reconstruct the global state. By offloading resource-intensive tasks such as dataset creation and neural network training to the offline phase, our approach ensures an efficient and adaptable clustering solution for dynamic network environments.

Motivation

For coalitional control to be effective in multi-agent systems, the accurate and efficient reconstruction of cluster structures following coalition formation is essential [32]. Although coalition control is acknowledged as a strong alternative to centralized and fully decentralized approaches, the ability to efficiently identify and maintain coalition structures in dynamic environments remains a significant obstacle to its practical deployment.

Two opposite paradigms are commonly recognized in traditional multi-agent control: centralized methods, such as model predictive control and optimal control, which necessitate solving complex, large-scale optimization problems resulting in substantial computational and communication overhead, rendering them impractical for large systems; and fully decentralized methods, which eliminate the need for global coordination but often fail to establish structured coalition formation, leading to suboptimal performance in cooperative tasks.

To address the limitations of current methods, this paper introduces a communication-efficient decentralized clustering method, enabling accurate and scalable reconstruction of cluster structures in multi-agent systems. In contrast to conventional clustering methods, which depend on global information or iterative optimization, our approach facilitates efficient coalition membership inference through local interactions, thereby minimizing communication overhead while preserving high cluster structure recovery accuracy.

Accurately reconstructing cluster structures is crucial for coalitional control, as it enables scalable decision-making by reducing computational complexity and facilitating real-time coalition-based control. Furthermore, it enhances communication efficiency by allowing agents to reconstruct coalition structures through localized interactions, minimizing unnecessary data exchange. Finally, it provides robust adaptation in dynamic environments, enabling agents to quickly and accurately reassess their coalition membership, thereby improving system resilience and coordination efficiency.

The proposed method is particularly beneficial for applications such as autonomous vehicle fleets, intelligent transportation systems, distributed sensor networks, and energy grid management, where maintaining coalitional structures is crucial despite dynamic agent behavior and communication constraints. For instance, in multi-agent robotic navigation within unknown environments, robots must adaptively form clusters to maneuver around obstacles. Traditional decentralized clustering methods would necessitate continuous updates, resulting in high communication costs and instability. However, this approach assumes that cluster evolution occurs at a slower timescale than local control actions and employs an accelerated consensus algorithm, ensuring stable clustering decisions during critical operations. This enables agents to coordinate efficiently without excessive communication overhead. By creating a decentralized clustering mechanism that reconstructs coalition structures with minimal communication overhead, this research aims to improve the scalability, adaptability, and efficiency of multi-agent control systems, fully leveraging coalitional control in real-world scenarios.

System description

Consider a multi-agent system consisting of a set of agents . Each agent has dynamics, which is modeled by the following state-space equation:

(1)

where is the state vector, is the control input, is the external disturbance, and is a possibly nonlinear function describing the dynamics of agent i.

By aggregating the local variables, we define global vectors:

The overall system dynamics can then be expressed as:

(2)

where is the aggregated mapping function. This model type is commonly used for controlling large-scale systems, such as vehicle platoons [33] and traffic networks [34].

Network layer.

We assume that agents communicate with each other through a network. In many multi-agent systems, the structure of the communication network is not static, but rather evolves over time. This evolution can be due to a variety of factors, such as mobility of agents, changing network conditions, or dynamic interactions between agents. In such systems, time-varying graphs are used to exhibit the time-dependent nature of communication. Formally, such a model can be represented as a sequence of graphs , where is the current graph at time t, with

• V is the set of vertices (agents), assumed to remain constant over time.
• is the set of edges at time t, which can change from one time step to another.

Each edge , signifies that agents and can communicate at time t, and these edges may be time-dependent, reflecting the presence or absence of communication links at specific time intervals. We assume that the communication is directed, meaning does not imply .

Assumption 1 (Expected strong connectivity) Let B(t) denote the adjacency matrix of the directed communication graph G(t). We assume that the expected graph , defined by the expected adjacency matrix , is strongly connected. That is, the graph induced by contains a directed path between every pair of nodes, ensuring that information can propagate globally in expectation.

The assumption implies that, although individual realizations G(t) may be disconnected at any given time, the network is connected on average, allowing for global information flow in the long run. This strong connectivity guarantees that information can propagate in both directions across the network, ensuring the effectiveness of distributed algorithms and the convergence of aggregation procedures.

The in(incoming)-neighbor set of an agent at time t, denoted by , is the set of agents to which is directly connected at time t. Mathematically:

The weighted in-degree of node i is , and the Laplacian matrix is , where B(t) corresponds to the weighted adjacency matrix at time t, .

Structural layer.

Agents in large-scale systems commonly group into coalitions or clusters to simplify operations and increase scalability. We define a coalitional structure as a partition of the global set into coalitions (or clusters), where each coalition C_l(t), , is suggested to include states sharing similar characteristics.

The coalitions must satisfy the conditions: for all l, , and for . Each coalition is considered as a subsystem with aggregated dynamics:

(3)

where , , and .

While conventional approaches assume a pre-defined coalitional structure with individual coalition control input synthesis, self-organizing systems necessitate agents to dynamically determine their cluster membership based on state vectors and neighbor measurements.

Assumption 2. (Cluster stability during clustering process). Once the clustering algorithm groups agents into clusters, the assignment of agents to clusters remains fixed throughout the process. There are no dynamic changes, such as a cluster merging, splitting, or shifting significantly once the initial groupings are established.

The rationale behind Assumption 2 is that state relationships are assumed to be stable throughout the clustering process. As a result, clustering processes are handled separately, and time indices are excluded for simplification. These processes may occur over longer durations, with timescales ranging from minutes to seasonal periods, such as indicated in [32].

Remark 1. Assumption 2 establishes a constraint of fixed cluster assignments during the clustering process. Although this constraint might not apply to highly dynamic decentralized networks, the analysis focuses on scenarios where cluster evolution occurs slower than local control actions. For example, in mobile robot navigation within an unknown environment, clusters dynamically form during obstacle avoidance maneuvers but remain stable between them. Re-clustering events, while possible, are infrequent compared to local control updates, rendering the assumption practical for applicable scenarios.

Objective

The objective is to discover the coalitional structure through communication-efficient local agent interactions. The central challenge in distributed clustering lies in developing algorithms that can effectively partition agents into meaningful clusters while minimizing centralized control and inter-agent communication overhead. Therefore, the problem encompasses the following subtasks:

• Coalition Formation: Based on a defined clustering criterion, agents are required to decide which states belong to a coalition. Each agent must perform this assignment locally, without knowledge of all states. The aim is to achieve an optimal partition that minimizes a clustering objective, which generally measures similarity within coalitions and dissimilarity between them.
• Communication Efficiency: Due to the cost or limitations of inter-agent communication, the goal is to minimize information exchange during clustering. Ideally, the data transmitted between nodes should scale sub-linearly with the number of agents or data points. The communication overhead can be embodied as:
• where represents the set of messages exchanged between agent i and its neighbors. The message size is defined as the number of packets exchanged between an agent and its neighbors. We assume that each packet carries one entry of a vector. The purpose is to minimize while ensuring convergence to an optimal or near-optimal clustering solution.

Decentralized compressed data aggregation

The solution presented comprises two primary components: the sensing and communication stages.

Sensing stage. Multi-agent systems involve deploying numerous agents across a spatial region, for example, in sensor networks or IoT deployments. These agents independently collect local data, which is frequently sparse or compressible using transforms like wavelets or Fourier. Due to their independent operation, agents might not have access to the entire dataset or all data at once. The data acquisition process is sparse, recording only notable events or changes.

Given a compressible signal , it can be represented sparsely as (a vector with many zero entries) using the sparsifying basis :

(4)

Agent state representations often include features like positions, sensor readings, or other parameters, which are high-dimensional, continuous, and dense. These states are commonly mapped onto a two-dimensional grid to improve processing, storage, or communication efficiency. This transformation frequently yields a sparse representation. Each agent’s state, such as its position or value of interest, is assigned to a grid cell based on its coordinates or features. For instance, a position can be mapped to the grid cell corresponding to the nearest grid coordinates. When the state is a multi-dimensional vector, dimensionality reduction techniques, like PCA or t-SNE, can project the state onto a 2D plane before grid mapping. Typically, this mapping is non-linear and not considered a sparsifying basis . However, our method bypasses the reconstruction step, enabling direct utilization of the sparse signal from the mapping, effectively treating as an identity matrix.

To represent agent i’s position on a 2D grid, we define a sparse vector , containing zeros and a single one, based on its state . Here, g_s denotes the total number of grid cells. From these vectors, we construct a common normalized vector , where .

The coalitional structure allows for a sparse encoding of . For each coalition , where , we define a one-hot vector that marks the grid position corresponding to the centroid of coalition . We also define the cumulative vector of agents in coalition as: .

Finally, we introduce the transformation given by:

where denotes the cardinality of the set , i.e., the number of agents in coalition l, is an arbitrarily small positive parameter.

Applying this transformation to the common vector yields the sparse representation:

The resulting vector approximates the common sparse centroid vector:

which has exactly n_c non-zero components corresponding to the coalition centroids.

The approximation error between and is bounded in the norm as:

Utilizing compressed sensing, each agent reduces its state representation by applying random projections, thus minimizing data transmission or storage. The measurements are obtained by multiplying the original data with a sensing matrix , where , resulting in a compressed data version, . This technique allows for acquiring fewer data points than the original dataset:

(5)

where is a column of corresponding to a nonzero entry of .

Communication stage. In traditional systems, each agent would send raw data to a central server or another aggregation point. However, in the multi-agent systems studied here, only the compressed measurements (random projections) are transmitted across the network that drastically reduces the bandwidth required for communication.

To facilitate independent cluster assignments, each agent must aggregate the compressed measurements from all agents, for example, by combining them into a single unified compressed representation. This aggregation can be achieved through various methods, such as linear combinations:

Consensus-based algorithms offer a decentralized solution for aggregation. Consensus involves agents sharing and combining information to reach a widespread agreement, ensuring system consistency while maintaining decentralization. In the context of compressed measurements, consensus-based aggregation allows agents to compute a shared representation, preserving individual data privacy. Typically, this is achieved through iterative algorithms where agents update their estimates based on neighbor measurements. Common consensus protocols include the Gossip algorithm and Weighted Average [35], designed for convergence to a common value. We employ an iterative scheme, starting with initial vectors and converge to the target vector as iterations increase. However, we restrict the process to a finite number of iterations due to practical limitations. Consequently, is approximated with an accuracy level of .

Reliable cluster structure recovery under incomplete and inaccurate measurements.

The following convex optimization problem is considered, where the objective is to find the cluster structure with the smallest -norm consistent with the observed data . The problem is formulated as:

(6)

where the matrix A is defined as , is the tolerance parameter account for measurement noise or inaccuracies in the observed data .

The uniform uncertainty principle asserts that the measurement matrix satisfies a restricted isometry property (RIP) [36]. Specifically, there exists a constant , known as the s-restricted isometry constant, such that for all vectors with sparsity at most s, the following inequality holds:

(7)

The smallest value of that satisfies inequality (7) is accepted as the assessment of .

Let denote the solution of the convex optimization problem (6). According to Theorem 2 from [22], the deviation between and the actual cluster structure is bounded as follows:

(8)

providing that the measurement matrix A satisfies the RIP condition (7) with . The term is chosen such that

In practical scenarios, the constants c₁ and c₂ in (8) remain within acceptable ranges for ; for instance, if , then and [22].

The objective, in line with the restricted isometry property, is to design a sensing matrix A with approximately orthogonal columns for any chosen subset.

Design of the measurement matrix.

A direct construction of the measurement matrix , such that satisfies the RIP condition, would require verifying the inequality (7) for all possible combinations of positions for the n_c nonzero entries in the vector . However, it has been shown that the RIP property can be fulfilled with a high probability by simply choosing as a random matrix using randomization of the observation process [37]. In this setting, the measurement vector consists of m linear combinations of the components of , with the weights in each combination selected randomly.

A random matrix , whose entries are independently and identically distributed (i.i.d.) as , exhibits two important and useful properties [22]:

• If and(9)
• then the matrix satisfies the RIP condition with probability at least , where c₃ and c₄ are positive constants that depend only on . This implies that any n_c-sparse or compressible vector of length g_s can be recovered with high probability from only random measurements.
• The matrix is universal in the sense that it preserves the essential information of any n_c-sparse signal during the dimensionality reduction from to . Furthermore, for any choice of orthonormal basis , the product is itself a random matrix with i.i.d. Gaussian entries and thus inherits the RIP property with the same high probability.

As shown in [38], the condition (9) can be derived directly from the well-known Johnson–Lindenstrauss lemma [39], which states that any set of n points in a d-dimensional Euclidean space can be embedded into an m-dimensional Euclidean space, where and is independent of d, such that all pairwise distances are approximately preserved.

In the context of the random matrix , the universality of its behavior and specific constraints for selecting the constants c₃ and c₄ are also discussed in [37,38]. These constants satisfy the following inequality:

RIP-preserving transformation.

The success of sparse recovery via convex optimization, as expressed in the bound (8), relies on the measurement matrix satisfying the restricted isometry property with a sufficiently small constant for sparsity level . While the random matrix satisfies the RIP with high probability under standard conditions as discussed above, it is crucial to ensure that the composite matrix A inherits this property after applying the structured transformation .

To ensure that the coalition-based transformation remains well-behaved and does not introduce excessive imbalance, we impose a mild regularity condition on the coalition structure. Specifically, we require that the coalition sizes are uniformly bounded around their average value. This prevents extreme disparities between large and small coalitions, which could otherwise affect the stability of the representation and the RIP property of the resulting measurement matrix.

Assumption 3. (Approximate balance of coalition sizes). The coalition structure is approximately balanced if there exists a constant such that for all , the size of each coalition satisfies:

That is, each coalition contains a number of agents that deviates from the average by at most a factor of .

Remark 2. (Interpreting the balance parameter ). The parameter determines how far individual coalition sizes are allowed to deviate from the average. When , all coalitions are perfectly balanced and have equal size. For , moderate deviations are allowed: the larger , the more heterogeneous the coalition structure may be. This formulation ensures that coalitions remain well-proportioned while still allowing flexibility. Importantly, the condition scales naturally with both the number of agents n and the number of coalitions n_c, making it suitable for large-scale systems.

The following proposition provides an explicit upper bound that plays a central role in ensuring that the measurement matrix preserves the restricted isometry property. The result relies on the structural assumptions introduced above, particularly the approximate balance of coalition sizes and the orthogonality of the centroid indicators.

Proposition 1. (IP-preserving bound). Under Assumption 3, the transformation matrix satisfies the operator norm bound

If is sufficiently small, then the matrix satisfies the RIP condition:

Proof 1. Let us write:

Since the vectors have disjoint support, each matrix has nonzero entries only in a single row (determined by the support of , i.e. the centroid vectors have non-overlapping support, which we express via orthogonality: ). As a result, the matrices are orthogonal in row space, and, using the definition of the spectral norm, we obtain:

Each is a sum of one-hot vectors, so its entries are non-negative integers counting the number of agents occupying each grid cell. Hence:

Next, we apply the Cauchy–Schwarz inequality and estimate the normalization factor:

By combining both bounds

we get

This completes the proof.

Optimization-based consensus

Standard consensus algorithms often suffer from slow convergence, requiring numerous iterations to reach a solution. In real-world applications, faster convergence is critical for system efficiency, responsiveness, and overall performance. This is especially true for real-time or near-real-time decision-making systems like autonomous vehicles, robotics, financial trading, and sensor networks. In such scenarios, rapid consensus guarantees swift and accurate system decisions. For instance, autonomous vehicle fleets require quick consensus on traffic conditions or road hazards. To address this challenge, we propose an extension of the Accelerated Local Voting Protocol (Accelerated-LVP) from [40] to a vector-valued setting.

Consensus through optimization in multi-agent networks involves designing an optimization problem whose solution guides agents towards a shared decision. This approach enables decentralized consensus while preserving individual agent autonomy.

We aim to compute at each agent simultaneously using an iterative method. Let represent the estimate of maintained by agent i at iteration k, where . The initial estimate is set as , as defined in (5).

In this paper, to reach agreement on a common consensus vector , we propose minimizing a regularized consensus objective function, given by:

(10)

where:

• denotes the vector of all compressed states.
• is a regularization parameter.
• denotes the Kronecker product, which is used to extend the Laplacian matrix to handle vector-valued states.
• is the identity matrix of dimension , corresponding to the fact that the state of each agent is a vector of dimension .

This objective function minimizes the differences between neighboring agents and while incorporating a strong convexity regularizer.

The next step is to take partial derivatives with respect to i-th components :

(11)

Finally, we need to determine the Lipschitz constant L and the strong convexity constant of the function to be optimized (10), as these values are essential for choosing the appropriate parameters:

• the Lipschitz constant L is:
• the strong convexity constant is:

where refers to the smallest eigenvalue of the matrix and refers to the largest eigenvalue of the matrix .

Next, we incorporate our modifications into the Accelerated-LVP framework, resulting in Algorithm 1. This algorithm serves as the aggregation step within the main algorithm. Crucially, all agents simultaneously generate the algorithm’s output, namely the compressed measurement , by processing only their local data and that of their immediate neighbors. After transmitting and aggregating these measurements, the subsequent stage involves reconstructing the centroids , .

Algorithm 1 Accelerated-LVP.

Inputs:

- Initial estimate of agent i

- Step-size h>0 of gradient-like iteration

- Lipschitz constant L, strong convexity constant

- Regularization parameter

- Number of iterations

Output: Compressed aggregated measurement

1: Pick , and set

2: Choose and set initial

3: Copy initial estimate to auxiliary variable

4: Define

5: for k = 1 to do

6:   Determine the local communication neighborhood

7:   Request the compressed states from the neighboring

  agents

8:   Find by solving equation:

9:

10:   Update :

11:   Calculate auxiliary variable :

12:   compute gradient at point using (11)

13:   Update the estimate:

14:

15:   Update auxiliary variable :

16: end for

Accelerated-LVP introduces an additional modeling layer that affects how well the aggregated state of the agents aligns with the linear observation model used in the recovery problem (6). This misalignment gives rise to an error that contributes to the effective residual . Understanding and bounding this term is essential to guarantee a stable and accurate recovery. The following proposition quantifies the Accelerated-LVP error, thus providing a theoretical justification for choosing a valid tolerance parameter in the recovery formulation.

Proposition 2. (Accelerated-LVP Error Estimation). Consider a sequence of estimates generated by Accelerated-LVP, and assume there exists a global optimum of the function

(12)

Suppose that for each iteration t, the following inequality holds [40]:

(13)

where:

• is an initial surrogate function,
• is a finite constant,
• characterizes the error decay rate,
• is a residual error, satisfying and as .

Then, the following statements hold:

1. Error bound after K iterations. For any :(14)
2. Asymptotic error bound. In the limit:(15)
3. Minimum number of iterations for a given accuracy. If a target accuracy is required, then the minimum number of iterations satisfies:(16)
   where .

Proof 2. Step 1: Bounding the error after K iterations.

By assumption, we have for any iteration t:

(17)

Setting t = K directly gives the first statement of the theorem.

Step 2: Asymptotic error bound.

Since and as , we take the limit superior on both sides:

(18)

Since and , we conclude that

(19)

Step 3: Estimating the number of iterations for accuracy .

To satisfy the accuracy requirement:

(20)

we require:

(21)

Then:

(22)

where . If , then:

(23)

Taking the logarithm on both sides:

(24)

Since , dividing by it reverses the inequality:

(25)

Thus, the minimum number of iterations is:

(26)

This completes the proof.

Traditional methods for determining cluster centroids, such as interior-point methods, are computationally expensive and unsuitable for agents with limited processing power. We propose a novel approach to overcome this limitation: pre-training a neural network on a diverse set of possible or likely scenarios. Instead of directly performing complex computations, agents can access a service that efficiently delivers the output of this pre-trained neural network. The following subsection details the reconstruction process using this neural network.

Reconstruction via a neural network

In compressed sensing, signal reconstruction typically involves solving an optimization problem to recover a sparse signal (where most components are zero or near– zero) from a set of measurements . Traditionally, neural networks can learn to map these under-sampled measurements back to the original signal . Our approach differs from this traditional method. First, each agent aggregates its local measurements using the Accelerated-LVP. Second, instead of reconstructing the original signals , we focus solely on reconstructing the cluster centroids.

A typical deep learning-based reconstruction process can be formulated as learning a mapping function such that:

where represents the estimated positions of the cluster centroids , for , along with the corresponding cluster magnitudes .

The actual network architecture is as follows:

1. Input Layer: Linear(compressed size, 512)
   • Fully connected layer mapping input to 512 dimensions.
2. Hidden Layers:
   • Layer 1:
     • Linear(512, 512)
     • ReLU()
     • BatchNorm1d(512)
   • Layer 2:
     • Linear(512, 512)
     • ReLU()
     • BatchNorm1d(512)
   • Layer 3:
     • Linear(512, 256)
     • ReLU()
     • BatchNorm1d(256)
   • Layer 4:
     • Linear(256, 128)
     • ReLU()
3. Output Layer:
   • Linear(128, 2 clusters_number)
   • Reshape with Unflatten(1, (2, clusters_number)) to create a (2, clusters_number) output structure.

The neural network receives the under-sampled measurement as input and produces an estimate of the centroids as output. The Hungarian Loss is applied to calculate the overall L₂ difference (like [41]) between the predicted centroids and the ground-truth centroids based on the optimal clusters’ assignment provided by the Hungarian algorithm. Standard Adam optimization is employed to train the network.

At its core, the application of neural networks in compressed sensing revolves around replacing conventional optimization or greedy algorithms with deep learning models. These models are specifically trained to discern the complex relationship between under-sampled measurements and the original sparse data. These networks effectively learn this intricate mapping by employing end-to-end training on datasets consisting of measurements paired with their corresponding ground truth signals. As a result, once trained, they offer a rapid and efficient approach for reconstructing signals within the framework of compressed sensing applications.

Compressed sensing theory provides guarantees for recovering sparse signals via -optimization, provided that the measurement matrix satisfies the RIP. In the absence of noise, this condition ensures that -minimization yields a unique sparsest solution. The reconstruction process can be viewed as a mapping, allowing us to define a function f that maps compressed measurements to the recovered sparse signal. The RIP condition implies that f is continuous, meaning that small changes in the measurement vector result in correspondingly small deviations in the reconstructed signal . By the Universal Approximation Theorem, such a continuous function f can be approximated by a neural network parameterized by , offering an efficient and learnable approach to sparse signal recovery from compressed data.

Proposed algorithm

To conclude, we outline our proposed method in Algorithm 2 being a combination of three consecutive steps:

Step 1: Sampling to obtain a Sparse State

Each agent quantizes its continuous state space by mapping it onto a discrete grid ensuring the data is sparse enough for effective representation, while capturing critical features, such as cluster centers. The procedure preserves the signal’s integrity using a sufficiently fine grid, facilitating accurate downstream processing. This discretization can be considered as a preliminary transformation to prepare the state for efficient compression.

Step 2: CS-Based Compression via Accelerated-LVP Aggregation

In this step, the compressed sensing principles are applied. Each agent compresses its grid-discretized state using a Gaussian measurement matrix. The CS technique capitalizes on the sparsity of the grid representation, significantly reducing data dimensionality while retaining the most relevant information. The resulted compressed portrayal serves as a reduced-dimensional stand-in for the global state, which can be used for further processing or reconstruction tasks.

In the context of aggregation, where the goal is to aggregate local measurements from multiple agents, we use accelerated optimization methods, i.e. Accelerated-LVP, to efficiently combine these measurements. The idea is to take advantage of the faster convergence of accelerated consensus methods to aggregate data quickly and accurately.

Step 3: Reconstruction via a neural network

Finally, we employ neural networks to reconstruct the centroids from the directly aggregated compressed measurements.

Algorithm 2 Decentralized clustering using compressed sensing and neural network.

Inputs:

- Agent local states

- Measurement matrix

- Communication topology

- Number of local voting protocol iterations

- Parameters of Accelerated-LVP:

Output: Cluster assignment for each agent i

Training Stage:

1: Generate synthetic dataset (states and centroids)

2: Map states to grid cells to obtain common vector

3: Compute compressed measurements

4: Train neural network to find a set of centroids based

  on

Inference Stage:

1: For each agent i do

2:   Obtain local state

3:   Map to grid cell and obtain representation

4:   Get local measurement

5:   Initialize aggregated measurement

6: end for

7: for each agent i do

8:   Update aggregated measurement using Accelerated-LVP

  (Algorithm 1) with parameters

9: end for

10: for each agent i do

11:   Find centroids set from aggregated compressed measurement

  :

12:   Determine cluster assignment:

13: end for

The proposed algorithm employs several strategies to accelerate the overall process:

• Leveraging Reduced-Dimensional Representations of States: Using lower-dimensional representations of the states substantially reduces the computational complexity of processing high-dimensional data. This facilitates more efficient data exchange and aggregation between agents, accelerating the convergence process.
• Applying Accelerated Optimization During Aggregation: During the aggregation step, accelerated optimization techniques are incorporated to speed up the convergence of the aggregation process improving the optimization efficiency, reducing the number of iterations required for consensus and enabling faster communication between agents.
• Directly Restoring Centroids Using Pre-Trained Neural Networks: The approach centers on reconstructing only the compact centroids that effectively represent the global estimate. By leveraging pre-trained neural networks for this purpose, traditional optimization methods are bypassed, enabling a highly efficient reconstruction process. These models efficiently map the aggregated measurements to the centroids, delivering a significant speed-up compared to conventional reconstruction techniques.

Generally, this combined approach not only reduces dimensionality and computational costs at each step but also accelerates aggregation and reconstruction, leading to a more efficient and scalable algorithm. Moreover, unlike traditional decentralized approaches that may require direct data sharing, our method ensures that individual sensor data is never transmitted between agents. Instead, nodes exchange only aggregated clustering-related information, significantly reducing privacy risks.

Analysis of the proposed algorithm.

Our analysis focuses on studying the impact of input errors in a neural network. These errors arise from inaccuracies in recovering the cluster structure and from the data aggregation procedure (see (8)).

Since input noise is considered in a neural network, the total network error will depend on several factors:

1. Model Error (): The error related to the architecture of the network, its parameters, and its ability to approximate the true function.
2. Training Error (): The error caused by a limited training dataset, insufficient optimization of weights, overfitting, or underfitting.
3. Input Error (): The error associated with noise or inaccuracy in the input data, i.e., and in Eq (8).

Thus, the total error of the network is:

(27)

We then explore the behavior of neural networks under input noise. First, we introduce an assumption regarding the neural network.

Assumption 4. (Lipschitz neural network). The considered neural network is supposed to be -Lipschitz continuous with respect to its input, i.e.,

Note that for a target function , the total error can be decomposed as:

Proposition 3. (Input Noise Propagation under Lipschitz Networks). Consider a base input , so that the actual (perturbed) input is . Under Assumption 4, we have:

1. Deterministic noise. If is a bounded, deterministic perturbation with , then
2. Random noise. If , then

Proof 3. 1. Deterministic Noise Case

For a target function f^*, we have

By Lipschitz continuity:

Since , we obtain

2. Random Noise Case

From Lipschitz continuity:

Taking the expectation on both sides, we derive:

Since , we get:

Then the total error satisfies:

This completes the proof.

Datasets

To evaluate the performance and robustness of the proposed approach, we conduct experiments on two types of datasets: one synthetic (artificial) and one derived from real-world observations. The synthetic dataset allows for controlled evaluation under known ground-truth coalition structures, while the real-world dataset demonstrates the method’s applicability in practical environments.

Synthetic dataset. The centroids for each cluster l are generated from a uniform distribution:

The number of points m_l for each cluster l is generated from a multinomial distribution with equal probabilities, ensuring a total of 3000 points for all clusters:

Points within each cluster are drawn from a multivariate normal distribution:

where is a small perturbation, and the covariance matrix is generated from a uniform distribution (with standard deviations in .

Real-world evaluation. We use the GeoLife GPS Trajectory Dataset developed by Microsoft Research Asia [42]. This dataset contains over 17,000 GPS trajectories collected from 182 users over a period of five years. The data was recorded with varying sampling rates and contains rich spatiotemporal information including latitude, longitude, altitude, and timestamp. GeoLife captures real human mobility patterns such as commuting, biking, and walking, making it a relevant and challenging benchmark for evaluating decentralized algorithms in location-aware and mobility-driven systems.

Baseline and metrics

We compare the method with the baseline distributed k-means algorithm presented in [7]. This method represents a strong baseline, particularly for distributed clustering under communication constraints.

We use the following metrics to compare the proposed algorithm with the baseline:

• Adjusted Rand Index (ARI) measures the similarity between the predicted clusters assignments and the actual labels, adjusted for the chance. A value of 1 indicates perfect agreement.
• Normalized Mutual Information (NMI) quantifies the mutual information between the predicted clusters and the actual labels, normalized to ensure values between 0 and 1, where 1 represents perfect agreement.
• Silhouette Score assesses how well-separated clusters are, with values ranging from -1 to 1, where higher values indicate better clustering.
• Mean Absolute Error (MAE) measures the average absolute error between the predicted and true cluster centroids.
• Dunn Index evaluates clustering quality by taking the ratio between the smallest distance between cluster centroids and the largest intra-cluster distance. Higher values reflect compact and well-separated clusters.
• Davies–Bouldin Index (DBI) assesses clustering validity based on the average similarity between each cluster and its most similar counterpart. Lower values indicate better separation and compactness.
• Communication Cost is the total number of exchanged messages across all nodes.

Remark 3. Notably, many existing methods, see, e.g., [5], are designed for scenarios with a relatively small number of agents, each having large local datasets. In contrast, our focus is on a setting where there is a large number of agents, but each possesses very limited local data. This fundamental difference makes direct comparisons with some state-of-the-art methods less applicable, as they rely on richer local data for effective clustering.

Experimental validation using synthetic data

The experiments are carried out with a batch size of 128. The Adam optimizer is used with an initial learning rate of 0.003, which linearly decreases to 0.0003. The model is trained on 1,000,000 samples and tested on a validation set of 10,000 samples. The setup involves point clouds from 1,000 agents mapped onto a grid, clustered into 10 clusters and 3 clusters. The source code is available here: https://github.com/vkpankov/cs_decentralized_clustering.

Random samples are shown in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Examples of three random samples from the validation dataset.

Each sample shows 1000 agents’ states mapped onto a ), together with the predicted centroids: 10 centroids for Samples 1 and 2, and 3 centroids for Sample 3.

https://doi.org/10.1371/journal.pone.0327396.g001

Impact of consensus procedure on the reconstruction quality. We perform distributed reconstruction of 10 clusters with simulated distributed compressed sensing, employing the Accelerated-LVP with step size . A random connectivity matrix B_t is utilized, in which two agents have 2% probability of being connected. As illustrated in Tables 1 and 2, the precision of the reconstruction increases with the number of iterations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of clustering metrics across Accelerated-LVP iterations. Part I.

https://doi.org/10.1371/journal.pone.0327396.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of clustering metrics across Accelerated-LVP iterations. Part II.

https://doi.org/10.1371/journal.pone.0327396.t002

Impact of compression on the reconstruction quality. In this experiment, we evaluate the reconstruction quality using the mean absolute error. We consider two compression modes: 14x and 7x compression, where the compressed signal is 14 times and 7 times smaller than the original signal, respectively. The communication topology is random, with a 0.02 probability that any two agents are connected. Fig 2 illustrates the relationship between clustering quality (measured by the relative MAE between predicted and ground-truth centroids) and the total volume of transmitted information across all agents. The shaded region around the curves indicates the variance observed across 30 runs with random topologies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Effect of compression on reconstruction quality under random communication topology.

The plot shows the relationship between clustering quality (measured by relative mean absolute error between predicted and ground-truth centroids) and the total transmitted information volume across all agents. Two compression modes are evaluated: 14x and 7x. The shaded areas represent variance across 30 runs with random topologies, where each pair of agents has a 0.02 probability of being connected.

https://doi.org/10.1371/journal.pone.0327396.g002

Comparison with the baseline. Next, we compare our method with the k-means-distributed baseline algorithm, aligning their communication costs to isolate and compare reconstruction accuracy. The communication cost differs by approximately 2%. Table 3 presents a comprehensive comparison of the proposed method with the distributed baseline [7] and the centralized ground-truth clustering (k-means). The proposed method demonstrates consistently superior clustering performance compared to the distributed baseline across all evaluation metrics, and it closely approaches the performance of the centralized ground-truth solution. Importantly, all differences between the proposed and distributed methods are statistically significant ( based on paired t-tests.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of clustering performance metrics for the proposed method, distributed baseline [7], and ground-truth (centralized k-means). Mean and standard deviation values are reported. Statistically significant differences (p < 0.01) were observed between the proposed and distributed methods using a paired t-test.

https://doi.org/10.1371/journal.pone.0327396.t003

The proposed method achieves high clustering accuracy, with the Adjusted rand index (ARI) is about 0.900 and the Normalized mutual information (NMI) of 0.940. These scores are significantly higher than those of the distributed baseline (ARI: 0.734, NMI: 0.877), and closely approach those of the centralized k-means solution (ARI and NMI: 0.994), indicating strong agreement with the true labels.

The proposed method also performs better in terms of structural metrics. It achieves a higher Silhouette Score (0.648 vs. 0.556) and Dunn Index (0.021 vs. 0.008), reflecting improved intra-cluster compactness and inter-cluster separation. Additionally, it yields a lower Davies–Bouldin Index (0.511 vs. 0.672), suggesting more coherent and distinct clusters. Although centralized k-means still performs best on these metrics, the gap is significantly reduced.

The Mean Absolute Error (MAE) between predicted and true cluster centroids is substantially lower for the proposed method (5.972) compared to the distributed baseline (11.107), demonstrating enhanced precision in centroid estimation. Centralized k-means achieves near-optimal MAE (0.837), but assumes full data centralization.

The proposed method also demonstrates improved communication efficiency, requiring only 8.3 million messages compared to 9.5 million for the distributed baseline. This reduction highlights the method’s scalability and practicality in decentralized settings where bandwidth is limited.

Ablation study. We also conducted experiments with alternative backbone architectures, all of which demonstrated notably lower performance compared to our proposed method. For example, a Transformer encoder with 6 layers, 4 attention heads, and a head dimension of 512 was trained on linearly projected compressed measurements for four times the number of iterations used in our model. Despite the extended training, the Transformer consistently underperformed across all clustering metrics (see Table 4). Additionally, we explored a diffusion-based method for reconstructing cluster positions and object detection-inspired approaches that treated the input as an image to identify the centers and sizes of ellipse-shaped clusters. These methods also failed to match the effectiveness of our approach.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Ablation study results.

https://doi.org/10.1371/journal.pone.0327396.t004

Scalability under varying agent counts. Fig 3 illustrates the metrics of our decentralized clustering algorithm evaluated with 1000, 3000, and 5000 agents. The overall clustering quality remains high across all tested configurations and consistently outperforms the baseline distributed method. A moderate decrease is observed at 5000 agents, suggesting that in large-scale scenarios, some architectural changes—such as increasing the network depth or the number of internal channels—should be explored to maintain quality.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Evaluation of the proposed algorithm across varying agent counts (1,000, 3,000, and 5,000), using four metrics: Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Silhouette Score, and Davies-Bouldin Index.

Each metric is plotted against the number of agents. The proposed method demonstrates consistently superior performance compared to the baseline distributed approach. A slight degradation in clustering quality at 5,000 agents indicates potential scalability limitations, which may be addressed through architectural modifications such as increased network depth or additional internal channels.

https://doi.org/10.1371/journal.pone.0327396.g003

Experimental validation using real-world data

Table 5 presents the performance metrics obtained from real-world experiments conducted on the GeoLife GPS trajectory dataset [42]. In each of the 20 runs, 3,000 data points were randomly sampled from the densest subregion of the dataset, defined by normalized longitude in the range [0.3, 0.5] and latitude in [0.5, 0.6]. The results compare the proposed method against both the distributed baseline and the centralized k-means algorithm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Real-world metrics.

https://doi.org/10.1371/journal.pone.0327396.t005

The results from the real-world evaluation confirm the effectiveness of the proposed method in producing well-structured clusters under decentralized conditions. The proposed algorithm consistently outperforms the distributed baseline across all structure-based clustering metrics, with all improvements being statistically significant .

The Silhouette score improves from 0.279 (distributed baseline) to 0.410, indicating better-defined and more coherent clusters. Although the centralized k-means still achieves the highest score (0.482), the proposed method substantially narrows the gap. The Dunn index also increases slightly from 0.002 to 0.003, suggesting improved inter-cluster separation and intra-cluster compactness relative to the baseline. The Davies–Bouldin index is reduced from 1.111 to 0.875, reflecting a more compact well-separated cluster configuration. While the centralized approach yields the lowest value (0.741), the proposed method offers a significant improvement over the baseline in a decentralized setting.

Overall, these findings demonstrate that the proposed method maintains strong performance in real-world environments and achieves a good balance between clustering quality and decentralized applicability.