1. Introduction

Text clustering is an improvement method in unsupervised natural language processing (NLP) and plays an essential role in arranging unstructured textual data in many arenas, and discovery of hidden patterns in uncategorized data [1,2]. Traditional approaches trust on static word representations like GloVe [3] or Word2Vec [4] that provide a fixed set of vectors to words and do not take context into account. However, the techniques are not easy to identify lexical ambiguity (e.g., a riverbank and a financial institution) and complicated semantic relations foremost to poor clustering results [5]. The occurrence of contextual embeddings in the model of BERT [6], RoBERTa [7], and their modifications has transformed NLP as it produces dynamic, context-sensitive representations that alteration to adapt with the surrounding text. Such models increase the linguistic nuances encoding with high clustering accuracy of intricate datasets [8]. Contextual embeddings (e.g., SBERT, DistilBERT, and RoBERTa) are different to fixed representations (GloVe, Word2Vec) as they produce dynamic and context-sensitive vectors that identify domain-specified semantics (e.g., technical jargon in Stack Overflow or similar topics in AG News). Despite the fact that contextual embeddings are superior to the static models, all of their performance vary in domains and tasks. To illustrate, RoBERTa is shown to be extremely effective in finding technical jargon in Questions in Stack Overflow [9], whereas DistilBERT is shown to be efficient in large-scale data [10]. This variability initiates the desire to use ensemble techniques where predictions of many models are used to mitigate bias and inconsistency [11]. But the averaging of the contribution of different models in a simplistic way often makes it hard to make the most of their corresponding capacities. To address this gap, this paper proposed the integration of Genetic Algorithms (GAs) [12] which is a group of evolutionary optimization methods, to dynamically optimize weights of contribution of each model. The GAs clearly overcome clustering challenges by exploiting the population-based exploration and diversity (through mutation), which otherwise cannot be achieved in traditional algorithms like a grid search or Bayesian optimization in the high-dimensional embedding spaces. Unlike extensive hyperparameter searches (grid search), or enclosed probabilistic searches (Bayesian optimization), GAs rely on tournament selection, cross-over, and Gaussian mutation to avoid local optima and evolve domain-specific fusion schemes. GAs enable adaptive combination of embeddings by refining weights with an iterative process of refinements using quality metrics of clustering (e.g., silhouette score or Adjusted Rand Index [ARI]) [13]. This GA-driven optimization, and ensemble learning have high clustering performance, which is domain-independent. To substantiate the thought of whether the framework could be replicated in the linguistic diversity, technicality and scale, the present study chose datasets (i.e., AG News, 20 Newsgroups and Stack Overflow). AG News investigates label placement of short headlines; 20 Newsgroups judges the strength of casual text where there is a topic overlap and Stack Overflow judges scalability of technical domains that need contextual accuracy. Because of these inadequacies, this paper has the following contributions towards science:

• Novel ensemble-based GA-driven unsupervised contextual embedding fusion. It suggests a novel framework that optimizes Genetic Algorithms to evolve fusion weights of heterogeneous contextual embedding models SBERT, RoBERTa and DistilBERT without ground-truth labels. Although previous GA algorithms in NLP are applied to supervised tasks and earlier evolutionary clusterization algorithms utilize single-model contextual embeddings, the suggested framework is the first to include multi-model contextual embeddings and apply GA-based weight evolution under an unsupervised goal, with L2-normalisation and a sum-to-one constraint to ensure fair and interpretable model integration.
• Simultaneous geometric and semantic optimization using dataset adaptive composite fitness. A novel fitness measure is constructed, which simultaneously maximizes the geometric cluster separation, label fidelity, and intra-cluster semantic coherence on one evolutionary signal. In contrast to earlier algorithms which made use of entirely geometric metrics, the function learns metric weighting on a per-dataset basis, allowing the framework to be used on both labeled and completely unlabeled corpora a property that has not been made possible by existing ensemble clustering methods.
• Cross-domain generalization and systematic benchmark of standalone contextual models. Standalone SBERT, RoBERTa and DistilBERT are rigorously empirically benchmarked on three linguistically different datasets AGNews, 20 Newsgroups and StackOverflow to quantify the performance limits of individual architectures and offer a community-reusable baseline. Cross-domain assessment then shows the versatility of the framework to formal, informal and technical text, and developed weights indicate domain-specific model roles that cannot be revealed by black-box ensemble methods.

This can be accomplished by reducing the distance between evolutionary computation and deep contextual learning so that this project can further extend the state-of-the-art text clustering to offer scalable and human-understandable solutions to real NLP problems. Integrating GAs and contextual embedding can be regarded as a paradigm shift in unsupervised learning. In the latter, evolutionary optimization allows for dynamic harmony in the capacities of diverse Transformer-based models (e.g., SBERT, RoBERTa, and DistilBERT). This hybrid approach surpasses conventional clustering pipelines by resolving two essential restraints of modern NLP: (1) the inadequate combination of heterogeneous embedding architectures and (2) the absence of flexible approaches for domain-specific text investigation. Distinct from static ensemble methods employing fixed weighting systems, the current GA-driven framework transforms model contributions following dataset-specific features, such as technical jargon in Stack Overflow, linguistic vagueness in 20 Newsgroups, or brief wording in AG News headlines.

2. Related studies

NLP has been transformed in contextual embeddings through substitution of static word depictions (e.g., Word2Vec [3], GloVe [4]) with dynamic, context-aware versions. This research comprehensively explain three basic models and their distinct contributions to clustering tasks as follows:

SBERT (Sentence-BERT) [14,15] SBERT adds to BERT [10,16] by presenting a Siamese or triplet network architecture to produce fixed-length sentence embeddings augmented for semantic resemblance. Unique from standard BERT, which yields token-level embeddings which necessitate manual pooling (e.g., mean or [CLS] token), SBERT adds a pooling layer (mean, max, or attention-weighted) in the pretraining for a direct production of sentence vectors. This model undergoes training on Natural Language Inference (NLI) datasets (e.g., SNLI, MultiNLI) using contrastive deficit, which takes full advantage of cosine resemblance between semantically analogous sentences while separating disparate pairs. Its construction removes repetitive BERT calculations for sentence pairs, markedly enhancing inference efficacy. In clustering tasks, SBERT highly performs owing to its uniform vector space, which contains spatially proximate texts that are semantically analogous. To illustrate, in Stack Overflow, SBERT can categorize questions on “React hooks” versus “Angular directives” by identifying complex semantic variances, rendering it exceptional for applications that need fast and interpretable clustering.

RoBERTa (Robustly Optimized BERT Approach) [7] RoBERTa is a further development of BERT, which optimizes training algorithms and hyperparameters. It substitutes BERT’s static masking with dynamically created masked tokens while training, boosting generalization by revealing the model to several contextual variations. RoBERTa also uses longer epochs (up to 500 k steps) and larger batch sizes (8,000 vs. 256 in BERT) and thus improves the convergence and stability. One notable structural change is the removal of the Next Sentence Prediction (NSP) task of BERT, replaced by segment-pair + −inference to downstream tasks. These transformations give RoBERT a deep contextual insight, making it outstanding in areas with technical or unclear language. In AG News, RoBERTa distinguishes between similar categories, like the world and business, by modeling abstruse discourse patterns. Its strength to polysemy (e.g., “Apple” as a company versus a fruit) guarantees uncluttered cluster boundaries, including datasets with high lexical vagueness.

DistilBERT [10,16] is a lightweight variant of BERT that has been kept in its purified form and retains 97% of the performance of BERT with minimal computational cost. It gets to this state through refining the knowledge by training a smaller student model (66 M parameters vs. BERT-base 110 M parameters) to behave like BERT through logit distillation (corresponding yield prospects) and attention distillation (positioning attention head weights). The result of this process is a 60x faster inference rate with an insignificant loss in precision, which makes DistilBERT extraordinary in large-scale clustering workloads like in queries to Stack Overflow (millions of queries). Despite the fact that the DistilBERT one is slightly less complex than the RoBERTa, its effectiveness enables real-time pipelines clustering in low-resource environments. Its trade-off between speed and veracity makes it particularly credible when paired with GA-driven ensemble models, which compensates low precision drops by a dynamic compensatory contribution of more high-fidelity models like RoBERTa.

Some methods that have been used to cluster texts using contextual embeddings are KMeans [17,18], DBSCAN [19], and hierarchical clustering. Nonetheless, such models are mostly treated as black-box feature extractors in research, in spite of the potential to dynamically combine their capabilities. An example would be that SBERT is preferred in semantic similarity-based clustering (such as collecting related news articles) whereas RoBERTa is better at domain-specific clustering (such as technical Stack Overflow threads). DistilBERT is comparable to large-scale clustering that basically demands speed (e.g., analysis of social media in real-time). Standalone models, although individually capable, have the challenge of adapting to problem-specifics of the data, including balancing efficacy and veracity or disentangling overlapping topics. In this regard, the study also advances previous techniques by proposing a GA-based ensemble that allows dynamically balancing the capacities of these models without constraints that limit them individually through evolutionary optimization.

Ensemble learning has emerged as a paradigm of authority in natural language processing (NLP), combining predictions of many models to enhance strength, generalizability and performance on a variety of tasks. Utilizing the respective capabilities of individual models enables ensemble techniques to reduce the danger of overfitting, decrease variance, and improve veracity. The three most prevalent strategies characterize this field: stacking, voting and hybrid architectures, and each has its own rewards and concerns of NLP application. Stacking is based on dynamic weight predictions of base models (e.g., BERT and RoBERTa) using a meta-model (e.g., logistic regression) as an input to their output [20]. The hierarchical model is effective in supervised tasks like classification, where labeled training data guide the meta-model to prefer high-performing models (e.g., use BERT on syntax, RoBERTa on semantics). Its reliance on ground-truth labels and computational complexity however limit its use in unsupervised clustering that does not have labels and critically needs to be calibrated dynamically [20].

Voting sums make predictions through majority voting (hard voting) or probability averaging (soft voting) [21]. Hard voting selects the most frequent forecast of the class, and soft voting selects the most average forecast of the class. Although its sorting method is simple and effective, its results depend on the existence of multiple models (homogeneous models, such as multiple variants of BERT) which regularly produce related errors. There is also a problem of adaptation to domain-specific issues in static systems (e.g., technical language vs. colloquial language), and therefore the lack of flexibility in dynamic clustering configurations [21,22].

Hybrid models join constructions (e.g., Transformers + CNNs) to take advantage of corresponding capacities, such as global attention (Transformers) and local pattern detection (CNNs) [23]. These systems enhance activities like machine translation but are faced by the issue of clustering that are brought about by skewed embedding spaces and training tasks. An unlabeled data that continues to make it challenging to optimize mismatched architectural objectives (e.g., the semantics of SBERT and the efficiency of DistilBERT), limiting their use in unsupervised contexts despite their hypothetical potential [23].

Despite their success in the supervised NLP tasks, the ensemble learning techniques are faced by two significant challenges in the unsupervised clustering namely that of two critical limitations. To begin with, they are limited by supervision bias: algorithms like stacking and voting require labeled data to normalize the weights of models or to support predictions [20,21]; but clustering problems do not have ground-truth labels, forcing them to rely on random or fixed weighting schemes. Second, fixed weighting impedes flexibility – existing ensembles repeatedly apply fixed, existing weights (e.g., equivalent shares of SBERT, RoBERTa, and DistilBERT) [11], without addressing dataset-specific issues such as technical jargon in Stack Overflow (where RoBERTa contextual complexity shows excellent results) or scale requirements in AG News (where DistilBERT efficacy is central). This dogmatism threatens inadequate clustering by its insensitivity to the tradeoffs between veracity, computational cost, and domain-specific language patterns. Such vulnerabilities highlight the urgency of using adaptive ensemble models that dynamically adapt model contributions when no labeled data is available- an issue directly addressed by the GA-driven methodology proposed in this study.

Inspired by the concepts of natural selection and genetic variation, GAs are population-based metaheuristics that give sound responses to complex optimization issues in machine learning (ML). GAs uses process simulation including, selection, crossover, and mutation, to explore iterative evolution of potential results (e.g., feature subsets, hyperparameters, or model weights) to maximize a given fitness function. Their adaptability has led to wide usage in the three central areas of ML listed below: Feature Selection: GAs are effective in identifying the best feature subsets of high-dimensional data to reduce computing costs, and retain prognostic control [24]. In text classification, to give an example, GAs promote mixtures of n-grams or TF-IDF features to promote discriminative expressions and eliminate noise. This feature is of particular importance in clustering, where the immaterial or repetitive attributes can falsely reflect the measures of resemblance and ruin cluster superiority.

Hyperparameter Tuning: GAs specialize in streamlining the procedure of optimizing the key hyperparameters (e.g., learning rates, regularization strengths, or the number of Transformer layers) by enhancing groups of potential structures [25,26]. Unlike gradient-based algorithms, GAs are able to guide non-convex search spaces, which makes them outstanding in tuning deep learning constructions with non-uniform performance landscapes. Existing uses include optimization of BERT fine-tuning parameters to domain-specific tasks [25,26], but the role of its use in unsupervised tasks like clustering remains uninvestigated.

Model Fusion: GAs enable adaptive ensemble learning, which can proceed by optimizing weights to combine the forecasts of heterogeneous models (e.g., BERT, RoBERTa, or CNNs) [27]. This approach outperforms its counterparts using static fusion schemes (e.g., averaging) in supervised tasks by dynamically choosing models with the capability to excel in a given situation. As an example, a GA-based ensemble has the potential to assign more weight to the RoBERTa on technical domains, e.g., Stack Overflow, and DistilBERT on large-scale AG News clustering. GAs thrives in the contexts in which the traditional optimization techniques are weakened. They also work with non-differentiable and discontinuous fitness landscapes like clustering measures (e.g., silhouette score or Adjusted Rand Index [ARI] with no gradient cues). Their crossover and mutation operators promote multiplicity of population, and dampen premature merging to local optima. More so, GAs enhances multi-objective optimization, balancing conflicting objectives (e.g., leveraging clustering veracity and minimizing computation cost) through Pareto front analysis [27].

Although GAs have been dominant in supervised NLP tasks, they have not been combined with unsupervised learning much. They are used in text summarization, where GAs develops sentence selection algorithms to achieve equalizing educational quality and brevity [28], and sentiment analysis; in the latter, feature weights are trained to classify by polarity [29]. In more recent work, GAs have been used to search the neural architecture of Transformer-based models, including to systematize the layer configuration of tasks like machine translation [30]. Nonetheless, there has been deep scrutiny as far as the application to contextual embedding fusion of clustering is concerned that is a core fault in unsupervised NLP. The existing GAs-based ensembles are often relying on fixed feature representations (e.g., TF-IDF) rather than on contextual embeddings, which tend to fail to optimize semantic richness of models like SBERT or RoBERTa. The present study bridges this gap by proposing the first GA-based framework of the dynamic weighting of contextual embeddings, which enables adaptive, label-free clustering on heterogeneous domains.

Despite the remarkable advancements in contextual embeddings and evolutionary optimization, three essential limitations remain in unsupervised text clustering, which inspires these research innovations. First, existing ensembles depend on static weighting systems (e.g., fixed averaging of SBERT, RoBERTa, and DistilBERT embeddings), which are unable to adjust to domain-specific intricacies. To demonstrate, static fusion can potentially weaken RoBERTa’s contextual accuracy in technical domains such as Stack Overflow or fail to see SBERT’s semantic clarity for AG News headlines, resulting in deficient cluster boundaries. Second, scarce exploration persists on GA-driven embedding optimization in unsupervised settings, although GAs are succeeding in supervised tasks (e.g., hyperparameter tuning [24,25,27,31,32]). rigid fusion methods are typically used in clustering frameworks (e.g., concatenation), disregarding the potential ability of GAs to transform dynamic weights that even out veracity, efficacy, and high-dimensional embedding spaces. Third, deficiency exists in cross-domain evaluation, as several frameworks corroborate on fine datasets (e.g., news articles), concealing generalizability across heterogeneous domains such as AG News (formal headlines), Stack Overflow (technical jargon), and 20 Newsgroups (casual text). Accordingly, this research further develops unsupervised text clustering via three essential innovations. First, the first GA-driven framework is presented for dynamic weight contextual embeddings (i.e., SBERT, RoBERTa, and DistilBERT), customizing model contributions to domain-specific requirements (e.g., choosing RoBERTa as the primary choice for technical Stack Overflow queries or SBERT for AG News’ semantic clarity). This adaptive fusion exceeds static ensembles through the iterative optimizing weights that utilize clustering metrics (e.g., silhouette score). Second, it sets up a standard for standalone contextual embedding clustering, measuring compromises such as DistilBERT’s 60% faster inference versus negligible precision deficits and determining domain-specific capacities (e.g., RoBERTa’s polysemy disambiguation). Finally, it proves the cross-domain strength by confirming the framework on three various datasets (20 Newsgroups, AG News, Stack Overflow), with ablation studies proving adaptability to linguistic diversity, technicality, and scale. By filling the fundamental gaps in dynamic weighting, GA optimization, and generalizability, this study offers a new meaning to unsupervised clustering as a flexible, effective, and domain-aware method.

3. Methodology

This study presents the integration of contextual embeddings, the Genetic Algorithm (GA) optimization, and clustering (Fig 1). To produce multi-model contextual embeddings, input raw text is transformed into semantic resemblance-based SBERT, contextual complexity-based RoBERTa and computational efficacy-based DistilBERT are chosen: SBERT, RoBERTa, and DistilBERT are corresponding to each other. Thereafter, an ensemble module where weights of the individual model are energetically adjusted based on optimized weights of the model, driven by the quality measures of the clusters (e.g., silhouette score and Adjusted Rand Index). This automatic combination will ensure the stabilizing procedure of domain-specific trades (e.g., preferring RoBERTa accuracy in technical Stack Overflow queries or DistilBERT speed in large-scale AG News groupings). The eventual weighted embeddings are then subjugated to a clustering algorithm, and semantically comprehensible and domain-aware clusters are made. The framework modularity makes it permit to modify and adding other clustering methods or models, and its scheme generated by GA merits robustness on heterogeneous data (e.g., AG News, 20 Newsgroups, and Stack Overflow). This is an end-to-end system that closes the divide between the static embedding fusion and dynamic, label-free optimization which allows us to have scalable and interpretable text clustering. The introduced framework integrates the three mechanisms through the stepwise process as follows:

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Flowchart of the proposed framework.

https://doi.org/10.1371/journal.pone.0353506.g001

Input: The system is fed with a set of raw text documents . The optimization process does not need any ground truth labels.

First stage: Embedding Extraction (Deep Models): Three pre trained models SBERT, RoBERTa and DistilBERT are applied individually to individual documents. The normalized embedding vectors of each document form the output of each model m. The result of this step is a collection of three embedding matrices ESBERT, ERoBERTa, EDistilBERT , with being the embedding dimension (768 in the case of SBERT/DistilBERT and 1024 in the case of RoBERTa). These matrices are the input in the GA module.

Second stage: GA Driven Ensemble Optimization: The GA works with a population of chromosomes, each representing a weight vector having and On a specific chromosome, the framework calculates fused embeddings of each document. Such fused embeddings are then clustered with the help of an adequate algorithm (e.g., K-means or DBSCAN or spectral clustering). The quality of the resultant clusters is measured through a composite fitness function which is a combination of Silhouette Score, ARI (in the event of the presence of labels) and Topic Coherence. To maximize this fitness, the GA successively mutates the population, which is selected to include tournament selection, uniform crossover and Gaussian mutation. This step produces the optimal weight vector w 0 which produces the best quality of clustering of the data set.

Third stage: Clustering and Evaluation: With the best weights, the resulting final ensemble embeddings are computed and sent to a clustering algorithm of choice on the dataset. The result is an output of cluster assignments of the input documents. The evaluation of these assignments is done based on internal (Silhouette Score, Topic Coherence) and external (ARI, NMI) measures.

This design has the advantage that the GA learns dynamically to integrate the strengths of the three deep models depending on the inherent structure of the data, and does not need to be manually tuned or labeled data.

3.1. Stage 2: Contextual embedding models

3.1.1. Model selection rationale.

I. SBERT (Sentence-BERT) [14] boosts the BERT [10] construction by presenting a Siamese or triplet network framework in the pretraining, directly yielding fixed-length sentence embeddings via integrated pooling layers (mean, max, or attention-weighted). This step eliminates the necessity of manually pooling at the token level (e.g., with [CLS] tokens) and accelerates inference, making SBERT outstanding in the context of fast clustering. SBERT is trained on Natural Language Inference (NLI) data (e.g., SNLI, MultiNLI) using contrastive deficit, which enriches the embeddings to exploit fully the cosine similarity between semantically similar sentences, and separates unrelated pairs with cosine similarity Eq. 1:

(1)

In which and are similar sentences, is a sentence that is not related to and , and m is the margin. This equation ensures a balanced space of vectors in which spatial proximity is based on semantics. The strengths of SBERT include efficiency (reducing repetitive BERT computations), semantic precision (complex texts like “React hooks” are clustered together, versus complex texts like “Angular directives” in StackOverflow), and interpretability (clean cluster boundaries due to reliable geometric properties). As an example, in Stack Overflow, SBERT embeddings are able to make discriminations between queries about React state management and Redux architecture by detecting context-absurdist differences in common phrases like state. Its effectiveness is due to its structure of grouping tasks that require fast and interpretable grouping, including creating Stack Overflow queries based on an intent or categorizing AGNews headlines based on a topic.

II. Robustly Optimized BERT Approach (RoBERTa) [7] enhances BERT through training process improvement and hyperparameter optimization, generating quality contextual understanding to cluster tasks. It replaces the masking of BERT with dynamic masking, where tokens are masked randomly in training with a Bernoulli distribution Eq. 2:

(2)

where is the masking of token i. This calculation divides contextual exposure thereby boosting generalization. Moreover, RoBERTa learns on 500,000 steps with higher batch sizes (8,000 vs. 256 of BERT); the weights are balanced during this process with gradient accumulation. In contrast to BERT, it does not have the Next Sentence Prediction (NSP) goal. On the other hand, it employs segment-pair ± inference to do downstream tasks. RoBERTa keeps BERT’s masked language modeling (MLM) deficit as follows Eq. 3, [6,7]:

(3)

where indicates an x masked with token i. This calculation trains the model to rearrange masked tokens based on the adjacent context and thus expands linguistic knowledge. RoBERTa is impressive in clustering by distinguishing overlapping subjects (e.g., world vs. business in AGNews) and polysemy (e.g., Apple as a company vs. a fruit) by modelling discourse patterns. Its domain-specific accuracy makes it outstanding in technical domains like StackOverflow, where jargon (e.g., “React hooks” vs. Angular directives) can only be evaluated in the context of a complex domain. As an example, in AG News, the difference between headlines like “Apple stock price” (business) and “Apple carbon footprint”(world) is determined by the presence of abstruse variances in modifier phrases (e.g., stock price vs. carbon footprint). These powerful capabilities make RoBERTa a better precision choice in clustering the tasks requiring a strong level of context clarification.

III. DistilBERT is a refined alternative of BERT [10], keeping 97% of BERT’s performance while minimizing constraints from 110 M (BERT-base) to 66 M, rendering it excellent for large-scale clustering tasks. Its configuration capitalizes on knowledge concentration, where a minor student model (DistilBERT) learns to mimic BERT’s behavior via two strategies Eq. 4:

(4)

where is cross-entropy deficit positioning output probabilities (), reduces Kullback-Leibler deviation between attention matrices (), and α = 0.5 evens out the two objectives. This procedure will ensure that DistilBERT gains BERT contextual insights and 60% faster inference. In clustering, DistilBERT shows very good results in large-scale datasets (e.g., Stack Overflow) where computational efficiency is paramount, but with small veracity losses compared to RoBERTa. Its speed vs. precision stability makes it suitable in resource-limited environments, like real-time social media exploration or processing millions of technical queries. As an example, in Stack Overflow, DistilBERT can quickly sort questions like React state management and Redux architecture and maintain semantic adherence. Despite being less efficient than RoBERTa with vague meanings (i.e., when asked to answer about Apple as a company, not a fruit), the effectiveness of DistilBERT ensures scalability without drifting away the main clustering objectives. With DistilBERT used as part of the GA-based ensemble, the system dynamically privileges its input in cases where speed is important, like clustering 120 k headlines of AG News or the queries in the giant warehouse of Stack Overflow.

3.1.2. Embedding extraction.

The framework selects contextual embeddings with a designed pipeline that is specific to a model design. First, tokenization transforms input text into model-specific token sequences (e.g., the BERT tokenizer to RoBERTa/DistilBERT, the SBERT tokenizer to sentence-level processing), ensuring that they are congruent with the training goals of the individual model [33,34]. Next, pooling techniques transform token-level representations into fixed-length representations:

1. Mean Pooling: Averages token embeddings to produce general-purpose vectors Eq. 5:

(5)

where denote token embeddings, and n represents sequence length. This process evens out efficacy and semantic rationality, excellent for broad clustering tasks.

1. [CLS] Token: Utilizes the first token’s embedding () to detect global semantics [35,36], essential for unravelling abstruse subtleties (e.g., differentiating “Apple’s stock price” vs. “Apple’s carbon footprint” in AG News).
2. Attention-Weighted Pooling uses attention scores to give attention to domain-specific features on the weights of token embeddings. A plan that has been supported in technical fields like Stack Overflow [37]. This procedure balances out productivity and situational conformity, especially to lightweight models like DistilBERT Eq. 6 [38].

(6)

This formula is especially compelling for technical domains (e.g., highlighting “React hooks” vs. “state management” in Stack Overflow). Finally, all embeddings undergo L2-normalization Eq. 7:

(7)

guaranteeing equitable weighting during GA-driven fusion. When normalization is absent, models such as RoBERTa with naturally larger embedding magnitudes may overpower optimization, consequently contorting the outcomes. This pipeline orchestrates construction capacities: SBERT’s mean-pooled embeddings favor semantic resemblance, RoBERTa’s [CLS] token detects hierarchical deliberation, and DistilBERT’s attention-weighted pooling evens out efficacy with contextual adherence. The outputs are normalized and their contributions are dynamically calibrated to dataset-specific needs (e.g., DistilBERT is favored in AG News because of its conciseness and RoBERTa is favored in Stack Overflow because of its technical acuity).

3.2. Stage 3: GA-Driven ensemble

The proposed framework incorporates GAs in the dynamically updated optimal weights of contextual embedding models (i.e., SBERT, RoBERTa, and DistilBERT) in clustering [12,33]. This concept through GA ensures adaptive embedding fusion by refining a model contribution iteratively on the clustering quality measure (e.g., Silhouette Score and Adjusted Rand Index) [13,39]. Unlike the more basic fusion techniques (e.g., uniform averaging), the GA leverages the power of population-based exploration and diversity generated by mutation to prevent local optima in high-dimensional embedding spaces, which is a limitation of grid search or Bayesian optimization [25]. It then proceeds to provide a detailed description of the architectural design of the GA, such as chromosome representation, fitness functions, and operator strategies, tailored to domain-specific problems [34,39].

3.2.1. Chromosome representation.

Each chromosome in the GA encodes a set of normalized weights that dynamically even out contributions from SBERT, RoBERTa, and DistilBERT Eq. 8:

(8)

where and ∑= 1. Equitable model incorporation and domain-specific flexibility are guaranteed in this incorporation, thereby solving fundamental problems in unsupervised clustering [12,40,41]. The chromosome design warrants equitable and interpretable model fusion through the enforcement of the following two key constraints: L2-normalization and a sum-to-one constraint. L2-normalization () inhibits models with naturally larger embedding magnitudes (e.g., RoBERTa) from overpowering the fusion, thus guaranteeing that weights follow semantic pertinence instead of scale divergence [34,36]. The sum-to-one constraint warrants interpretability (e.g., a RoBERTa weight of 0.6 means 60% contribution) while circumventing deteriorating outcomes (e.g., over-reliance on a single model) [34]. Further streamlining optimization requires weights to be bounded between 0 and 1, limiting the search space to a low-dimensional simplex manifold (∑ = 1); the latter removes fallacious regions (e.g., negative weights) while keeping the right combinations [36]. This interpretable gene mapping (e.g., gene 1 = SBERT weight, gene 2 = RoBERTa weight) facilitates intuitive investigation of evolved strategies, such as recognizing RoBERTa dominance in technical domains such as Stack Overflow. The GA takes advantage of this architecture to permit iterative refinements of weights based on clustering quality measures (e.g., Silhouette Score and Adjusted Rand Index) to use fusion strategies to domain-specific issues: favoring DistilBERT’s efficacy for AG News’ conciseness (e.g., short headlines such as “Apple stock drops 5%”), highlighting RoBERTa’s contextual exactness in unraveling Stack Overflow’s technical jargon (e.g., “React hooks” vs. “Redux architecture”) [7] and equalizing SBERT’s semantic resemblance with DistilBERT’s speed to address overlapping topics in 20 newsgroups (e.g., “religion” vs. “politics”) [14,34]. Through the encoding of model contributions as a bounded, interpretable vector, the chromosome design, adaptive optimization, is made possible through the encoding of model contributions to a restricted, interpretable vector, applicable in heterogeneous datasets. Therefore, it bridges the gap between rigid and fixed fusion of static learning and flexible and label-free evolutionary learning.

3.2.2. Fitness function.

The fitness function is used to measure the quality of clustering to guide the GA to the best model fusion. It uses three metrics that are complementary, Silhouette Score, Adjusted Rand Index (ARI) and Topic Coherence to ensure a high degree of domain-sensitive optimization. Their roles and application can be described in detail as shown below:

1. Silhouette score

Silhouette Score measures the interconnection and divergence of clusters, ensuring that embeddings form distinct, accurate clusters Eq. 9 [13]:

(9)

is the mean distance intra-cluster (similarity to its own cluster), is the minimum average distance inter-cluster (dissimilarity to the nearest competing cluster). The range of scores is between −1(poor clustering) and 1(optimal separation) with the values being close to 0. The latter means overlapping clusters. This measure is biased in the Genetic Algorithm (GA) to favor solutions that exploit intra-cluster similarity (e.g., cluster together Stack Overflow queries with semantic similarity) and minimize inter-cluster overlap (e.g., distinguish between React hooks and Redux architecture). The Silhouette Score ensures evolved weights (e.g., RoBERTa dominance in technical domains) to make clusters more clear, which makes it highly suitable to datasets that have overlapping or poorly defined clusters (e.g., 20 Newsgroups). This emphasis on geometric consistency makes the fusion constraints of a static system limited, and permits adaptive optimization of contextual embedding contributions to solid, explainable clustering.

2. Adjusted rand index

The ARI compares clustering accuracy to ground-truth labels (where available) Eq. 10:

(10)

RI is the Rand Index (the fraction of correctly labeled pairs), and E[RI] is the expected Rand Index under random labeling, ARI condones chance-based groupings as an excuse to justify robustness [12]. This scale gives a preference to those answers that align with domain-specific labels, e.g., distinguishing between stock price (business) and carbon footprint (world) in AG News. The GA uses the capitalization of ARI to turn weights that maintain label adherence and minimize biases that arose when using the plain fusion approaches. This scenario enables adaptive optimization of contribution of contextual embeddings in labeled datasets.

3. Topic coherence

In case of unlabeled datasets, UMass Topic Coherence measures the semantic coherence within clusters by computing term co-occurrenceEq. 11 [42,43]:

(11)

In which is the likelihood of the co-occurrence between terms and in cluster , and epsilon is a smooth parameter that does not allow to be divided by 0. This measure ensures clusters contain semantically related phrases (e.g., “React hooks” and state management in StackOverflow) by declaring high co-occurrence rates and approving independent term assignments. Topic Coherence is fundamental in the GA to optimize unlabeled datasets (e.g., 20 Newsgroups) in which no ground-truth labels are available. The exploitation of coherence leads the framework to induce the weights that lead to focusing on contextual correspondence, e.g., the meticulousness of RoBERTa in technical subjects or the semantic similarity of SBERT in vague issues. This focus on the term-level interactions enhances interpretability, ensuring that clusters have applicable linguistic patterns as opposed to arbitrary associations.

3.2.3. Implementation considerations.

The GA uses weighted fitness functions and dynamically selected metrics to ensure the flexibility in both labeled and unlabeled datasets. The fitness function combines the three most important metrics in a composite objective as follows Eq. 12:

(12)

where α + β + γ = 1, and weights (α,β,γ) favor objectives based on domain requirements (e.g., α = 0.5, β = 0.3, γ = 0.2 for AG News) [12,40]. This procedure balances out accuracy (through Silhouette Score and ARI), effectiveness (through Topic Coherence), and domain-related tradeoffs (such as preferring the accuracy of RoBERTa to technical domains). In the case of labeled data (like AG News), the GA prefers ARI to match clusters with the predetermined categories (e.g., unraveling [business] Apple stock price vs. unraveling [world] Apple carbon footprint) [44]. In the case of unlabeled data like Stack Overflow or 20 Newsgroups, the framework experiences a dynamic change in priorities to Silhouette Score and Topic Coherence, highlighting geometric divergence and semantic coherence (e.g., covering similar topics like religion vs. politics in 20 Newsgroups) [42]. This flexible design guarantees strong optimization across heterogeneous domains while sustaining computational tractability, linking static embedding fusion and adaptive, label-free learning.

3.2.4. Genetic algorithm operators.

Three core operators are employs in GA, specifically, selection, crossover, and mutation for the reiterative refinement of chromosome weights and optimization of clustering achievement. Selection employs tournament/k-tournament strategies, where k chromosomes undergo arbitrary sampling, and the most suitable solution is kept. This process evens out exploration and exploitation by keeping high-performing solutions while sustaining population multiplicity [24]. Crossover presents hybrid solutions through the following:

1. Single-point crossover: Merges parent chromosomes at an arbitrary axis (e.g., swapping weights after the second gene), keeping partial results while presenting new combinations [40,44].
2. Uniform crossover: Arbitrarily switches genes between parents to discover wide-ranging solution spaces, allowing for the merging of capabilities from multiple models (e.g., SBERT’s semantic resemblance + RoBERTa’s contextual precision) Eq. 13 [40,44].

Lastly, mutation guarantees strength by upsetting weights via Gaussian noise:

(13)

with σ controlling the strength of mutation, this mechanism maintains population multiplicity, prevents merging prematurely, and allows finding new weight combinations (e.g., finding the best trade-offs between the speed of DistilBERT and the accuracy of RoBERTa) [25]. All these operators make the GA dynamically optimize the incorporation of fusion strategies to domain-specific considerations, e.g., preferring RoBERTa to the technical language of Stack Overflow or DistilBERT to the scalability needs of AG News. The entire ensemble optimization (combining the operators above) based on a GA is given in Algorithm 1.

Algorithm 1: GA-Driven Ensemble Optimization

Input: documents emdedding_models M = {SBERT, RoBERTa, DistilBERT}, Population size P, Generations G, Mutation Rate μ, Crossover Rate χ, Elitis m Rate

Output: optimal weight

1. //stage 1: embedding extracting

2. for each emdedding_models m M do

3.   for each d

4.     ex_dim[d] m.ExtractEmbedding(d)   // dim:767 or 1025

5.     ex_dim[d] L2Normaliztion(ex_dim[d])

6.    end for

7.   end for

8.   //stage 2: GA_initialization

9.   Population InitializePopulation(p)//each_chromosome= with

10.   //stage 3: Evolutionary_Optimization

11.   for generation 1 to G do

12.    for each chromosome population do

13.     // weighted_embedding_fusion

14.     for each document d

15.      e_ensemble[d] SBERT[d] + RoBERTa[d] + DistilBERT [d]

16.    end for

17.    //Clustering and fitness evaluation

18.    clusters cluster(e_ensemble)

19.    fitness[c] α.Silhouette+β.ARI+.Coherence

20.   end for

21.   //Genetic Operators

22.   New_ Population elite_selection (population, )

23.   while |New_ Population| < P do

24.    parent1 tournament_slection(population, )

25.    parent2 tournament_slection(population, )

26.    child UniformCrossover(parent1, parent2) with probability χ

27.    child GaussianMutation(child, μ, )

28.    Child EnforceSUM_constaint_(child)   //renormalization to

29.    New_ Population.add(child)

30.    end while

31.    Population New_ Population

32.    if fitness improvement < for 10 generations then break

33.   end for

34.   //stage 4: output

35.    argmax fitness(population)

36.   return

3.3. Ensemble clustering strategy

Weighted Embedding Fusion: The final ensemble embeddings are created via the incorporation of normalized contextual embeddings from SBERT, RoBERTa, and DistilBERT using Genetic Algorithm (GA)-evolved weights Eq. 14:

(14)

where and The GA optimizes these weights dynamically, balancing domain-specific tradeoffs, like attention to RoBERTa contextual accuracy to Stack Overflow technical jargon or to the effectiveness of DistilBERT to AG News in conciseness. Each embedding is L2-normalized, prior to fusion, as follows Eq. 15:

(15)

assure fair weighting by removing extent biases (e.g., not permitting larger embedding norms in RoBERTa to control as a result of architectural measure). This normalization phase conserves the geometric relational properties between embeddings. However, it allows comparisons of weights in an explainable manner.

4. Experiment results

4.2. Baseline models

To evaluate the proposed GA-based ensemble model, this work conducted a compare with three categories of baselines which include separate models, static fusion models, and state of the art clustering models. To begin with, individual contextual embedding frameworks, i.e., SBERT, RoBERTa, and DistilBERT, are evaluated individualistically with K-means, DBSCAN and spectral clustering. The models below can be used as a benchmark:

• SBERT shows strong performance in semantic resemblance tasks (e.g., clustering similar Stack Overflow questions like: React hooks vs. Redux architecture).
• RoBERTa is biased towards contextual specificity, responding to the overlapping issues in AG News (e.g., “Apple stock price [business] vs. Apple carbon footprint [world]) with a response.
• DistilBERT balances large-scale efficiency (e.g., when working with millions of Stack Overflow queries) at the cost of less trivial accuracy compared to RoBERTa.

The Static Ensemble (Equal Weighting) is then a combination of SBERT, RoBERTa, and DistilBERT with equal weights to test whether dynamic GA-enhanced fusion is better than naive averaging. This baseline is based on an equal contribution of all models, ignoring domain-specific tradeoffs (e.g., the contextual complexity of RoBERTa with technical domains or the efficiency of DistilBERT with conciseness in AG News). This paper substantiate how adaptive weighting enhances clustering quality (e.g., resolving vague polysemous words like Apple in AG News or state management in Stack Overflow) by contrasting it with this approach. Finally, comparison to state-of-the-art clustering algorithms is carried out:

• HDBSCAN (hierarchical density-based clustering) processes noise and overlapping topics in casual text (e.g., 20 Newsgroups).
• BERT-CC (contrastive clustering) exploits BERT’s masked language modeling to polish cluster boundaries.
• GA-Optimized TF-IDF utilizes evolutionary algorithms to conventional feature selection and functions as a baseline for label-free optimization.

Table 1 measures their performance across AG News, 20 Newsgroups, and Stack Overflow using Silhouette Score, ARI, NMI, and Topic Coherence. These comparisons highlight the superiority of the GA-driven framework to level out accuracy, efficiency, and domain-specific adaptability and reduce the discrepancy between fixed fusion and dynamic, label-free optimization.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of baseline models vs. proposed GA-Driven ensemble.

https://doi.org/10.1371/journal.pone.0353506.t001

Each of the baselines will be tested using the same metric on the same three data (Silhouette, ARI, NMI, Topic Coherence). The statistical significance of the proposed method with each of the baselines is determined. Table 1 summarizes results indicating that GA driven ensemble is more successful than any of the baselines in all the datasets, the difference being significant. This extended is a good indication of the excellence of the proposed framework.

The scores of UMass Topic Coherence (Table 1) demonstrate the level of effectiveness, when using the proposed GA-driven ensemble, to ensure semantic consistency of clusters, particularly with unlabeled datasets like Stack Overflow. The metric where clusters of high-frequency terms have a significant co-occurrence (e.g., React hooks and state management in Stack Overflow or carbon footprint and environmental impact in AG News) are rewarded, and expressions arbitrarily linked to each other are sanctioned. The GA-based framework achieves much greater scores in coherence than any baselines, and therefore demonstrates that it is able to effectively convert domain-aware fusion strategies.

The UMass Topic Coherence score of GA-driven ensemble is 0.78 which is significantly higher than each of the models (SBERT: 0.65, RoBERTa: 0.70, and DistilBERT: 0.58) and the unchanging ensemble (0.62). The GA works perfectly since it is capable of dynamically biased towards the contextual accuracy of RoBERTa (=0.6) that decontextualizes similar categories, like Apple stock price (business) and Apple carbon footprint (world). The strength of RoBERTa is that it models the modifier phrases (e.g., stock price vs. carbon footprint) in a complex way that ensures that clusters are tied to domain-specific semantics rather than to the overlap of these words. The framework, through successive improvement of weights in the context of coherence-driven fitness, justifies the emphasis of the evolved results on the rich contextual knowledge of RoBERTa, which is capable of addressing the vagueness not addressed by a fixed fusion process.

In the case of 20 Newsgroups, the GA-based ensemble has a Topic Coherence score of 0.70, which is higher than individual models (SBERT: 0.50, RoBERTa: 0.56, and DistilBERT: 0.46) and the default ensemble (0.51). This improvement is explained by the fact that the GA fairly allocates weights between SBERT (=0.4) and DistilBERT (=0.4) that deal with vague concepts (e.g., a religion vs. a political party) mutually. The framework guarantees that clusters keep the semantically comprehensible expressions (e.g., “prayer” and “faith” in religion vs. “government” and “elections” in politics).

On the Stack Overflow, GA-driven ensemble (0.60) has higher score in Topic Coherence compared to SBERT (0.42), RoBERTa (0.49), DistilBERT (0.42), and static ensembles (0.43). The GA is dynamic in assigning weights, SBERT =0.4 and RoBERTa =0.5 to untangle the technical jargon (e.g., Redux architecture vs React hooks). On the other hand, to fixed fusion, which makes contextual accuracy less accurate, the GA prefers RoBERTa to its capability to identify discourse patterns in technical settings (e.g., React and Redux actions and differentiating between state management). By emphasis on domain-specific co-occurrence of terms and symmetrical consistency, the framework can ensure that clusters are directed by intent-based clustering rather than the insignificant similarities, and ensure that the GA is adaptive optimization rather than inelastic averaging.

4.3. Results and discussion

As identified in Table 2, there are better results in the ensemble due to the GA: + 12% in Topic Coherence over the ensemble of the static models and over single models +10%−15% achieved in ARI and NMI. RoBERTa is the most resilient standalone model (ARI = 0.67) and it makes use of its contextual accuracy to disentangle overlapping groupings (e.g., Apple stock price [business] vs. Apple carbon footprint [world]). Nonetheless, the GA-based framework beats RoBERTa with the addition of weights () to incentivize the complexity of the context and equalize SBERT () to semantic stability. This combination ensures that clusters match with the abstruse modifier phrases (e.g., stock price vs. carbon footprint) and avoids inherent vagueness of the fixed averaging. The level of statistical significance tests prove that p is less than 0.05 which proves the effect of the GA of adhering to labels and the level of semantic consistency.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Clustering performance comparisons on AG News dataset.

https://doi.org/10.1371/journal.pone.0353506.t002

As shown in Table 3 in the 20 Newsgroups dataset, the GA-based ensemble is significantly better than all baselines which achieve +16% greater Silhouette Score and +15% greater Topic Coherence than the fusion with a static ensemble. Individual models like RoBERTa (Silhouette = 0.47) and SBERT (Topic Coherence = 0.50) are reasonable but can not work with informal writing and similar issues (e.g., religion vs. politics). The GA-based structure takes care of this issue by balancing out the contributions ( with SBERT and with DistilBERT) to leverage semantic congruency and scaleability. As an example, the semantic similarity in SBERT ensures that the term pairs are intelligible (e.g., in religion clusters, there are terms such as prayer and faith), whereas the efficiency of DistilBERT handles the size of the dataset (e.g., in religion clusters). The ablation experiments demonstrate that dynamic weighting prevents the negative results of over-reliance on the size of RoBERTa (e.g., over-reliance on magnitude) and that clusters are driven by domain-specific intent rather than architecture biases.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Clustering performance comparisons on 20 Newsgroups.

https://doi.org/10.1371/journal.pone.0353506.t003

Table 4 on Stack Overflow, where there are no ground-truth labels, the improvements of the GA-driven ensemble in Silhouette Score and Topic Coherence are + 14% and +17% over the figures of the static fusion, respectively. RoBERTa (Silhouette = 0.43) and SBERT (Topic Coherence = 0.42) demonstrate wise work but can unlock technical terminology (e.g., “React hooks” vs. Redux architecture) only to some extent. The GA-based approach addresses this problem through dynamic weight change ( in RoBERTa, in SBERT) to prefer precision in context in technical areas. As an example, evolved weights ensure clusters keep apart expressions like state management (React) and Redux actions, which are mixed with other expressions due to inflexible fusion. Topic Coherence improvements highlight the fact that GA can optimize the co-occurrence of terms (e.g., useEffect and useState in React cluster) and thus transform raw embeddings into domain-specific categories. These results approve the usefulness of the framework in high-variability contexts that are unlabeled and where geometric isolation and semantic uniformity are critical to actionable information.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Clustering performance comparisons on Stack Overflow.

https://doi.org/10.1371/journal.pone.0353506.t004

5. Discussion

As we can observe in the summary of the experimental results presented in Figs 2, 3, and 4, and Tables 2, 3 and 4, proposed GA-driven ensemble performs better than standalone models and standalone fusion across all data sets. The following is a breakdown of the reasons behind this, to tie the observed performance to the developed weight distributions and the characteristics of each dataset.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Clustering performance comparison on AG News.

https://doi.org/10.1371/journal.pone.0353506.g002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Clustering performance comparison on 20 Newsgroups.

https://doi.org/10.1371/journal.pone.0353506.g003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Clustering performance comparison on Stack Overflow.

https://doi.org/10.1371/journal.pone.0353506.g004

AG News Formal Polysemic Headlines. The GA weight the most to RoBERTa , intermediate to SBERT, and least to DistilBERT , as illustrated in Fig 2. This distribution indicates the better disambiguation of polysemous words by RoBERTa due to its ability to dynamically mask and train longer, which increases the focus on the modifier phrases. Considering the example of distinguishing between what is known as Apple stock price (Business) and Apple carbon footprint (Science) is based on the identification of contextual cues that RoBERTa is more successful at capturing compared to SBERT. Nevertheless, it can occasionally be the case that RoBERTa misclusters very short headlines that are not richly contextualized (e.g., the release of a new iPhone by Apple), and the global semantic similarity of SBERT offers a stabilizing influence. The uniform weights ensemble (static) dilutes the accuracy of RoBERTa, which results in a reduced ARI and coherence (see Table 2). The GA therefore effectively augments the strengths of RoBERTa whilst keeping the safety net of SBERT and cannot be achieved in fixed weight schemes.

20 Newsgroups, overlapping topics. Fig 3 shows the balanced distribution of the GA between SBERT and DistilBERT with RobERTa being the sole recipient of the allocation. This trend is because deep contextualization of RoBERTa is more prone to overfitting the Usenet posts with their noisy and colloquial language in clusters. The contrastive training model of SBERT generates a homogeneous space of embedding, where similar words in terms of semantics, such as the notions of prayer and faith, are clustered even in the non-scholarly text, whereas DistilBERT provides scalability to the 18,846 documents in the dataset. The GA consequently inhibits the contribution of RoBERTa since it does not want to be fragmented and uses the complementary benefits of SBERT (cohesion) and DistilBERT (efficiency). The non-dynamic unified weighting conditions the RoBERTa to add coherence with an equal probability, which lowers the coherence (Table 3). The outcome of this experiment shows that the GA can down weight models which do not have a good fit in a domain.

Stack Overflow: Technical Jargon. The GA values RoBERTa and SBERT with little contribution by DistilBERT (Fig 4). RoBERTa is also good at capturing complex discourse patterns of technical forums, i.e., the difference between closely similar words such as useEffect (React) and dispatch (Redux). The semantic similarity of SBERT can be used to cluster conceptually similar but lexically dissimilar words (e.g., didMount and useEffect) so that they are not disintegrated. The fact that DistilBERT has lower weight indicates that it has less contextual fidelity to subtle technical differences. By making all the contributions equal, the static ensemble degrades the accuracy required to make correct grouping (Table 4). The weights developed by the GA therefore attain a synergy which trades off context-depth and semantic stability, which is important in unlabeled technical data.

In all the datasets, the quality of weights dynamically adjusted by the GA according to the indicators of the quality of clustering (Silhouette, ARI, Coherence) is critical. The results of the analysis prove that the evolved weight distributions are explainable and correspond to the strengths of each embedding model that have been known, which confirms the framework as adaptable and offers to act on the similar unsupervised tasks.

6. Conclusion

The study introduces a GA-based ensemble model which provides a novel definition to unsupervised text clustering as it addresses fundamental weaknesses of fixed fusion models and independent contextual embedding models. The framework bridges the gap between inflexible, uniform averaging and domain-aware optimization by dynamically changing the weights of SBERT, RoBERTa, and DistilBERT using GAs. Critical innovations involve L2-normalization and a sum-to-one constraint to ensure equitable integration of models, without models with inherently larger embedding magnitudes (e.g., RoBERTa in AG News) dominating and sum-to-one constraint ensuring interpretable weight values (e.g., is contextual precision of RoBERTa). Unlike fixed ensembles, which undermine model performance (e.g., by uniformly averaging in Stack Overflow), the GA-driven method progressively updates weights depending on dataset-specific tasks: giving preference to contextual complexity of RoBERTa with technical jargon in Stack Overflow, semantic similarity of SBERT with similar topics in 20 Newsgroups, and performance of DistilBERT with large-scale AG News cl Such dynamic fusion ensures that clusters may adhere to domain intent (e.g., decoupling of react hooks and Redux architecture) and retain computational tractability.

The cross-domain generalizability and strength of the framework to high-dimensional spaces are supported by a systematic ablation and case studies. The mechanisms of tournament selection, Gaussian mutation and elitism ensure that the diversity of the population remains, without premature amalgamation in complex fitness scenarios (such as vague polysemous terms like Apple in AG News). In the case of labeled datasets (AG News), ARI and NMI indicate an increase of +10%−15% over baselines, whereas unlabeled datasets (StackOverflow) achieve 0.60 Topic Coherence, which is 0.17 higher than static fusion. These results highlight the capacity of the GA to supplement accuracy, efficiency, and domain-specific compromises but not relying on ground-truth labels. The flexibility of the framework to linguistic diversity, technicality, and scale is supported by cross-dataset substantiation of AG News (formal headlines), 20 Newsgroups (casual text), and Stack Overflow (technical jargon). As an example, SBERT and DistilBERT jointly deal with casual vagueness in 20 Newsgroups (=0.4, =0.4) whereas RoBERTa performs best in technical ones (=0.5) to disentangle complex patterns of discourse.

This paper advances unsupervised clustering of heuristic pipelines to adaptive, label-free optimization by systematically outperforming the static ensembles and standalone models on heterogeneous datasets. The modular construction of the GA-driven framework makes it easier to combine the emerging models (e.g., BERT-wwm, ELECTRA) and the hybrid goals (e.g., Pareto front analysis in multi-objective optimization). Real-life uses include NLP in daily applications, such as real-time question routing over technical fora, or cross-domain news classification. Future research can generalize the framework to semi-supervised setting and explore distributed GA optimization on huge datasets. This work reduces the gap between evolutionary computation and deep contextual learning by transforming raw embeddings into actionable groupings with no labeled data, and sets a new benchmark of a solid and understandable clustering in unsupervised environments.