2.1 The random lasers intensity fluctuations theory

Random Laser Intensity Fluctuations (RLIF) theory, discussed by Anderson S.L. Gomes and others, explores the fluctuations in light intensity caused by multiple reflections and interference in random Laser without fixed cavities and random scattering media [30]. It suggests that the probability density function of laser intensity is represented by an exponentially truncated Lévy-like distribution, which depicts a wide range of intensity fluctuations [31]. In such a distribution, as the input excitation power increases, the accumulation of electromagnetic energy also increases, thereby raising the tail index of the intensity distribution [32]. This means the fluctuations in intensity are quite significant, and the intensity distribution tends toward a Gaussian distribution, due to the Central Limit Theorem [30]. Additionally, the collective effect of pseudo-modes in random Laser is considered. Due to the varying contributions from each scattering center, the interference patterns of light waves exhibit high randomness and irregularity, which are characteristic of the intensity fluctuations in random Laser [31]. Random laser intensity fluctuations can be expressed as the superposition of multiple independent scattering events, each of which can be represented by a sine wave [32]. The mathematical formula is as follows:

(1)

where, I₀ represents the average light intensity, while and denote the amplitude, frequency, and phase of the i-th sine wave, respectively [33].

2.2 Chaos theory

Chaos theory studies pseudo-random, unpredictable behaviors in deterministic nonlinear systems, with its core being the system’s extreme sensitivity to initial conditions (the butterfly effect) and inherent pseudo-randomness [34]. This pseudo-randomness is not true randomness, but rather a sequence generated by deterministic equations, which is ergodic and non-periodic [35]. Chaotic systems effectively represent high-dimensional, nonlinear, strongly coupled iterative processes, similar to the search trajectory of swarm intelligence algorithms [36]. When the system’s state deviates from a stable path due to parameter disturbances or boundary condition changes, the chaotic mechanism quantifies its fluctuations, revealing hidden patterns not captured by traditional linear analysis [37]. This paper uses the Logistic map to generate chaotic sequences for discrete systems, with the formula as follows:

(2)

where, Seed is the chaotic seed within the range (3,4), and in this paper, it is set to 3.9, where x_n is the current value and x_n + 1 is the next value. At this point, the system enters chaos, showing extreme sensitivity to initial values [38].

Firstly, rounding errors inevitably occur when initializing the population in a wrapper algorithm, leading to significant deviations in long-term behavior. The algorithm then searches the solution space diffusely, making it difficult to concentrate locally. However, chaotic sequences can infinitely approach any point within a finite range, offering more efficient search space coverage than uniform distribution. Therefore, this paper considers using chaos theory to improve the population initialization part. Secondly, chaotic sequences have an almost non-repetitive characteristic. They can inject continuous diversity into the algorithm, suppressing premature convergence. Thus, this paper considers guiding the positions of individual agents in swarm intelligence algorithms through chaotic sequences to enhance their development ability in smaller feature spaces.

2.3 RF-FIM

The feature importance in random forests is calculated by adding the gains from features at each node across all decision trees. entropy of information is often used to measure feature contributions in classification problems [39].

(3)

(4)

where, H(D) is the Entropy of the dataset D, C is the Category set, D_j is the Subset of the category j, is the Conditional Entropy of after given a subset of feature , is the size of the feature subset, is the size of the dataset. The contribution of each decision tree to feature importance is the average reduction in information entropy of that feature across all trees [40]. Feature importance evaluation in random forests is done through the contribution of node splits in the integrated decision trees, providing quantifiable screening criteria for high-dimensional features. Its advantage is that it can handle nonlinear relationships without requiring independent feature distribution assumptions [41].

The purpose of using the RF-FIM algorithm in the feature selection process is to select a small number of features with high information gain from the original dataset. This helps reduce the complexity of model training and improves efficiency [42]. The number of selected features is based on the total number of features in the dataset and depends on the size of the dataset and samples.

2.4 The gray wolf optimization algorithm

As a typical swarm intelligence algorithm, the Grey Wolf algorithm (GWA) [43] simulates and optimizes the social hierarchy and group hunting behaviors of Grey Wolf populations. The gray wolf population has a strict social hierarchy, categorized into four types: α wolves, β wolves, δ wolves, and ω wolves, with lower ranks having more wolves. ω Grey Wolf individuals X_i in the pack update their positions based on the locations of α wolf, β wolf, and δ wolf, , , [43], with the update formula as follows:

(5)

(6)

(7)

(8)

(9)

(10)

(11)

Where, D represents the distance between the grey wolf and its prey, and t represents the iteration. X and X_p represent the position vectors of the Grey Wolf and the prey, while A and C are vector coefficients, with their specific mathematical expressions given by formulas (8) and (9) [43]:

(12)

(13)

Where, a is the convergence factor, r₁ and r₂ are random vectors taking values in [0,1]. As the number of iterations increases, the value of a gradually decreases from 2 to 0 [44].

When handling feature selection problems, the GWA algorithm needs to be converted into a binary form to facilitate feature selection. However, existing methods of binary feature selection usually rely on random number comparisons, making it difficult to avoid selecting all or no features. While the GWA algorithm has excellent global search capabilities, it tends to get trapped in local optima in the later stages of iteration. Therefore, this paper first considers adopting a more robust approach to convert continuous algorithms into discrete ones, and then applies a strategy that combines two swarm intelligence optimization algorithms to improve the grey wolf algorithm, forming a new algorithm that balances global search and local development as part of the wrapper.

2.5 Dung beetle algorithm

The Dung Beetle Algorithm (DBA) [14] is a population based swarm intelligence optimization algorithm proposed in 2022. This algorithm is mainly influenced by the behaviors of dung beetles such as rolling, reproduction, foraging, and stealing, with the dung beetle population divided into different roles in a 6:6:7:11 ratio [14]. DBA has advantages in convergence, search effectiveness, and exploration capabilities, but some phases of behavior that develop around local optima often remain at local optima [14], The following Fig 1. shows the conceptual model of the dung beetle’s boundary selection strategy [16].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Strategy of Dung Beetle’s boundary selection.

Conceptual model of Dung Beetle’s boundary selection strategy.

https://doi.org/10.1371/journal.pone.0338051.g001

The foraging behavior of dung beetles selects safe areas, and the foraging area for larvae dynamically updates [14], represented equation as follows:

(14)

Here, X_b is the local optimal position of the current population, Lb^b and Ub^b define the lower and upper bounds of the optimal foraging area [16], The position update equation for the foraging dung beetles is as follows:

(15)

Where Y₁ is a random number following a normal distribution, and Y₂ is a random vector between 0 and 1, with a size of 1 × Dim [16].

Although DBA has a unique boundary selection strategy, it struggles with exploring the solution space. Since the optimal search in each iteration is updated around the optimal individual, the position of the optimal individual is crucial. The original DBA algorithm struggles to deeply analyze the optimal individual, leading to some waste in the utilization of the optimal individual.

To improve DBA’s ability to explore neighborhood optima and solve the premature convergence problem, this study first considers introducing the chaos theory introduced in Sect 2.2 to guide the position of dung beetle individuals, hoping to further enhance DBA’s local development ability. Then, it considers adopting a unique embedded structure that combines the improved DBA and the GWA algorithm introduced in the previous section, which can minimize the premature convergence issue. This forms a novel wrapper feature selection algorithm based on a hybrid swarm intelligence algorithm.

2.6 Previous hybrid feature selection methods

In the field of feature selection, previous researchers combined the advantages of filter and wrapper methods to develop many excellent hybrid algorithms.They have made significant contributions to the research field. The following Table 1 summarizes the five hybrid feature selection algorithms from 2021 to 2024.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Recent hybrid feature selection methods.

https://doi.org/10.1371/journal.pone.0338051.t001

In 2021, Got et al. attempted to merge filter and wrapper methods, using Whale Optimization Algorithm (WOA) to address the multi-objective nature of feature selection [47]. However, multi-objective optimization remained at the algorithm level, with experimental validation only designed using 2 multi-objective methods and 3 datasets [47]. This makes the results difficult to generalize. In 2022, Zhu et al. proposed an HFSIA algorithm to address the challenges of high-dimensional data. The algorithm combines filter methods and introduces a deadly mutation mechanism and Cauchy operator to enhance algorithm performance for dimensionality reduction [48]. However, the high redundancy of features led to poor classifier performance. At the same time, its fitness evaluation method was inefficient, leading to high computational complexity.

In 2023, Xu et al. discussed the method of using Markov blankets for feature selection, focusing on the relationship between features and classes [49]. They emphasized computing symmetrical uncertainty to evaluate the shared information between features and classes, helping to identify relevant features. Meanwhile, the study acknowledged the limitations of the datasets used. Secondly, the algorithm is prone to local optima due to neglecting feature interactions. Lastly, manually set thresholds are required, making it difficult to find the global optimal solution. In the same year, Deng et al. proposed a strategy involved continuously updating the feature threshold set to guide the evolution of genetic algorithms, enhancing search capability [50]. However, the unguided crossover and mutation in the genetic algorithm led to excessive randomness and the generation of poor solutions. Its randomness causes slow convergence of the algorithm.

In 2024, Li et al. designed a heuristic search based forward feature selection algorithm to enhance the exploration of feature subsets [51]. However, the cooperative swarm intelligence feature selection algorithm has limitations in searching the entire solution space, making it easy to overlook better solutions. Secondly, the designed algorithm often sacrifices the globally optimal feature combination to meet output conditions. Lastly, the heuristic algorithm suffers from early termination, leading to repeated feature subsets and invalid features.

Previous researchers’ work demonstrated the optimization potential of hybrid feature selection but still had the following shortcomings:

1) Static thresholds or fixed output conditions are difficult to adapt to changes in data distribution.

2) Most methods focus on classification accuracy, ignoring multi-objective collaborative optimization of feature subset length and feature redundancy.

3) Meta-heuristic methods have low computational efficiency in ultra-large feature spaces, leading to a surge in computational costs.

To address these issues, the proposed HMF-W algorithm first uses an extremum control strategy in the proposed process optimization mechanism to adaptively adjust the threshold, thus accommodating real-time changes in the optimization process. Next, the proposed algorithm uses an innovative algorithmic framework, where the univariate Filter method is responsible for significant dimensionality reduction. The POM mechanism regulates the alternating execution of the bivariate Filter method and the improved wrapper method to repeatedly optimize the result feature subset of the univariate Filter method. This method not only ensures a high ACC while selecting shorter feature subsets, but also significantly reduces computational complexity when dealing with high-dimensional data.

3.1 mSMMI filter algorithm

Feature selection algorithms, specifically filter algorithms, rank features in a dataset using specific metrics. These include univariate and multivariate methods, such as deep learning based feature selection and LASSO regression. LASSO regression performs feature selection using L1 regularization, which ensures sparsity in high-dimensional data. However, it relies on linear assumptions and is limited in handling nonlinear relationships.

Based on the renowned multivariate mRMR concept, to better consider both relevance and redundancy simultaneously, this paper introduces a new filter algorithm, mSMMI, which uses Maximum Mutual Information to measure the relevance between features and labels, and minimum Spearman to measure redundancy among features, dynamically adjusting the ratio of mutual information score to redundancy using two parameters, s and y.

3.1.1 Maximum Mutual Information (MMI).

Mutual Information (MI) measures the dependency between two variables, indicating how much information about one variable can be obtained from the value of another. Traditional methods of measuring relevance, such as the Pearson correlation coefficient, can only capture linear relationships. Mutual information can measure complex nonlinear relationships. This makes mutual information extremely robust when dealing with datasets that have highly complex structures.

For two random variables X(x₁, x₂, ..., x_n) and Y(y₁, y₂, ..., y_n), mutual information MI(X;Y) is defined as:

(16)

Where, p (x, y) is the joint probability distribution of X and Y, and p(x) and p(y) are the marginal probability distributions of X and Y, respectively.

The value of mutual information can be interpreted as how much information about variable Y can be obtained if we know the value of variable X (and vice versa). The higher the value, the stronger the dependency between the two variables. If the mutual information is zero, it indicates that the two variables are independent.

Suppose we have two vectors a and b,where a = , b = . Using Eq (13), we calculate MI(a, b) = 1.609. The high mutual information indicates a strong nonlinear relationship between a and b, as shown in Fig 2. If the two vectors are c = and d = , using Eq (13), we calculate MI(a, b) = 0.673. The low mutual information indicates a weak positive correlation between c and d, as shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Relationship between vectors a and b.

Nonlinear Relationship between vectors a and b.

https://doi.org/10.1371/journal.pone.0338051.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Relationship between vectors c and d.

Weak positive correlation between vectors c and d.

https://doi.org/10.1371/journal.pone.0338051.g003

In high-dimensional raw datasets, there are thousands of features, which are reduced to hundreds of features by the M1 module to form a feature subset Here, the feature f_i with the highest relevance to the label L is calculated using Eq (17):

(17)

where, MMI(F_object, L) represents the maximum mutual information value between features and the label, F_object denotes the feature subset obtained after dimensionality reduction by module M1, L represents the label vector in the dataset, MAX is the maximum function, MI(L, f_i) is calculated by Eq (16). and f_i denotes the features in F_object, where i = 1, 2, ..., n.

3.1.2 Minimum Spearman (mS).

In feature selection, the concept of maximum relevance and minimum redundancy considers both the correlation between features and labels and the redundancy among features.

The Spearman coefficient [38] is primarily used to measure the monotonic relationship between two variables, where one variable increases or decreases as the other does. The Spearman coefficient is less sensitive to the distribution and form of data, focusing more on the ordinal relationships between variables. This allows it to be used in analyzing both linear and non-linear monotonic relationships, making it more effective in handling non-normal distributions or data with outliers.

For any two features in the feature set , denoted as f_i and f_j, their Spearman coefficient is calculated by Eq (18).

(18)

where, the Spearman coefficient between feature [-1,1], n is the number of features, d_ij is the difference in observation sequences of features f_i and f_j,i = 1,2,...,n.

Given two features F₁(0.8,0.8,0.7,0.4,0.9,0.6,0.7), F₂(0.5,0,0,0.1,0.5,0.1,0.1), the Spearman coefficient =0.384, indicating a certain monotonic relationship between the two features, though it is relatively weak.

Eq (18) calculates the Spearman coefficient between known features and each feature in the set. Known features are denoted as f_know, and a set of candidate features are denoted as The feature f_k with the smallest Spearman coefficient value with the known features in the set is represented by Eq (19):

(19)

Where, f_know denotes a Known feature, currently f₁. F_object is the target feature set, at this point F_{object−know}, and denotes the minimum Spearman coefficient between the known feature and the target features set. The MIN function denotes the minimum value, f_k is a feature in F_object, and the Spearman coefficient between f_k and f_know is the smallest within F_object. The value of is calculated using Eq (18).

The minimum Spearman (mS) between two feature sets is calculated as follows: the selected feature set is represented as P={p_1, p_2,..., p_i,..., p_m}, i=1,2,,…,m And the target set is denoted as A feature is selected from the target set to satisfy the requirement that the Spearman coefficient is the minimum value between the selected feature and the known feature subset.The Spearman coefficient between the two feature sets is represented by Eq (20).

(20)

Where, P is the selected feature set, F_object is the target feature set, mS2(P,F_object) is the minimum Spearman coefficient between the target and selected feature sets. MIN is the minimum value function, f_min is the feature from F_object with the smallest Spearman coefficient, mS1(f_min,P) calculated by Eq (19).

3.1.3 Minimum Spearman Maximum Mutual Information (mSMMI).

In the M1 module, the RF-FIM algorithm reduces the number of features, cutting thousands down to a smaller set of hundreds with high information gain. In the M2 module, the bivariate filter algorithm further improves the selection, picking dozens of features that have high relevance and low redundancy. The number of features selected by the RF-FIM and mSMMI algorithms depends on the number of features and samples in the dataset, as well as the adjustment factor p and the mapping function f(x). This helps determine how many features will be selected by both algorithms. The formula for the number of features k2 selected by filter is:

(21)

Where, the random number RandnumA, as the baseline, can provide a certain number of features for the filter algorithm. rand()(1,10), p(0,1) with p set to 0.8 in this case. where f and e represent the number of features and samples in the dataset, respectively. The cosine function term can decide the number of features to select based on the sample size of different datasets. In larger datasets, more diverse features can be selected to avoid an overly concentrated distribution of feature numbers across datasets. The number of features K1 selected by the RF-FIM algorithm is also calculated using Eq (21). where the range of the baseline RandnumA is expanded by 10 times. This allows for more high quality features to be provided for multiple runs of the mSMMI algorithm’s feature selection, with the rest remaining unchanged.

Taking the Co and Ov datasets as examples, we use Eq (21) to calculate 100 iterations, and the variation of K1 and K2 values is shown in Fig 4 below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Evolution of K1 and K2 on the Co and Ov datasets.

The trend of K1 and K2 values in the Co and Ov datasets.

https://doi.org/10.1371/journal.pone.0338051.g004

From Fig 4, it can be seen that the K1 and K2 values obtained from Eq (21) in the Ov dataset are significantly greater than those obtained in the Co dataset. This is because the Ov dataset has more features and samples than the Co dataset. Therefore, on different datasets, the filter algorithm can select an appropriate number of high quality features using Eq (21).

We combine the previously mentioned maximum mutual information score and minimum Spearman coefficient to select features. After formulating MMI and mS into a single equation, we opt for subtraction to reduce computation. Subsequently, we introduce parameters into by Eq (22) to adjust the influence weights of mutual information scores and redundancy scores, thereby avoiding the situation where the influence weights of correlation and redundancy remain constant during the filter algorithm. In this way, we can select features one by one from the candidate feature subset.

(22)

Where, s and y are iteratively generated based on Eqs (23) and (24), F_object as the candidate feature set, L is the label vector in the feature subset, P is the selected feature set, p₁ is the first selected feature, and i denotes the number of selected features.

This study uses the mSMMI algorithm to find the best feature combination, which is then improved by the HGW-CDBW wrapper algorithm for precise feature selection. We use parameters s and y to adjust the influence of relevance and redundancy. Since the values of MMI and mS differ significantly across different datasets, the values of s and y cannot be 1. The value of s decreases while y increases, and they are calculated using Eqs (23) and (24).

(23)(24)

Where i is the current iteration of the wrapper algorithm, and I is the maximum number of iterations. s decreases from 1 to 0, and y increases from 0 to 1 during the wrapper algorithm’s iterations, as shown in Figs 5 and 6 shows the flowchart of mSMMI. Below is the pseudocode:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Trend of s and y.

During the maximum number of iterations, s decreases from 1 to 0, and y increases from 0 to 1.

https://doi.org/10.1371/journal.pone.0338051.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Flowchart of mSMMI algorithm.

This flowchart is an introduction to the proposed filter algorithm mSMMI.

https://doi.org/10.1371/journal.pone.0338051.g006

Algorithm 1 mSMMI Algorithm Pseudocode.

3.2 The hybrid gray wolf and chaotic dung beetle wrapper algorithm

To synergize the global convergence of the gray wolf optimization algorithm and the local exploitation advantages of the DBA algorithm, while avoiding the premature convergence flaw of DBA, this study improves the dung beetle and gray wolf optimization algorithms through a hybrid algorithmic strategy. Specifically, it combines the fine development ability of the chaotic dung beetle algorithm (CDBA) with the hierarchical leadership structure of the binary gray wolf algorithm. It then combines with a support vector machine classifier to form the Hybrid Gray Wolf and Chaotic Dung Beetle Wrapper (HGW-CDBW) algorithm.

3.2.1 Chaotic dung beetle algorithm.

The Dung Beetle algorithm is inspired by the behavior of dung beetles, which use their antennae to sense their surrounding environment and push spherical dung to find the best burial spot. The algorithm solves optimization problems by simulating this behavior. This paper applies it to hybrid feature selection, using average ACC as the fitness function to evaluate the performance of feature subsets and find the optimal subset.

The dung beetle primarily searches around locally optimal solutions. However, this search is too blind and singular, often neglecting the exploration of nearby areas while approaching locally optimal solutions, reducing the versatility of the individual and frequently resulting in settling at local optima. This paper introduces chaos sequences into the dung beetle search process to increase chaotic disturbances, using Logistic mapping to generate chaos sequences, with the specific formula as follows:

(25)

Where, Seed is a chaotic seed with a value range of (3,4), set to 3.9 in this paper. x_n is the current value, δ is the next value in the chaotic sequence. n is the number of search agents, set to 30 in this paper.

One weakness of traditional DBA is its poor exploration of solution spaces, as each iteration optimally searches around the best individual. Thus the position of the best individual is crucial, but traditional DBA fails to analyze the best individual in depth, leading to some waste of the best individual’s utility.

To enhance DBA’s capability to explore local optima and address premature convergence, we employed chaotic sequences as the adjustment factor for search steps, achieving adaptability and diversity in the search process. Compared to the traditional DBA algorithm, the introduction of chaotic mapping increases the randomness and exhaustiveness of the search, allowing for more effective local exploration within the feature space while maintaining diversity and convergence in the search. In this paper, the initial position of the dung beetles is determined by the top three optimal solutions from the grey wolf algorithm. The following is the formula for the improved dung beetle position update:

(26)

Where, x_i is the current position of the dung beetle, is the new dung beetle position, and i is the individual dung beetle’s index number. X_best is the known best position, and δ is the step length factor generated based on Eq (25).

Each dung beetle having an independent step length factor. The sign function determines the direction of movement, and the log function adjusts the movement step length, decreasing as the distance increases, allowing for a precise approach to the optimal solution.

The introduction of chaotic theory aims to increase the diversity and local exploitation ability of the algorithm, avoiding premature convergence in CDBA algorithm and exploring more potential optimization areas. Below is the pseudocode for the CDBA algorithm:

Algorithm 2 CDBA algorithm pseudocode.

3.2.2 Embedded dung beetle binary grey wolf algorithm.

In the discrete feature selection problem, each position vector dimension in the binary grey wolf algorithm corresponds to the selection status of a feature, where 1 indicates the feature is selected, and 0 means it is not selected. Position selection is usually assisted by random numbers, with the threshold generally fixed at 0.5. Although this position update method uses randomization to determine feature selection, it may result in cases where all random numbers are greater than or less than 0.5, leading to the selection of all features or no features being selected. To overcome it, we use a set of random numbers in ascending order, selecting the median from the sequence as a threshold to determine positions, as per the following formula:

(27)

where is the selection status of feature j by wolf i in iteration t+1 , rand() is a random number generated from a uniform distribution, and μ is the median of an ascending sequence of random numbers; features greater than μ are selected, otherwise, they are not.

In Fig 7, in the feature selection stage, this study adopts a median based threshold selection strategy. Under this selection mechanism, a feature is selected only when its corresponding value exceeds the threshold. This strategy effectively avoids the extreme cases of not selecting any features or selecting all features, ensuring that the algorithm always obtains a sufficient and high quality feature subset. Secondly, it can adaptively regulate the feature space dimension, reducing the final feature subset dimension. This binarization strategy improves the algorithm’s convergence performance and stabilizes the size of the feature subset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Feature correspondence position selection.

This figure is an explanation of Eq (27).

https://doi.org/10.1371/journal.pone.0338051.g007

To overcome the limitations of the GWA algorithm in local development capabilities, we combined the precise development ability of the chaotic dung beetle algorithm with the hierarchical leadership structure of the GWA. The chaotic dung beetle algorithm focuses on and optimizes the top three individuals in the grey wolf population, conducting local development in their vicinity areas. It generates non-periodic search paths through chaotic sequences and adjusts the dung beetles’ movement direction and step size using Eq (22), achieving precise local development. This approach can improve the quality of solutions while maintaining diversity.

Specifically, we employ the chaotic dung beetle algorithm to perform a search in the vicinity of Alpha, Beta, and Delta wolves, respectively. After three searches, we sort and compare the fitness values of the three best dung beetle individuals with those of the current top three alpha wolves, thereby dynamically updating the positions and fitness values of the top three grey wolf individuals. On this basis, other wolves in the pack are attracted to approach the direction of the optimal solution, and the positions of the wolf pack are adjusted based on the repositioned top three alpha wolves. This approach can preserve the diversity of the solution space. The proposed embedding strategy not only retains the excellent global search capability of the Grey Wolf algorithm but also significantly improves the precision of local searches. Ultimately, a balance between exploration and exploitation is achieved. Fig 8 is a flowchart of the HGW-CDBW; below is the pseudocode:

Algorithm 3 HGW-CDBW algorithm pseudocode.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Flowchart of HGW-CDBW algorithm.

This flowchart is an introduction to the HGW-CDBW algorithm.

https://doi.org/10.1371/journal.pone.0338051.g008

3.3 The proposed algorithm

The HMF-W algorithm consists of M1 and M2 modules. In the M1 module, the RF-FIM algorithm mentioned in Sect 2.3 is used for preliminary dimensionality reduction, achieving a significant reduction in dimensions. In the M2 module, the proposed Filter mSMMI and wrapper HGW-CDBW algorithms are used. Through the proposed POM mechanism, we dynamically adjust the execution frequency of Filter and wrapper algorithms to escape local optima. When the wrapper algorithm is executed multiple times and its search effect approaches saturation, the algorithm will execute the Filter algorithm to reinitialize the population. This allows for more effective approaches to the global optimum. Let us now provide a detailed introduction:

3.3.1 Population initalization combining chaos theory.

In swarm intelligence optimization algorithms, population diversity affects the convergence speed and search efficiency. Populations are usually initialized randomly, but this method may lead to uneven population distribution, thus affecting the algorithm’s global search capability and convergence speed. To address this issue, this paper proposes a population initialization method based on chaos theory, enhancing diversity and coverage by mapping chaotic sequences into the search space. By assigning an independent random seed to each individual, the diversity of the initial population is ensured. To initialize the position of each grey wolf using the following formula:

(28)

Where, where N is the number of populations, J is the dimension of the search space, p_ij is the value of the i-th individual in the j-th dimension. , where , . l and u are the lower and upper bounds of the search space, respectively; δ is an element from the chaotic sequence, calculated according to Eq (25).

Chaotic systems exhibit better uniformity, sensitivity to initial values, and pseudo-randomness, which help the population to thoroughly explore the solution space. The chaotic sequence is transformed into the problem’s solution space through linear mapping, ensuring the feasibility of the initial solutions. Algorithm 4 demonstrates the specific steps for initializing the population based on chaos theory.

Algorithm 4 Chaos based population initialization.

3.3.2 Process optimization mechanism.

In feature selection, the previously mentioned filter and wrapper methods can be combined in various ways to form a hybrid algorithm. This paper introduces the theory of random laser intensity fluctuations, combined with the proposed average improvement rate and mechanism control strategy, forming a POM mechanism that can dynamically adjust the execution frequency of filter and wrapper algorithms. The wrapper algorithm is used for multiple optimizations of the results obtained from the filter algorithm executions.

According to Eq (1), when A₁ = 2.743, A₂ = 1.372, A₃ = 0.914; f₁ = 0.1, f₂ = 0.2, f₃ = 0.4; and , the random laser intensity fluctuation model produces the results shown in Fig 9.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. The RLIF function.

The Figure demonstrates the variation trend of the random laser intensity fluctuation function.

https://doi.org/10.1371/journal.pone.0338051.g009

The horizontal axis in the diagram represents time, showing two periods, while the vertical axis represents the intensity of the laser. We can see that there are six extremums in the intensity during each cycle. These six extremums are used to adjust the execution of the filter and wrapper algorithms.

Average Improvement Rate(AIR), as proposed in this paper, is a key performance metric that measures the degree of performance improvement from one iteration to the next. We use it to determine whether the wrapper algorithm is trapped in a local optimum. The specific formula is as follows:

(29)

where, AIR is the Average Improvement Rate, f_i and f_i + 1 are the fitness scores of the current (i^th) and next ((i + 1)^th) iterations respectively, N is the total number of iterations, and i denotes the current iteration number, .

In the early iterations of the HGW-CDBW algorithm, due to the randomness of the initial solutions and the exploration of the solution space, the search effects are more significant, as indicated by a higher AIR value. As the algorithm approaches the optimal solution or saturation point, the improvement in fitness decreases and may even decline. Consequently, the AIR value naturally decreases. The POM mechanism monitors whether the wrapper algorithm’s search results are approaching saturation or getting trapped in local optima by dynamically comparing the count of AIR decrease (CAD) and extrema.

When the CAD value exceeds the current extremum, the algorithm will call the mSMMI filter algorithm again to select a new set of candidate features to reinitialize the population for the HGW-CDBW algorithm, and the CAD value is reset to zero. This achieves the purpose of escaping the local optimum and providing a diverse set of candidate features.

The extremum control (EC) strategy proposed in this paper is used to regulate the progress of the wrapper algorithm’s search. It dynamically adjusts the current extremum used for comparison by combining the six light intensity extrema mentioned before and the AIR value calculated by Eq (29) to determine the search status of the wrapper algorithm, thus ending the search earlier or later. The specific logic is as follows:

If the AIR value continues to increase, indicating that the search results of the algorithm are stronger than expected, the current extremum is increased. Then the wrapper algorithm will perform more iterations, allowing it to search more solution spaces, making it easier to find the global optimum. If the AIR value continues to decline to zero or even below zero, it indicates that the search effect of the algorithm is weaker than expected. Depending on the extent to which it is below expectations, the current extremum is dynamically decreased to allow the wrapper algorithm to end the current search sooner and revert to the filter algorithm to reinitialize the population. This allows the wrapper algorithm to quickly escape local optima. Fig 10 below shows the changes in the values of CAD and Extremum in the Colon dataset over iterations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Trend of the values of CAD and Extremum in Colon dataset.

The figure demonstrates the variation trends of both the performance metric CAD and the extremum set values during 100 algorithm iterations on the Colon dataset.

https://doi.org/10.1371/journal.pone.0338051.g010

Fig 10 shows that in the early stages of iteration, the search for solutions is ideal, and the EC strategy gradually increases the current extremum, allowing the wrapper algorithm to iterate more times and fully explore the solution space. As the search deepens, the likelihood of finding better solutions increasingly decreases. The EC strategy gradually lowers the extremum to expedite the completion of the wrapper algorithm, which then triggers the bivariate filter algorithm to refresh the population. This achieves the goal of quickly escaping local optima and providing a more diverse set of features. The EC strategy determines when to adjust the process of the algorithm, enabling self-optimization adjustments based on the current search status and historical data, and effectively avoiding premature convergence to local optima.

This paper, based on the theory of random laser intensity fluctuations, combines the proposed average improvement rate and extremum control strategy to form a POM mechanism. Unlike traditional multicollinearity techniques, which mainly focus on handling linear correlations between features, this mechanism can promptly call the proposed bivariate filter algorithm to more effectively help the wrapper algorithm escape local optima. It can also adaptively extend the search process of the wrapper algorithm, improving global search capability.

Below is the pseudocode of the HMF-W algorithm, and Fig 11 is its flowchart. From it, we can see that the HMF-W algorithm first uses the RF-FIM algorithm to reduce the original feature set significantly and obtain the candidate feature subset F₁, as Module M1. Module M2 includes the remaining part of the algorithm, in which Filter Algorithm 1 (mSMMI) is used to perform multi-angle comprehensive consideration on F₁, resulting in the candidate feature subset F_j. Then, based on F_j, Algorithm 4 is used to initialize the population, which is handed over to Algorithm 3 (HGW-CDBW) for iterative optimization. In addition, the proposed POM is responsible for real-time monitoring of the optimization process of Algorithm 3. When Algorithm 3 gets trapped in a local optimum, it switches to Algorithm 1 to generate a new candidate feature subset F_j, which is then given to Algorithm 4 to initialize the population and continue the iterative process of Algorithm 3 until the optimal feature subset is output. Fig 11 below shows the flowchart of the entire HMF-W algorithm:

Algorithm 5 HMF-W algorithm pseudocode.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Flowchart of HMF-W algorithm.

This figure is an introduction to the proposed algorithm HMF-W.

https://doi.org/10.1371/journal.pone.0338051.g011

4.1 Experimental dataset and algorithm environment settings

4.1.1 Experimental datasets.

To validate the effectiveness of the HMF-W algorithm, 10 high-dimensional datasets from internationally recognized Microarray Data repositories and the Gene Expression Model Selector are used. Table 2 provides a brief description of these datasets, including the dataset name, sample size, number of features, and feature size. The sample size ranges from 62 to 253, the number of features from 2,000 to 24,481, and the number of classes from 2 to 5. Datasets with fewer than 100 features are considered low-dimensional, those with fewer than 2,000 are medium-dimensional, and the rest are high-dimensional. Datasets with two class labels are binary, while those with more than two labels are multi-class.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of the experimental datasets.

https://doi.org/10.1371/journal.pone.0338051.t002

4.1.2 Algorithm parameter settings.

In the next experiments, we compare the proposed HMF-W algorithm with other wrapper and hybrid feature selection algorithms. The SVM classifier uses the radial basis function as the kernel, and its radial basis function and penalty parameters are determined using grid search. Additionally, we used ten-fold cross-validation to compute the classification accuracy (ACC). The dataset is divided into 10 equally sized groups. One group is used as the test set, while the other 9 are combined to form the training set. The average of these classification accuracies and the length of the obtained feature subset (LEN) are used as the result of the fitness function. Based on reference [43], the best classification performance happened when there are 100 iterations and 10 runs. To improve classification performance, this study set the population size to 30 for all algorithms, used 100 iterations, and averaged ACC after testing each dataset 10 times. Table 3 shows the detailed parameter values for each wrapper algorithm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Parameter configurations of compared algorithms.

https://doi.org/10.1371/journal.pone.0338051.t003

In this study, the Support Vector Machine (SVM) classifier is used as the evaluation criterion for the fitness function. For testing the ACC of the dataset, we used the 10-fold cross validation method. We divide the dataset into ten equal parts using this method, where 9 parts (90%) are merged as the training set, and the remaining part (10%) is used as the test set. This process is repeated 10 times to ensure that every sample is tested. The parameters of the SVM classifier, including the kernel function, penalty parameter, and Radial Basis Function (RBF) kernel parameters, are selected using the Grid Search method. To ensure fairness, except for Sect 4.6, which uses nested cross-validation (i.e., the parameters of the SVM classifier are set within each training fold), the remaining experimental results all use 10-fold cross validation.

4.2 Ablation experiments

In order to verify the superiority of the dual-module hybrid algorithm HMF-W, the improved components of the algorithm were isolated for processing on the 10 test datasets given in Table 2. A total of 7 ablation hybrid algorithms were formed. These include the HMF-W algorithm proposed in this paper, the HMF-W_CDB algorithm (removing the chaotic Dung Beetle algorithm), the HMF-W_GWA algorithm (removing the binary grey wolf algorithm), the HMF-W_M1 algorithm (removing the RF-FIM algorithm or M1 module), the HMF-W_mSMMI algorithm (removing the mSMMI algorithm), the HMF-W_Chaos algorithm (removing chaos theory), and the HMF-W_POM algorithm (removing the POM mechanism). Each algorithm was run 10 times to get the average ACC, average LEN, and Relative Speedups (RS), as well as the Standard Deviation (SD) of all results. In this paper, the shortest average RunTime (RT) and its lowest SD for each dataset are taken as T and t, respectively, and the comparison results are shown in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Performance comparison of the average ACC (%), LEN, and RS ± SD of 7 ablation hybrid algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t004

From Table 4, it can be seen that the HMF-W algorithm performs better in classification accuracy on all datasets, but on the Ly dataset, the proposed algorithm reaches the same accuracy as the HMF-W_CDB algorithm. Then, in most datasets, the feature subset length chosen by HMF-W is better than others, except for Ly, BT, and CS datasets, where the proposed algorithm is slightly worse than HMF-W_M1 and HMF-W_CDB. Finally, on the DL dataset, the proposed algorithm’s RS is significantly better than the other algorithm components. On most datasets, the relative speedup of the proposed algorithm is slightly worse than that of the HMF-W_GWO algorithm. Overall, the HMF-W algorithm achieves excellent classification accuracy while selecting a feature subset with lower length, balancing both classification precision and convergence. Moreover, the relative speedup of the proposed algorithm is also relatively better, surpassing at least half of the ablation hybrid algorithms. This shows the proposed algorithm has excellent classification accuracy, strong convergence, and good computational efficiency.

To more comprehensively assess the algorithm’s robustness, Table 5 presents the worst, best, and average ACC of the HMF-W algorithm on 10 datasets, as well as the corresponding feature number selected for the worst and best ACC. Additionally, we plot the convergence of the performance metrics of the 7 algorithms across all datasets, as shown in Figs 12 and 13.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. The convergence trends of ACC of the 7 algorithms on 10 Datasets.

This figure demonstrates the iterative convergence of classification accuracy of the proposed algorithm and six ablation algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g012

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. The convergence trends of LEN of the 7 algorithms on 10 Datasets.

This figure demonstrates the iterative convergence of feature subset lengths of the proposed algorithm and six ablation algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Experimental results of HMF-W algorithm on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t005

By combining Table 4 and Fig 12, it can be observed that the proposed algorithm has a better ACC convergence trend across all datasets compared to other ablation hybrid algorithms. Specifically, on the SR dataset, the ACC convergence trend of the proposed algorithm is similar to that of HMF-W_Chaos, HMF-W_GWA, HMF-W_M1, and HMF-W_mSMMI algorithms, all of which outperform other algorithms significantly. On the DL dataset, the ACC convergence trend of the proposed algorithm is almost identical to that of the HMF-W_GWA algorithm, both significantly outperforming other algorithms. On the Ov dataset, the ACC convergence trend of the proposed algorithm is similar to that of the HMF-W_M1 algorithm, both clearly outperforming other algorithms. Therefore, the improvements in the proposed algorithm clearly enhance its ACC.

From Fig 13, it can be seen that on the Ly, BT, and CS datasets, the LEN convergence trend of the proposed algorithm is slightly inferior to the ablation hybrid HMF-W_CDB and HMF-W_M1 algorithms, but still significantly better than other algorithms. Therefore, the improvements in the proposed algorithm effectively enhance its convergence.

Based on Table 4, Figs 12 and 13, it can be concluded that the proposed process optimization mechanism helps improve the algorithm’s ACC, demonstrating its excellent algorithmic control capability. Additionally, the improvements in the other five algorithm components have also enhanced the algorithm’s classification capability to varying degrees, which proves that the improvements in the components of the proposed HMF-W algorithm are effective.

4.3 Filter algorithm comparison experiment

To verify whether the proposed Filter method, mSMMI, performs excellently. On the 10 test datasets given in Table 2, the mSMMI algorithm was compared with 10 other Filter methods by running each one once under conditions of selecting 25, 50, 75, 100, 150, 175, 200, 250, 275, and 300 features. The average ACC, RS, and their SD were obtained. In this paper, the shortest average RT and its lowest SD for each dataset are taken as T and t, respectively, and the comparison results are shown in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison of the average ACC (%) and RS ± SD of 11 filter algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t006

From Table 6, we can see that, on the Br dataset, MkmR and mSMMI algorithms are comparable. Except for the BT and CS datasets, where mSMMI performs slightly worse than FisherScore and ReliefF, mSMMI outperforms other Filter algorithms on the remaining 8 datasets. This proves that the proposed Filter method has superior classification performance.

Due to the excellent computational cost of filter algorithms, the computational efficiency gap between filter algorithms is difficult to demonstrate. Therefore, in the Wilcoxon test for RS results, only algorithms such as ChiSquare, FisherScore, Relief, MKMC, and mKMR show significant differences with the proposed algorithm on up to 4 datasets. Through comparing RS, we can observe. On the CS dataset, the RS of the mSMMI algorithm is significantly better than the others, reducing the average runtime by 25%. On the Ly and Ov datasets, the proposed algorithm’s RS is on par with most algorithms and better than a few others. On the Co, SR, DL, and Br datasets, the RS of all algorithms is the same. However, on the BT, ML, and SC datasets, the proposed algorithm is on par with other bivariate filter algorithms but slightly worse than most univariate algorithms. It is clear that the proposed bivariate filter method has an advantage in computational efficiency.

To provide a more comprehensive evaluation of the algorithm’s robustness, Table 7 shows the worst, best, and average ACC of the mSMMI Filter algorithm on 10 datasets, as well as the number of selected features corresponding to the worst and best ACC.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Experimental results of mSMMI algorithm on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t007

4.4 Wrapper algorithm comparison experiment

To verify the superiority of the proposed algorithm’s wrapper part. On the 10 test datasets given in Table 2, we compared 6 other wrapper algorithms with the wrapper HGW-CDBW algorithm proposed in this paper. To improve computational efficiency, each algorithm randomly selects hundreds features before performing wrapper search. Each algorithm was run ten times to get the average ACC, average LEN, and RS, along with the SD of all results. In this paper, the shortest average RT and its lowest SD for each dataset are taken as T and t, respectively, and the comparison results are shown in Table 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Comparison of the average ACC (%), LEN, and RS ± SD of 7 wrapper algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t008

From Table 8, we can see that, on most datasets, the improved algorithm HGW-CDBW shows higher average ACC than other wrapper algorithms. Only on the SR dataset, the improved algorithm is on par with the GWA+SVM algorithm. Also, on the ML dataset, the improved algorithm is slightly worse than the CDBA+SVM algorithm. On most datasets, the improved algorithm’s average LEN is better than other algorithms, except on the Br dataset, where the improved algorithm is slightly worse than the GWA+SVM algorithm. On most datasets, the RS of the improved method is significantly better than other wrapper algorithms. Except on the Co dataset, where the improved method is slightly worse than the GWA+SVM algorithm. On all datasets, the improved algorithm shows statistically significant differences in average ACC, LEN, and RS compared to most other algorithms. This proves that the improved wrapper algorithm in this paper has superior convergence and classification performance.

To provide a more comprehensive evaluation of the algorithm’s robustness, Table 9 shows the worst, best, and average ACC of the HGW-CDBW algorithm on 10 datasets, as well as the corresponding LEN.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Experimental results of HGW-CDBW algorithm on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t009

4.5 Hybrid algorithm comparison experiment

To validate the superiority of the proposed HMF-W algorithm, we compare it with four recent hybrid algorithms on the 10 test datasets given in Table 2. Each algorithm was run ten times to get the average ACC, average LEN, and RS, along with the SD of all results. In this paper, the shortest average RT and its lowest SD for each dataset are taken as T and t, respectively, and the comparison results are shown in Table 10.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Comparison of the average ACC (%), LEN, and RS ± SD of 5 hybrid algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t010

From Table 10, we can see that the proposed HMF-W algorithm performs much better in accuracy on most datasets, except that it is slightly worse than DFDW on the Ly dataset. Then, its chosen feature subset length is shorter on most datasets, except for Co where it’s slightly worse than DFDW, showing good convergence and classification performance. Next, the proposed algorithm’s RS is significantly better than most other algorithms on most datasets, except on the Ov dataset, where it is slightly worse than the IGMPMMIAPSO algorithm. From a statistical Wilcoxon test perspective, the proposed algorithm’s results show significant differences from most other algorithms across all datasets. Except for the proposed algorithm’s ACC, which does not show significant differences from at least two algorithms on the SR, Br, and DL datasets, the LEN does not show significant differences from two algorithms on the CS and Br datasets, and the RS does not show significant differences from the DFDW and IGMPMMIAPSO algorithms on the Ov dataset. Finally, the proposed algorithm shows a relatively low standard deviation in most datasets, showing its results are robust.

To further validate the classification performance of the proposed algorithm, Fig 14 shows the ACC values obtained by these five hybrid algorithms on ten datasets. In this figure, the vertical coordinate of the radar chart is the ACC value, and the number at the edge is the number of runs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. The single ACC values of the 5 algorithms running on 10 datasets.

This figure displays the classification accuracy values of 5 hybrid algorithms over 10 executions on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g014

From Fig 14, we can see that the proposed HMF-W algorithm performs much better in ACC on most datasets, except that it is slightly worse than DFDW on the Ly dataset. Then, its chosen feature subset length is shorter on most datasets, except for Co where it’s slightly worse than DFDW, showing good convergence and classification performance. Next, the proposed algorithm runs faster than most others, except on Ov where it is slightly slower than IGMPMMIAPSO. Based on the Wilcoxon test, the results show significant differences from most other algorithms on all datasets. Except that its ACC on SR, Br, and DL is not significantly different from at least two algorithms, its subset length differs insignificantly only on CS and Br, and its running time only on Ov compared with DFDW and IGMPMMIAPSO. Finally, the proposed algorithm shows a relatively low standard deviation in most datasets, showing its results are robust.

To further describe the differences between HMF-W and the other 10 hybrid algorithms. The minimum and maximum ACC rates obtained by the 11 algorithms combined with the SVM classifier on ten datasets are shown in Figs 15 and 16, where the horizontal axis represents the datasets and the vertical axis represents the ACC values. The points in the figure represent the lowest or highest ACC values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Comparison of the lowest ACC values of 11 algorithms on 10 Datasets.

This figure compares the minimum classification accuracy values achieved by 11 hybrid algorithms on all 10 experimental datasets.

https://doi.org/10.1371/journal.pone.0338051.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Comparison of the Highest ACC values of 11 algorithms on 10 Datasets.

This figure visualizes the maximum classification accuracy values attained by 11 hybrid algorithms across the 10 experimental datasets.

https://doi.org/10.1371/journal.pone.0338051.g016

From Figs 15 and 16. It can be seen that the classification ability of the HMF-W algorithm, compared to other algorithms, is the best across all five percentiles (minimum, quartile (25th percentile), median, quartile (75th percentile), and maximum). Therefore, based on Table 10 and Figs 14 to 16, it can be seen that the HMF-W algorithm has significant advantages in reducing feature dimensions and finding the optimal ACC. The standard deviation of the classification results of the proposed algorithm is relatively low on most datasets, so it can be concluded that the dynamic feature selection strategy used by the filter method is robust. Moreover, this algorithm, with its outstanding search ability, can effectively solve the problems faced during the feature selection process. In summary, this algorithm can balance the goals of ACC and feature count.

4.6 Nested cross-validation hybrid algorithms comparison experiment

To further demonstrate the superiority of the proposed algorithm, this section presents new results by applying nested cross-validation to five hybrid algorithms. Specifically, on 10 test datasets, the proposed algorithm was compared with four recent hybrid algorithms. Each algorithm ran 10 times to obtain average ACC, average LEN, RS, and the SD of all results. In this paper, the shortest average RT and its lowest SD for each dataset are taken as T and t, respectively, and the comparison results are shown in Table 11:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Nested cross-validation comparison of the average ACC (%), LEN, and RS ± SD of 5 hybrid algorithms on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.t011

As seen in Table 11, all hybrid algorithms experience a significant increase in running time under nested cross-validation, but the proposed HMF-W algorithm remains the best. On most datasets, the proposed algorithm’s average ACC and RS are better than all other hybrid algorithms, except on the Co and Br datasets, where the proposed algorithm’s average ACC is slightly worse than the TMKMCRIGWO, DFDW, and IGMPMMIAPSO algorithms. Also, on the Ov dataset, the proposed algorithm’s RS is slightly worse than the DFDW and IGMPMMIAPSO algorithms. Moreover, by comparing Tables 10 and 11, the average ACC of the proposed algorithm under nested cross-validation is 0.92% higher than that under 10-fold cross-validation. Specifically, on most datasets, the proposed algorithm achieves a higher average ACC under nested cross-validation than under 10-fold cross-validation, with two more datasets achieving 100% average ACC under nested cross-validation compared to 10-fold cross-validation. However, the LEN selected by the proposed algorithm under nested cross-validation is longer, and the running time increases significantly, but it still outperforms the other hybrid algorithms.

4.7 Biological genomic explanation

From a biological genomics perspective, only a few microarray genomic data are relatively important for disease diagnosis [54,55]. The proposed algorithm can find the smallest gene subset with the highest ACC. It is crucial to analyze these genes by identifying the obtained genes, as well as their contribution to medical diagnosis and biological significance. Table 12 shows the optimal gene subsets obtained using the proposed algorithm on each dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 12. The best gene subsets obtained by the proposed algorithm on all datasets.

https://doi.org/10.1371/journal.pone.0338051.t012

As shown in Table 12, the proposed algorithm, when combined with SVM, achieves high ACC and TPR on the optimal gene subset selected for high-dimensional microarray data. Secondly, on most datasets, the proposed algorithm selects the best features at least 5 times in 10 runs. Except for the CS dataset, where the best features were selected only 2 times. The results show that the proposed algorithm has good ability to identify and select optimal features. Finally, the gene features selected by the algorithm have biological significance. We conducted enrichment analysis on the optimal genes selected from the Co, SR, and Ov datasets. Details are as follows:

First, the Co dataset includes 30 cases of colorectal cancer polyps at various stages. It also contains variations in preparation procedures and observations using different surgical instruments during complete colonoscopy. Shieh et al. noted that polyps usually make up a small proportion in complete colonoscopy records [56]. In the 10 runs of the proposed algorithm in this study, the best 9 genes were selected 6 times, with both ACC and TPR being 95.3%. Furthermore, DAVID’s (GOTERM_CC_DIRECT) GO cellular component analysis shows that this gene list is enriched in extracellular exosome functions, with an enrichment score of P=1.95E-2 and Benjamini=9.74E-1.

The SR dataset includes 83 samples, each with 2,308 gene expression values. It covers four types of small round cell blue tumors: 29 cases of Ewing’s sarcoma, 18 of neuroblastoma, 25 of rhabdomyosarcoma, and 11 of Burkitt lymphoma. This is a well-known pediatric tumor classification dataset [59]. In the 10 runs of the proposed algorithm, the best 4 genes were selected 6 times, achieving both an ACC and TPR of 100%. Furthermore, DAVID’s (GOTERM_CC_DIRECT) GO cellular component analysis shows that this gene list is enriched in intracellular membrane bounded organelle functions, with an enrichment score of P=7.00E-3 and Benjamini=0.301.

The Ov dataset consists of surface-enhanced laser desorption ionization time-of-flight protein mass spectrometry data, including 162 ovarian cancer samples and 91 normal tissue samples [23]. The optimal gene subset selected by the proposed algorithm achieved an ACC and TPR of 100%. In the 10 runs of the proposed algorithm, the best 3 genes were selected 6 times, achieving both an ACC and TPR of 100%. Furthermore, DAVID’s (UP_SEQ_FEATURE) annotation enrichment shows that this gene list is enriched in disulfide bond functions, with an enrichment score of P=7.63E-2 and Benjamini=1.14E-1.

The biological interpretation for other biological datasets is as follows:

The Ly dataset includes gene expression profiles for three common adult lymphatic malignancies: 46 cases of diffuse large B-cell lymphoma, 9 of follicular lymphoma, and 11 of chronic lymphocytic leukemia. In 2014, Zhang, Y et al. analyzed certain activities of B cells [57]. The gene subset selected by the proposed algorithm achieved an ACC of 98.6% and a TPR of 98.5%. The DL dataset represents a common type of non-Hodgkin lymphoma, accounting for of all such cases. It can be classified into two subtypes based on molecular phenotype: germinal center B-cell-like and activated B-cell-like. In 2017, R.T. et al. studied various types of lymphoma. The ABC subtype has a poor prognosis, with a 5-year survival rate of only 26%, while the GCB subtype has a 62% survival rate [58]. The optimal gene subset selected by the proposed algorithm achieved an ACC and TPR of 100%. The BT dataset includes gene expression profiles for malignant gliomas, recording four types of brain cancer samples: 14 typical brain tumors, 15 lesions, 14 glioblastomas, and 7 anaplastic oligodendrogliomas [2]. The optimal gene subset selected by the proposed algorithm achieved an ACC of 98.9% and a TPR of 98.8%. The CS dataset covers different genetic subtypes of CLL, a mature B lymphocyte malignancy characterized by prolonged lymphocyte survival, immune dysfunction, and treatment resistance [61]. The optimal gene subset selected by the proposed algorithm achieved an ACC and TPR of 94.6%. The ML dataset is a gene microarray for mixed leukemia caused by chromosomal translocation in acute lymphoblastic leukemia. It records three leukemia subtypes: 24 cases of MLL chromosomal translocation leukemia, 20 of acute lymphoblastic leukemia, and 28 of acute myelogenous leukemia [62]. The optimal gene subset selected by the proposed algorithm achieved an ACC and TPR of 100%.

The SC dataset is a Phenome-Wide Association Study related to smoking and cannabis use, involving polygenic risk scores for alcohol, opioids, smoking initiation, and lifetime cannabis use disorders. The optimal gene subset selected by the proposed algorithm achieved an ACC and TPR of 87.7%. The Br dataset includes gene expression profiles from breast cancer patients: 46 cases of recurrent breast cancer and 51 of non-recurrent breast cancer. Due to RNA degradation in the samples [59], the proposed algorithm achieved an ACC of 74.9% and a TPR of 74.8%.

This demonstrates that the proposed HMF-W algorithm can accurately identify and select biologically significant genes in most genomic or proteomic datasets.

4.8 Analysis of dimensionality reduction effects

The HMF-W algorithm can obtain short feature subsets with high ACC, which is attributed to the algorithm’s good convergence and its dual-module structure design, ensuring the selected feature subsets are shorter. As shown in Figs 17 to 19, the x-axis represents all datasets, while the y-axis represents the number of features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 17. The three-level dimensionality reduction effects of HMF-W on 10 datasets.

This figure demonstrates the three-layer dimensionality reduction effects of the HMF-W algorithm on 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g017

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 18. The first-level dimensionality reduction effects of HMF-W on 10 datasets.

This figure showcases the primary dimensionality reduction performance of HMF-W during its first-layer processing across 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g018

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 19. The second-level dimensionality reduction effects of HMF-W on 10 datasets.

This figure details the secondary dimensionality reduction results achieved by HMF-W at its second optimization layer over 10 datasets.

https://doi.org/10.1371/journal.pone.0338051.g019

In Fig 17, we can see that the proposed algorithm achieves excellent dimensionality reduction results on 10 datasets through the dimensionality reduction operations of Modules M1 and M2. The number of features in most datasets drops to single digits. Among them, the Ov dataset shows the most significant effect, reducing from 15,154 features to 3, lowering the dimension to 0.02% of the original. The final LEN of 7 datasets is less than 10. Only the Ly, CS, and Br datasets have a final feature count slightly above 10, with dimensions reduced by at least 0.45% of the original. Therefore, the algorithm has a lower time complexity, with its wrapper part adopting a unique embedded algorithm structure, making its time complexity slightly higher than that of hybrid algorithms such as DFDW, IGMPMMIAPSO, and TMKMCRIGWO.

Figs 18 and 19 display the number of features (K1, K2) after dimensionality reduction by the RF-FIM and mSMMI algorithms, following 10 runs on each dataset. As shown in Fig 18, the average number of features obtained by Module M1 was between 183. 3-245. 3, with all maximum values above 200, and two datasets (CS, SC) exceeding 250. Fig 19 shows that after the execution of the filter algorithm, the average number of features selected by Module M2 ranges from 15.9 to 18, with the maximum values around 20. Among them, three datasets (Co, DL, Br) have fewer than 20 features. The number of selected features can be adjusted in the algorithm’s coding to avoid selection failures. Overall, multiple K values ensure the algorithm can select diverse features, demonstrating the strong convergence of the dimensionality reduction algorithm in Module M2.

4.9 Computational complexity

Time complexity is an important metric for measuring the resources (such as time and space) required during algorithm execution. Time complexity and space complexity reflect the time efficiency and memory consumption of the algorithm, respectively. These metrics are used together to evaluate the computational efficiency and performance of the algorithm, providing a basis for comparing different algorithms. The Computational complexity of the 10 hybrid algorithms used for comparison with the proposed HMF-W algorithm is presented in table form, as shown in Table 13.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 13. Computational complexity comparison of algorithms.

https://doi.org/10.1371/journal.pone.0338051.t013

Where, T represents the total number of iterations, K represents the number of selected features, n represents the number of features in the dataset, and S represents the time spent executing the SVM classifier. n×K represents the Computational complexity of updating the algorithm’s position, and K×S represents the Computational complexity of the SVM classifier during one iteration.

In the proposed HMF-W algorithm, RF-FIM and mSMMI algorithms are first used for dual dimensionality reduction on the original dataset. If the original dataset has 2000 features, after the first dimensionality reduction, 200 features may remain, and after the second reduction, 20 features may remain. Then, these 20 features are used as the original feature subset for the wrapper algorithm, which is more beneficial for selecting the most relevant and least redundant features. As the number of features decreases, the computation time also decreases, leading to a corresponding reduction in Computational complexity. In Table 13, since the number of features selected is much smaller than the original number of features in the dataset, and K is much smaller than n, K×n is smaller than n². Therefore, this algorithm has relatively low Computational complexity. Except for the ablation hybrid algorithm without chaos theory, which has nearly the same Computational complexity as the proposed algorithm, the Computational complexity of the proposed algorithm is slightly higher than that of its remaining 5 ablation hybrid algorithms.

Because the wrapper part of the proposed algorithm uses a unique embedded structure, its Computational complexity is slightly higher in theory than hybrid algorithms such as DFDW, IGMPMMIAPSO, and TMKMCRIGWO. However, as shown in Sect 4.5, the proposed algorithm runs faster than most other hybrid algorithms on almost all datasets. This is because we carefully optimized the wrapper process and achieved good results in Relative Speedups. Specifically, the DBO itself is efficient, and we improved its local search ability using chaos theory. Then, the CDBO was applied in the local search phase, where the gray wolf optimizer is least efficient. In this way, the strong classification performance of the gray wolf optimizer is kept, while the computation efficiency is greatly improved.

When dealing with most high-dimensional omics data, the HMF-W algorithm shows higher computing efficiency and faster speed than others, due to its lower complexity and optimized process. This means that within the same time, the proposed algorithm can handle more data or perform more iterations, improving both performance and efficiency. Moreover, its lower Computational complexity shows that the algorithm is more practical for large-scale omics data. Under limited resources, the proposed algorithm may be more suitable for most high-dimensional omics data, as it can run with less memory. However, when handling extremely large omics data, the algorithm might face difficulties due to memory size limits.