On the performance improvement of Butterfly Optimization approaches for global optimization and Feature Selection

Butterfly Optimization Algorithm (BOA) is a recent metaheuristics algorithm that mimics the behavior of butterflies in mating and foraging. In this paper, three improved versions of BOA have been developed to prevent the original algorithm from getting trapped in local optima and have a good balance between exploration and exploitation abilities. In the first version, Opposition-Based Strategy has been embedded in BOA while in the second Chaotic Local Search has been embedded. Both strategies: Opposition-based & Chaotic Local Search have been integrated to get the most optimal/near-optimal results. The proposed versions are compared against original Butterfly Optimization Algorithm (BOA), Grey Wolf Optimizer (GWO), Moth-flame Optimization (MFO), Particle warm Optimization (PSO), Sine Cosine Algorithm (SCA), and Whale Optimization Algorithm (WOA) using CEC 2014 benchmark functions and 4 different real-world engineering problems namely: welded beam engineering design, tension/compression spring, pressure vessel design, and Speed reducer design problem. Furthermore, the proposed approches have been applied to feature selection problem using 5 UCI datasets. The results show the superiority of the third version (CLSOBBOA) in achieving the best results in terms of speed and accuracy.


Introduction
In recent years, the complexity of real-world engineering optimization problems has been increased rapidly due to the advent of the latest technologies. In order to find the optimal solutions to these problems, many optimization methods have been introduced to find the optimal solutions. These algorithms can be divided into 2 major categories: deterministic and stochastic. In the formal category, for example Linear and non-linear programming [1], the solution of the current iteration is used in the next iteration to get the updated solution. The methods in this category have some limitations such as falling into local optima, single based solutions, and other issues regarding search space as mentioned in [2]. In the latter category stochastic methods, also known as metaheuristics, which generate & use random variables. This category has many advantages such as flexibility, simplicity, gradient-free and independently to the

Preliminaries
In this section, the BOA inspiration and mathematical equations are shown first. Then, the basics of Opposition-based Learing and Chaotic Local Search are presented.

Butterfly optimization algorithm
The BOA equations and complexity is described in details in the following subsections.

Inspiration & mathematical equations.
Butterflies belong to the Lepidoptera class in the Animal Kingdom Linnaean system [50]. In order to find food/mating partner, they used their sense, sight, taste, and smell. Butterfly Optimization Algorithm (BOA) is a recent naturebased algorithm developed by Arora and Singh in 2018 [31]. BOA simulates the behavior of butterflies in food foraging. Biologically, each butterfly has sense receptors that cover all butterfly's body. These receptors are considered chemoreceptors and are used in smelling/sensing the food/flower fragrance. To model butterflies' behavior, it's assumed that each butterfly produce fragrance with some power/intensity. if a butterfly is able to sense fragrance from the best butterfly, it moves towards the position of the best butterfly. On the other hand, if a butterfly can't sense fragrance, it moves randomly in the search space. In BOA, the fragrance is defined as a function of physical intensity as given in 1.
where pf i refers to the amount of fragrance perceived by other butterflies, c is the sensory modality, I and a refer to stimulus intensity and power exponent respectively. Global search (exploration) and local search (exploitation) phases are given respectively by Eqs 2 and 3.

Complexity.
To be able to compute the BOA complexity, assume the population size is (P), maximum iteration number (N), the problem dimensions (D). Then, the BOA complexity can be calculated as follows O(N(D × P + D × C)) where C refers to the cost of the fitness function = O(NDP + NDC).

Opposition-based Learning
Tizhoosh in [51] introduced Opposition-based learning (OBL) to accelerate the convergence by calculating the opposite solution of the current one and taking the best of them. In [47] a mathematical proof is given to show that the opposite solutions are more likely to be near optimal than totally random. The opposite solution � X i can be calculated from the following equation where a, b is the lower bound and the upper bound respectively.

Chaotic local search
Chaotic system characteristic can be used to make local search operator in order to strengthen the exploitation abilities in solving optimization tasks. Chaos is based on the navigation of deterministic nonlinear complex behavior. There are many chaotic maps in literature such as logistic, singer, tent, piecewise, and sinusoidal. This is because of the efficiency of chaotic map is related to the problem itself as mentioned by Fister et al. [52,53]. Logistics map is used in this paper and its sequequence can be obtained from the following equation.
2 The proposed approaches

Opposition-Based BOA (OBBOA)
The first version is called OBBOA which improves the performance of BOA by using OBL strategy. OBL enhance the BOA algorithm by improving its ability to explore search space deeply and speed up the reaching to optimal value. This version consists of 2 stages: First, at the initialization stage by calculating the opposite solution to each one in the initialization, then selecting the best N values. Second OBL is embedded in the updating stage. The pseudocode of this version is given in Alg. 3: Define sensory modality c, stimulus intensity I, and switch probability p 4: calculate stimulus intensity I i at x i using f(x i ) 5: Compute � X 6: Select best N from X [ � X 7: while (curr_Iter < Max_Iter) do 8: for each butterfly in (X) do 9: Calculate fragrance using Eq 1 10: end for 11: g � = best butterfly 12: for each butterfly in (X) do 13: r = rand() 14: if r � p then 15: Update butterfly position using Eq 2 16: else 17: Update butterfly position using Eq 3 18: end if 19: Calculate � x 20: end for 22: Update value of a 23: end while 24: Return g � .

Chaotic Local Search BOA (CLSBOA)
In the second version which is called CLSBOA, Chaotic Local Search is integrated with BOA to make a proper balance between exploration and exploitation. The pseudo-code of this version is introduced in Alg. 3.

Chaotic Local Search Opposition-Based BOA (CLSOBBOA)
In this version, both of the 2 previous modification has been added together to enhance BOA and get the most near optimal solution. Complexity: To be able to compute the BOA complexity, assume the population size is (P), maximum iteration number (N), the problem dimensions (D). Then, the CLSOBBOA complexity can be Calculate fragrance using Eq 1 10: end for 11: g � = best butterfly 12: for each butterfly in (X) do 13: r = rand() 14: if r ¡ p then 15: Update butterfly position using Eq 2 16: else 17: Update butterfly position using Eq 3 18: end if 19: Calculate � x 20: end for 22: Generate the candiate solution CS by performing CLS strategy 23: Update value of a 25: end while 26: Return g � .

Experiments
In this section, the proposed algorithms are tested using CEC as shown in the first subsection after that these algorithms are applied to 5 UCI datasets.

Benchmark functions
In this subsection, 30 functions have been used to compare algorithms using many statistical measure.

Test functions.
A set of 30 functions from CEC 2014 are used to compare the performance of the proposed algorithms with other state-of-art algorithms. This benchmark functions have new characteristics such as rotated trap problems, graded level of linkage, and composing functions through dimensions-wise properties. This benchmark can be categorized to the following (Unimodal, Multi-modal, Hybrid, and Composite functions) and the definition of these function can be shown in Table 1 where opt. refers to the mathematical optimal value and the bound of the variables in the search space falls in the interval 2[−100, 100].   [54].
The individual search agent is set to 50 and the maximum number of iteration is fixed to 500. The parameters setting of all comparative algorithms is given in Table 2.

Results & discussion.
In this section, the proposed versions (OBBOA, CLSBOA, and CLSOBBOA) are presented and compared with the original BOA as shown in Table 3. From this table, it has been noticed that the 3rd proposed version called (CLSOBBOA) have achieved the best results in terms of Average/Mean, Best, Worst, and Standard Deviation (SD). Table 4 shows the comparison of CLSOBBOA (the best proposed version) with other stateof-art metaheuristics algorithm. It's noticed that CLSOBBOA achieve best results and ranked first in almost half of the benchmark functions. Figs 1, 2 and 3 show the convergence curve of these functions. Also, Wilcoxon rank sum [55,56] test has been performed between CLSOB-BOA and the native BOA as given in Table 5 where the significance level has been considered 5%.

Engineering problem
In order to evaluate a metaheuristics algorithm, a common approach is testing it on real constrained Engineering problems. These engineering problems have many equality and inequality. In addition, the optimal parameter values of almost engineering problems are unknown. In this subsection, 4 engineering optimization problems are used to test CLSOBBOA. These problems are welded beam engineering design, tension/compression spring, pressure vessel design, and Speed reducer design problem.
3.2.1 Welded beam design problem. This engineering problem proposed by Coello in [57] has 4 parameters. These parameters are design thickness of the weld h, clamped bar length l, bar thickness b, and the height of the bar t. The mathematical representation can be expounded in Appendix 6.1. Table 6 shows the results of CLSOBBOA compared with Animal  Migration Optimization (AMO) [58], Water cycle algorithm (WCA) [59], Lightning search algorithm (LSA) [60], Symbiotic organisms search (SOS) [61], and Grey Wolf Optimizer (GWO) [62].

Tension/Compression spring.
The second engineering constrained problem is called Tension/Compression spring proposed by Arora [63]. The main goal of this problem is to minimize the weight of design spring by find the optimal values for the 3 parameters: the diameter of the wire d, the mean diameter of the coil D and the active coil numbers N. Also, Appendix 6.2 gives its mathematical definition. Table 7 compares the results of CLSOBBOA algorithm with WCA, ABC [64], TLBO [65], and SOS.

Pressure vessel design.
One of the most famous engineering problem is the pressure vessel design introduced by Kannan and Kramer in [66] which aims to minimize the cost of materials, welding, and forming This problem has 4 parameters: the thickness T s , head's thickness T h , the inner radius R, and cylindrical length L. Mathematical definition of this problem is shown in Appendix 6.3. Results of CLSOBBOA compared to other state-of-art algorithms LSA, SOS, ABC and GWO is shown in Table 8.

Speed reducer design problem.
The last engineering problem introduced in this section is the speed reducer problem The objective of the function ids to fond the best parameter which are face weight, teeth on pinion number, teeth module, shaft length 1 between bearings and the shaft length 2 between bearings. The Mathematical representation is shown in Appendix 6.4. Table 9 compare the results of CLSOBBOA with GWO, AMO, WCA, and SOS.

CLSOBBOA in Feature Selection (FS)
In this subsection CLSOBBOA is used in order to solve FS using 5 different datasets.

CLSOBBOA architecture of FS.
To be able to solve feature selection (FS), we regard it as a binary optimization since the solutions are limited to 0, 1 where "0" refers to the corresponding attribute hasn't be selected whereas "1" is its contrary. To convert continous solution to binary one, a transfer function is needed. In this paper, we use sigmoid function as shown in the following equation where x k i refers to the position of i-th agent at dimension k. The output from the previous equation is still continuous and to have binary-valued one, the following stochastic equation is used FS fitness function is finding the small feature number and achieving the highest accuracy. So the FS fitness equation is as follows: where γ(D) refers to error rate, C is the features total number, R is the length-size of selected features. α and β can be calculated as α 2 [0, 1] and β = 1 − α

Experimental setup & results.
Here, 5 different datasets from UCI have been used to evalute the CLSOBBOA performance in solving FS problem. The details of each dataset can be found in Table 10. The results of CLSOBBOA in solving FS problem. The results of CLSOB-BOA compared with original BOA, PSO, and GWO are shown in Tables 11-13 in terms of average fitness, feature size length, and classification accuracy. From these results, we can conclude the significant of CLSOBBOA in solving FS

Conclusion & future work
In this paper, a 3 variants of BOA algorithm have been introduced to improve its performance and preventing it from getting trapped in optimal subregion. These version merge the original BOA with Chaotic local search strategy and Opposition-based Learning concepts. The results show that the algorithm named CLSOBBOA have ranked first in more than half of CEC2014 benchmark functions. Although, the proposed algorithm tested using 4 different constrained engineering problems.