^{1}

^{1}

^{2}

^{3}

The authors have declared that no competing interests exist.

Global optimization is challenging to solve due to its nonlinearity and multimodality. Traditional algorithms such as the gradient-based methods often struggle to deal with such problems and one of the current trends is to use metaheuristic algorithms. In this paper, a novel hybrid population-based global optimization algorithm, called hybrid firefly algorithm (HFA), is proposed by combining the advantages of both the firefly algorithm (FA) and differential evolution (DE). FA and DE are executed in parallel to promote information sharing among the population and thus enhance searching efficiency. In order to evaluate the performance and efficiency of the proposed algorithm, a diverse set of selected benchmark functions are employed and these functions fall into two groups: unimodal and multimodal. The experimental results show better performance of the proposed algorithm compared to the original version of the firefly algorithm (FA), differential evolution (DE) and particle swarm optimization (PSO) in the sense of avoiding local minima and increasing the convergence rate.

Global optimization is crucially important in many applications, such as image processing [

Technically speaking, optimization methods can be divided into two main parts: deterministic algorithms and stochastic algorithms [

Nowadays, most stochastic algorithms can be called meta-heuristic algorithms [

The paper is organized as follows: the main idea of the standard firefly algorithm and standard differential evolution are illustrated in Section 2, and then the details of our proposed hybrid firefly algorithm are described in Section 3. In Section 4, we will demonstrate and carry out the analysis of the experimental results. Finally, Section 5 concludes the work.

Firefly algorithm (FA) [

The Firefly Algorithm (FA) is based on the communication behaviour of tropical fireflies and the idealized behaviour of the flashing patterns. FA uses the following three idealized rules [

All fireflies are unisex so that one firefly will be attracted to other fireflies regardless of their sex;

Attractiveness is proportional to their brightness. Thus for any two flashing fireflies, the less bright one will move towards the brighter one. The attractiveness is proportional to the brightness and they both decrease as their distance increases;

The brightness of a firefly is affected or determined by the landscape of the objective function. (Thus, for a maximization problem, the brightness can simply be proportional to the value of the objective function.)

In the standard firefly algorithm, there are two important points. One is the formulation of the light intensity and another is the change of the attractiveness. Firstly, we can always assume that the brightness of the firefly can be determined by the encoded objective function landscape. Secondly, we should define the variation of light intensity and formulate the change of the attractiveness. As we know that in nature the light intensity decreases with the distance from its source and the media will absorb the light, so in our simulation we suppose the light intensity _{0} is the original light intensity at the source (i.e., at the distance _{0} is the original light attractiveness at

The Cartesian distance is used to calculate the distance between any two fireflies _{i} and _{j}
_{i} being drawn from different distributions such as the Uniform distribution, Gaussian distribution and Lévy flight. In the third term, α is a scaling parameter that controls the step size and it should be linked with the interests of the problems.

According to above idealization and approximations rules, the pseudo-code of standard firefly algorithm can be summarized in Algorithm 1.

Objective function _{1},⋯,_{D})^{T}

Initialize a population of fireflies _{i} (

Calculate the light intensity _{i} at _{i} by _{i})

Define light absorption coefficient

f

Calculate the distance _{i} and _{j} using Cartesian distance equation

_{j} > _{i})

Attractiveness varies with distance

Move firefly

Evaluate new solutions and update light intensity

Rank the fireflies and find the current best

Post-process results and visualization

Differential evolution (DE) was proposed by Storn and Price in 1996, which uses a vectorized mutation operator and two forms of crossover (either exponential or binomial) to evolve from the randomly generated, initial starting points to the potentially optimal solution. There are many DE variants. In this paper, we use the so-called DE

For a given _{i} can be defined as follows:
_{1},_{2}, _{3} ∈ [1,

The binomial crossover operation tries to produce a new trial vector from the perturbed or mutated vector _{i,g+1} = [_{i1,g+1},_{i2,g+1},⋯,_{iD,g+1}] and the target vector _{i,g} = [_{i1,g},_{i2,g},⋯,_{iD,g}]
_{r} ∈ [0,1] is the so-called crossover constant. Here, _{i,g+1} gets at least one character from the mutated vector _{i,g+1}.

The selection mechanism is similar to those of other algorithms where a greedy acceptance is performed:

This means that the update is accepted only if a better objective is achieved.

Algorithm 2 summarizes the basic steps of the standard differential evolution algorithm.

Initialize the population _{i} (

Set the perturbation parameter _{r}

f

For each _{i}, randomly choose 3 different vectors

Use mutation to generate a new vector _{i}

Generate a random index

Generate a randomly distributed number

Crossover operation, for each parameter _{ij}, update

Select operation, select and update the solution _{i}

Post-process results and visualization

Both the firefly algorithm and differential evolution have their own advantages and they both work well for a wide range of optimization problems. In this paper, we propose a new hybrid algorithm based on FA and DE by combining some of the advantages of both algorithms. We call the proposed approach the hybrid firefly algorithm (HFA) that combines the attraction mechanism of FA with the mixing ability of DE so as to increase the speed of convergence and the diversity of the population. The major difference between firefly algorithm and differential evolution is how new individuals are generated and then used at each iteration.

Among the many components of algorithms, intensification and diversification (also called exploitation and exploration) are the two major components of any meta-heuristic algorithm [

Firstly, the earlier observations and studies in the literature indicated that the firefly algorithm can subdivide the whole population into subgroups automatically in terms of the attraction mechanism via the variation of light intensity and one of the FA variants can escape from the local minima owing to long-distance mobility by Lévy flight [

Based on above descriptions, the fundamental steps of the HFA can be summarized as the pseudo-code shown in Algorithm 3 where we can see that the parallel use of FA and DE can strike a good balance between exploration and exploitation during the whole iteration process.

Divide the whole group into two groups: _{1} and _{2}

Initialize the populations _{1} and _{2}

Evaluate the fitness value of each particle

Perform FA operation on _{1}

Perform DE operation on _{2}

Update the global best in the whole population

Mix the two groups and regroup them randomly into new groups: _{1} and _{2}

Evaluate the fitness value of each particle

Post-process results and visualization

Though the detailed computational complexity may depend on the structure of the implementation, however, for three meta-heuristic algorithms used in this paper, their complexities can be easily estimated. For FA, the time complexity is O^{2}^{2}

Benchmark functions are useful to evaluate new algorithms and their features such as the precision, the rate of convergence, the robustness and the general performance. To evaluate the performance of our proposed algorithm and other existing algorithms, a set of 13 standard benchmark functions is used and such benchmarks have been chosen with a diverse range of properties. Theoretically speaking, if a small number of the benchmark functions are used, the experimental results may be potential biased due to the limited diversity of the problem objective landscape and in this case it would be very difficult to draw any convincing conclusions. Therefore, we have chosen test functions based on the characteristics, modality and other properties so as to provide a fairly rich set of functions with varied difficulties. In essence, we used the same test functions as those used in [_{min} is the global optimum value in the variable scales.

Function Name | Function | _{min} |
||
---|---|---|---|---|

Sphere | 30 | [−100,100]^{D} |
0 | |

Schwefel’s 2.22 | 30 | [−10,10]^{D} |
0 | |

Schwefel’s 1.20 | 30 | [−100,100]^{D} |
0 | |

Schwefel’s 2.21 | _{4}(_{i}{|_{i}|,1 ≤ |
30 | [−100,100]^{D} |
0 |

Rosenbrock | 30 | [−30,30]^{D} |
0 | |

Step | 30 | [−100,100]^{D} |
0 | |

Quartic Noise | 30 | [−1.28,1.28]^{D} |
0 |

Function Name | Function | D | S | _{min} |
---|---|---|---|---|

Schwefel’s 2.26 | 30 | [−500,500]^{D} |
-12569.5 | |

Rastrigin | 30 | [−5.12,5.12]^{D} |
0 | |

Ackley | 30 | [−32,32]^{D} |
0 | |

Griewank | 30 | [−600,600]^{D} |
0 | |

Pendlized | 30 | [−50,50]^{D} |
0 | |

Generalized Pendlized | 30 | [−50,50]^{D} |
0 |

The test benchmark functions can be divided into two groups in terms of the number of local minima: unimodal functions and multimodal functions. The unimodal test functions have one global optimum, so they are suitable for benchmarking the local exploitation ability of algorithms. This kind of functions will allow to focus more on the convergence rates of the tested algorithms other than the final results. Multimodal test functions have many local minima, and the number of local optima usually increases exponentially with the problem dimension, so they are suitable for benchmarking the global exploration ability of algorithms. This kind of multimodal functions can test the exploration ability which can make the algorithm escape from local optima. In some applications, to find a good optimal or suboptimal solution is more important, while other applications may place the emphasis on the accuracy of the solutions. So the quality of final results is more of concern in such applications.

From _{1}-_{7} are unimodal, high-dimensional problems. Function _{5}, also namely the ‘banana function’, has a global optimum inside a long but flat, narrow, parabola-shaped valley. To find the location of the valley is non-trivial, though not too difficult. However, to converge to the global minimum with a high accuracy is more difficult, especially for gradient-based algorithms. Function _{6} is the step function, characterized by plateaus and discontinuities. In addition, function _{7} is a noisy quadratic function.

Functions _{8}–_{13} in _{8} is a non-convex, multimodal and additively separable function. This seemingly simple function can be deceptive because the global minimum at (420.9687,⋯,420.9687) is geometrically distant from the next best local minima in the domain [−500,500]^{D} where _{9} is also challenging as it is one of the most difficult benchmarks commonly used in the literature because it has multiple, steep wells with multiple local minima. Another widely used multimodal benchmark function is _{10}, namely the Ackley function, which can be characterized by a deep valley at the centre and an almost flat outer zone. Consequently, it is quite challenging to solve because it is easy for most optimization algorithms to get trapped in one of its many local minima due to the multimodality.

For the verification purpose of the algorithms and the analysis of the experimental results, our proposed hybrid firefly algorithm is compared to the standard FA and DE as well as PSO to benchmark the performance and to see if there is any improvement.

In all cases, the population size is set to 40, and the dimension of the benchmark functions is equal to 30. We also set the maximum number of iterations, as the stopping criteria, equal to 2000. The initial population is generated using uniformly distributed random initialization within the ranges or limits of the design variables. In addition, 30 independent runs have also been carried out for each function and each algorithm with completely different initial settings. The results from the algorithms are accompanied according to four standard statistical measures: the Minimum, the Maximum, the Mean, and the Standard Deviation (Std) of the fitness values calculated over 30 independent runs.

For the firefly algorithm, we set the initial attractiveness _{0} = 2 * ^{2} where ^{iter} where 0.2 is the initial randomness factor and _{r} = 0.9 as the crossover constant [_{1} and _{2} are both set as 2, the inertia weight _{max} = 0.9 to _{min} = 0.4 [

It is worth pointing out that in our proposed HFA, the parameters, _{0}, _{i},

All of the algorithm-dependent parameters are summarised in

Algorithm | Control parameters | |||
---|---|---|---|---|

FA | _{0} = 2 * |
^{2} |
^{iter} |
_{i} = |

DE | _{r} = 0.9 |
|||

PSO | _{max} = 0.9 |
_{min} = 0.4 |
_{1} = 2 |
_{2} = 2 |

In the first series of experiments, the aim is to compare the exploitation ability and convergence rate of the mentioned algorithms for functions _{1}-_{7}. The statistic results of 30 independent runs are given in

# |
Statistics | HFA | FA | DE | PSO |
---|---|---|---|---|---|

_{1} |
Min | 1.07E-193 | 1.02E-87 | 1.3949e-09 | 6.33E-13 |

Max | 7.84E-170 | 1.95E-87 | 6.1265e-08 | 6.86E-10 | |

Mean | 1.57E-87 | 1.4155e-08 | 9.43E-11 | ||

Std | 0 | 1.89E-88 | 1.2949e-08 | 1.48E-10 | |

_{2} |
Min | 1.40E-117 | 1.56E-44 | 1.7950e-04 | 3.21E-09 |

Max | 7.39E-102 | 1.95E-44 | 0.0013 | 2.48E-07 | |

Mean | 1.73E-44 | 6.0678e-04 | 3.66E-08 | ||

Std | 1.35E-102 | 8.52E-46 | 2.8188e-04 | 4.69E-08 | |

_{3} |
Min | 1.97E-66 | 2.3801 | 0.0454 | 127.3 |

Max | 1.30E-55 | 97.594 | 1.2297 | 1237.8 | |

Mean | 25.714 | 0.2789 | 459.69 | ||

Std | 2.42E-56 | 24.013 | 0.2844 | 241.9 | |

_{4} |
Min | 2.05E-05 | 1.37E-44 | 0.3745 | 2.787 |

Max | 2.5528 | 1.86E-44 | 2.3055 | 12.205 | |

Mean | 0.7115 | 0.8832 | 6.5934 | ||

Std | 0.76784 | 1.30E-45 | 0.3877 | 2.2382 | |

_{5} |
Min | 2.47E-29 | 26.346 | 14.9121 | 1.8755 |

Max | 0.53092 | 89.131 | 25.2670 | 114.49 | |

Mean | 29.053 | 21.9994 | 49.686 | ||

Std | 0.16183 | 11.348 | 2.1032 | 34.029 | |

_{6} |
Min | 0 | 0 | 0 | 0 |

Max | 0 | 0 | 0 | 0 | |

Mean | |||||

Std | 0 | 0 | 0 | 0 | |

_{7} |
Min | 7.15E-05 | 0.000518 | 0.0029 | 0.013704 |

Max | 0.000296 | 0.003894 | 0.0239 | 0.04622 | |

Mean | 0.001582 | 0.0113 | 0.031475 | ||

Std | 5.07E-05 | 0.000797 | 0.0045 | 0.008425 |

As can be seen from _{4}. For _{4}, our proposed HFA cannot tune itself successfully, whereas FA solves this function quite accurately. In essence, this case is consistent with the so-called no-free-lunch (NFL) theorems. This means that there is no universally superior algorithm for all types of problems [

In the rest of this section, we use Freidman tests to test which of the mentioned algorithms are statistically better in the solution of benchmark functions [

# |
HFA | FA | DE | PSO |
---|---|---|---|---|

_{1} |
2.64E-171 | 1.57E-87 | 1.4155e-08 | 9.43E-11 |

_{2} |
2.46E-103 | 1.73E-44 | 6.0678e-04 | 3.66E-08 |

_{3} |
0.7115 | 25.714 | 0.2789 | 459.69 |

_{4} |
5.30E-57 | 1.68E-44 | 0.8832 | 6.5934 |

_{5} |
0.077152 | 29.053 | 21.9994 | 49.686 |

_{6} |
0 | 0 | 0 | 0 |

_{7} |
0.000183 | 0.001582 | 0.0113 | 0.031475 |

T-test | HFA -FA | HFA -DE | HFA-PSO |
---|---|---|---|

P | 0.0143 | 0.1025 | 0.0143 |

Figs _{1}, _{3} and _{5}. In these figures, the horizontal axis is the number of iterations and the vertical axis is the fitness value of the benchmark function. It is can be seen that HFA performs significantly better than FA, DE, and PSO. For example, _{1}, namely the simple sphere function, is a famous benchmark function. During the whole generations, HFA displays a faster convergence rate than those of FA, DE, and PSO due to its better exploitation search ability. It is clear that HFA quickly reaches the neighborhood of the global optimum and gets approximately 10^{−16} with only 200 iterations, while DE and PSO can only reach approximately 10^{−12} and 10^{−8}, respectively after the final 2000 iterations. In fact, HFA has a nearly constant convergence rate throughout the whole iteration for most of the unimodal benchmark functions. And the experiment results of HFA after 200 iterations (which means only one iterative repetition time) are better than the final results of DE and PSO after 2000 iterations. Hence from Figs

For the second series of experiments, we use multimodal functions to compare the exploration ability of the compared algorithms. The statistical results of comparing the mentioned algorithms with 30 independent runs are presented in

# |
Statistics | HFA | FA | DE | PSO |
---|---|---|---|---|---|

_{8} |
Min | -12569 | -10596 | -5627.9 | -10001 |

Max | -12214 | -8424.1 | -4515.8 | -7093.8 | |

Mean | -9469.5 | -5016.3 | -9020.8 | ||

Std | 133.24 | 515.47 | 260.0659 | 515.74 | |

_{9} |
Min | 1.08E-08 | 3.9798 | 143.8889 | 17.909 |

Max | 4.36E-08 | 15.919 | 196.3629 | 44.773 | |

Mean | 9.3858 | 175.9112 | 30.15 | ||

Std | 7.29E-09 | 3.0436 | 12.24334 | 7.1079 | |

_{10} |
Min | 4.44E-15 | 7.99E-15 | 4.3632e-05 | 4.75E-07 |

Max | 6.13E-05 | 1.51E-14 | 0.0031 | 6.15E-05 | |

Mean | 1.31E-05 | 3.1272e-04 | 7.10E-06 | ||

Std | 2.33E-05 | 3.36E-15 | 5.4874e-04 | 1.47E-05 | |

_{11} |
Min | 0 | 0 | 1.8299e-07 | 3.74E-12 |

Max | 5.64E-08 | 0 | 0.1005 | 0.046483 | |

Mean | 5.86E-09 | 0.0132 | 0.013444 | ||

Std | 1.19E-08 | 0 | 0.0220 | 0.012311 | |

_{12} |
Min | 1.57E-32 | 1.57E-32 | 1.8989e-09 | 2.34E-12 |

Max | 1.57E-32 | 1.57E-32 | 0.0036 | 0.31096 | |

Mean | 2.2768e-04 | 0.024188 | |||

Std | 5.57E-48 | 5.57E-48 | 6.8779e-04 | 0.064897 | |

_{13} |
Min | 1.35E-32 | 1.35E-32 | 6.6202e-08 | 2.42E-11 |

Max | 1.35E-32 | 1.35E-32 | 7.1684e-05 | 0.010987 | |

Mean | 1.1956e-05 | 0.0032963 | |||

Std | 5.57E-48 | 5.57E-48 | 1.7650e-05 | 0.0051211 |

From the statistic results in _{8} and _{9}. The FA is the best for solving the functions _{10} and _{11}. In addition, HFA and FA have almost equal optimization abilities for solving the functions _{12} and _{13}. Both can obtain the accurate results of these functions.

Similar to what we have done for the unimodal test functions, the Friedman tests using the significance level of 0.95 (or α = 0.05) are also conducted for all the multimodal benchmark functions.

# |
HFA | FA | DE | PSO |
---|---|---|---|---|

_{8} |
-12439 | -9469.5 | -5016.3 | -9020.8 |

_{9} |
3.39E-08 | 9.3858 | 175.9112 | 30.15 |

_{10} |
1.31E-05 | 1.25E-14 | 3.1272e-04 | 7.10E-06 |

_{11} |
5.86E-09 | 0 | 0.0132 | 0.013444 |

_{12} |
1.57E-32 | 1.57E-32 | 2.2768e-04 | 0.024188 |

_{13} |
1.35E-32 | 1.35E-32 | 1.1956e-05 | 0.0032963 |

T-test | HFA -FA | HFA -DE | HFA-PSO |
---|---|---|---|

P | 1.0000 | 0.0143 | 0.1025 |

At the same time, the convergence curves of different algorithms for _{9} and _{10} have been shown in Figs

In this paper, we have proposed a novel hybrid firefly algorithm (HFA) by combining some of the advantages of both firefly algorithm and differential evolution. Based on the theoretical analysis and the problem solving ability of metaheuristic algorithms, we can summarize that HFA has three advantages or improvements: the first strategy is equipped with a better balance between exploration and exploitation due to the parallel use of FA and DE and the population information-sharing. The experimental results illustrated that FA can provide an excellent convergence rate and a strong exploration ability, whereas DE is good at exploitation by using mutation and crossover operators. Ideally, an algorithm should explore the search space as extensively as possible to find all the promising regions and simultaneously it should conduct a more refined search in the promising areas so as to improve the precision of the solutions.

The second improvement is that the selection mechanism used in the proposed approach can enable the solution to converge to the optimum in a better way. This is achieved by first mixing the two subpopulations that are independently evolved using either FA or DE, and then selecting the best solutions among both subpopuations. Thus, it is more likely to find the global optimum than each individual algorithm involved in the hybrid. The third strategy improvement is that the hybrid can increase the diversity of solutions efficiently and can also help the algorithm avoid the stagnation problem by using a mixing and regrouping mechanism. It can be observed that the attraction operator in FA is a double-edged sword. To some extent, it can accelerate the convergence speed, but may also mislead the algorithm to get stuck into some local optima if the diversity of the population becomes low. Technically speaking, this hybrid mechanism can liberate the population from sub-optimal solutions and enable a continued progress toward the true global optima as have been observed in the simulations.

The statistical analyses have also confirmed the theoretical insight in this paper that the three enhancements in the combined approach can explore and exploit the search space more efficiently. It has been seen from the above results that the proposed HFA can indeed work well compared to FA, DE and PSO, which has been further confirmed by the results obtained from the Friedman tests.

Future work will explore different ways of mixing and regrouping the population so as to enhance the performance even further. In addition, it will be useful to carry out a more detailed parametric study to see how different sub-stages of iterations can be used to maximize the parallelism and also to reduce the overall number of iterations. Furthermore, it will also be useful to automatically tune these parameters depending on the modality of the problem and thus can solve problems more effectively in real-world applications.

This work has been supported financially by the National Natural Science Foundation of China under the Grant 51109041, by the Fundamental Research Funds for the Central Universities under the Grant HEUCF160405 and also supported by the China Scholarship Council.