A Novel Flexible Inertia Weight Particle Swarm Optimization Algorithm

Particle swarm optimization (PSO) is an evolutionary computing method based on intelligent collective behavior of some animals. It is easy to implement and there are few parameters to adjust. The performance of PSO algorithm depends greatly on the appropriate parameter selection strategies for fine tuning its parameters. Inertia weight (IW) is one of PSO’s parameters used to bring about a balance between the exploration and exploitation characteristics of PSO. This paper proposes a new nonlinear strategy for selecting inertia weight which is named Flexible Exponential Inertia Weight (FEIW) strategy because according to each problem we can construct an increasing or decreasing inertia weight strategy with suitable parameters selection. The efficacy and efficiency of PSO algorithm with FEIW strategy (FEPSO) is validated on a suite of benchmark problems with different dimensions. Also FEIW is compared with best time-varying, adaptive, constant and random inertia weights. Experimental results and statistical analysis prove that FEIW improves the search performance in terms of solution quality as well as convergence rate.


Introduction
Swarm intelligence is an exciting new research field still in its infancy compared to other paradigms in artificial intelligence [1]. One of the research areas within computational swarm intelligence is particle swarm optimization (PSO), which developed by Eberhart and Kennedy in 1995 [2,3], inspired by intelligent collective behavior of some animals such as flocks of birds or schools of fish. In PSO, each individual represents a potential solution and is termed as "particle" and the flock of particles called "swarm" represents the population of individuals, so a population of potential solutions is evolved through successive iterations. The most important advantages of the PSO, compared to other optimization strategies, lies in its speedy convergence towards global optimum, easily implementable code, complex computation free environment and having few parameters to adjust. Accelerating convergence speed and avoiding the local optima have become the two most important and appealing goals in PSO research. A number of variant PSO algorithms have, hence, been proposed to achieve these two goals [4,5]. It is seen to be difficult to simultaneously achieve both goals. For example, the comprehensive-learning PSO in [5] focuses on avoiding the local optima, but brings in a slower convergence as a result. Therefore, despite being having several attractive features and a potential global optimizer, PSO alike several other populations based search algorithms have certain drawbacks associated with it. To overcome the drawbacks caused by "stagnation of particles", several attempts have been made to enhance the performance of PSO and the improved variants superseded the standard one. Some of these include, proposing inertia weight (IW) [6,7], introducing constriction factor based PSO [8], weighting particle's own experience and neighbors experience [9], fine tuning of various PSO parameters [10], proposing different interaction methods among PSO particles [11,12]. Moreover PSO has been hybridized [13] with concepts borrowed from other heuristic and deterministic algorithms to improve its searching ability and enhancing its convergence towards global optima. As we know, IW can balance the proportion of global search ability and local exploration ability. When its value is bigger, the algorithm has a stronger global search ability and poorer local exploration ability. When IW value is smaller, global search ability and local exploration ability are just reverse. In the other word, IW controls the particle's momentum and so many strategies have been proposed in previous studies to choose a suitable IW that maintains the exploration-exploitation trade-off throughout the searching process. In this paper we propose a flexible exponential inertia weight (FEIW) PSO algorithm (FEPSO) for optimization problems. This work differs from the existing time-varying IW strategies at least in two aspects: firstly, it proposes a flexible IW, which can adapt with each problem, i.e., for a certain optimization problem, with suitable parameter selection, we can get a special IW strategy that has best performance for solving it. The second is to compare the best time-varying, adaptive and primitive IW strategies with FEIW and obtain that FEPSO is more efficacious for optimization problem.
The rest of this paper is organized as follows: Section 2 presents the principles of particle swarm optimization algorithm. A review on inertia weight strategies is stated in section 3. Proposed inertia weight and its properties will be discussed in section 4. In Section 5, parameter settings and performance evaluation criteria is introduced. The numerical analysis, statistical tests and discussion of results is performed under section 6 and the conclusions are given in section 7.

The Principles of Particle Swarm Optimization Algorithm
The basic idea of the PSO algorithm is to search out the optimum value by collaborating and sharing information between the individuals, and the particle's quality could be measured according to the fitness value of particles. First, the positions and velocities of a group of particles are initialized randomly, and then the optimal solution can be searched out by updating generations in the search space. Suppose that the size of the swarm is M and the search space is D − dimensional. The position of the ith particle is presented as [1, D], and l d and u d are the lower and upper bounds of the dth dimension of the search space. The velocity of each particle is represented with a vector. The ith particle velocity is presented as v i = (v i1 , v i2 , . . ., v iD ). At each time step, the position and velocity of the particles are updated according to the following equations [2]: n ij ðt þ 1Þ ¼ n ij ðtÞ þ c 1 r 1ij ½p bestij ðtÞ À x ij ðtÞ þ c 2 r 2ij ½g bestj ðtÞ À x ij ðtÞ ð1Þ x ij ðt þ 1Þ ¼ x ij ðtÞ þ n ij ðt þ 1Þ ð 2Þ where r 1ij , r 2ij are two distinct random numbers [2], generated uniformly from the range [0,1], the acceleration coefficients c 1 , c 2 are two positive constants [3] and t is the current iterative time.
The best previous position found so far by this particle is denoted as p besti = (p i1 , p i2 , . . . ,p iD ), and the best previous position discovered by the whole swarm is denoted as g best = (g 1 , g 2 , . . . ,g D ).
The velocity of particle should be under the constrained conditions [v min , v max ] D . The balance between global and local search throughout the course of a run is critical to the success of an optimization algorithm [14]. Almost all of the evolutionary algorithms utilize some mechanisms to achieve this goal. To bring about a balance between the exploration and exploitation characteristics of PSO, Shi and Eberhart proposed a PSO based on inertia weight (ω) in which the velocity of each particle is updated according to the following equation [15]: n ij ðt þ 1Þ ¼ on ij ðtÞ þ c 1 r 1ij ½p bestij ðtÞ À x ij ðtÞ þ c 2 r 2ij ½g bestj ðtÞ À x ij ðtÞ ð3Þ They claimed that a large IW facilitates a global search while a small IW facilitates a local search. By changing the IW dynamically, the search capability is dynamically adjusted. This is a general statement about the impact of ω on PSO's search behavior shared by many other researchers. However, there are situations where this rule cannot be applied successfully [16].
The PSO procedure can be divided into the following steps: 1. Initialize the original position and velocity of particle swarm; 2. Calculate the fitness value of each particle; 3. For each particle, compare the fitness value with the fitness value of p best , if current value is better, then renew the position with current position, and update the fitness value simultaneously; 4. Determine the best particle of group with the best fitness value, if the fitness value is better than the fitness value of g best , then update the g best and its fitness value with the position; 5. Check the finalizing criterion, if it has been satisfied, quit the iteration; 6. Update the position and velocity of particle swarm, return to step 2.

Review on Inertia Weight Strategies
Since the initial development of PSO, several variants of this algorithm have been proposed by researchers. The basic PSO, presented by Kennedy and Eberhart in 1995 [2], has no IW. The first modification introduced in PSO was the use of an IW parameter in the velocity update equation of the initial PSO resulting in Eq (3), a PSO model which is now accepted as the global best PSO algorithm [15]. In this section, the various IW strategies are categorized into three classes. The "primitive class" contains strategies in which the value of the IW is constant during the search or is determined randomly. None of these methods uses any input. The "adaptive class" contains those methods which use a feedback parameter to monitor the state of the algorithm and adjust the value of the IW. The "time-varying class" is defined as a function of time or iteration number.

Primitive class
IW parameter was originally introduced by Shi and Eberhart in [15]. They used a range of constant IW (CIW) values and with low values of this parameter, i.e. ω > 0.8, PSO tends to traps in local optima. They suggest that with a ω within the range [0.8,1.2], PSO finds the global optimum in a reasonable number of iterations. Shi and Eberhart analyzed the impact of the IW and maximum velocity on the performance of the PSO in [6]. In [17], a random IW (RIW) is used to enable the PSO to track the optima in a dynamic environment.
where Rand() is a random number in [0.1]; ω is then a uniform random variable in the range [0.5,1].

Adaptive class
Adaptive IW strategies are those that monitor the search situation and adapt the IW value based on one or more feedback parameters. In [18], Arumugam and Rao use the ratio of the global best fitness and the average of local best fitness of particles to determine the IW in each iteration with where f(.) is the fitness function. The inertia weight in (6) is termed global-average local best IW (GLBIW). Clerc [19] proposes an adaptive inertia weight (AIW) approach where the amount of change in the inertia value is proportional to the relative improvement of the swarm. Let x i (t) denote the position of particle i in the search space at time step t. The inertia weight is adjusted according to where the relative improvement, m i , is estimated as with ω(I max ) % 0.5 and ω(0) < 1.

Time-varying class
Most of the PSO variants use time-varying IW strategies in which the value of the IW is determined based on the iteration number. Time-varying IW strategies have important applications in various fields yet [20,21]. These methods can be either linear or non-linear and increasing or decreasing. In [8], a linear decreasing IW (LDIW) was introduced and was shown to be effective in improving the fine-tuning characteristic of the PSO. In this method, the value of ω is linearly decreased from an initial value (ω max ) to a final value (ω min ) according to the following equation: where t and I max are the current iterative time and the maximum iterative time, respectively. This strategy is very common and most of the PSO algorithms adjust the value of IW using this updating scheme.
Accepting the general idea of decreasing the IW over iterations, some researchers proposed nonlinear decreasing strategies. Chatterjee and Siarry [22] propose a nonlinear decreasing variant of IW in which at each iteration of the algorithm, ω is determined based on the following equation: where n is the nonlinear modulation index. Different values of n result in different variations of IW all of which start from ω max and end at ω min . Feng et al. [23,24] use a chaotic IW (CHIW) in which a chaotic term is added to the LDIW. The proposed ω is as follows.
where ω 1 and ω 2 are the original value and the final value of IW and z = 4z (1 − z). The initial value of z is selected randomly within the range(0,1). Chen et al. [25] propose a natural exponential inertia weight (NEIW) strategy according to the following equation: where ω min = 0.4 and ω max = 0.9, which is found to be very effective for NEIWPSO.
Li and Gao [26] give a kind of exponent decreasing inertia weight (EDIW) The massive experiments indicate the algorithm performance can enhance greatly when ω min = 0.4, ω max = 0.95, d 1 = 0.2 and d 2 = 7. In [27], Bansal et al. implemented a comparative study on fifteen IW strategies to select best IW strategies. With c = 7 for CIW, ω min = 0.4, ω max = 0.9 for LDIW and ω 1 = 0.9, ω 2 = 0.4 for CHIW, They concluded that CHIW is the best strategy for better accuracy and RIW strategy is best for better efficiency. Also it is shown that CIW and LDIW are best inertia weights based on minimum error. Arasomwan and Adewumi [28] established the fact that LDIW is very much efficient if its parameters are properly set. They showed that with good experimental setting, LDIW will perform competitively with similar variants. Thus in this paper, for comparative studies, we use of CIW, RIW, LDIW, CHIW, NEIW, EDIW, GLBIW and AIW as eight well-known primitive, time-varying and adaptive IW strategies.

Proposed Inertia Weight and Its Properties
In order to overcome the premature convergence, low efficiency or low accuracy of the other IW strategies, we introduce a novel IW strategy for improving the performance of PSO. In this section, first this new IW will be introduced then its properties will be analyzed. At the end, we introduce the IW strategy parameters.

Proposed inertia weight strategy
Definition. Suppose ω 1 , ω 2 and ψ are positive real numbers. We define an inertia weight strategy by where and t 2 [0,I max ] is an integer number. In this strategy, t and I max are the current iterative time and the maximum iterative time, respectively. The parameters ω 1 and ω 2 are inertia weight at the start and inertia weight at the end of a given run, respectively. In the other word and We call ω(t), the Flexible Exponential Inertia Weight (FEIW) strategy because it can adapt with each problem, i.e., with suitable parameters selection, we can construct many increasing or decreasing inertia weights, or even a lot of strategies with one global minimum in [0,I max ], thus FEIW encompasses a wide range of IW strategies. There is a trade-off between accuracy and efficiency of the PSO algorithm and one of the most important of applications of FEIW is that according to each problem, one can easily change the parameters ω 1 , ω 2 and ψ, to achieve better accuracy or better efficiency or both of them. Fig 1 shows the flow-chart for PSO based on the FEIW technique used in this paper.

Flexible exponential inertia weight analysis
Before using FEIW, we should have some information about its behavior. In particular, to select its parameters, we need a careful analysis of the function ω(t) In this subsection, for a mathematical analysis of FEIW, suppose that t 2 [0,I max ] be a real number instead of integer number. We define a new function by and call it as "check function". Also the notation sgn(.) means the sign function is as follows: sgnðxÞ ¼ x jxj The check function has the following properties: and Proof. According to definition of FEIW, ψ > 0 thus 1 -e 2ψ < 0, therefore based on Eq (15), Similarly one can prove the other term.
Proof. By using relation (14), we have From Lemma1 and relation (22), we can conclude Àa 1 a 2 > 0, hence the proof is complete. On the other hand, ψ > 0 and I max > 0, thus Using Eqs (15), (16) and (19), we have : ω(t) < 0. First note that based on relations (17) and (18), the end points of curve of ω(t) have positive values. Since ω(t) is a continuous function, thus it has at least two roots, a contradiction, because according to Lemma 2, the equation ω(t) = 0 has at most one root.
Proof. Suppose that ω 1 = ω 2 = O. From Eqs (15) and (16), we have α 1 = e ψ α 2 , thus using Eq (14), it is concluded that In this special case, the check functions are as follows: By Theorem 1, has a minimum at t ÃÃ ¼ I max 2c ln a 1 1þe c e c 2 . Lemma 4. As ψ approaches 0 from the right, FEIW function approaches linear inertia weight function. If ω 1 > ω 2 , then this linear function is decreasing, while if ω 1 < ω 2 , the function is increasing.
Proof. Differentiating ω(t) with respect to t, from Eqs (14)-(16), we get where m is the slope of line through (0,ω 1 ) and (I max ,ω 2 ). Thus the limit of FEIW function as ψ approaches 0 from the right equals o c lim ðtÞ as follows: Since I max > 0, relation (37) implies o c lim ðtÞ is decreasing if ω 1 > ω 2 , and is increasing if All of above results are summarized in Table 1.

Flexible exponential inertia weight parameters
The massive experiments indicate the proposed algorithm performance can enhance greatly for most problems when ω 1 % 0, ω 2 % 1, ψ % 2.6 for increasing FEIW and ω 1 % 1, ω 2 % 0, ψ % 2.6 for decreasing FEIW and ψ % 5 for cases ω 1 % ω 2 . In this paper, the parameters of different variations of FEIW strategy are selected such that include all the different situations such as increasing (decreasing) functions and functions with a global minimum.
In this strategy, according to Table 1, we experimentally select three values for ψ as follows: Also six pairs of positive numbers are selected for (ω 1 ,ω 2 ). These variations of FEIW strategies in Table 2 will be used for comparison with four best IW strategies [27] i.e., CIW, RIW, LDIW and CHIW and four well-known strategies i.e., NEIW, EDIW, GLBIW and AIW. As shown in Fig 2, unlike other inertia weights, the FEIW strategies are either increasing functions or decreasing functions or none.

Parameter Settings and Performance Evaluation Criteria
From the standard set of benchmark problems available in the literature, twenty six problems are selected to test efficacy and accuracy of the proposed variants with other existing variants. These problems are of continuous variables and have different degrees of complexity and multimodality. These functions are shown in Tables 3 and 4 along with their range of search space.

Parameter settings
For implementing these fourteen strategies in PSO, a code has been developed in MATLAB 1 2014. For a fair comparison, all the fourteen variants are run with the same parameter setting and on same computing environment. Each PSO variant is run 100 times with random initial population.

Performance evaluation criteria (PEC)
According to the "no free lunch theorem" [34], one optimization algorithm cannot offer better performance than all the others on every aspect or on every kind of problem. Thus the efficiency and accuracy of all algorithms is tested against a set of well-known standard benchmark unimodal and multimodal functions given in Tables 3 and 4. Also we use of different evaluation criteria to obtain valid results. A run in which the algorithm finds a solution satisfying |f out − f min | < ε, where f out is the best solution found when the algorithm terminates and f min is the known global minimum of the problem, is considered to be successful. In this case, ε is error of the algorithm. In order to evaluate the performance of different IW strategies, we need to define different terms for termination of the PSO algorithm, so the termination criterion for all considered PSO variants is one of the following conditions: ➢ Condition 1: achieving to I max .
➢ Condition 2: achieving to I max or when the known optimum is within 1 -ε of accuracy, whichever occurs earlier.
For each method and problem the following are recorded: 1. Success rate (SR) is number of successful runs (S run ) per total number of runs (T run ) 2. Average number of iterations of successful runs (ANS).  Table 4. Benchmark functions formula.

Minimum number of iterations of successful runs (MNS).
No.
6 Results, Analysis and Discussions

Numerical results
In this subsection, a comprehensive comparative study of IW for fourteen strategies is carried out. The computational results for all the considered set of benchmark functions using all the

Comparison Analysis of IW Strategies
According to the numerical results obtained from this study (Tables 5-14), we can compare IW strategies with each other based on any benchmark function. For each problem and each PEC, the best and worst IW strategies have been determined in Tables 15-22. The following notation is used in these tables: S-IW indicates several inertia weights except variations of FEIW. Also S-FEIW indicates several inertia weights including some variations of FEIW. For example in Table 17, the worst IW strategies for Pinter function (f 23 ) in terms of ANS, are GLBIW and AIW, also in Table 20, the best IW strategies for Quintic function (f 22 ) in terms of AE, are FEIW-3, FEIW-5 and Table 5. Comparison of success rate, average and minimum number of iterations of successful runs for considered PSO variants with condition 2, I max = 1000, D = 10, ε = 10 −1 for f 2 , f 3 , f 4 , f 10 functions and ε = 10 −10 for others (υ > I max ).

IW
PEC  Table 23, it could be concluded that FEPSO seems to be more efficient and has good convergence compared to other IW strategies.
In the next subsection, we will show that statistical tests confirm that the variations of FEIW significantly improves results. Wilcoxon sign rank test is nonparametric statistically hypothesis test which can be used as an alternative to the paired t-test when the results cannot be assumed to be normally distributed. The results for Wilcoxon's test are summarized as R + and R − , which represent the sum of positive and negative ranks of an algorithm in comparison to other algorithms in the column. During statistical analysis on Table 5, we have considered two performance criteria, average and minimum number of iterations of successful runs, which evaluate the convergence speed of a given algorithm. Table 24 comprises results of wilcoxon signed rank test for these two performance criteria taken I max = 1000 and D = 10. Table 24 shows that the variations of FEIW win over other strategies in 23 of 24 tests in terms of average number of iterations of successful runs. Also the p-value in most of the cases is less than 0.01. Thus in terms of average number of iterations of successful runs, all the six variations of FEIW are significantly better than CIW, RIW, LDIW and CHIW. According to Table 24, this is true for minimum number of iterations of successful runs. Therefore the wilcoxon sign rank test on Table 5 clearly proves the superiority of FEIW over other IW models in terms of convergence speed. Table 25 shows the results for wilcoxon signed rank test for average and minimum number of iterations of successful runs according to Table 6. Table 25 shows that FEIW-1, FEIW-5 and FEIW-6 win over GLBIW, AIW, NEIW and EDIW in the all cases and also the p-value is less than 0.01 and thus these three variations of FEIW are significantly better than other IW strategies in terms of convergence speed. With applying statistical analysis on Tables 7 and 8, we can evaluate the solution precision of FEPSO algorithm. Table 26 comprises results of wilcoxon signed rank test for average and minimum error taken for I max = 1000 and D = 10. Table 26 shows that except in FEIW-4, the other variations of FEIW win over other strategies in most of the cases with p-value<0.05. Thus in terms of average and minimum error, FEIW is significantly better than CIW, RIW, LDIW and CHIW. Therefore the wilcoxon sign rank test on Tables 7 and 8 clearly proves the superiority of FEIW over other IW models in terms of solution precision. Table 27 shows the results for wilcoxon signed rank test for average and minimum error according to Tables 9 and 10. The observation of results in Table 27 confirms that FEIW-1 wins in the all cases with p-value less than 0.05 and is significantly better than GLBIW, AIW, NEIW and EDIW. Using wilcoxon signed rank test from Tables 11 and 12, the solution precision of FEPSO algorithm for I max = 500 and D = 50 can be evaluated. Table 28 contains results of this test for average and minimum error. In terms of average error, all the variations of FEIW win over CIW, RIW and LDIW strategies in all the cases with p-value<0.05. Also FEIW-2 wins over CHIW strategy in all the cases with p-value< 0.05. In   [38,39]. The goal of this test is to determine whether there are significant differences among the algorithms considered over given sets of data. The Friedman test determines the ranks of the algorithms for each individual data set, i.e., in the minimization problems, the best performing algorithm getting minimum rank. Outcomes of Friedman test on Tables 5-14  are shown in Tables 30-35. The results of Friedman test are used to observe whether there is overall difference among IW strategies. In all tables the p-value of Friedman test is lower than the level of significance considered α = 0.05 and α = 0.01 thus there are significant differences among the observed results. The speed in obtaining the global optimum is a salient yardstick for measuring the algorithm performance. From Table 30, FEIW-1 has the best performance among all IW strategies, in terms of average and minimum number of iterations. Also FEIW-5 Table 11. Comparison of average, minimum and standard deviation of error for considered PSO variants with condition 1, I max = 500 and D = 50.

IW
PEC has the highest rank of success rate. Similarly, Table 31 shows that FEIW-1 has the best rank among all IW strategies in terms of success rate, average and minimum number of iterations. Thus with condition 2, I max = 1000 and D = 10, Friedman test proves the advantage of FEIW-1 and FEIW-5 over other IW strategies in terms of convergence speed and solution precision. From Table 32, FEIW-6 and FEIW-1 have the best performance among all IW strategies, in terms of average and minimum error, respectively. Also Table 33 shows that FEIW-1 has the best rank in terms of average and minimum error. Thus with condition 1, I max = 1000 and D = 10, Friedman test proves that FEIW-6 and FEIW-1 are the best strategies for better accuracy. Under condition 1, I max = 500 and D = 50, from Table 34 one can observe that FEIW-1 and FEIW-6 have the highest performance since these strategies have minimum rank, in terms of average and minimum error, respectively. With condition 1, I max = 1000 and D = 50, from Table 35 one can conclude that FEIW-1 is the best IW strategy in both average and minimum error test. Therefore, FEPSO significantly outperforms CIWPSO, RIWPSO, LDIWPSO, CHIWPSO, GLBIWPSO, AIWPSO, NEIWPSO and EDIWPSO in terms of solution quality and convergence rate using the Friedman test.  [40] to detect significant differences for the considered variants. The Bonferroni-Dunn test is used to compare an IW strategy with all the other strategies. The performance of two strategies is significantly different if the corresponding mean ranks differ by at least the critical difference (CD): where N i and N f are number of IW strategies and benchmark functions, respectively. Also critical values q α at the probability level α is given in [35] as follows q 0:05 ¼ 2:773 ; q 0:1 ¼ 2:539 ð42Þ    Table 15. Best and worst IW strategies for each benchmark function in terms of success rate, average and minimum number of iterations of successful runs according to Table 5.

PEC
Case The difference among mean ranking of PSO variants is illustrated by Bonferroni-Dunn's graph in Figs 3-5. In Bonferroni-Dunn's graph, we have drawn a horizontal star-line which represents the threshold for the best performing algorithm (the one with the lowest ranking bar in minimization problems) for a better comparison of variants. A line is drawn for each Table 16. Best and worst IW strategies for each benchmark function in terms of success rate, average and minimum number of iterations of successful runs according to Table 6.

PEC
Case  Table 18. Best and worst IW strategies for each benchmark function in terms of average, minimum and standard deviation of error according to Tables 7 and 8.

PEC
Case level of significance considered in this study, at a height equal to the sum of minimum ranking and the corresponding CD computed by the Bonferroni-Dunn method. The bars exceeded these lines are associated to an algorithm having worst performance. In Fig 3, Bonferroni-Dunn bar charts for average and minimum iterations prove that FEIW-1 has the best speed in Table 22. Best and worst IW strategies for each benchmark function in terms of average, minimum and standard deviation of error according to Tables 13 and 14.

PEC
Case    Table 25. Wilcoxon-ranks and p-value on the average and minimum number of iterations of successful runs according to Table 6.

Mode Average Iterations Minimum Iterations
For average error criteria, CIW, RIW, LDIW, CHIW, GLBIW, AIW, FEIW-4 and FEIW-5 emerge as worst performers and FEIW-1 and FEIW-6 as best performers; For minimum error criteria, CIW, RIW, LDIW, CHIW, GLBIW, AIW, FEIW-3 and FEIW-4 come as worst performers and FEIW-1 and FEIW-6 as best performers. For standard deviation criteria, CIW, RIW, LDIW, GLBIW, AIW, FEIW-3, FEIW-4 and FEIW-5 emerge as worst performers and FEIW-1 and FEIW-2 as best performers. Therefore, in general manner, Bonferroni-Dunn bar charts show that FEIW-1 strategy has the best performance among all considered strategies.   6.3.4 Boxplot. In addition to using statistical tests to observe the performance of considered PSO variants, boxplot analysis is also performed for benchmark functions and shown in Figs 6-8. In Fig 6, boxplots of average and minimum iterations show that medians of FEIW-1, FEIW-5 and FEIW-6 are smaller than others. Thus these boxplots show that FEPSO is faster than CIWPSO, RIWPSO, LDIWPSO, CHIWPSO, GLBIWPSO, AIWPSO, NEIWPSO and  Table 6. (E) Success rate based on Table 5. (F) Success rate based on Table 6. EDIWPSO. The results of boxplots of average and minimum error in Figs 7 and 8, indicate the superiority of FEIW-1, FEIW-5 and FEIW-6 strategies over other approaches in terms of accuracy. These boxplots prove that FEIW strategy is a reliable IW and has better performance than other considered IW strategies.

Convergence graph
The convergence graph for FEIW-1, FEIW-3, FEIW-5 and FEIW-6 is demonstrated in Fig 9. The termination criterion for these graphs is condition 2, where D = 10 and I max = 30000. From convergence graph, we can discover that the convergence rate of the mentioned IW strategies is clearly faster than the other strategies on the benchmark functions. At the same time, the best solution get by FEPSO is more optimum than by CIWPSO, RIWPSO, LDIWPSO, CHIWPSO, GLBIWPSO, AIWPSO, NEIWPSO and EDIWPSO.

Conclusion
There are many modifications have been done to the standard PSO algorithm. Some of modifications to the basic PSO are directed towards introducing new strategies of inertia weight  Table 6. (E) Success rate based on Table 5. (F) Success rate based on Table 6. which tuned based on trial and error. Suitable selection of the inertia weight provides a balance between global and local searching. This paper proposed a new flexible exponential time-varying inertia weight (FEIW) strategy to improve the performance of PSO. The algorithm named as FEPSO is proposed based on FEIW strategy. We confirmed the FEPSO's validity in terms of convergence speed and solution precision by testing it with a suit of well-known standard benchmark unimodal and multimodal functions and by comparing obtained results with eight inertia weight strategies of the best time-varying, adaptive and primitive inertia weight strategies. The comparisons are made in terms of convergence speed and solution accuracy and the results are tabulated and graphs are plotted for dimensions 10 and 50 separately. Statistical tests show that this novel strategy converges faster than others during the early stage of the search process and provide better results for problems. Thus experimental results clearly prove the superiority of the proposed model over other inertia weight models. The future work includes the implementation of the FEPSO to solve a real world problem with lots of complexity such as brain MR image segmentation to compare the efficiency of the FEPSO with other recent optimization techniques.