A Differential Evolution Algorithm Based on Nikaido-Isoda Function for Solving Nash Equilibrium in Nonlinear Continuous Games

A differential evolution algorithm for solving Nash equilibrium in nonlinear continuous games is presented in this paper, called NIDE (Nikaido-Isoda differential evolution). At each generation, parent and child strategy profiles are compared one by one pairwisely, adapting Nikaido-Isoda function as fitness function. In practice, the NE of nonlinear game model with cubic cost function and quadratic demand function is solved, and this method could also be applied to non-concave payoff functions. Moreover, the NIDE is compared with the existing Nash Domination Evolutionary Multiplayer Optimization (NDEMO), the result showed that NIDE was significantly better than NDEMO with less iterations and shorter running time. These numerical examples suggested that the NIDE method is potentially useful.


Introduction
Nash equilibrium(NE) is one of the most important concepts in game theory, since it reveals the intrinsic link between the game equilibrium and economic equilibrium. NE is widely used in practical applications, therefore, it is a key issue to find NE. However, infinite strategies and complex payoff functions (such as non-concave, non-differentiable, multi-modal, etc.)make it difficult to solve NE. In fact, the difficulty of solving the NE has greatly limited its application in economics, management and sociology.
Economists mainly studied continuous games, in which there are infinite strategies and the payoff functions are also continuous. As we all know, the purpose of game analysis is to predicate a result of a game, which is NE. At present, the commonly used methods for locating NE of a continuous game are the following.
(1) Analytical method is the most straightforward approach, which sets the partial derivatives of every player to zero [1], thus we can obtain the optimum strategy for one player if the other players do not change their strategies. This is also known as player's best-response function. Solving all the best-response function can obtain the optimal strategy of each player, that is NE [2,3]. The analytical method is useful for providing insights into the behavior of players, however, it is difficult to apply this method to deal with multi-player or non-differential payoff function [4,5].
(2) Mathematical programming is the most widely used method in detecting NE, which is convenient to add certain constraint to the model, and easy to extend to a big system. The mathematical programming method can be divided into the following categories: sensitivity information based [6]; non-interior point algorithm [7]; nonlinear complementarity approach [8], etc. Furthermore, a variational inequality approach [9] has also been applied in Cournot model. However, the mathematical programming method is heavily dependent on the choice of the initial point. If we set an improper initial point, the solution may only be a local optimal solution.
(3) Matrix game is a method which disperses the strategy space, therefore, each player's strategy space is composed of several typical strategies [10]. In addition, the matrix game is also used in the mixed NE [11].
(4) Evolutionary algorithm is a stochastic optimization method, which is not strictly limited to the objective function and can also handling non-differentiable, nonlinear and other complex payoff functions. Son [12] proposed an improved co-evolutionary algorithm-hybrid coevolutionary programming to find a global optimal NE, which overcame the local optimal shortcoming and improved the convergence of the co-evolutionary algorithm. There are also other optimization algorithms such as invasive weed optimization [13]; cooperative co-evolutionary invasive weed optimization [14]; PSO [15], etc.
In many economic models, there are some special functions to convert NE into a optimization problem, such as Nikaido-Isoda function [16] and Lyapunov function [17]. Moreover, equilibrium also can be described as solution of polynomial equations [18].
When building a game model, economists generally assume that the demand curves are monotonically decreasing functions, and the cost functions are constant or represented by a quadratic functions. The advantage of this assumption is to guarantee a unique NE, which is convenient to find NE. In real economic activity, however, the nonlinear demand functions and cost functions made the model more consistent with the real market competition, this will inevitably lead to the payoff functions become extremely complex, thus making the NE difficult to be solved.
A differential evolution algorithm(DE) for solving Nash equilibrium in nonlinear continuous games is presented in this paper, called NIDE. In each generation, parent and child strategy profiles are compared one by one pairwisely with Nikaido-Isoda as fitness function. In applied research, the NE of nonlinear game model with cubic cost function and quadratic demand function(even some other non-concave payoff functions) are solved. Compared with the existing Nash Domination Evolutionary Multiplayer Optimization (NDEMO), the results suggested that the NIDE was significantly better than NDEMO with less iterations and shorter runtime for solving the NE problems. Meanwhile, it illustrated that the NIDE has high efficiency and wide applicability.
The organization of this paper is as following. Section 2 explains the basic concepts of game theory and provides a searching algorithm based on DE. Section 3 presents numerical examples and compared with NDEMO, and finally conclusions are drawn afterwards

Prerequisites
The normal-form game usually consists of three parts: N = {1,Á Á Á,n} denotes the set of players, n is the number of players; S i ¼ fS i1 ; S i2 ; Á Á Á ; S im i g denotes the strategies set of player i; u i : S ! R denotes the payoff function for each player i 2 N, where S = S 1 × S 2 × Á Á Á × S n is a Cartesian product and represents all possible situation of the game, and R is set of real number.
Given a particular topology for a set S i , there is corresponding direct product topology for the set S = S 1 × S 2 × Á Á Á × S n . For example, if S i is a compact metric space, then S is also a compact metric space. As players' strategies can be quantified, so the pure strategy of each player S i can be regarded as a bounded and closed convex subset in Euclidean space. If the number of strategies S i for player i is infinite, it is called "Infinite game". Definition 1: The infinite game G = {s 1 , Á Á Á, s n ; μ 1 , Á Á Á, μ n } is called continuous game, if given a certain topology for a set S, and the payoff function u i : S ! R is a continuous function for player i 2 N.
In an N-player game, when player i chooses his pure strategy s i 2 S i given a pure strategy set of others s −i = (s 1 , Á Á Á, s i−1 ,s i+1 Á Á Á, s n ) 2 s −i , he receives a profit of u i (s i ,s −i ). In order to study the equilibrium of continuous game, we need introduce the concept of NE.
Definition 2: In the normal-form game G = {s 1 , Á Á Á, s n ; μ 1 , Á Á Á, μ n }, a combined strategy pro- NE is a point where no player can obtain a higher profit by unilateral movement. That is, a profit-maximizing player will not deviate from NE. In fact, if a strategy profile s 0 ¼ ðs n Þ is not NE, that means there must exist a player i who can obtain a higher profit by choosing another strategy s } i when others choose s Accordingly, the strategy s 0 i is called dominated strategy. From the definition 2, if s Ã i 2 S is a dominant strategy for player i, the outcome resulting from the strategy s Ã i is better than the outcome resulting from any strategy s 0 2 S i ðs 0 6 ¼ s Ã i Þ, regardless of what the other players are doing.
Sometimes, a strategy to be a dominant strategy is only relative to the specific strategy, thus we have the following definition.
Definition 4: In a game, a player may have two strategies s 0 i and s } i , then strategy s } i is said to relatively dominant strategy s 0 i , than is It means that there may exist another better strategy than s } i . Next, we will introduce the concept of Nikaido-Isoda function [19], which plays an important role in the study of NE.
Definition 5: Let u i be the payoff function of player i, then the Nikaido-Isoda function ψ: Each term of the series in the Nikaido-Isoda function represents the change of one player's gains from strategy x i to y i , while the strategies of the others are held fixed. Nikaido-Isoda function represents the sum of the change in gains or losses of all players.
Based on the definition 4 and definition 5, given two strategy profiles fx; yg 2 S; ðx ð x 1 ; Á Á Á ; x n Þ; y ð y 1 ; Á Á Á ; y n ÞÞ; if u i (y i |x) > u i (x) always hold for each player i 2 N, we say the strategy profile y is dominant to the strategy profile x. at this point, we have the following proposition hold. Proposition 1: If the strategy profile y is dominant to the strategy profile x, then cðy; xÞ < cðx; yÞ: Prove: If the strategy profile y is dominant to the strategy profile x, then Similarly, Therefore, ψ(y,x) < ψ(x,y) holds. If x Ã is NE, for any y 2 S, and y 6 ¼ x Ã , then ψ(x Ã ,y) < ψ(y,x Ã ). Therefore the Nikaido-Isoda function plays the role of a "merit function" measuring the proximity of a strategy to NE.

NIDE algorithm
In 1995, Storn and Price [20][21][22] proposed a advanced evolutionary algorithm, called differential evolutionary(DE), which was a relatively new stochastic optimization method, and has been shown to be a simple and effective algorithm for the problem of continuous global optimization [21]. The DE requires few control variables, is robust, easy to use and lends itself very well to parallel computation.
There are three important variables in DE: Mutation Factor F, Probability of Crossover CR and Population size NP. Mutation Factor F is positive real number which controls the amplification of the differential variation. Experience has shown that, it is best to choose F between 0.4 and 1.0. In addition, a crossover operation is performed, using the parameter CR, the value of CR is smaller, the robustness of DE will be better. Generally, CR 2 [0,1]. If the population size NP is larger, the search ability of DE will be intensified, and the calculated amount of DE will be increased. According to literature [21], NP usually to be chosen between 5n-10n, where n is the number of variables.
Based on Proposition 1, by redefinition of selection criteria used in evolutionary methods with the Nikaido-Isoda function, we propose a differential evolution(NIDE) to find the NE. At each generation, parent and child strategy profiles are compared one by one pairwise, using Nikaido-Isoda function to play for the part of fitness function. The whole process can be summarized into the following steps(Algorithm1). Algorithm1: A differential evolution based on Nikaido-Isoda function (NIDE) 1: Input: the maximum number of iterations Max it , the population size NP, Mutation Factor F, Probability of Crossover CR, the convergence tolerance ε 2: it 1 3: Randomly initialize populations of NP parent strategy profiles P 4: Evaluate fitness to players with P 5: while it < Max it do 6: for i = 1 to NP do 7: x P it i 8: Use Algorithm 2 to create child strategy profiles y 9: if ψ(y,x) < ψ(x,y) Child strategy profiles y are created by applying the DE operators via the stochastic combination of randomly chosen parents as discussed in Storn(2003) [23]. Through tested on five different forms of DE operators(DE/rand/1/bin, DE/best/1/bin, DE/rand/2/bin, DE/best/2/bin, DE/rand-to-best/1/bin), we found the operator "DE/rand/1/bin" possessed faster convergence. In order to show the performance of NIDE, we use the same algorithm in literature Koh(2012) [24] to create child strategy profiles, just describe by the following(Algorithm 2). Randomly choose 3 integers: r 1 , r 2 , r 3 between 1 and NP 5: r 1 6 ¼ j, r 2 6 ¼ j and r 3 6 ¼ r 2 6 ¼ r 1 6 ¼ j 6: x P j 7: a P r 1 8: b P r 2 9: c P r 3 10: Mutation: Produce a mutant vector z via a stochastic combination of donor vectors 11: z a + F(b−c) 12 In order to check convergence to the NE, randomly select a individual s and compute the Euclidean norm between s and every member of P(lines 13 to 14 of Algorithm1). If the maximum distance is less than ε, the population is judged to have converged to a NE and the algorithm can terminated. Otherwise the process is repeated.

Results and Discussion
In order to test the efficiency of the NIDE, we make a comparison between NIDE and NDEMO [24]. Table 1 gives the parameters used for the numerical experiments just the same in Koh(2012) [24]. All numerical experiments were conducted using Matlab R2013a running on a 32 bit Windows 7 machine with 2 GB of RAM.

Oligopolistic game model with cubic cost function
When building a game model, Economists generally assume that the demand curves are monotonically decreasing functions, and the cost functions are constant or represented by a quadratic functions. The advantage of this assumption is there exists a unique NE, and it is convenient to finding the NE. In real economic activity, however, the nonlinear demand functions and cost functions make the model more consistent with the real market competition, this will inevitably lead to the payoff functions become extremely complex, thus making the NE difficult to be solved.
In microeconomics, however, the total cost function is always non-negative in its definitional domain and decreases to zero on the left side of the inflection point, then it gradually increases while on the right side of the inflection point [25]. The literature [25] pointed out the total cost curve is the analogy of cubic function, so we employ There are four forms of cubic function. d > 0 and Δ 0; d < 0 and Δ 0; if d < 0 and Δ > 0; d < 0 and Δ > 0 and Δ = 4c 2 − 12bd.
The total cost function is a cubic function as a > 0, d > 0, Δ 0, and the inflection point (À c 3d , f À c 3d À Á ) falls in the first quadrant, that is, 8 > > > > > > > < > > > > > > > : Case 1: Example 1 with two players. Here we first consider the situation with two players i, i = 1,2. Suppose that the output of player i is q i , each player maximizes individual profits from the sale of the homogeneous good. The total outputs are and the inverse demand function [26] is The total cost curve is Hence the profit function of player i is given by We calculate the first order of the profit, i.e.
Based on Eq (9), we can find out the player's response function, It is difficult to obtain the analytical solutions in Eq (10), so we assign a value to each parameter, then use NIDE to detect the NE. Set m ¼ 5; n ¼ 1; where a i , b i , c i , d i submit to the conditions (7). The NIDE identified the NE(0.7563, 0.7697) successfully. In order to compare the iterations and runtime, the NIDE and NDEMO were both runned 10 times, the results were showed in (Fig 1 and Fig 2). The compare results produced by the NIDE and NDEMO were given in Table 2.
Case 2: Example 1 with three players. For a more complex case, we consider the three player model. We use the same forms of cost and demand function as in Case 1. Three players maximize their profit by adjusting their strategic variable q 1 , q 2 and q 3 . The profit functions of each player are analogous to those in (8). where a i , b i , c i , d i submit to the conditions (7). The NIDE could successfully identify the NE(0.5450, 0.5581, 0.5712) (Fig 3 and Fig 4)as well as Table 3 also show the comparative results of iterations and runtime.
Next, in order to show the high efficiency and wide applicability of NIDE, we introduce two more complex models.

A game model with local NE
This example is taken from [12]. Suppose that there are two players in a market, and the nonconcave profit functions for players are defined by where pi = 3.1415, and the strategic choice of the two players between 0 and 7.5. We calculate the first order of the profit, i.e.
Through (Fig 5), we know that the Eq (12) have many solutions(the points of intersection of Fig 5) in the domain [0, 7.5], and it is difficult to differentiate real NE and"local NE traps" use the conventional NE search algorithms. The NIDE overcomes the local optima and detects the NE which agrees with previous results. The comparative results of iterations and runtime are shown in (Fig 6 and Fig 7) and Table 4.
Electricity market with two players. Electricity market is a classical linear Cournot model [12]. For sample, we assume there are only two players, and they submit bids to generate quantities s1 and s2, respectively.
In order to analyze the strategy of each player, we define each player's profit by where λ = [2750-4(s 1 +s 2 )]/75, cost functions C1 = 0.01s 1 2 +10 s1, and C2 = 0.01s 2 2 +10 s2.    Table 7 reports results obtained from applying NIDE and NDEMO to this game. From Table 7, NIDE's average running time is 0.2s with only 66 generations of evolution, while NDEMO took 0.5s with 122 generations. It proved the efficiency of NIDE algorithm is still superior to NDEMO in classical liner demand function model.

Conclusions
In this paper, we proposed an evolutionary algorithm for solving NE in nonlinear continuous games. The algorithms of NIDE enabled us to handle the NE problems of nonlinear game model with the cubic cost function and the quadratic demand function, and even other nonconcave payoff functions. In order to assess the performance of NIDE, we compared the NIDE with the existing differential evolution algorithm(NDEMO), comparison of the results showed that the NIDE was significantly better than NDEMO with less iterations and runtime for solving the NE problems. It is more efficient in locating NE in practice and will not result in local optimal.
Meanwhile, through these numerical examples, suggesting that the NIDE could be a potentially useful method.