Robust network data envelopment analysis approach to evaluate the efficiency of regional electricity power networks under uncertainty

In this paper, new Network Data Envelopment Analysis (NDEA) models are developed to evaluate the efficiency of regional electricity power networks. The primary objective of this paper is to consider perturbation in data and develop new NDEA models based on the adaptation of robust optimization methodology. Furthermore, in this paper, the efficiency of the entire networks of electricity power, involving generation, transmission and distribution stages is measured. While DEA has been widely used to evaluate the efficiency of the components of electricity power networks during the past two decades, there is no study to evaluate the efficiency of the electricity power networks as a whole. The proposed models are applied to evaluate the efficiency of 16 regional electricity power networks in Iran and the effect of data uncertainty is also investigated. The results are compared with the traditional network DEA and parametric SFA methods. Validity and verification of the proposed models are also investigated. The preliminary results indicate that the proposed models were more reliable than the traditional Network DEA model.


Introduction
Efficiency evaluation of the electricity power networks is an important task for managers for better understanding the past accomplishments of the network, which helps them plan for its future development. In fact, understanding the efficiency level of homogenous electricity power networks may help decision makers and regulators adopt better management strategies and make realistic policies.
Several methods have been developed to evaluate the efficiency of decision making units (DMUs). These methods can be generally classified as parametric and non-parametric methods. While in the parametric methods a cost or production function is estimated, the advantage of non-parametric methods is that it is not necessary to estimate the cost or production function.
Data envelopment analysis (DEA) model, developed by Charnes et al. [1] is a non-parametric methodology which has been widely recognized as an effective technique for measuring and evaluating the relative efficiency of a set of decision making units in the presence of multiple inputs and outputs. DEA applies an efficient frontier made up of the most efficient decision making units to measure the relative efficiency of different decision making units. The basic idea behind DEA method is application of mathematical optimization for evaluating relative efficiency of DMUs with multiple inputs and outputs.
In traditional DEA models, internal structure of DMUs is ignored and each DMU is considered as a black box. However, there are studies that show this assumption may be misleading [2,3]. In literature, DEA models that consider the internal structure of DMU's are called the Network DEA models.
In many practical applications, data are subject to uncertainty. According to [4], sometimes even small data perturbation yields suboptimal or infeasible solutions. It means that considering certain data in DEA, may lead to misleading efficiency results. Since, a small change in data may change the classification of decision making units (DMUs) from an efficient to an inefficient status and vice versa [5].
In this paper, Robust Network DEA models are developed to evaluate the efficiency of some electricity power networks with uncertain data. Robust optimization is a relatively new methodology for dealing with data uncertainty. While there are many methods to deal with uncertainty in data, an important motivation for applying robust optimization method is that in many applications the probability distribution of the data in unknown. Furthermore, in some applications infeasibility is not allowed at all. The proposed models are based on the adaptation of robust optimization methods developed by Ben-Tal and Nemirovski [4] and Bertsimas et al. [6]. The proposed robust optimization techniques have not been applied with Network DEA in any other previous studies.
This paper is organized into seven sections. Literature review is presented in section 2. Mathematical details of the network DEA model and proposed Robust Network DEA (RNDEA) models are illustrated in sections 3 and 4, respectively. The proposed models are based on Ben-Tal and Nemirovski (BN) and Bertsimas et al. (BA) approaches. Section 5 represents the stochastic frontier analysis (SFA) method. A case study of 16 regional electricity power networks is presented in section 6. Moreover, in this section the results of the implementation of Network DEA, Robust Network DEA based on BN and BA approaches and SFA method are illustrated and compared. Finally, the conclusions are summarized in Section 7.

Literature review
Mathematical optimization is a branch of applied mathematics that generally tries to optimize a real objective function by selecting best values for decision variables. Mathematical optimization techniques can be applied to improve electricity power systems in different ways (e.g., [7,8]). Hu et al. [9] applied a mathematical optimization framework to optimize plug-in hybrid electric vehicles. Hu et al. [10] developed a multicriteria optimization approach for evaluating the optimal tradeoffs between the fuel-cell durability and hydrogen economy in the fuel-cell hybrid bus. DEA transforms the problem of efficiency evaluation to a relatively simple linear programming optimization model, in which, the value of objective function is the relative efficiency of under consideration DMU.
In the previous studies, DEA was applied for measuring the relative efficiency of the electricity utilities. Edvardsen and Førsund [11] studied a sample of large electricity distribution utilities from Norway, Sweden, Denmark, Finland and The Netherlands for the year 1997, using input-oriented DEA and Malmquist productivity index. They found that electricity distributors of Finland maintained the highest productivity compared with other countries. Estache et al. [12] used DEA and stochastic frontier analysis to evaluate the efficiency of the main electricity distribution companies in South America. They found a low correlation between DEA and stochastic frontier analysis. Giannakis et al. [13] applied DEA to study service quality of electricity distribution utilities in UK. They found that cost-efficient companies had not necessarily shown high service quality. They also concluded that integrating service quality in regulatory benchmarking was preferable to cost-only approaches. Ramos-Real et al. [14] estimated changes in the productivity of the 18 Brazilian electricity distribution companies using DEA. They found that the incentives generated in the reform process would not have led the companies to behave in a more efficient manner. Sadjadi and Omrani [15] developed a new DEA method with the consideration of uncertainty in the outputs to evaluate the efficiency of 38 electricity distribution companies in Iran. Tavana et al. [16] proposed a DEA model to investigate the impact of IT investment on productivity of 20 Iranian power plants. Sözen et al. [17] applied CRS and VRS models to evaluate the efficiency of power plants in Turkey with respect to the cost of electricity generation and the environmental effects. Furthermore, they investigated the relationship between efficiency scores and input/output factors. Vazhayil and Balasubramanian [18] grouped Indian electricity sector strategies into three portfolios and employed deterministic and stochastic DEA models for efficiency optimization of electricity sector strategies. Their analysis showed that weight-restricted stochastic DEA model was more appropriate than deterministic method.
Existing DEA approaches for evaluating the efficiency of electricity utilities are under some serious criticisms: In large body of literature, only one stage (component) of electricity network is evaluated. For example power plants [16,17] or electricity distribution firms [11][12][13][14][15]. Furthermore, in most of the existing approaches, data uncertainty is ignored. Data uncertainty is considered in robust DEA model proposed by Sadjadi and Omrani [15] to evaluate the efficiency of stage distribution, but in this study only outputs were considered to be uncertain and uncertainty in inputs was ignored. In this paper, network DEA is applied for evaluating the efficiency of entire electricity networks and robust optimization methodology is used for dealing with data uncertainty.
The idea of network DEA was first developed by Charnes et al. [19]. They discussed two processes of army recruitment: a) awareness creation through advertising and b) contract creation. Since then, several studies have been accomplished to measure the efficiency of systems taking into account the internal structures. Färe et al. [20] introduced the basic network DEA models to investigate the efficiency of sub DMUs. Prieto and Zofío [21] developed a network DEA model to deal with different sub-technologies corresponding to alternative production processes, to evaluate the efficient resource allocation among them. They applied their model to a set of OECD countries (OECD -The Organization for Economic Co-operation and Development-is founded in 1960 to promote policies that will improve the economic and social well-being of people around the world. Today, 35 countries are members of OECD). Cook et al. [22] proposed several network DEA models to examine a more general problem of an open multistage process. Kao [23] developed a relational network DEA model by considering interrelationship of the sub-DMUs, to measure the efficiency of the whole system and sub-systems at the same time. They also introduced dummy processes to transform a complicated system to a series system. The model evaluated both overall efficiency and divisional efficiency. Saranga and Moser [24] applied an external assessment survey methodology that complements the internal perceptional measures of purchasing, supplied management (PSM) performance and developed an efficiency measurement framework using the classical and two-stage value chain data envelopment analysis models. Yu et al. [25] designed information-sharing scenarios to analyze the efficiency of supply chain through a simulation model. They applied a cross-efficiency DEA approach to deal with both desirable and undesirable measures. Chen and Yan [26] took the perspective of organization mechanism to deal with the complex interactions in supply chain. Accordingly, they introduced three network DEA models, under the concepts of centralized, decentralized and mix organization mechanisms. A comprehensive review of the network DEA models was accomplished by Kao [27]. He classified different network DEA models with regard to both the models developed and structures examined.
In DEA literature, stochastic approach [18], interval model [28,29] and fuzzy method [30,31] are also applied to model data uncertainty. One of drawbacks of stochastic approach is that the decision maker is required to assume a distribution function for the error process [32]. However this assumption may not be realistic because it is very difficult to choose one distribution over another. The interval approach was first proposed by Cooper et al. [33]. One of difficulties of this approach is difficulty in the evaluation of the upper and lower bounds of the relative efficiencies of the DMUs. The fuzzy DEA was first proposed by Sengupta [32]. In some cases the complexity of fuzzy approach can grow exponentially. Pitfalls of some fuzzy DEA models are addressed by Soleimani-Damaneh et al. [34]. Because of these drawbacks of existing methods for dealing with data uncertainty, robust optimization method is applied in this paper. Robust optimization was originally presented by Soyster [35]. El-Ghaoui and Lebret [36] and El Ghaoui et al. [37] and Ben-Tal and Nemirovski [4,38,39] presented a new idea for dealing with the data uncertainty based on ellipsoidal uncertainty sets. Recently, Bertsimas and Sim [40][41][42] and Bertsimas et al. [6] introduced a robust optimization approach based on polyhedral uncertainty set. Robust optimization theory has been applied in many practical applications. Such examples include project management (e.g., [43][44][45]), inventory management (e.g., [46,47]), portfolio optimization (e.g., [48][49][50]), environmental management (e.g., [51,52]). Comprehensive information about the history and the growth of the robust optimization can be found at [53] and [54].

Network data envelopment analysis
To evaluate the efficiency of electricity power networks, it is not adequate to only consider the initial inputs and final outputs of networks and ignore the internal linking activities among different stages, because ignoring the operations of components may lead us to misleading results. More significantly, a network may be efficient while all components are not [55].
Consider the D-stage process pictured in Fig 1. The input vector of each stage is denoted by X d (d = 1, . . ., D). The outputs of stage d (d = 1, . . ., D) take two forms, Y d and Z d . Y d represents outputs leaving the process at stage d and Z d represents the outputs of stage d that becomes inputs to the next stage, d+1. These types of outputs are intermediate measures.
Linear programming form of the network DEA model was proposed by Cook et al. [22] to evaluate the efficiency of the general serial structures presented in model (1). In this model the Model (1) is an output-oriented network DEA model. A DEA model is output-oriented if it maximizes outputs without increasing inputs. In this model, the objective function is maximizing the summation of outputs of different stages of DMU O. First constraint keeps the level of inputs at a constant level. Constraint 2 and constraint set 3 state that the aggregated system output must be less than or equal tothe aggregated system input for all DMUs. Constraint 2 refers to stage 1 and constraint set 3 refers to other stages.

Robust optimization
Robust optimization provides risk-averse models to deal with uncertainty in optimization problems. In traditional optimization methodologies, data are assumed to be known with certainty. In fact in traditional methods a small data uncertainty is ignored hoping that small data uncertainties would not have significant impact on optimality and feasibility of the solution, but as illustrated by Ben-Tal and Nemirovski [4], sometimes even a small data perturbation deserves suboptimal or infeasible solutions.
In order to present the robust methods proposed by Ben-Tal and Nemirovski [4] and Bertsimas et al. [6], consider the following linear optimization problem: Let J i be the set of uncertain coefficients in i th row of matrix A andã ij (j 2 J i ) be the true values of the parameters which are subject to uncertainty and take value in [a ij Àâ ij ; a ij þâ ij ], where a ij andâ ij are the nominal value and the constant perturbation of variableã ij , respectively.
Since the equality constraints are not allowed in robust optimization methods, model (1) is transformed into the following form, which can be solved easily using parametric programming method.
In model (3), the π O is overall efficiency of DMU O. The first constraint is transformation In this model, when we maximize π O , the output/input ratio of DMU O (efficiency of DMU O) will be calculated. Same as model (1), other constraints state that the aggregated system output must be less than or equal to the aggregated system input for all DMUs.

Robust network DEA based on BN approach
Ben-Tal and Nemirovski define the uncertain dataã ij as follows [4]: where z ij represent independent random variables which are symmetrically distributed between -1 and 1 and e>0 is the percentage of perturbations (uncertainty level). Robust form of model (2) based on BN approach is as follows: In model (5), robust form of general linear programming problems based on BN approach is presented. This model shows that BN approach adds a term to the left side of the constraints.
In this approach decision maker can control the reliability level by varying parameter O.
The proposed robust network DEA based on BN approach is expressed as follows. This model is a general form that considers inputs, outputs and intermediate measures as In model (6), π O is the efficiency of under consideration DMU and x dij , y drj and z dkj are i th input, r th output and k th intermediate measure of division d in j th DMU, respectively. Constraints of model (6), are robust form of constraints of model (3) based on BN approach (see model (5)). This model is in nonlinear form and can be solved using nonlinear programming techniques.
Unfortunately the BN approach suffers from the following disadvantages: • The approach increases the number of variables and makes the models more complicated.
• BN approach transfers linear programming models into nonlinear forms which are more difficult to obtain optimal solutions.

Robust network DEA based on BA approach
Let λ ij be the scaled deviation of parameterã ij from its nominal value as l ij ¼ ðã ij À a ij Þ=â ij . Clearly, η ij is unknown and symmetrically distributed in the interval [-1,1]. In addition, J i is the set of coefficients in constraint i which are uncertain. Moreover, the parameter Γ i is called budget of uncertainty and introduced for constraint i, to adjust level of protection against uncertainty. The summation of scaled variation of the parameters cannot exceed Γ i , i.e.,  (2) can be reformulated as follow: In model (7), the ith constraint is equivalent to minã i 2J iã 0 i x ! b i . As a result, for constraint i, the following auxiliary problem has to be solved: Dual form of Eq (8) is as follow: where q i and r ij are the dual variables. Substituting Eq (9) in Eq (7), the robust approach based on [40][41][42] and [6] is as follow: À y j x j y j 8j; The proposed robust network DEA based on BA approach is expressed in Eq (11).

Stochastic frontier analysis (SFA)
Stochastic Frontier Analysis (SFA), which was independently proposed by Aigner et al. [56] and Meeusen and Van den Broeck [57] is a statistical method based on the regression analysis for estimating the efficient frontier and efficiency scores [58]. Statistical nature of SFA allows for the inclusion of stochastic errors in the analysis. SFA decomposes the error term in two parts, one represents the statistical noise and another represents the inefficiency.
The stochastic frontier production function model presented by Aigner et al. [56] and Meeusen and Van den Broeck [57] is in following form: where x i , q i , and e i = v i − u i are input vector, output vector and error term for DMU i , respectively. β i is the vector of unknown parameters that should be estimated and v i is the symmetric error term and u i is asymmetric non-negative inefficiency term. Kumbhakar and Lovell [59] defined the relationship between technical efficiency and −u i as: TE i = exp(−u i ) where TE i is the technical efficiency of DMU i . In order to compute the technical efficiency of decision making units, the probability function for the distribution of the errors and distribution of inefficiencies is required. The probability distributions function for v i and u i are normally assumed as follows: where Nð0;s 2 v Þ and N þ ðm;s 2 u Þ in Eqs (13) and (14)represent that distribution functions of v i and u i are normal and half-normal [59]. Also, γ is defined as follow: In Eq (15), the parameter γ is the relative importance of inefficiency. This parameter must be between 0 and 1 and shows the percentage of error that the efficiency may have [59]. In models with single output, functions such as Cobb-Douglas are applied to estimate the efficiency but in multi-input and multi-output state, distance function is required. As in [60] and [61], for distance function the translog form is applied. Translog form of input distance function is as below: where i denotes i th decision making unit and M and K are the number of inputs and outputs and x Ã k ¼ x k =x K . Eq (16) may be more clearly expressed as ln( Where v i is the symmetric error term and u i is asymmetric nonnegative inefficiency term.

Case study
In this section, the proposed models are applied to evaluate the efficiency of 16 Iranian regional electricity power networks. Iranian regional electricity power networks consist of three stages: Generation, Transmission and Distribution. Each stage has inputs and outputs and specific energy is transmitted between stages. In generation stage, power plants consume gas oil and fuel oil (liquid fuel) and natural gas (gas fuel) to produce electricity power. Generation has two outputs: mean practical power and specific energy. Practical power is maximum power of generators considering environmental situation (temperature, humidity, etc.) and specific energy is total energy produced excluding the electrical energy consumed in power plants. Transmission stage includes stations, overhead lines, cables and other electrical equipment to transit energy from power plants. A station includes a series of electrical equipment e.g. transformers, circuit breakers, disconnectors, instrument devices, in/out-coming feeders etc. number of employees, capacity of transmission stations, length of transmission network and energy delivered from nearby networks are inputs of transmission stage. This stage delivers energy to nearby networks and distribution stage. Stage 3 is distribution stage. This stage consists of a series of medium and low voltage overhead lines and underground cables to distribute electricity energy in an area. Inputs of this stage are: number of employees, length of distribution network and transformers capacity. Transformer is a static electrical device that transfers energy by inductive coupling between its winding circuits. Number of customers and total energy sales are outputs of this stage.
The regional networks act under the supervision of TAVANIR(Iran power Generation, Transmission and Distribution Management Company). TAVANIR is responsible for managing regional electricity power networks and acts under the supervision of Ministry of Energy. Our data series involves the annual data on 16 regional networks observed in 2014. These data are retrieved from Iran Power Generation, Transmission and Distribution Management Company annual publications. Structure of regional electricity power networks and considered inputs, outputs and intermediate measures for each stage is illustrated in Fig 2. Summary statistics over data set of case study is shown in Table 1 (to obtain full data see S1 File). Table 2 reveals the results of applying Network DEA and Robust Network DEA models for 16 regional electricity power networks (DMUs). As shown in column four of this table, in term of technical efficiency for Network DEA in 2014, five regional networks obtained efficiency score equal to one. These networks can be considered as reference set to the others. Other networks obtained efficiency scores between 0.803 (Kerman) and 0.999 (Tehran, Zanjan and Gilan).  Table 2. The perturbation e is considered to be 0.01, 0.05 and 0.1. For example when e = 0.05, the efficiency scores  Fig 3. In this case as perturbation increases from 0.01 to 0.1, the mean of efficiency measures is decreased from 0.954 to 0.898. In fact, the efficiency score of each network decreases when perturbation increases. For the implementation of Robust Network DEA based on BA approach (RNDEA-BA) it is sufficient to choose Γ at least equal to Eq (17):

Proposed network DEA models results
Where F is cumulative distribution of the standard Gaussian variable, e is perturbation and n is the sources of uncertainty. When the problem has few number of uncertain data, it is necessary to ensure full protection [41].  Table 2 (columns 9-11) show the results of RNDEA-BA approach. In this case, as perturbation increases from 0.01 to 0.1, the mean of efficiency measures is decreased from 0.924 to 0.670 and for each DMU, when the perturbation increases, the efficiency score decreases.
The process of implementing proposed framework is as follows: first managers collect data from 16 regional electricity power networks. Then, according to their knowledge about Robust NDEA approach to evaluate the efficiency of regional electricity power networks under uncertainty DMUs, they determine that data from which DMUs are subject to uncertainty. After that, they decide which method (RNDEA-BA or RNDEA-BN) should be applied (for DMUs with certain data they use conventional network DEA method). Then, for each DMU with uncertain data, they determine the uncertainty level. Finally, they use Table 2, or Figs 3 or 4, to measure the efficiency of DMUs and rank them.
Choosing between applying RNDEA-BN or RNDEA-BA approaches is a managerial decision and we cannot determine which one is more accurate. Note that if decision makers decide to use RNDEA-BN (or RNDEA-BA) approach, they should use it for all DMUs with uncertain data, i.e., it is not correct to use BN approach for some DMUs with uncertain data and use BA approach for other DMUs with uncertain data.

SFA results
The SFA is applied as an alternative method to measure the efficiency of 16 electricity distribution networks. As our case is in multi-input and multi-output state, a translog distance function is used to estimate the parameters of SFA. The SFA method is implemented by using the FRONTIER 4.1 [62] to measure the efficiency of generation, transmission and distribution stages and then the efficiency of each network is average of efficiency scores in stages. The parameter γ for generation, transmission and distribution stages is 0.78, 0.99 and 0.82 respectively. The results of SFA method for 2014 are presented in third column of Table 2. As shown in this table, networks obtained efficiency scores between 0.776 and 0.996 and Fars and Azarbayejan are the least and the most efficient networks. The average of efficiency scores for 16 networks in 2014 is 0.927 and the standard deviation is 0.060.

Comparison between BN and BA approaches
The results of the Network DEA, BA and BN approaches are compared in Fig 5. The figure shows that efficiency scores in Robust Network DEA models are less that Network DEA. Also, efficiency scores of DMUs in BN method are less that BA method. While BN approach changes the class of problems, the BA approach preserves the class of problems, e.g., in BA approach the robust form of a linear programming model remains in linear programming form. However BN approach changes a linear programming model to a nonlinear one. Hence, if the number of constraints and variables increase, the BA approach is better than BN. In this study proposed robust network DEA models are in nonlinear form but the model based on BN approach is more complicated to solve.
Another indicators to compare BA and BN approaches, are the number of constrains and variables. Assume that there are k coefficients for the matrix A with m × n dimensions which are subject to uncertainty. Given that the original model has n variables and m constraints, BN approach has m+2k constraints and n+2k variables where k is the number of uncertain data. The BA approach has m+k+n constraints and n+k+1 variables [6]. Therefore, BA has fewer variables than BN and when k > n the number of constraints in BA approach is less that BN.
Comparison between SFA and robust network DEA approaches  Robust NDEA approach to evaluate the efficiency of regional electricity power networks under uncertainty

Validation
To verify the results of proposed models, the Pearson test of correlation (ρ) and the Spearman test of correlation (r s ) are employed. Correlation coefficient describes both the strength and the direction of the relationship between two variables and can range in value from −1 to +1, where 1 is total positive correlation, −1 is total negative correlation and 0 is no correlation. Pearson correlation coefficient is a measure of the linear correlation between two variables. The Spearman correlation coefficient is often used to evaluate relationship between ranked variables rather than the raw data. In this study, the Pearson test is applied to compare proposed models with network DEA model and Spearman test is employed to compare proposed models with results of SFA method. Table 3 represents the Pearson and Spearman test between proposed models and NDEA and SFA models.
The Pearson test statistics is 0.871 and 0.923 for RNDEA-BN and RNDEA-BA approaches, respectively. The result indicates a strong direct relationship between NDEA and proposed models based on BN and BA approaches which results in the rejection of H 0 at 1% level. To  Robust NDEA approach to evaluate the efficiency of regional electricity power networks under uncertainty measure the Spearman correlation coefficient first regional electricity power networks should be ranked based on their efficiency scores. The r s for RNDEA-BN and RNDEA-BA approaches is 0.576 and 0.554, respectively that shows a relatively strong direct relationship between SFA and proposed models. The results are significant at the 5% level.

Conclusion
In real-world problems, sometimes data are imprecise or vague. This study has conducted a framework to evaluate the efficiency of DMUs with network structure under uncertainty when the distribution of uncertain parameters is unknown. The proposed approach is based on the recently developed robust optimization approaches presented by Ben-Tal and Nemirovski [4] and Bertsimas et al. [6]. In this paper robust network DEA models were developed which can handle uncertainty of inputs, outputs and intermediate measures. While in literature the efficiency of components of electricity power networks was evaluated, in this paper the efficiency of the whole network was measured. Developed models were verified and validated by Pearson and Spearman correlation techniques. Robust network DEA models developed in this paper was applied to evaluate the efficiency of 16 regional electricity power networks. The Pearson test was applied to compare the proposed models with network DEA model and the Spearman test was employed to compare the proposed models with the results of the SFA method. The results of Pearson and Spearman tests were significant at 1% level 5% level, respectively; hence, there was a direct relationship between proposed models and network DEA and SFA methods.
The results show that the Robust Network DEA models are more reliable than Network DEA model.