Solving multi-objective optimization problems in conservation with the reference point method

Managing the biodiversity extinction crisis requires wise decision-making processes able to account for the limited resources available. In most decision problems in conservation biology, several conflicting objectives have to be taken into account. Most methods used in conservation either provide suboptimal solutions or use strong assumptions about the decision-maker’s preferences. Our paper reviews some of the existing approaches to solve multi-objective decision problems and presents new multi-objective linear programming formulations of two multi-objective optimization problems in conservation, allowing the use of a reference point approach. Reference point approaches solve multi-objective optimization problems by interactively representing the preferences of the decision-maker with a point in the criteria (objectives) space, called the reference point. We modelled and solved the following two problems in conservation: a dynamic multi-species management problem under uncertainty and a spatial allocation resource management problem. Results show that the reference point method outperforms classic methods while illustrating the use of an interactive methodology for solving combinatorial problems with multiple objectives. The method is general and can be adapted to a wide range of ecological combinatorial problems.


Introduction
In recent years, the benefits of using optimization methods to solve decision problems have been widely acknowledged in conservation biology. For example, optimization methods have been developed to best allocate limited resources to protect threatened species [1], protect interacting species [2], design reserves [3,4], eradicate invasive species [5], restore habitat [6] or translocate species [7]. In behavioral ecology, optimization is used to test evolution by natural selection [8,9]. The control of disease across meta-populations can also be optimized to ensure fastest recovery [10]. Such optimization methods are needed because decision problems are often combinatorial: the possible decisions we have to choose from are combination of smaller ones, which makes the number of possible decisions too large to attempt an exhaustive approach (one cannot generate every possible decision and compare them). a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 Additionally, many decision problems in conservation involve several conflicting objectives [11]. For example, when managing interacting species simultaneously in a complex ecosystem, increasing the abundance of one species can result in the decrease of another [2]. Management cost can also be considered as an additional objective. However, these problems are generally converted to single-objective optimization problems, either considering only one objective or considering an a priori aggregation of the objectives [1,12,13], but see [14,15] for some exceptions. In contrast to these single-objective approaches, multi-objective combinatorial optimization aims to solve multi-objective combinatorial decision problems without such reduction.
Here, we show that it is possible to solve classic multi-objective combinatorial optimization problems in conservation using a cutting edge approach from multi-objective optimization. The reference point method is an interactive approach that provides optimal solutions while accounting for multiple individual objectives. The preferences of the decision-maker are directly expressed as desired values on each objective. These preferences constitute the components of a reference point. Then, an optimization algorithm calculates the closest possible feasible solution to these preferences. If the computed solution is deemed unsuitable, the decision-maker can update his/her preferences and a new solution is calculated. This process can be repeated iteratively until satisfaction of the decision-maker is reached. This type of method is attractive because it does not need any assumptions about the structure of preferences of the decision-maker, i.e. preferences can be handled even if they are complicated and do not follow a fixed scheme such as a linear trade-offs. Additionally, associating the reference point method with an exact optimization method allows us to provide optimal guarantees on solutions computed.
The reference point method has yet to be used in conservation. In this paper, we present the reference point method after introducing some concepts of multi-objective combinatorial optimization and providing a brief critical review of classic approaches. We then demonstrate the benefits of applying the reference point method to two classic combinatorial problems encountered in conservation: a dynamic multi-species decision problem under uncertainty [2] and a spatial resource allocation problem involving several objectives including biodiversity [16]. We show that the reference point method, used in conjunction with an exact optimization method, outperforms the current approaches used in conservation for solving such multiobjective problems, in term of both optimality and guidance for the decision-maker. Finally, we show that the formulation of the multi-species dynamic problem can be easily extended to any problem using a Markov decision process (MDP) formalism.
which can only take value in the set X of feasible (i.e. possible) decisions. Because we are in a combinatorial context, X is assumed to be discrete.
Any decision x 2 X matches with a point z 2 In contrast to single-objective optimization problems, which admit at most one optimal value, multi-objective optimization problems often admit several optimal points, i.e. points of Z X that cannot be outperformed by another point of Z X . These points are called non-dominated points. Formally, a non-dominated point z is a point (z 1 , . . ., z p ) 2 Z X such that there is no z 0 2 Z X with the property z 0 ! z, where inequality ! between two points of Z is defined in Box 1.
Non-dominated points are essential in multi-objective optimization, since they represent the best (optimal) feasible points in terms of a component-wise ordering relation. The set of non-dominated points is also called the Pareto frontier [19]. Consequently, a Pareto frontier represents the set of the best (optimal) feasible points. Most of the multi-objective combinatorial optimization approaches aim to discover non-dominated points and their corresponding decisions, called efficient decisions. Indeed, multi-objective combinatorial optimization is often related to one of the following well-known underlying challenges: 1. Find a particular non-dominated point of the Pareto frontier, according to the preferences of a decision-maker (called local approach in this paper); 2. Discover the entire Pareto frontier, or an approximation of this set (called global approach in this paper).
When the number of criteria is large (> 3), it becomes difficult to calculate, represent and analyze the Pareto frontier. Consequently, the local approach should be preferred for problems where the number of criteria may be more than 3.
Finding a non-dominated point according to preferences of decision-makers can be tackled using an aggregation function [20], sometimes also called a scalarizing function. The role of aggregation functions is to discriminate non-dominated points according to some preferences. More precisely, an aggregation function is a function s from Z to R, which associates a unique real value to every point of the criteria space. In multi-objective combinatorial optimization, s also depends on parameters called preferential parameters, representing the preferences of a decision-maker [21]. The decision-maker can be a person, a group of persons or any entity able to provide preferences.

Classic multi-objective optimization approaches in conservation
Multi-objective optimization has been used for a long time in fisheries [22], forestry [23], natural resources management [16] or molecular biology [24]. In these fields multi-objective optimization is referred to multi-objective programming if not interactive and interactive processes Box 1 8 z; z 0 2 Z; z 0 ! z if and only if 8j 2 f1; . . . ; pg; z 0 j ! z j and 9k 2 f1; . . . ; pg; z 0 k > z k : otherwise. Goal programming, and compromise programming, which aim to minimize the deviation between the achievement of goals and their aspiration levels (fixed by the decisionmaker in the goal programming case and computed in the compromise programming case), are also popular in these fields [22,23]. This section will focus on multi-objective optimization in conservation. Multi-objective optimization is less developed in conservation than in forestry or fisheries but see [15] for an exception. For example, the well-known approximate solver Marxan [4] in conservation is not a multi-objective solver, because the multiple objectives called "targets" are considered as constraints and not as objectives, and no multi-objective optimization framework is yet considered.

Explicit approaches.
Finding optimal solutions when explicitly accounting for multiple objectives in combinatorial problems is a mathematically challenging endeavor. A way to avoid this mathematical challenge is to use what we call explicit approaches in this paper, i.e. generate a few feasible solutions and compare their performance either by sampling using a model [25,26] or empirically by asking experts [27,28]. Although this approach is not, strictly speaking, multi-objective optimization, it is very common in conservation. Indeed, in some cases the structure of the system to optimize prohibit the use an exact approach, except for small problems.
The explicit approach allows us to perform multi-criteria decision analysis (MCDA), which is very powerful where the number of possible decisions is small [29][30][31]. The goal of MCDA methods is to determine a best decision or strategy among a reasonable number of possible ones, given that these decisions/strategies are evaluated on several criteria Another usual approach in conservation is to try to establish correlations between criteria (trade-off analysis), via exhaustive approaches [32], or heuristic approaches [33]. Unfortunately, there is no reason that criteria of combinatorial problems have the same correlation from one instance to another (changing the data could result in a complete different correlation). Additionally, the lack of scalability of exhaustive approaches and the lack of optimality of heuristic approaches make them very limited approaches to solve combinatorial problems.

Implicit approaches.
When the multi-objective problem can only be implicitly defined (see Section 2.1 for a formal definition), we are then confronted to a multi-objective optimization problem.
Local approaches can be used to perform two types of methods: a priori methods and interactive methods. A priori methods use a unique aggregation function, fixed and defined once by the decision-maker, while interactive methods allow the decision-maker to iteratively change his/her preferences. Global approaches, which generate the entire Pareto frontier, are often called a posteriori methods.
Several approaches in conservation aim to find a unique objective summarizing the individual objectives, and then treat the problem as a single objective. Reducing several objectives in one is usually done using an a priori aggregation function, i.e. an aggregation function with fixed preference parameters. The cost-benefit approach is probably the most used approach applying this principle. The cost-benefit approach is an economic approach where every criteria is considered as having an economic counter-part [34]. Such functions are often used to perform a "cost-benefit" analysis [1,35] or a simple weighted sum of the objectives [13]. Other aggregation functions of the objectives have been studied in conservation [12,36]. Several major well-known drawbacks occur in these approaches. Using economic values of species is ethically controversial because it requires associating an economic value to species [37]. Additionally, in practice, depending on the economical evaluation methods, the value of a species can vary significantly, sometimes from one to tenfold [38]. The second drawback is related to the subjectivity and the complexity of the fixed aggregation function. Choosing among a set of potential aggregation functions can be difficult to justify [12,36]. Finally, reasoning with one objective (aggregated function) instead of several, reduces considerably the role of the decision-maker in the optimization process. Indeed, his/her role is then limited to define the problem. A prescribed solution is then provided by the scientists, missing an opportunity to involve the decision-maker in the decision-making process itself.
A posteriori methods aim to generate the Pareto frontier or an approximation of it. Some methods in conservation can be classified as a posteriori methods [39,40]. A posteriori methods corresponds to trade-off analysis methods for implicit approaches. Generating the Pareto frontier is only possible and relevant for problems with a small number of objectives.
Generally based on the use of parametric aggregation functions, interactive methods aim to interactively find the non-dominated point that corresponds the most to the preference of a decision-makers [20,21]. In these methods, the decision-maker preferences can evolve according to the following iterative procedure: • Optimization results are obtained using current preferences; • New preferences are obtained by eliciting feedback from the decision-maker on current results. Multi-objective optimization interactive methods are not very common in conservation but see [15] for an exception.
• Every solution provided by the algorithm corresponds to a non-dominated point of the multi-objective problem.
• Every non-dominated point of the multi-objective problem can be generated by the algorithm.
These requirements are very important, because when we run a multi-objective optimization process (1) to save time we want to generate only non-dominated points and (2) we don't want that a non-dominated point can be missed, because the point could correspond the best point according to the preferences of a decision-maker. The reference point method is one of the only multi-objective optimization methods to satisfy the requirements [18]. This is due to the aggregation function used in the method, called achievement function and was created specifically for the reference point method [41]. The formal formulation of the (augmented) achievement function is as follow [21]: where " z ¼ ð" z 1 ; :::; " z p Þ is the reference point and λ = (λ 1 , . . ., λ p ) is the direction of projection of " z to the Pareto frontier, with l j ¼ Conversely, if we use a weighted sum as an aggregation function, the second requirement is not satisfied because only the convex hull of the Pareto frontier can be generated. Moreover, weights have no significance, and transforming them into meaningful values [43] can be obscure for the decision-maker [44], because the true preferential parameters are hidden. More importantly, the weighted sum method is well-known to not provide good compromise solutions [45], i.e. solutions which are well balanced when considering their criteria values. Finally, this method makes the strong assumption that one objective can always linearly compensate linearly another objective. However, as raised in [46], the following question has in general no answer: "how much must be gained in the achievement of one objective to compensate for a lesser achievement on a different objective?". Human preferences are often much more complicated than linear trade-offs and may require more elaborate methods.
In this paper we will use the linear programming formulation of the reference point method, for reasons explaned in Section 2.3.

The linear programming formulation of the reference point method
Linear programming is well-known in conservation [47], but less in the context of multiobjective optimization, especially approaches. A linear programming approach is however interesting because it is an implicit approach and it can solve optimally combinatorial problems. In this paper we use linear programming in its general sense, i.e. this also includes integer linear programming and mixed integer linear programming. Thus, nonlinear and even non-convex optimization problems can also be tackled with this approach (thanks to some linearization to perform). Of course, the difficulty to solve the program will depend on the the nature of the optimization problem and how hard it is to lieanrize expressions.
Using the reference point method requires solving an optimization problem at every iteration of the interactive process. Although other optimization methods are possible, linear programming (LP) is particularly well suited to solve this problem. The LP formulation with p objectives, n variables and m constraints is: z l j ðf j ðxÞ À " z j Þ; j 2 f1; :::; pg a i Á x b i ; i 2 f1; :::; mg The variables are z and the components of vector x. x represents the decision, f j (x), j 2 {1, . . ., p} are the objective values, and z = min j2{1,. . .,p} f j (x). If some f j are not linear, then it is necessary to linearize them. Every inequality a i Á x b i represents a constraint of the problem, while z l j ðf j ðxÞ À " z j Þ are constraints implying that z = min{λ j (f j (x) − z j )}.
The simplicity of the formulation is one reason of the popularity of LP to solve the reference point method, which has been implemented in many fields where combinatorial problems occur, for example in telecommunication [48], finance [49] or transportation [50].

Adapting the reference point method to two classes of problems in conservation
The LP formulation of the reference point method requires finding good LP formulations of the optimization problem we want to solve. In the case of discrete optimization, this is in general a hard task and requires a strong knowledge of the problem and LP techniques. In this section we present LP formulations for a multi-species dynamic conservation problem and a multi-objective environmental spatial resource allocation problem, so that the reference point method can be applied in both cases.

Dynamic problem in conservation.
In [2], the authors propose a method for solving a sequential decision problem under uncertainty, aiming to conserve simultaneously two interacting endangered species: Northern abalone and sea otters. This bi-objective is solved using (indirectly) an a priori weighted sum of the objectives. Different weights are tested to generate and explore alternatives. Weighting the criteria allows the use of classic MDP solution methods such as dynamic programming [51]. More specifically, the problem is a predatorprey problem where interactions between sea otters and their preferred prey abalone are described using a MDP formalism. Every year, managers must decide between 4 actions: introduce sea otters, enforce abalone anti-poaching measures, control sea otters, half enforce antipoaching measures and half control sea otters. The time horizon is 20 years. The original problem aims to maximize the density of abalone and abundance of sea otters.
Because weighting the objectives of an optimization problem can be controversial (see Section 2.2), we propose to use the linear programming reference point method. Adapting Program 1 directly to a LP formulation of MDPs is challenging because rewards appear only in the constraints and not in the objective. In [45], the authors were confronted with the same situation when they tried to apply a similar multi-objective optimization technique (the Chebyshev method) to MDPs in a robotic context. The Chebyshev method aims to minimize the Chebyshev norm between the reference point and the decision space. The reference point method is different for several reasons. First, in the reference point method, preferences of the decision-maker are directly expressed as values on every criterion, while in the Chebyshev method preferences are expressed as weights. Second, the reference point method allows the decision-maker to choose values inside the feasible space, which is not the case in the Chebyshev method. However, the same idea as in [45] can also be used for the linear programming reference point method formulation. We first wrote the single-objective dual LP formulation of MDPs and then adapted Program 1 to it. LP and dual LP formulations of MDPs are available in [52].
Formally, the multi-objective Markov decision process related to our multi-species problem is defined by the tuplet: {S, A, H, R A , R SO , Tr}. S is the state space, A is the action space and H = {0, Á Á Á, T − 1} is the time-horizon of size T. Taking action a 2 A when in state s 2 S leads to an immediate reward R A (s, a) for abalone and R SO (s, a) for sea otters. Tr is the transition matrix. Further details and values are available in [2]. Program LP DP is the linear programming formulation reference point method we wrote.
The main variables are the dual variables x t,a,s of the initial problem. Variables C A and C SO represent respectively the normalized density of abalone over 20 years and the normalized number of sea otters over 20 years. ð " C A ; " C SO Þ is the reference point which corresponds to the current preferences of the decision-maker. Note that this LP formulation can easily be generalized for any multi-objective Markov decision process problem, which makes our approach very general (see Box 2).

Spatial allocation of resources.
Spatial allocation of resources is an important challenge in conservation including, but not limited to, reserve design [4,53] or environmental investment decision-making problems [16]. In this section, we provide a linear programming reference point formulation of the problem, and demonstrate the use of the reference point method to tackle a spatial resource allocation problem.
In our model, we consider an environmental investment decision-making problem inspired by [16]. We considered a map of 3600 cells, where a decision consists in selecting a subset of 120 cells for management under a budget constraint. In [16] only three objectives were considered, which allows an a posteriori approach. As discussed in Section 2.1, this approach has limitations. In particular, a posteriori approaches are relevant only for few criteria (e.g. 3 criteria) while the reference point approach can deal with a large amount of criteria.
We extended the model proposed in [16] by considering five criteria. The first criterion is related to the minimization of the total travel time of water. Selected cells will benefit from management allowing prevention of fast runoff from the highest cells to the water points. The second criterion is related to the maximization of carbon sequestration. Every selected cell contributes to an improved carbon sequestration in different ways. These two criteria are explained in details in [16]. We define three additional criteria related to biodiversity. Each of these criteria represent the contribution of the selected cells to the conservation of a different species.
We considered a map of |I| × |J| cells where I = J = {1, . . ., 60}. We first generated an elevation map, i.e. for every cell (i, j) 2 I × J we generated an elevation e i,j . According to this elevation map, water runoffs were computed, such that for every cell, the water comes from the highest neighbor (in case of several highest neighbors, one is picked randomly). Thus, every cell (i, j) has a unique antecedent A((i, j)), except the peaks of the map which have no antecedent, where we set A((i, j)) = ;.
For every cell (i, j), x i,j is a 0-1 variable taking the value 1 if (i, j) is managed, and the value 0 otherwise.
For every cell (i, j), t i,j is the average time the water stays on the cell when not managed. d i,j is the additional time water stays on the cell when managed. In our experiments, t i,j and d i,j are random values. For every cell (i, j), T i,j is the time for water to travel the path from the origin cell to the cell (i, j). T i,j is then equal to the time needed to reach the antecedent cell A((i, j)) plus the time of staying on the cell. The Water Traveling Time criterion WTT is the total time needed for water to reach every cell.
For every cell (i, j), managing (i, j) increases its carbon sequestration value by c i,j . The carbon sequestration criterion CS is equal to the sum of c i,j over the managed cells (i, j).
For every cell (i, j) and every species S 2 {1, 2, 3}, managing (i, j) increases the number of individuals of species S by n S i;j . For every species S, the biodiversity criteria N S is equal to the total number of the saved individuals by management.
Finally, the cost of managing any cell (i, j) is denoted by cost i,j . The management is constrained to respect a budget B.
J is the set of objective R j is the reward function associated with objective j 2 J " C j is the current preference on objective j 2 J Tr is the transition matrix of the MDP LP RA below is the (mixed integer) linear program associated to the multi-objective resource allocation problem considering the 5 criteria WTT, CS and N S , S 2 {1, 2, 3}, and a budget equal to B. This is the application of Program 1 to our resource allocation problem. WTT is represented by variable z WTT . CS is represented by variable z CS . Each N S is represented by variable Our approach is exact and accounts for more objectives than in [16]. One can also compare the optimal solutions with the solutions provided by the usual explicit approaches. Given the combinatorial nature of the problem, an exhaustive search is of course not possible. We tested a possible explicit approach consisting in randomly generating 10,000 feasible decisions, i.e. respecting the budget constraint. This can be done easily by randomly choosing 120 cells in the grid and compute the values of the objectives afterwards. From these decisions we kept 300 points which are non dominated by other generated points. The most simple approach consists in comparing every feasible point to every other feasible point, which can be done in O(pn 2 ) in the worst case, where p is the number of criteria and n is the nuber of feasible points. In practice however, one can take advantage of the fact that if a point is declared dominated during the process, then we can remove it from the list of the points to compare. In doing so, our aim is to perform a MCDA approach using the 300 points as feasible decisions.
We applied the reference point method using every generated point of the explicit approach as a reference point. In other words, we projected the points of the explicit approach to the Pareto frontier using our LP formulation.

Results
For both case studies, we used the optimization solver Cplex (version 12) to solve the corresponding linear programs.

Dynamic problem in conservation
Our experiments consisted in comparing the weighted sum approach with the reference point approach for solving the multi-objective dynamic problem in conservation (Section 2.4.1) .  Fig 2(a) shows the resulting non-dominated points using the weighted sum method applied to our dynamic problem in conservation. Twenty equally distributed pairs of weights from (0, 1) to (1, 0) were used, generating only 4 distinct non-dominated points (the 20 points match to the 4 distinct points). In the context of an interactive procedure, the guidance provided to the decision-maker is then limited. Additionally, none of the non-dominated points represents a good compromise solution between the two objectives since no point has similar values on xaxis and y-axis. Fig 2(b) shows the resulting non-dominated points using the reference point method applied to the same problem and using 20 equally distributed reference points in the criteria space: we computed the extreme points A and B of the Pareto frontier and subdivided the segment [AB] into 20 points. This time 19 distinct non-dominated points were obtained and interesting good compromise solutions can be identified (similar values on both criteria). Although the method we used is similar to Pareto-frontier generation methods [54][55][56], in practice we do not need to systematically generate such a representation of the Pareto-frontier and the reference point method can be used interactively to only find the point corresponding the most to the preferences of the decision-maker. These result show how the reference point method is good at generating non-dominated points that are different and well spread, providing a good guidance for the decision-maker in a potential interactive context.
Note that in both cases the computing time was very small and not reported here. This is because both cases were modelled by linear programs using only continuous variables, typically fast to solve [52].

Spatial allocation of resources
We compared an explicit approach with the reference point approach for solving the multiobjective spatial resources allocation problem (Section 2.4.2).
An analysis of the 300 pairs of points corresponding to the explicit approach and the reference point approach revealed that points generated by the reference point method were, on average, at least 27.74% greater on every criteria than the points provided by the explicit method.
This result is not surprising, because the reference point generates only non-dominated points (see the guarantees of Section 2.2.2), while the explicit method has a very low probability of generating a non-dominated point. Table 1 illustrates the superiority of the reference point method compared to the explicit method.
The total computation time for both methods was very low. For the reference point method, which is of course the slowest of the two methods, generating all the 300 points took only 84 seconds, i.e. 0.28 seconds per point on average.

Discussion
Two main types of method for solving multi-objective problems exist in conservation: methods solving simplistic decision problems but using elaborate multi-objective decision-making processes, e.g. [57] and Section 2.2.1, and optimization methods solving complex problems but using simplified and inaccurate decision-making process, e.g. [2] and Section 2.2.2. This paper considers a new approach for reconciling these two extreme types of approaches: the reference point method coupled with linear programming. The method can optimally solve multi-objective combinatorial problems while using an accurate interactive decision-making process.
The theoretical features of the reference point method unlock a large range of important issues of multi-objective decision-making in conservation such as ethics, significance, transparency, convenience, interactivity and optimality (see Section 2). Additionally, the method avoids classic assumptions about the decision-maker's preferences. Results from the two problems in conservation show that the method outperforms classic approaches by providing either better guidance for the decision-maker or better solutions on every criteria (Section 3). Table 1. Comparison between a sampling-based multi-objective explicit approach and the reference point method through a spatial resource allocation problem. Among all generated pairs of points, three randomly selected pairs are compared in the criteria space (pairs 1, 11 and 22). Units are not relevant in this table since the data was randomly generated. The main caveat of the method is the need for an efficient linear programming formulation of the problem. The development of such formulations needs strong linear programming modelling techniques [52,58]. However, in the particular case of multi-objective problems using a Markov decision process formalism, one can directly use our general formulation provided in Section 2.4.1 (Box 2).

Pair
As [17,59] and more recently [60] emphasize, there is a real need to find good compromise solutions for multi-objective problems in conservation and in ecology in general. The approach could also be used to extend single-objective optimization techniques that tackle adaptive management problems [11] and decision problems under partial observability [1], where interactive methods seem particularly relevant. For adaptive management, methods that have been investigated to date are either based on the explicit approach methodology [17], or based on the weighted sum method [11,13]. Recent approaches to find good compromise solutions between simplicity and optimality in conservation [61] should also benefit from our approach.
With the increasing need to account for multiple objectives in conservation, the linear programming reference point approach should positively impact the way of solving multi-objective decision problems involving complex systems.