1 Introduction

In recent times, the successful application of CG methods in solving vector optimization problems (VOPs) has attracted considerable attention as detailed in [1]. Since then, these approaches have gained recognition for their simplicity and minimal memory requirements, thereby proving effective (see, for example, [2, 3] and their references).

Before exploring VOPs, let us consider some well-known CG parameters related to the natural unconstrained optimization problem, which focuses on minimizing The parameters include the β_k of Polak-Ribiére–Polyak (PRP) [4], Hestenes-Stiefel (HS) [5], Dai–Liao (DL) [6], and Hager–Zhang (HZ) [7, 8]. Other well-known CG methods include: a survey on DL [9], Fletcher-Reeves (FR) [10], Conjugate Descent (CD) [11], Dai-Yuan (DY) [12], and Liu-Storey (LS) [13]. In most cases, the convergence of the CG method based on these parameters is achieved only if the search direction attains a decent property or sufficient descent condition.

Another crucial iterative technique for optimization is the spectral gradient method introduced in [14]. This method has substantial performance. Later, in [15] the spectral gradient and CG method were combined to give the first spectral conjugate gradient (SCG) method. The method used the following search direction (1) with spectral parameter θ_k−1 and β_k, see [15]. The SCG method has been extensively investigated by several authors, including spectral CG with sufficient descent property [16, 17], spectral CG involving RMIL [18], and self-adjusting spectral involving hybrid DL CG [19].

Most of the parameters mentioned above are considered for VOP, in which the objective function F represents a function that maps vectors from an n-dimensional space of real numbers to vectors in an m-dimensional space of real numbers. F is also assumed to be a continuously differentiable function. Let Q be a subset of the m-dimensional space of real numbers, closed, convex, and a pointed cone with a nonempty interior. The unconstrained VOP is defined as (2) which optimizes F, where Note that, F = (F₁, ⋯, F_m)^T for some i = 1, ⋯, m. Moreover, if Q is the m-dimensional space of real numbers with nonnegative components, then VOP reduces to multi-objective optimization (MOO). Also, if Q consists of only nonnegative real numbers and m = 1, then (2) reduces to single-objective optimization (SOO).

Several applications in industry and finance are considered instances of VOP, where multiple objective functions are optimized concurrently. Consequently, it becomes imperative to determine a set of optimal points for VOP [20–26]. Because a total order is lacking in where m ≥ 2, the solution to VOP consists of a set of non-dominated points, often referred to as Pareto optimal or efficient points. The challenge lies in identifying the solutions that strike the most advantageous balance. It is important to mention that, (2) signifies minimizing F with respect to the ordering cone Q.

One way to approach VOPs is the scalarization techniques, which parameterized single-objective optimization problems to yield Pareto-optimal points. The decision-maker must choose the parameters, as they are not predetermined. Making this choice can pose significant challenges or become impossible for some problems [27–29]. Consequently, to overcome these drawbacks, some descent-based algorithms have been suggested as solution approaches for VOP. Thanks to the works in [30, 31]. Subsequently, numerous other studies have followed this trajectory, exploring similar directions, see the survey on MOO descent methods in [32] and the references [33–36].

In [1], the conjugate parameters of [4, 5, 10, 12, 37] are considered for VOPs. Their study encompassed numerical implementations of these methods, which were analyzed and discussed. Among these methods, as per the considered test problems, the nonnegative PRP and HS showed exceptional performance in comparison to the others. Conversely, the CD and DY methods outperformed FR in terms of efficiency. Thereafter, Goncalves and Prudente [38] extended the Hager-Zhang CG method for VOPs. For this method, the search direction does not guarantee the descent condition, even with an exact LSE. To address this issue, the authors proposed a self-adjusting HZ method, utilizing a sufficiently accurate LSE, which possesses the descent property. Other works in this direction include the work of [39] based on sufficiently accurate LSE, the first hybrid CG methods proposed for VOPs in [3] and some modified CG methods in [40].

Following the works in [15, 41, 42], He et al. [43] proposed the SCG methods for VOPs. In contrast to scalar optimization, the extension of SCG does not yield a descent property. As a result, the authors provided a modified self-adjusting SCG algorithm to induce the property through the algorithm. They established the convergence using a sufficiently accurate LSE that satisfies the Wolfe LSE. It is therefore natural to ask if there could be an SCG method for which the descent property is guaranteed without inducing it into the algorithm.

In this paper, we affirmatively address the above question. We define a new form of search direction in the vector context, inspired by the work of [44]. Our method yields a sufficient descent property independent of any LSE for arbitrary nonnegative conjugate parameters. We consider four of these parameters and establish their convergence using Wolfe LSE. We provide computational experiments to validate our findings. The outcomes here are compared with HZ and SP methods, showing that our proposed methods are promising.

The presentation of the work proceeds as follows: Section 2 discusses the basic notions and preliminaries. Section 3 presents the proposed algorithm and its convergence properties. Section 4 presents and discusses computational experiments, and Section 5 provides substantial remarks.

2 Preliminaries

This section presents the basic notions related to VOPs. For further details, see [1, 31, 45–47]. Throughout the subsequent sections, (2) signifies minimizing F with respect to the ordering cone Q. Moreover, for a generic Q, the partial order in generated by Q, ≼_Q is in the sense that z ≼_Q y ⇔ y − z ∈ Q, and ≺_Q, is given by z ≺_Q y ⇔ y − z ∈ int(Q). Moreover, the idea of optimality is substituted with “Pareto-optimal or efficient” and “weak Pareto-optimal or efficient” in VOP.

Definition 1 [32] A vector is Pareto-optimal (efficient) if and only if ∄ a vector s.t and .

Definition 2 [32] A vector is weak Pareto-optimal (weak efficient) if and only if ∄ a vector s.t .

Remark 3 Definition 1 implies Definition 2, but the converse is not generally true.

Below are some properties of the cone Q, including its positive polar cone: Notice that Q = Q**, considering the convexity and closedness of Q, The cone formed by is denoted as cone(W), and conv(W) represents the convex hull of W. Now, consider K ⊆ Q*, where 0 ∉ K and K is compact. We define Q* as follows: (3) Now, for a generic set Q, we defined K as follows: (4) which satisfies the condition in (3). In this work, we adopt the definition of K provided in (4). The Jacobian of F at t is denoted as JF(t) and the image of JF(t) on is represented as Image(JF(t)).

The following (5) is considered the necessary condition for Q − optimality of If condition (5) is satisfied, the point is termed as stationary or Q-critical. Conversely, if is not Q-critical, there exists a vector s.t , indicating that b is a Q-descent direction (Q-DD) at . See, for example [47].

Define as The ζ is well-defined, since K is compact. We observed that ζ also provides some features of −Q and −int(Q) as follows: and .

Next, let us define by (6)

Definition 4 We have a Q-DD if f(t, d) < 0, and a Q-critical point if f(t, d) ≥ 0 for all d.

Definition 5 If satisfies (7) then we have a sufficient descent condition (SDC), where c > 0.

Lemma 6 [31]. If is in C¹. Then, the following statements hold:

(a) , for and ℓ ≥ 0;

(b) The mapping (t, d) ↦ f(t, d) is continuous;

(c) |f(t, d) − f(t′, d)| ≤ ‖JF(t) − JF(t′)‖|‖d‖, ;

(d) If ‖JF(t) − JF(t′)‖ ≤ L‖t − t′‖, then ‖f(t, d) − f(t′, d)‖ ≤ L‖d‖‖t − t′‖.

For CG method of VOPs, we define by (8) and by (9) The iterate begins with arbitrary and updated (10) where ℓ_k > 0 is called the step size, computed via an LSE method, and the search direction d_k is defined as (11) with β_k as the conjugate parameter. The VOPs versions of parameters proposed in [1] are as follows: (12) where f(⋅, ⋅) is defined by Eq (6). The vector version of Hager-Zhang was given in [38] as follows: (13) where and ω₁ ≔ −f(t_k, h(t_k)) + f(t_k−1, h(t_k)), ω₂ ≔ f(t_k, d_k−1), ω₃ ≔ f(t_k, h(t_k−1)) − f(t_k−1, h(t_k−1)), ω₄ ≔ f(t_k−1, d_k−1), with (14) where

Now, consider the problem: (15) see for instance, [46].

Here, we define the most commonly known LSE used for conjugate gradient algorithms, namely the exact and inexact LSE, which are defined as follows: we have an exact LSE if ℓ > 0 is computed as follows: (16) As stated in [1], the standard Wolfe LSE and the strong Wolfe LSE for VOPs. It state that ℓ > 0 fulfills the standard Wolfe condition (WWC) if (17) We find ℓ > 0 by means of the strong Wolfe condition (SWC) if the following conditions are satisfied: (18) where e ∈ Q s.t (19) and 0 < ρ < σ < 1.

Lemma 7 [31] Consider h(t) and v(t) as defined in (8) and (9) respectively, then we have

1. (a). if t is a Q-critical point, then h(t) = 0 and v(t) = 0,
2. (b). let t be not Q-critical point, then h(t) ≠ 0, v(t) < 0, and h(t) is a Q-DD,
3. (c). the maps, h and v are continuous.

3 Spectral-like algorithm and convergence properties

In this section, we present the main algorithm and its convergence properties. However, before delving into details, it is important to consider the following throughout this work: according to Lemma 7 (b), we have h(t_k) ≠ 0. Otherwise, t_k is a Q-critical or stationary point.

Consider the vector version of the Dai–Liao conjugate parameter as follows: (20) where α > 0. Now, we consider the following nonnegative parameters (21) and the iterative: (22) where d_k is defined as follows: (23) As mentioned in the preliminary section, ℓ_k > 0 is computed via an LSE strategy, s_k−1 = t_k − t_k−1 = ℓ_k−1d_k−1, and β_k is a nonnegative parameter. Observe that by Lemma 7 (b), we have f(t_k, h(t_k)) ≠ 0.

The following sufficient descent condition follows from Lemma 6(a) and (23) (24) This implies that we always have (7) with c ≤ 1, irrespective of LSE and the β_k parameter.

Remark 8 It is easy to see that (23) can be expressed as (25) where This implies that our method is a special case of SCG method with . Thus, we now have a spectral CG that achieved (7) without any LSE. Note that when employing an exact LSE here, (23) becomes the well-known nonlinear CG method (11) with d_k−1 replaced by s_k−1.

Before we proceed to the convergence analysis, we will require the following significant assumptions.

Assumption 9 Let Q be composed of a finite number of elements and ∃ an open set Ω s.t where and ∃ L > 0 s.t ‖JF(t) − JF(t′)‖ ≤ L‖t − t′‖ for all t, t′ ∈ Ω.

Assumption 10 Let and D_k+1 ≼_Q D_k, for all k, then ∃ s.t .

Assumption 11 The set is bounded.

Note that, by Assumption 11 we have that this implies that, ∃ s.t (26) for all k. Therefore, we have from Lemma 6(d) that ∃ γ > 0 s.t (27) for all k. Also, by the boundedness of {f(t_k, h(t_k))} and Lemma 7(b), there exists δ > 0 s.t (28) for all k. By (27) and (28), we have (29) with ‖q‖ = 1. We stress that these assumptions naturally extend those considered in single-objective optimization.

We will need the following Zoutendijk Lemma in our convergence analysis.

Lemma 12 [1] Let Assumptions 9 and 10 hold. Consider (10), with Q-DD d_k and ℓ_k satisfies (17). Then, (30)

Next, we present the spectral-like algorithm for the nonnegative CG methods for VOPs.

Algorithm 1: A spectral-like algorithm for VOPs

Step 0: Take and initialize k ← 1.

Step 1: Compute h(t_k) and v(t_k) as in (8) and (9), respectively.

Step 2: If v(t_k) = 0, then stop. Otherwise, compute ℓ_k > 0 s.t condition (18) is satisfies.

Step 3: Compute d_k as defined in (23), where β_k is a nonnegative parameter.

Step 4: Set t_k+1 = t_k + ℓ_kd_k, for k ← k + 1 and move to Step 1.

Remark 13 (i) Step 1 is well explained by Lemma 7.

(ii) In Step 2, we use a LSE procedure to compute ℓ_k > 0 fulfilling (18). We emphasize there exist ℓ_k > 0 fulfilling (18) under Assumptions 9 and 11 as detailed in [1, 48].

(iii) In Step 3, we compute d_k and utilize any of the conjugate parameters in (21) one at a time and move to the next step, where the iterates are updated continuously.

One of the sufficient conditions for establishing the convergence here is to estimate the norm of its search direction as follows: if k = 0 or f(t_k, s_k−1) < 0, then we have the following estimate

Otherwise, we have from (23) that (31)

Moreover, we get (32)

Note that, by (27) and (28), we have (33) and (34)

Now, applying (33) and (34) in (32), we have (35)

Now, using (35) in (31), we have

This can further be written as (36) where .

Next, we estimate the modulus of ϕ_k by means of a so called property (*), this property was introduced by Gilbert and Nocedal [49] and recently extended in [1]. This property(*) indicates that β_k is small whenever s_k−1 is small.

Property (*): Consider Algorithm 1 and assume that there exists s.t (37)

Then we have property (*) if there exist p > 1 and λ > 0 s.t (38) and

We have by (28) and (38) that (39) holds with c₁ = p/γ.

The result below indicates that, given some mild assumptions, a CG method fulfilling property (*) converges.

Theorem 14 [1] Given a CG Algorithm, assuming that Assumptions 9 and 11 are satisfied for all k, s.t (a) β_k ≥ 0; (b) d_k is a Q-DD of F at t_k; (c) ℓ_k satisfies (18); (d) property (*) hold. Then,

Remark 15 It is evident that standard Wolfe (17) holds whenever strong Wolfe (18) is assumed. Thus, we assume only strong Wolfe condition for the subsequent results.

Theorem 16 Let Assumptions 9 and 11 hold. Consider Algorithm 1, and let the sequence {t_k, d_k} be generated using any of the following:

1. (i) ;
2. (ii) ;
3. (iii) ;
4. (iv) .

If ℓ_k satisfies (18), then, (40)

Proof We prove that condition (d) of Theorem 14 holds for all the cases (i)-(iv). To show this, we recall from [39] that, it is enough to show existence of a nonnegative constant ϵ s.t (41)

Note that, by (37), Lemma 7(b), and (29), we get (42)

(i) Beginning with we have

Now, by Lemma 6(c)-(d) and (28), we get (43) where ‖s_k−1‖ = ‖t_k − t_k−1‖.

Again, by (6) and (27), we estimate the following, (44)

Thus, by (43) and (42), we have where Therefore, we have property (*).

(ii) we have

Thus, by (43), (42) and (17), we have where Therefore, we have property (*).

(iii) Following similar arguments with previous estimate, we have

Thus, by (43), we have (45) and by Lemma 6 (c)-(d) (46)

Observe that, by (17) and (24), we have

By Lemma 7 (b) and (37), we have (47)

Again, by (17) and from (24), we have that ω₄ < 0 for all k. Moreover, using the definition of ω₂ and ω₄ and the fact that 0 < σ < 1, we get (48)

Again, since ω₄ < 0, for all k, we have that ω₂ ≤ ω₂ − ω₄. Thus, by (47) and (48), we get (49)

Therefore, from (45), (46), (47), (48), and (49), we have where Hence, we have property (*).

(iv) Then, we have

By (17) and (24), we have (50) for all k ≥ 1. Thus, (51)

Using (44) and (43) in (51), we get

We see by (42) that Therefore, Hence, thereby completing the proof.

The lemma presented herein aligns with Lemma 6 as documented in [38]. It plays an important role in proving the convergence of the proposed CG methods.

Lemma 17 Suppose Assumptions 9 and 11 hold. Consider Algorithm 1, where ℓ_k is obtained by (18). If there exists s.t (52) and property (*) holds with β_k ≥ 0. Then, d_k ≠ 0 and where .

Proof Notice that d_k ≠ 0, otherwise (24) is not true. Thus, w_k is well-define. From the fact that d_k is Q-DD at t_k and ℓ_k fulfills (18), we have the Zoutendijk condition (30). Now, applying (52), Lemma 7 (b), and (7), we get (53)

Next, we define and Then, (23) is rewritten as

Observe that (54) and by applying (54), we have

Now, we have (55)

It follows from β_k ≥ 0, (55) and the triangle inequality, that (56)

Since s_k−1 = t_k − t_k−1, then from (26), we have that Moreover, it follows from (28), (42) and (44) that

Also, from (56), we have

By (28) and (53), we get

This complete the proof.

Next, we present the main convergence theorem, considering ϕ instead of β_k. The proof follows directly from [Theorem 2, p. 905, [38]], and thus, it is omitted.

Theorem 18 Given Algorithm 1 and assuming that Assumptions 9 and 11 hold. Then

4 Numerical results and discussions

In this section, we evaluate the performance of the proposed spectral-like Algorithm 1 by examining the following methods: PRP+, HS+, HZ, and DL+. We aim to gauge their efficiency and robustness in addressing benchmark test problems sourced from various MOO research articles. The algorithms were coded in Fortran 90. Subsequently, in the context of MOO, we define e as Q as and K as .

Below, we present a summary of the methods under consideration, including their initial parameter values. This encompasses both our proposed methods and those employed for comparison purposes:

• SPRP+: a spectral-like PRP+ method given by Algorithm 1 with β_k in (21);
• SHS+: a spectral-like HS+ method given by Algorithm 1 with β_k in (21);
• SHZ+: a spectral-like HZ method given by Algorithm 1 with β_k in (21) with μ = 1.0;
• SDL+: a spectral-like DL+ method given by Algorithm 1 with β_k in (21) and α = 0.1.

Our findings are compared with the following CG methods:

• HZ: a Hager-Zhang CG algorithm given in [38] with μ = 1.0,;
• SP: a spectral CG method (SCG) given in [43].

An essential part of the algorithms include computing the steepest descent direction, h(t). To achieve this, we utilize Algencan to solve problem (15); for more details, refer to [50]. In addition, the selection of the step size was performed using a LSE strategy that fulfills (18). The same LSE, employed for both HZ and SP, was used for all the proposed methods. Below are the initial parameters utilized in the LSE procedure for the implementation of the proposed methods: On the other hand, we have by Lemma 7 that is a stationary point if and only if v(t) = 0. Consequently, the experimentation was conducted by running all the implemented method up to the point of convergence, which is assumed to be or whenever, the maximum number of iterations, #maxIt = 5000 is exceeded. In this case, the v(t) is defined by (9) and the machine precision, eps ≈ 2.22 × 10⁻¹⁶.

Details of the test problems under consideration are provided in Table 1. The first column provides the names of the problem, for instance, “Lov1” aligning to the first problem introduced by A. Lovison in [51], and “SLCDT1” corresponding to the first problem given by Schütze, Lara, Coello, Dellnitz, and Talbi in [52]. All the remaining problems follow the same pattern with their corresponding references. The second column gives the corresponding references, while the third column is assigned for “n” the number of variables and the fourth column is assigned as “m” the objective functions of the problems. A box constraint was utilized for the starting points, defined as , where the lower bound is indicated in the fifth column and the upper bound is indicated in the last column.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The considered test problems.

https://doi.org/10.1371/journal.pone.0302441.t001

In Table 2, the results of the proposed algorithms under the respective test problems are presented in comparison with HZ and SP CG methods. All the methods ran 100% successfully or reached a critical point. The table arrangements are: ‘Iter,’ ‘FunEva,’ and ‘GradEva.’ Thus, they denote the median number of iterations, functions, and gradient evaluations, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Performance of the proposed spectral-like methods in comparison with HZ and SP.

https://doi.org/10.1371/journal.pone.0302441.t002

In a VOP setting, the primary objective is to approximate the Pareto frontier of the given problem. To achieve this, we employed a methodology where each implemented method underwent 200 runs for each problem, and Iter, FunEva, and GradEva were recorded for each run. The methods began with initialization using uniformly distributed random points within the problem’s specified bounds, as detailed in Table 1. The comparison metrics employed here include the Iter, FunEva, and GradEva.

In order to guarantee an equitable and significant comparison of algorithms, we employed the well-established performance profile as documented in [53]. The profile visually represents algorithm performance, which compares algorithmic performance across various metrics, offering a comprehensive assessment of efficiency and robustness. This tool enables us to concisely summarize the experimental data showcased in Tables 2 and 3. Furthermore, the performance profile offers insights into the effectiveness of the methods proposed in this study in comparison to the HZ and SP CG methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Continuation of Table 2.

https://doi.org/10.1371/journal.pone.0302441.t003

Based on the considered test problems, the performance profile for Iter is displayed in Fig 1. It is observed that the SPRP+, SHZ+, SDL+, and SHS+ methods exhibit the best performance according to their appearance, outperforming the other compared methods. On the other hand, the HZ and SP methods are the least-performing. Additionally, Fig 2 illustrates FunEva, showing that the SPRP+, SHZ+, SDL+, and SHS+ methods required fewer function evaluations compared to the HZ and SP methods. Finally, Fig 3 presents GradEva, where the SPRP+, SHZ+, SDL+, and SHS+ methods evaluated fewer gradients than the HZ and SP methods. This observation aligns with the fact that our methods satisfy the SDC independently of any LSE.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Performance on Iter.

https://doi.org/10.1371/journal.pone.0302441.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Performance on FunEva.

https://doi.org/10.1371/journal.pone.0302441.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Performance on GradEva.

https://doi.org/10.1371/journal.pone.0302441.g003

5 Concluding remarks

We introduced new spectral-like CG methods that achieve sufficient descent property independently of any LSE and for arbitrary nonnegative CG parameters. Four well-known conjugate parameters, PRP+, HS+, HZ+, and DL+ are considered and thus are referred to as SPRP+, SHS+, SHZ+, and SDL+, respectively. We established the convergence of the proposed methods using Wolfe LSE. Our algorithms achieved this without regular restart and assumption of convexity regarding the objective functions. The sequences generated by our algorithm identify points that satisfy the first-order necessary condition for Pareto optimality. We conducted computational experiments which show the implementation and efficiency of the methods with a promising performance. The proposed spectral-like methods, SPRP+, SHZ+, SDL+, and SHS+, exhibited the best performance according to their appearance, outperforming HZ, and SP methods in all the considered metrics, the number of iterations, function, and gradient evaluations.

A challenging task to consider in the future is the three-term CG method, which is of special interest in yielding sufficient descent of the search direction. This work may be challenging, considering that f as defined in (6) is only sublinear with respect to the second variable. However, the work herein provides insight into the three-term method.

Acknowledgments

The first author acknowledges the support provided by the Petchra Pra Jom Klao Ph.D. scholarship of King Mongkut’s University of Technology Thonburi (KMUTT) with No. 23/2565 and Center of Excellence in Theoretical and Computational Science (TaCS-CoE), KMUTT. Additionally, the authors acknowledge the references [1, 39] for making their Fortran codes available: https://github.com/lfprudente/CGMOP.