Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

An efficient Dai-Yuan projection-based method with application in signal recovery

  • Jamilu Sabi’u ,

    Contributed equally to this work with: Jamilu Sabi’u, Ado Balili

    Roles Data curation, Formal analysis, Methodology

    Affiliation Department of Mathematics, Faculty of Science, Yusuf Maitama Sule University, Kano, Nigeria

  • Ado Balili ,

    Contributed equally to this work with: Jamilu Sabi’u, Ado Balili

    Roles Writing – original draft

    Affiliation Department of Mathematics, Faculty of Science, Yusuf Maitama Sule University, Kano, Nigeria

  • Homan Emadifar

    Roles Writing – review & editing

    homan_emadi@yahoo.com

    Affiliations Department of Mathematics, Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Saveetha University, Chennai, Tamil Nadu, India, MEU Research Unit, Middle East University, Amman, Jordan, Department of Mathematics, Hamedan Branch, Islamic Azad University, Hamedan, Iran

Abstract

The Dai and Yuan conjugate gradient (CG) method is one of the classical CG algorithms using the numerator ‖gk+12. When the usual Wolfe line search is used, the algorithm is shown to satisfy the descent condition and to converge globally when the Lipschitz condition is assumed. Despite these two advantages, the Dai-Yuan algorithm performs poorly numerically due to the jamming problem. This work will present an efficient variant of the Dai-Yuan CG algorithm that solves a nonlinear constrained monotone system (NCMS) and resolves the aforementioned problems. Our variant algorithm, like the unmodified version, converges globally when the Lipschitz condition and sufficient descent requirements are satisfied, regardless of the line search method used. Numerical computations utilizing algorithms from the literature show that this variant algorithm is numerically robust. Finally, the variant algorithm is used to reconstruct sparse signals in compressed sensing (CS) problems.

Introduction

Firstly, we consider the nonlinear minimization problem (1) in which f is an n-variables smooth function from . The following notations will be employed throughout this work: ∇f(xk) = g(xk) = gk, and f(xk+1) = fk+1. It has become a trend in nonlinear optimization to solve (1) by employing the conjugate gradient (CG) method, which is an algorithm that requires minimum memory to implement. Given an arbitrary , the method takes the following form: (2) where xk stands for the kth iterate, ϑk > 0 is a steplength that is often obtained by applying an appropriate line search procedure [16]. The CG direction dk given by (3) in which βk has a significant impact on the robustness of different CG algorithms. Over the decades, various formulations of the parameter have been proposed. The classical βk updates are defined as follows: (4) (5) in which ‖⋅‖ stands for the 2norm of vectors. For more βk updates, the reader should see [1, 2, 14, 15]. A descent condition (DC) with respect to a CG direction is satisfied if (6) and also satisfied the sufficient DC if (7)

In most circumstances, proving the sufficient DC (7) is enough to analyze global convergence for numerous CG algorithms.

Now, observe that the parameters in (4) and (5) have the common numerator ‖gk+12 and respectively. It was noted in [14] that the global convergence theorem of CG methods with βk defined in (4) depends only on the Lipschitz condition and is independent of the boundedness assumption. Also, CG schemes with the update parameters in (4) converge globally to the minimizer of objective function f in (1) when the step-size ϑk is computed exactly, namely, ϑk is determined as

Despite this, the performance of methods using parameters in (4) is not encouraging. This is because they are prone to jamming, which occurs frequently when the algorithms perform multiple steps that do not approach the minimizer. On the other hand, compared to the schemes in (4), the CG methods with the parameters in (5) exhibit nice numerical performance because they possess an automatic restarting mechanism that avoids the jamming phenomena experienced by the methods with βk in (4). However, these methods converge globally only when f is a strongly convex function, namely, (8) where is a positive-definite and symmetric matrix, the exact line search is employed and the step size approaches zero whenever the Lipschitz condition is assumed. Now, it is important to note that the methods with βk defined by (4) and (5) are equivalent whenever f satisfies (8) and the exact line search is employed. The schemes also generate the pure conjugacy condition (9) which ensures the minimization of f in at most n steps holds with as defined earlier. Now, let yk be as defined in (1). Then utilizing the mean-value theorem for general nonlinear functions, there exists τ ∈ (0, 1) such that (10) holds. Considering (9) and (10), it is ideal to consider the conjugacy condition (11) which, combined with (3) yields the update parameter presented in (5). Furthermore, both conjugacy condition (9) and (11) are implemented only with the exact line search condition which is not feasible in practical computations. Following this development, Perry employed the quasi-Newton condition to develop a modification of (11), which is used with inexact line searches namely (12)

As a consequence of (12), Dai and Liao [16] incorporated a parameter t ∈ (0, ∞) in (12) and presented another variant of (12) as (13)

As the Dai-Yuan(DY) method possesses shortcomings of the CG methods with βk in (4), attempts have been made over the years by researchers to develop its modifications for solving (1) with better performance and global convergence theorem [1720]. In line with this, Andrei [21] proposed a scaled DY-type method for solving (1) with sufficient descent and conjugacy conditions. Andrei [22] also proposed a hybrid method for solving (1), where the parameter βk of the scheme is a convex combination of the DY and HS parameters. Based on ideas of the quasi-Newton method by Li and Fukushima [23], Wei et al. [24], Zhang et al. [25], and Zhang [26] proposed two DY-type algorithms for solving (1). One of the schemes converges globally for nonconvex functions, while the second method exhibits the good performance properties it inherits from the HS method. For other DY-type methods proposed in the literature, see [2734] and the references therein.

It has become a trend for nonlinear real-life problems, which are often modelled as (1), to also be formulated as the convex-constrained nonlinear problem (14) in which is a nonempty, closed convex subset of , and is continuous and monotone, i.e., F satisfies the inequality (15)

Furthermore, due to the notion of optimality condition and nice properties of the CG methods for solving (1) and (14), i.e., when ∇f = F, is regarded as a gradient mapping of (1), different methods for solving (14) have been proposed in recent decades by scholars working on large-scale problems. Search directions of the methods are defined by where βk is determined by (4), (5) and their modifications. The problem expressed in (14) arises in applications such as the 1norm regularization problems in CP [3540], the equilibrium problems in [41, 42], etc.

In recent years, variants of DY algorithms [30] have been developed to solve the problem in (14). This includes the spectral DY projection approach by Liu and Li [43], where the algorithm was obtained by combining the classical DY parameter [9], the spectral gradient method [44], and the projection algorithm proposed in [45]. In this scheme however, the . Moreover, without considering the system’s differentiability assumption, the authors were able to prove the global convergence of their method. Also, in [46], Liu and Li combined the multivariate spectral gradient method [47] with the DY scheme [9] to present a spectral DY-type method for solving problem (14). Motivated by the work in [43, 48], Liu and Feng [49] presented a modified DY-type method for solving problem (14). The scheme’s derivative-free structure makes it ideal for solving large-scale non-smooth problems. By employing the Lipschitz continuity assumption, the authors proved the global convergence of the new method. Inspired by the work in [49], Sani et al. [50] presented a DY-type method for solving (14), where the search direction of the scheme is obtained as a convex combination of the unmodified DY and CD parameters. The authors proved the global convergence of the scheme under mild assumptions. Only recently, Alhobaiti et al. [51] proposed two scaled DY-type algorithms for solving (14), where two different approaches were applied to compute the scaling parameter. The authors also showed that both methods satisfy (7) irrespective of the line search strategy used.

We outline the work’s aims as:

  • To develop an efficient DY-type scheme for solving the problem (14), which not only inherits nice properties of the classical DY method but avoids its shortcomings.
  • To present a DY-type algorithm that ensures sufficient DC.
  • To develop a method with restart and scaling properties for accelerating the convergence.
  • To demonstrate global convergence and analyse the proposed method’s convergence rate.
  • To demonstrate the application of the method in sparse signal reconstruction.

The paper is organised as follows: The motivation and details of the proposed method are discussed in the next section. Section three contains the convergence results. Section four presents a detailed discussion of numerical results. part five gives the scheme’s applications, while part six draws conclusions.

Motivation and details of the method

We begin this section by stating the Cauchy-Schwartz inequality that will occasionally be recalled in the derivation as well as convergence analysis of the proposed algorithm. Let u and v be vectors in an inner product space, then (16) where 〈u, v〉 stands for inner product between u and v.

As known theoretically, the sufficient DC (7) holds for all the classical CG methods when the objective function is convex quadratic and the exact line search procedure is employed. In actual computations, however, where inexact line searches are used, (7) does not hold in general for these methods. For example, by substituting the HS parameter in (3) and multiplying through by , we obtain (17)

Clearly, (17) satisfies the sufficient DC if and . On the other hand, if and the condition may not hold as the quantity may become larger than −‖gk+12. To remedy this defect of the HS method, Dong et al. [52] made some modifications to the method that not only satisfies the sufficient DC but also inherits the nice attributes of the scheme. The authors proposed the following modified HS search direction: (18)

In (18), λk+1 is given by where and

Motivated by this approach, Aminifard and Babaie-Kafaki [53] made an almost similar modification to the classical PRP method, which is also a scheme with the same built-in mechanism as the HS method but also fails to satisfy the condition (7) when inexact line searches are employed in general as demonstrated in the case of the HS method. The search direction put forward by the authors in [53] for which (7) holds, while retaining nice attributes of the unmodified PRP method is defined as follows: (19) where and

Inspired by (18), (19), and the classical DY method, we propose the following DY-type search directions: (20) in which (21) where (22) with (23) and t is a positive parameter in the interval (0, 2). Also, (24) and

The parameter and are the improved Dai-Yuan CG parameters and are both constructed in a similar with the proposed by Dong et al. [52] and the proposed by Aminifard and Babaie-Kafaki [53]. The direction (20) has both the scaling and restarting advantages that made it satisfy the sufficient descent condition without the aid of any line search procedure, and the accelerated convergence rate both numerically and theoretically. From (24) and monotonicity of F, we obtain (25) where in the second equality we used the definition of sk = zkxk = −ϑkdk, and the inequality follows from the monotonicity of F. The steps of the spectral-modified Dai-Yuan algorithm are as follows:

Algorithm 1 (Spectral Modified Dai-Yuan Method)

Initial Data: Initialized ϵ > 0, , ζ ∈ (0, 1), σ ∈ (0, 1), μ > 2, t ∈ (0, 2).

Initialization: Set k = 0 and d0 = −F0.

1: Determine F(xk) and verify if ‖F(xk)‖ ≤ ϵ. If yes, stop; otherwise go to 2.

2: Set zk = xk + ϑkdk, and find , where mk is the smallest nonnegative integer m that satisfies (26) holds.

3: If and ‖F(zk)‖ ≤ ϵ, stop, otherwise obtain (27) (28)

4: Determine dk+1 by the formula (20).

5: We repeat the procedure from 1 by setting k = k + 1 until the stopping condition is satisfied.

Convergence results

This will give the convergence rate and the global convergence for the spectral-modified Dai-Yuan algorithm. Now, we start with the following helpful assumptions:

  1. 1. We assumed exists such that .
  2. 2. Given L > 1, we have (29)

Before we proceed, it is required to state an important property of the projection operator which will be recalled later.

Lemma 1 Given that is as defined in (14), then and (30)

Lemma 2 The sequence of search directions for the spectral-modified Dai-Yuan algorithm satisfies the inequalities (31) where μ > 2, and t ∈ (0, 2).

Proof: It can be seen from (20) that for k = 0, . Next, we analyze the two cases presented in (20) for k = 1, 2, ….

First case: . By pre-multiplying (20) by Fk+1, we get (32)

Second case: . Given that ‖Fk+12 and are both greater than zero, pre-multiplying the dk+1 for the second case by Fk+1, we obtain (33)

Therefore, utilizing (32) and (33), we obtain the inequality in (31), which establishes the proof.

Now, from Cauchy Schwartz inequality (16) and (31), we obtain the first inequality. It can also be seen from that from (20) and for k = 0, d0 = −F0 which implies that ‖d0‖ = ‖F0‖. Now, for k = 1, 2, …, we analyze two cases.

First case: . From (20), (21), (22), (25), and (16), we have (34)

Case (2) . From (20), (23) and (25), we have (35)

Hence, from (34) and (35), we get (36)

This Lemma will be used to illustrate the next Lemma result.

Lemma 3 (1) Let dk and xk be sequences generated using the spectral-modified Dai-Yuan algorithm. Then a non-negative integer exists such that (26) holds.

(2) Assuming that Assumption 2 holds true for {xk} and {zk} obtained using the spectral-modified Dai-Yuan algorithm. Then αk > 0 satisfies (37)

Proof: Proof of (1) is omitted since it has been proven in [54]. To show (2), we assume that Algorithm 1 terminates at some point xk. If so, it follows that F(zk) = 0 or F(xk) = 0. If F(xk) ≠ 0, then the direction dk ≠ 0 as well. We will demonstrate that (26) always stops within finite iterations. If F(xk) ≠ 0, then dk ≠ 0 follows from (31). We now demonstrate that the line search (26) always ends within finite iterations. Suppose ϑk = 1 in (26), then (26) holds, otherwise, will not satisfy (26), namely, with, . From Assumption 2 and (26), we can write

Therefore, we have with the second inequality obtained from (36).

Lemma 4 Assume that Assumptions 1 and 2 are true. Therefore is convergent for any arbitrary solution in and consequently (38)

Proof: To begin the proof, we start with the boundedness of {xk} and {zk}. From (15) and zk, we obtain (39)

From (15) and , we get (40)

Utilizing (30), (27), (28), (39) and (40), we have (41)

So that we obtain

Therefore, is a decreasing sequence which shows it is bounded and {xk} is also bounded. From the boundedness of {xk} and continuity of F, we have that there exists a constant ξ1 such that for all k ≥ 0, (42)

Also, from (36) and (42), we obtain

Hence, there exists a constant ξ2 such that where . From (15), (16), (39) and (42), we have which further implies that

By setting , the boundedness of {zk} is established. Also, the continuity of F, implies that there exists a constant such that

This with (41) yields (43)

Considering that is convergent and {F(zk)} is bounded, we have from (43) that which leads to

Theorem 5 Considering the Assumptions 1 and 2, the sequence {xk} converges.

Proof: Utilizing (37) and (38), we see that when 0 ≤ ϑdk‖ ≤ ϑkdk‖ → 0. In addition, limk→∞dk‖ = 0. From (31), it yields the relation

This indicates that limk→∞Fk‖ = 0. From (38) and boundedness of {xk}, we have that a cluster point of {xk} exists. Suppose represents a cluster point of , where is an indexing set for which

Then, by continuity of F, we get which implies that is a solution of (14), i.e., . Furthermore, is convergent from Lemma 3.3. Therefore, setting , we obtain

So, we conclude that the sequence {xk} goes to .

Assumption 3. Given that τ ∈ (0, 1), , and satisfying (44) where stands for the neighbourhood of , given as

Thus, specifies the distance between x and .

Theorem 6 If Assumptions 1, 2, and 3 are true for {xk} given by Algorithm 1, then converges Qlinearly to 0, implying that {xk} converges to Rlinearly.

Proof: Suppose that . Then (45)

By replacing with in (41), we have (46)

From (29), (36), (45) and since for a solution , , we can write (47) where .

Next, from (31) and Cauchy Schwartz inequality (16) by assuming c = 1 to have (48)

Thus, since , using (44), (45), (46), (47), (48), and (37), we obtain

Since τ, σ, ϑ ∈ (0, 1), and L > 1, φ > 1; then . It follows that converges Q-linearly to 0, meaning that {xk} is R-linearly convergent to .

Numerical results

In this part, we give the numerical findings for the spectral-modified Dai-Yuan algorithm to demonstrate its usefulness, which we identify as SDYM. The proposed algorithm is evaluated via some current spectral-like methods from the literature:

  • A method for solving nonlinear monotone equations using scaled gradient projections (SCGP) [55].
  • The prominent spectral CG DY-type for NCMS (MDY) [56].
  • A convex constraints RMIL CG method for monotone equations (SRMIL) [57].

We choose to compare the SDYM algorithm with these algorithms because they are also derivative-free algorithms with good convergence properties and are numerically efficient, for more details on the compared algorithms, we refer readers to the highlighted references. The four algorithms are written in Matlab R2015 on a PC with the following specifications (4 GB RAM, 2.30 GHZ CPU). The methods are also implemented using the line search given in (26), with the SCGP, MDY, and SRMIL parameters applied as they appear in each of the papers. Parameters for SDYM are given as ζ = 0.4, σ = 10−3, ψ = 1.9, μ = 2.5, t = 0.6. Each Matlab program completes when ‖F(xk)‖ ≤ 10−10 or ‖F(zk)‖ ≤ 10−10 or 1000 iterations are reached. Six examples of the operator F, which are presented as Example 1 to Example 6 are chosen for the experiments with dimensions 103, 104, 5 × 104 and the following initial starting points:

, , , , ,

The numerical findings using six examples of monotone operator equations are given by considering as a convex-constrained set. The examples are as follows:

Example 1. This is a Nonsmooth Function that can be found in [58].

Fi(x) = 2xi − sin|xi|, i = 1, 2, …, n,

where .

Example 2. This is a Nonsmooth Function that can be found in [59].

, i = 2, …n

where .

Example 3. This is a Penalty Function 1 that can be found in [50].

,

where .

Example 4. This is a Nonsmooth Function that can be found in [47].

Fi(x) = 2xi − sin|1 − xi|, i = 1, 2, …, n,

where .

Example 5. This is an artificial problem that can be found in [60].

F1(x) = x2 − 1 + 2.5x1,

Fi(x) = 2.5xi + xi+1 + xi−1 − 1,

Fn(x) = 2.5xn + xn−1 − 1 i = 2, 3, …, n − 1,

where .

Example 6. This is an artificial problem that can be found in [61].

F1(x) = sin x1 − 1 + 2x1,

Fi(x) = 2 sinxi + 2xi + 2xi−1 − 1,

Fn(x) = 2xn + sin xn − 1, i = 2, …, n − 1,

where .

The numerical findings carried out for the considered algorithms are given by Tables 13, this reads as follows as to what each column’s labels mean: “VAR” and “PN” stand for the dimension and number of the example solved, “IG” and “NOI” are the initial guess and number of iterations respectively. “FV” and “Ptime” denote function values and the processing time recorded in seconds, while “Norm” and “***” stand for the termination point and failure of an algorithm after reaching the maximum iteration (1000).

thumbnail
Table 1. SDYM against SCGP, SRMIL, and MDY numerical results for examples 1 and 2.

https://doi.org/10.1371/journal.pone.0300547.t001

thumbnail
Table 2. SDYM against SCGP, SRMIL, and MDY numerical results for examples 3 and 4.

https://doi.org/10.1371/journal.pone.0300547.t002

thumbnail
Table 3. SDYM against SCGP, SRMIL, and MDY numerical results for examples 5 and 6.

https://doi.org/10.1371/journal.pone.0300547.t003

From Tables 13 we observe that SDYM algorithms are more effective compared to the other three algorithms since they successfully solved all six examples. To further explain the performance of each algorithm, the performance tool developed in [62] by Dolan and Moré to plot Figs 13, with respect to our metrics performance, namely; function values, iterations, and processing time for each algorithm. The y-axis of each figure represents the examples solved with a minimum value of the aforementioned metrics; the right side of each figure represents the percentage of the examples solved by each algorithm successfully, while the algorithms that maintain the topmost curve, indicate the scheme that solves the majority of considered examples within a factor τ of the best time. Fig 1 indicates that the SDYM algorithm solved 62.96% of the test examples with minimum iterations, while SCGP, MDY and SRMIL recorded 19.45%, 1.85% and 15.74% respectively. As exhibited by Fig 2, the SDYM algorithm solved 79.63% of the test examples with fewer function values, while SCGP, MDY and SRMIL recorded 12.96%, 6.48% and 0.93% respectively. Regarding minimum CPU time metric, Fig 3 showed that SDYM is the fastest since it solved 73.15% of the test examples compared to SCGP, MDY and SRMIL algorithms recorded 24.07%, 0.93% and 1.85% respectively. In addition to the above analysis, the curve representing the SDYM algorithm remains at the top of the curves representing the SCGP, MDY and SRMIL algorithms. Based on this discussion, it can be concluded that the SDYM algorithm is promising for solving (14).

thumbnail
Fig 1. Performance of SDYM against SCGP, SRMIL, and MDY for iterations.

https://doi.org/10.1371/journal.pone.0300547.g001

thumbnail
Fig 2. Performance SDYM against SCGP, SRMIL, and MDY for function evaluations.

https://doi.org/10.1371/journal.pone.0300547.g002

thumbnail
Fig 3. Performance SDYM against SCGP, SRMIL, and MDY for CPU time.

https://doi.org/10.1371/journal.pone.0300547.g003

Sparse signal reconstruction

Here, we discuss the application of the SDYM algorithm in solving signal reconstruction problems. To that end and to further test its performance, the SDYM algorithm is compared with HTTCGP [40] and PCG [38] algorithms. Sparse signal reconstruction refers to a technique of reconstructing a disturbed signal in a sparse scenario. This procedure comes up often in practical disciplines, which include astrophysical signals, machine learning, wireless sensor networks, video coding, medical imaging, radar, and compressive imaging [63]. The target of the technique is obtaining sparse solutions to the under-determined linear system , in which is a sampling matrix, x denotes the original sparse signal and stands for an observed value. Hence, in order to reconstruct x from the system , the 1norm regularization problem (49) is solved where γ > 0 is a parameter.

In solving (49), the authors in [64] reformulated it as a convex quadratic model by first splitting a vector into two parts, i.e., where vi = (xi)+, wi = (−xi)+, ∀i = 1, 2, …, n and (.)+ = max{0, x}. By this formulation, we can write where . Consequently, (49) can be re-formulated as (50)

By setting the variables: (50) can be reformulated as (51) as a standard bound-constrained quadratic programming problem. This problem was restarted as which clearly represents the convex quadratic programming problem, see [65]. Since it was proven in [65, 66] that F is monotone and Lipschitz continuous, then (49) can equivalently be represented as (14), which can be solved with the SDYM algorithm.

Experiments and reported results

Here, the SDYM algorithm is applied from some k observations to reconstruct a sparse signal of length n with kn. For the experiments, the HTTCGP and PCG algorithms were employed to compare the performances of SDYM since they are used to solve signal problems. The PCG and HTTCGP algorithms were run with the same values as used in the original papers. Apart from ζ = 0.4, the SDYM parameter values stay the same as in the previous experiment. The quality of restoration is tested in the experiments by the mean square error (MSE), which is defined by where is the reconstructed signal and is the actual one. This definition shows that a scheme with lower MSE yields better-quality reconstructed signals. We used n = 212 as the size of the signal with k = 210, and 26 as the randomly nonzero elements contained in the original signal. In Matlab, the Gaussian matrix is produced using the function randn(k, n). Furthermore, noise, particularly interferes with the measurement b, where η is the Gaussian noise distributed as N(0, 10−4). To initiate the experiment, we used the measurement image, i.e., , which terminates if the inequality holds, where the function , with the parameter γ > 0.

The actual sparse signal, its measurement, and reconstructed version by the three algorithms are displayed in Fig 4. It can be observed from plots 5–8 in descending order that the three algorithms were able to reconstruct the signal almost exactly from the measurement, with SDYM achieving that much faster. In addition, Figs 58 consist of four graphs that depict the convergence performance of the three methods in terms of mean square error (MSE), function values (ObjFE), processing time and number of iterations metrics. Thus, as observed from Figs 58, the rates of descent of MSE and ObjFE for SDYM are much faster than HTTCGP and PCG methods. In addition, the experiment was repeated times, and the results are given in Table 4. The results displayed HTTCGP and PCG in three of the four metrics considered. These results clearly showed that the SDYM algorithm is effective for decoding sparse signals in CP.

thumbnail
Table 4. The signal reconstruction report of twelve noisy experiments.

https://doi.org/10.1371/journal.pone.0300547.t004

thumbnail
Fig 4. Original, measurement and restored signals comparison for the SDYM, HTTCGP, and PCG algorithms.

https://doi.org/10.1371/journal.pone.0300547.g004

thumbnail
Fig 5. The comparison for the number of MSE against the number of iterations.

https://doi.org/10.1371/journal.pone.0300547.g005

thumbnail
Fig 6. The comparison for the number of MSE against processing time (seconds).

https://doi.org/10.1371/journal.pone.0300547.g006

thumbnail
Fig 7. The comparison for the number of the objective function against the number of iterations.

https://doi.org/10.1371/journal.pone.0300547.g007

thumbnail
Fig 8. The comparison for the number of the objective function against processing time (seconds).

https://doi.org/10.1371/journal.pone.0300547.g008

Conclusion

In this paper, we presented a spectral-type Dai-Yuan algorithm for a constrained NCMS and reconstructed sparse signals in CP. The new algorithm fulfils the sufficient DC irrespective of the line search technique applied. The proposed algorithm’s global convergence was shown using a monotone line search approach and certain fundamental assumptions. Furthermore, a comparison of the proposed approach to three efficient methods shows that it is efficient.

References

  1. 1. Ahmed K, Waziri MY, Halilu AS, Murtala S. On two symmetric Dai-Kou type schemes for constrained monotone equations with image recovery application. EURO Journal on Computational Optimization. 2023;11:100057.
  2. 2. Ahmed K, Yusuf Waziri M, Sani Halilu A, Murtala S. Sparse signal reconstruction via Hager–Zhang-type schemes for constrained system of nonlinear equations. Optimization. 2023:1–32.
  3. 3. Salihu N, Kumam P, Sulaiman IM, Seangwattana T. An efficient spectral minimization of the Dai-Yuan method with application to image reconstruction. AIMS Mathematics. 2023;8(12):30940–62.
  4. 4. Salihu N, Kumam P, Awwal AM, Arzuka I, Seangwattana T. A Structured Fletcher-Revees Spectral Conjugate Gradient Method for Unconstrained Optimization with Application in Robotic Model. InOperations Research Forum 2023 Oct 18 (Vol. 4, No. 4, p. 81). Cham: Springer International Publishing.
  5. 5. Salihu N, Kumam P, Awwal AM, Sulaiman IM, Seangwattana T. The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery. Plos one. 2023;18(3):e0281250. pmid:36928212
  6. 6. Sabi’u J, Shah A, Waziri MY. Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations. Applied Numerical Mathematics. 2020;153:217–33.
  7. 7. Fletcher R, Reeves CM. Function minimization by conjugate gradients. The computer journal. 1964;7(2):149–54.
  8. 8. Fletcher R. Practical methods of optimization. John Wiley & Sons; 2000.
  9. 9. Dai YH, Yuan Y. A nonlinear conjugate gradient method with a strong global convergence property. SIAM Journal on optimization. 1999;10(1):177–82.
  10. 10. Hestenes MR, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of research of the National Bureau of Standards. 1952;49(6):409–36.
  11. 11. Polak E, Ribiere G. note sur la convergence de directions conjugees, Rev. Francaise Inform. Recherche Operationelle 3 (1969), 35–43.
  12. 12. Polyak BT. The conjugate gradient method in extremal problems. USSR Computational Mathematics and Mathematical Physics. 1969;9(4):94–112.
  13. 13. Liu Y, Storey C. Efficient generalized conjugate gradient algorithms, part 1: theory. Journal of optimization theory and applications. 1991;69:129–37.
  14. 14. Hager WW, Zhang H. A survey of nonlinear conjugate gradient methods. Pacific journal of Optimization. 2006;2(1):35–58.
  15. 15. Narushima Y, Yabe H. A survey of sufficient descent conjugate gradient methods for unconstrained optimization. SUT journal of Mathematics. 2014;50(2):167–203.
  16. 16. Dai Y-H., Liao L-Z. New conjugacy conditions and related nonlinear conjugate gradient methods. Applied Mathematics and Optimization. 2001;43:87–101.
  17. 17. Waziri MY, Ahmed K, Sabi’u J. A Dai–Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arabian Journal of Mathematics. 2020;9:443–57.
  18. 18. Waziri MY, Ahmed K, Sabi’u J, Halilu AS. Enhanced Dai–Liao conjugate gradient methods for systems of monotone nonlinear equations. SeMA Journal. 2021;78:15–51.
  19. 19. Sabi’u J, Sulaiman IM, Kaelo P, Malik M, Kamaruddin SA. An optimal choice Dai-Liao conjugate gradient algorithm for unconstrained optimization and portfolio selection. AIMS Mathematics. 2024;9(1):642–64.
  20. 20. Yakubu UA, Mamat M, Mohamad MA, Rivaie M, Sabi’u J. A recent modification on Dai-Liao conjugate gradient method for solving symmetric nonlinear equations. Far East J. Math. Sci. 2018;103:1961–74.
  21. 21. Andrei N. A Dai–Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization. Applied Mathematics Letters. 2008;21(2):165–71.
  22. 22. Andrei N. Another hybrid conjugate gradient algorithm for unconstrained optimization. Numerical Algorithms. 2008;47(2):143–56.
  23. 23. Li DH, Fukushima M. A modified BFGS method and its global convergence in nonconvex minimization. Journal of Computational and Applied Mathematics. 2001;129(1-2):15–35.
  24. 24. Wei Z, Yao S, Liu L. The convergence properties of some new conjugate gradient methods. Applied Mathematics and computation. 2006;183(2):1341–50.
  25. 25. Zhang L, Zhou W, Li D. Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search. Numerische mathematik. 2006;104(4):561–72.
  26. 26. Zhang L. Two modified Dai-Yuan nonlinear conjugate gradient methods. Numerical Algorithms. 2009;50(1):1–6.
  27. 27. Babaie-Kafaki S. A hybrid conjugate gradient method based on a quadratic relaxation of the Dai–Yuan hybrid conjugate gradient parameter. Optimization. 2013;62(7):929–41.
  28. 28. Jiang XZ, Jian JB. A sufficient descent Dai–Yuan type nonlinear conjugate gradient method for unconstrained optimization problems. Nonlinear Dynamics. 2013;72:101–12.
  29. 29. Zhou G, Ni Q. A spectral Dai-Yuan-type conjugate gradient method for unconstrained optimization. Mathematical Problems in Engineering. 2015 Jan 1;2015. http://dx.doi.org/10.1155/2015/839659
  30. 30. Jiang X, Jian J. Improved Fletcher–Reeves and Dai–Yuan conjugate gradient methods with the strong Wolfe line search. Journal of Computational and Applied Mathematics. 2019;348:525–34.
  31. 31. Zhu Z, Zhang D, Wang S. Two modified DY conjugate gradient methods for unconstrained optimization problems. Applied Mathematics and Computation. 2020;373:125004.
  32. 32. Mishra SK, Samei ME, Chakraborty SK, Ram B. On q-variant of Dai–Yuan conjugate gradient algorithm for unconstrained optimization problems. Nonlinear Dynamics. 2021;104:2471–96.
  33. 33. Narayanan S, Kaelo P. A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization. Numerical Mathematics-Theory Methods and Applications. 2021;14:527–39.
  34. 34. Laylani Y, Khudhur HM, Nori EM, Abbo KK. A hybridization of the Hestenes-Stiefel and Dai-Yuan Conjugate Gradient Methods. European Journal of Pure and Applied Mathematics. 2023;16(2):1059–67.
  35. 35. Halilu AS, Majumder A, Waziri MY, Ahmed K. Signal recovery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach. Mathematics and Computers in Simulation. 2021;187:520–39.
  36. 36. Halilu AS, Majumder A, Waziri MY, Awwal AM, Ahmed K. On solving double direction methods for convex constrained monotone nonlinear equations with image restoration. Computational and Applied Mathematics. 2021;40:1–27.
  37. 37. Kiri AI, Waziri MY, Ahmed K. A modified Liu-Storey scheme for nonlinear systems with an application to image recovery. Iranian Journal of Numerical Analysis and Optimization. 2023;13(1):38–58.
  38. 38. Liu JK, Li SJ. A projection method for convex constrained monotone nonlinear equations with applications. Computers & Mathematics with Applications. 2015;70(10):2442–53.
  39. 39. Xiao Y, Zhu H. A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. Journal of Mathematical Analysis and Applications. 2013;405(1):310–9.
  40. 40. Yin J, Jian J, Jiang X, Liu M, Wang L. A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numerical Algorithms. 2021;88:389–418.
  41. 41. Dirkse SP, Ferris MC. MCPLIB: A collection of nonlinear mixed complementarity problems. Optimization methods and software. 1995;5(4):319.
  42. 42. Meintjes K, Morgan AP. A methodology for solving chemical equilibrium systems. Applied Mathematics and Computation. 1987;22(4):333–61.
  43. 43. Liu J, Li S. Spectral DY-type projection method for nonlinear monotone system of equations. Journal of Computational Mathematics. 2015:341–55.
  44. 44. Barzilai J, Borwein JM. Two-point step size gradient methods. IMA journal of numerical analysis. 1988;8(1):141–8.
  45. 45. Fukushima M, Qi L, editors. Reformulation: nonsmooth, piecewise smooth, semismooth and smoothing methods. Springer Science & Business Media; 1999.
  46. 46. Liu J, Li S. Multivariate spectral DY-type projection method for convex constrained nonlinear monotone equations. Journal of Industrial & Management Optimization. 2017;13(1).
  47. 47. Yu G, Niu S, Ma J. Multivariate spectral gradient projection method for nonlinear monotone equations with convex constraints. Journal of Industrial and Management Optimization. 2013;9(1):117–29.
  48. 48. Li Q, Li DH. A class of derivative-free methods for large-scale nonlinear monotone equations. IMA Journal of Numerical Analysis. 2011;31(4):1625–35.
  49. 49. Liu J, Feng Y. A derivative-free iterative method for nonlinear monotone equations with convex constraints. Numerical Algorithms. 2019;82:245–62.
  50. 50. Ding Y, Xiao Y, Li J. A class of conjugate gradient methods for convex constrained monotone equations. Optimization. 2017;66(12):2309–28.
  51. 51. Althobaiti A, Sabi’u J, Emadifar H, Junsawang P, Sahoo SK. A Scaled Dai–Yuan Projection-Based Conjugate Gradient Method for Solving Monotone Equations with Applications. Symmetry. 2022;14(7):1401.
  52. 52. Dong XL, Liu HW, He YB, Yang XM. A modified Hestenes–Stiefel conjugate gradient method with sufficient descent condition and conjugacy condition. Journal of Computational and Applied Mathematics. 2015;281:239–49.
  53. 53. Aminifard Z, Babaie-Kafaki S. A modified descent Polak-Ribiére-Polyak conjugate gradient method with global convergence property for nonconvex functions. Calcolo. 2019;56:1–1.
  54. 54. Waziri MY, Ahmed K, Halilu AS, Sabi’u J. Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing. RAIRO-Operations Research. 2022;56(1):239–73.
  55. 55. Koorapetse M, Kaelo P, Offen ER. A scaled derivative-free projection method for solving nonlinear monotone equations. Bulletin of the Iranian Mathematical Society. 2019;45:755–70.
  56. 56. Aji S, Kumam P, Awwal AM, Yahaya MM, Sitthithakerngkiet K. An efficient DY-type spectral conjugate gradient method for system of nonlinear monotone equations with application in signal recovery. Aims Math. 2021;6(8):8078–106.
  57. 57. Lei S. A spectral conjugate gradient method for convex constrained monotone equations. Science Asia. 2021;47(4):514–519.
  58. 58. Cruz WL. A spectral algorithm for large-scale systems of nonlinear monotone equations. Numerical Algorithms. 2017;76:1109–30.
  59. 59. La Cruz W, Martínez J, Raydan M. Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Mathematics of computation. 2006;75(255):1429–48.
  60. 60. Abubakar AB, Kumam P, Awwal AM. A descent dai-liao projection method for convex constrained nonlinear monotone equations with applications. Thai Journal of Mathematics. 2018:128–152.
  61. 61. Gao P, He C. An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints. Calcolo. 2018;55:1–7.
  62. 62. Dolan ED, Moré JJ. Benchmarking optimization software with performance profiles. Mathematical programming. 2002;91:201–13.
  63. 63. Aminifard Z, Hosseini A, Babaie-Kafaki S. Modified conjugate gradient method for solving sparse recovery problem with nonconvex penalty. Signal Processing. 2022;193:108424.
  64. 64. Figueiredo MA, Nowak RD, Wright SJ. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE Journal of selected topics in signal processing. 2007;1(4):586–97.
  65. 65. Xiao Y, Wang Q, Hu Q. Non-smooth equations based method for l1-norm problems with applications to compressed sensing. Nonlinear Analysis: Theory, Methods & Applications. 2011 Jul 1;74(11):3570–7.
  66. 66. Pang JS. Inexact Newton methods for the nonlinear complementarity problem. Mathematical Programming. 1986;36(1):54–71.