https://orcid.org/0000-0001-8024-5056
Figures
Abstract
In this paper, a hybrid conjugate gradient projection method for finding solutions of constrained nonlinear equations is proposed by integrating both hyperplane projection and hybrid techniques. The key features of this method are as follows: (1) It is characterized by a low storage requirement and relies solely on function values; (2) The designed search direction ensures the sufficient descent property without the need for line search approaches; (3) Under certain reasonable assumptions, the global convergence of the method is established; (4) Experimental results demonstrate that the proposed method outperforms the two existing methods about 75.71%, 85.36%, and 86.43% of benchmark problems in terms of CPU time, the number of function evaluations, and iterations. Furthermore, it is applied to successfully solve the sparse signal restoration problems.
Citation: Xia Y, Li D, Wang S (2025) Global convergence in a hybrid conjugate gradient projection method for finding solutions of constrained nonlinear equations with applications. PLoS One 20(10): e0335265. https://doi.org/10.1371/journal.pone.0335265
Editor: Mohamed Kamel Riahi, Khalifa University of Science and Technology, UNITED ARAB EMIRATES
Received: March 15, 2025; Accepted: October 8, 2025; Published: October 28, 2025
Copyright: © 2025 Xia et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: This work is supported by the National Natural Science Foundation in China (grant number 11661009), the Natural Science Foundation in Guangxi Province, PR China (grant number 2024GXNSFAA010478; 2020GXNSFAA159069), the Special projects in key areas of ordinary universities in Guangdong Province, PR China (grant number 2023ZDZX4069), the Research Team Project of Guangzhou Huashang University (grant number 2021HSKT01), and the Guangzhou Huashang College Dao shi Project (grant number 2024HSDS28).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Numerical applications for nonlinear equations span various domains, including financial forecasting [1], compressed sensing problems [2], subproblems in generalized proximal methods with Bergman distances [3], machine learning [4], and robot arm control [5,6]. Motivated by these applications, the goal of this paper is to finding solutions of constrained nonlinear equations:
where is a closed convex and non-empty set, and
is a continuous and monotone function. Throughout the paper, we denote
as
, and
as the Euclidean norm on
. To establish a foundation for the discussion, we now review some key concepts related to nonlinear equations:
(1) A function is said to be monotone if:
(2) A function satisfies the Lipschitz continuity condition with L > 0 if:
Since conjugate gradient methods [7,8] are known for their simplicity and low storage requirements, they are widely used to solve the problem (1). The sequence of iterates {xk} generated by these methods is updated as:
where is the step-size determined by a line search approach, and dk is the search direction. Various methods can be employed to generate the search direction dk, such as Newton methods [9–11], quasi-Newton methods [12,13], Levenberg-Marquardt methods [14,15], and their modifications. For example, in [11], Waziri et al. proposed a method that approximates the Jacobian matrix as a diagonal matrix, significantly reducing storage requirements, computational cost, and CPU time. In [12], Ai-Baali et al. explored the properties of quasi-Newton methods, originally introduced by Broyden as an alternative to Newton’s method and later extended to nonlinear equations. In [14], Ou and Lin presented a hybrid method for solving non-smooth equations with box constraints, combining the Levenberg-Marquardt-like techniques and non-monotone strategies. Their method demonstrates both global and local superlinear convergence with promising numerical results. Despite the advantages of these methods, they generally require solving linear equations using either the Jacobian matrix or its approximation at each iteration, which makes them less suitable for large-scale nonlinear problems. Consequently, many researchers have turned to conjugate gradient methods for generating the search direction, as these methods do not rely on such costly operations and can efficiently handle large-scale equations.
Recently, the hybridization of conjugate gradient methods [16,17] has emerged as an interesting and effective technique for designing search directions. Several researches have explored different combinations of conjugate parameters to improve performance across various applications. For example, in [18], Waziri et al. applied the convex combination technique to develop an improved search direction based on Fletcher-Reeves (FR) and Polak-Ribière-Polyak (PRP) conjugate parameters. Similarly, in [19], Sun et al. introduced a hybrid conjugate gradient-based projection method that combines Hestenes-Stiefel (HS) and Dai-Yuan (DY) conjugate parameters. In another approach, Yusuf et al. [20] proposed a hybrid method for solving large-scale nonlinear equations, which merges FR and PRP parameters. Additionally, in [21], Danmalam et al. developed a hybrid conjugate residual method for nonlinear monotone equations with convex constraints, combining FR and a new residual parameter to ensure descent direction without line search approaches. In [22], Koorapetse et al. proposed a hybrid conjugate gradient projection method for large-scale nonlinear monotone equations, ensuring descent property and global convergence. Furthermore, in [23], Li et al. presented a hybrid conjugate gradient-based projection method with modified PRP and DY parameters, and proved global convergence and superior efficiency in numerical experiments, particularly in sparse signal and image restoration. In [24], Li et al. also developed a derivative-free descent method, which incorporates spectral gradient methods, projection techniques, and monotone line searches. In a different approach, Abubakar et al. [25] proposed a hybrid three-term conjugate gradient method that ensures sufficient descent and trust region properties without line search approaches. Lastly, in [26], Yin et al. introduced a hybrid three-term conjugate gradient projection method with adaptive line search for large-scale nonlinear monotone equations with convex constraints. These diverse hybrid methods highlight the growing potential and versatility of conjugate gradient methods in optimization, particularly for large-scale and constrained problems.
Motivated by the works in [27–29], we adopt the hybridization techniques to modify a novel conjugate parameter, which is designed to ensure the global convergence of the method. Based on this parameter, we then propose a search direction that satisfies the sufficient descent property. By incorporating effective line search approaches and projection techniques, we propose a hybrid conjugate gradient projection method specifically tailored for finding solutions of constrained nonlinear equations.
Algorithm
To establish a coherent framework for formulating the search direction in constrained optimization, we first review the foundational works in unconstrained optimization, specifically addressing the problem of minimizing a function f(x), where . The conjugate gradient method, a widely-used iterative approach in unconstrained optimization, depends heavily on the choice of conjugate parameters that govern the search direction. Hager et al. [27] developed a novel conjugate parameter, which is defined as:
where is the gradient of the function at the current point xk,
, and
. Similarly, Yu et al. [28] developed a modified version of the PRP conjugate parameter:
where the first term promotes the classical PRP conjugacy, while the second term introduces a correction to account for the history of the gradients, providing a more adaptive search direction. Moreover, Li et al. [29] presented a modified Liu-Story (LS) conjugate parameter, which is expressed as:
Notably, all of these conjugate parameters share a common structural pattern, which suggests that their forms can be unified into a more general framework for optimization. Motivated by these findings, we extend these conjugate parameters to address constrained optimization problems, particularly those involving nonlinear equations as in problem (1). This insight leads us to propose a hybrid conjugate parameter to ensure the global convergence of the method:
where ,
,
, and
. To ensure that the search direction satisfies the sufficient descent property, we define the following search direction:
In this section, we propose a hybrid conjugate gradient projection (HCGP) method to find solutions of constrained nonlinear equations. To ensure clarity and precision, we first discuss the line search strategy employed in our method, as well as the projection operator used to enforce the feasibility of iterates.
We adopt a line search strategy based on the approach outline in the work [30]. This approach determines the optimal step-size through the following inequality condition:
where is a small constant, and the step-size is selected according to the rule
with parameters
and
. This approach ensures a gradual and controlled descent along the search direction while maintaining stability in the optimization process.
A critical aspect of constrained optimization is ensuring that each iterative point lies within the feasible region E. To address this, we define the projection operator PE that projects any point onto the feasible set. This operator minimizes the Euclidean distance between x and the feasible region E:
The projection operator exhibits a well-known non-expansive property, which ensures that the distance between projections of two points is no greater than the distance between the original points themselves:
With the line search strategy and projection operator, our method computes the trial point at each iteration. If the trial point wk does not lie within the feasible region E, we update the next iterative point xk + 1 using the following update rule:
where is a scaling factor.
The iterative process continues until the term falls below a given threshold δ, indicating convergence to a solution. The detailed steps of the method are summarized in Algorithm 1.
Algorithm 1. HCGP algorithm.
1: Initialization: Parameters: ,
,
,
,
,
. Set
.
2: while do
3: Compute the search direction dk by (5).
4: Determine the step-size by (6) and compute the trial
point .
5: if and
then
6: Break.
7: else
8: Update the next iteration xk + 1 by (8).
9: end if
10: Set .
11: end while
Analysis of convergence properties
In order to demonstrate that the sequence {xk} generated by the HCGP method converges to its optimal solution of the problem (1), we must establish the following general assumptions:
(A1) The solution set of the problem (1) is non-empty;
(A2) The function e(x) is both monotone and Lipschitz continuous on .
Search direction analysis
With these assumptions in place, we proceed to analyze the properties of the search direction sequence {dk}, which is a key element in proving convergence. The following lemma characterizes the sufficient descent property of the search direction sequence.
Lemma 1. The search direction sequence {dk} satisfies the sufficient descent property for any :
Moreover, for , we have:
Proof: When k = 0, we have . When
, multiplying both sides of (5) by ek, we have:
Thus, the sufficient descent property (9) holds. Additionally, by applying the Cauchy-Schwartz inequality and using the sufficient descent property, we have:
which implies that . □
Line search analysis
Lemma 2. Suppose that Assumptions (A1) and (A2) hold. Then the following conclusion are satisfied:
(1) The line search strategy (6) is well-defined.
(2) The step-size satisfies the following relation:
Proof: To prove the lemma, we address each part of the statement separately.
Part 1: Suppose, by contradiction, that there exists a positive integer such that the line search strategy (6) dose not hold for any integer i > 0. This means that for all i > 0, we obtain:
Given the continuity of , we can take the limit as
, yielding:
Besides, from the sufficient descent property (6), we know that:
which contradicts (12). Thus, such a k0 cannot exist, and we conclude that the line search strategy (6) is well-defined.
Part 2: Assume that . In this case, we define
, which does not satisfy the line search strategy (6), implying that:
Using the sufficient descent property (13) and combining with (9), we obtain the following inequality:
which is rewritten as:
□
Global convergence analysis
Lemma 3. Suppose that Assumptions (A1) and (A2) hold. Then, the sequence is convergent for any
. Furthermore, the sequences {xk} and {wk} are bounded, and
Proof: The result follows directly from a similar result presented in Lemma 4 of [23]. Therefore, the proof is omitted here for brevity.
Lemma 4. Suppose that Assumptions (A1) and (A2) hold. If , there exists a positive constant N such that, for all
,
Proof: From (3), (8), we have the following relationship:
Utilizing this in conjunction with (4), we obtain:
Additionally, using Lemma 3, we know that the sequence {ek} is bounded due to the continuity of and the boundedness of {xk}. That is, there exists a constant r>0 such that
holds. From this result, (5), and (16), it follows that
Thus, we have with
. □
Theorem 1. Suppose that Assumptions (A1) and (A2) hold. Then, we have:
Proof: We proceed by assuming the contrary, i.e., (17) does not hold. This implies the existence of a constant such that
holds for all k. Given this assumption, and using the result (10), we immediately deduce the following inequality:
Next, by multiplying both sides of (11) by , we obtain the following expression:
Substituting (15) and (18) into (19), we obtain:
This leads to a contradiction with (14), which shows that our assumption that for all k must be false. □
Numerical experiments
In this section, we compare the performance of the HCGP method with that of two similar methods: HSDY method [31] and DCG method [32]. All experiments are conducted on a Lenovo PC equipped with an Intel(R) Core(TM) i7-12700F processor (2.10GHz), 16GB of RAM, and Windows 11 operation system. For the HCGP method, the following parameters are used:
The parameters for the HSDY and DCG methods are chosen according to the settings specified in the original literature.
In this experiment, we compare the performance of the HCGP, HSDY, and DCG methods on eight benchmark problems. For all methods, the programs are terminated when one of the following criteria is met:
where “NI” is the number of iterations. The dimensions of the benchmark problems is chosen from the set: [1000, 5000, 10000, 50000, 100000]. The initial points for the benchmark problems are selected as follows: ,
,
,
,
,
,
. The specific formulations of the benchmark problems are as follows:
Problem 1:
and .
Problem 2:
and . Clearly, this problem has a unique solution
.
Problem 3:
and .
Problem 4;
and .
Problem 5:
and .
Problem 6:
and .
Problem 7:
and .
Problem 8:
and .
The experiment results are summarized in Tables 1–8, where “CPUT” represents the CPU time in second, “NFE” represents the number of function evaluations, “NI” represents the number of iterations, “NORM” represents the norm value of the function at the approximative point, and “Init(n)” represents the initial point and the dimension (multiplied by 1000). To provide a visual representation of the computational differences, we utilize the performance profiles introduced by Dolan and Moré [33]. These profiles allow us to assess the relative performance of the methods, where higher curves indicate better numerical efficiency. In Fig 1, the HCGP method demonstrates the lowest CPU time in nearly 75.71% of the benchmark problems, outperforming the HSDY and DCG methods, which achieve this in approximately 8.21%, 16.79% of cases, respectively. Similarly, in Fig 2, the HCGP method consistently shows the fewest number of function evaluations, excelling in almost 85.36% of the benchmark problems. In contrast, the HSDY and DCG methods reach this efficiency in 18.21% and 13.21% of cases, respectively. Lastly, Fig 3 illustrates that the HCGP method also requires the fewest number of iterations in 86.43% of the benchmark problems, while the HSDY and DCG methods perform better in only in 17.86% and 23.57% of cases, respectively.
Application on sparse signal restoration
In statistical inference and signal processing, solving linear systems of equations that yield sparse solutions poses a significant challenge. These systems are frequently ill-conditioned, which implies that they are highly sensitive to small perturbations in the input data. This characteristic can lead to instability in the solution, which makes it challenging to compute the solution accurately. To address this, a common approach is to formulate an objective function that combines a sparse -norm regularization term and a quadratic
-norm error term. This approach ensures that the solution remains sparse while minimizing the reconstruction error. The corresponding optimization problem can be expressed as follows:
where represents the observed signal,
is the unknown signal to be estimated, and
is a regularization parameter that controls the balance between the error and the sparsity of the solution. The matrix
defines a linear transformation.
To simplify the optimization, we introduce the decomposition:
Here, and
for all
, where
. By using this decomposition, the
-norm of v can be rewritten as:
where is a vector of ones. This leads to a reformulation of the original problem (20) as:
which is a bound-constrained quadratic program. The next step is to express this problem in a form that is more conducive to numerical solution. By defining , we can rewrite the problem (21) as:
where
Since H is positive semi-definite, the problem (22) represents a convex quadratic program, which guarantees the existence of a unique optimal solution. Xiao et al. [34] showed that this quadratic program can be transformed into the following simpler form:
They further demonstrated that is a continuous and monotone function, which provides useful properties for solving the optimization problem. Consequently, solving the original problem (20) is equivalent to solving the simpler problem (23), which can be expressed in the form of the problem (1). Thus, the proposed method efficiently solves the sparse signal restoration problem.
In this experiment, we compare the performance of the HCGP method against the HSDY and DCG methods. The goal is to restore a sparse signal of length n from an observed signal of length m. The mean squared error (MSE), used to evaluate the quality of the restoration, is defined as:
where and v represent the original signal and the recovered signal, respectively. The dimensions of the signal is set to n = 6144, m = 1536, and k = 192 non-zero elements are randomly selected. The matrix A is randomly generated in MATLAB, and
is the observed data where
is the Gaussian noise. The value η is obtained by the same continuation technique for the tested methods, i.e.,
. The initial point is selected as
and the numerical experiments terminate when the relative change between successive iterative falls below 10−5. The stopping criterion for all methods is defined as:
where fk denotes the function value f(v) at .
The sparse signal restored by all methods is shown in Fig 4, where it is clear that the HCGP, HSDY, and DCG methods are all capable of successfully restoring the original signal. In addition, Fig 5 illustrates the comparative performance of these methods in terms of convergence properties, including MSE, the CPU time (CPUT), the number of iterations (NI), and the values of the objective functions (ObjFun). To further assess their effectiveness, the experiment (as shown in Table 9) is repeated multiple times, allowing for a robust comparison of these methods. Specifically, in terms of NI and CPUT, the HCGP method outperforms the other two methods, requiring fewer iterations and less computational time to recover the signal.
The x-axis represents the number of iterations (top right and bottom right) and the CPU time in seconds (top left and bottom left). The y-axis represents the MSE (top left and top right) and the objection value (bottom left and bottom right
Conclusions
In this paper, we propose an extension of a hybrid conjugate gradient projection method for finding solutions of constrained nonlinear equations, leveraging hyperplane projection and hybrid techniques. The proposed method is distinguished by its low storage requirement and its exclusive reliance on function values, making it both efficient and practical for large-scale problems. The search direction designed in this method ensures the sufficient descent property, effectively eliminating the need for line search methods. Under suitable assumptions, we prove the global convergence of the proposed method. The experimental results, including those for large-scale constrained nonlinear equations and sparse signal restoration, demonstrate that the proposed method is numerically competitive and efficient, which outperforms similar methods from the literature in terms of both convergence rate and computational efficiency.
References
- 1. Dai Z, Zhou H, Wen F, He S. Efficient predictability of stock return volatility: The role of stock market implied volatility. The North American Journal of Economics and Finance. 2020;52:101174.
- 2. Figueiredo MáAT, Nowak RD, Wright SJ. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J Sel Top Signal Process. 2007;1(4):586–97.
- 3. Iusem NA, Solodov VM. Newton-type methods with generalized distances for constrained optimization. Optimization. 1997;41(3):257–78.
- 4. Boyd S. Distributed optimization and statistical learning via the alternating direction method of multipliers. FNT in Machine Learning. 2010;3(1):1–122.
- 5. Muhammad L, Waziri MY, Sulaiman IM, Moghrabi IAR, Sambas A. An improved preconditioned conjugate gradient method for unconstrained optimization problem with application in Robot arm control. Engineering Reports. 2024;6(12).
- 6. Ibrahim SM, Muhammad L, Yunus RB, Waziri MY, Kamaruddin SBA, Sambas A, et al. The global convergence of some self-scaling conjugate gradient methods for monotone nonlinear equations with application to 3DOF arm robot model. PLoS One. 2025;20(1):e0317318. pmid:39854395
- 7. Liu W, Jian J, Yin J. An inertial spectral conjugate gradient projection method for constrained nonlinear pseudo-monotone equations. Numer Algor. 2024;97(3):985–1015.
- 8. Salihu SB, Halilu AS, Abdullahi M, Ahmed K, Mehta P, Murtala S. An improved spectral conjugate gradient projection method for monotone nonlinear equations with application. J Appl Math Comput. 2024;70(4):3879–915.
- 9. Darvishi MT, Barati A. A third-order Newton-type method to solve systems of nonlinear equations. Applied Mathematics and Computation. 2007;187(2):630–5.
- 10. Shin B-C, Darvishi MT, Kim C-H. A comparison of the Newton–Krylov method with high order Newton-like methods to solve nonlinear systems. Applied Mathematics and Computation. 2010;217(7):3190–8.
- 11. Waziri M, Leong W, Hassan M, Monsi M. A new Newton’s method with diagonal Jacobian approximation for systems of nonlinear equations. Journal of mathematics and statistics. 2010;6(3):246–52.
- 12. Al-Baali M, Spedicato E, Maggioni F. Broyden’s quasi-Newton methods for a nonlinear system of equations and unconstrained optimization: a review and open problems. Optimization Methods and Software. 2013;29(5):937–54.
- 13. Waziri MY, June LW, Hassan MA. Diagonal Broyden-like method for large-scale systems of nonlinear equations. Malaysian Journal of Mathematical Sciences. 2012;6(1):59–73.
- 14. Ou Y, Lin H. A hybrid method for solving systems of nonsmooth equations with box constraints. Optim Lett. 2020;14(8):2355–77.
- 15.
Kanzow C, Fukushima M, Yamashita N. Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties. Inst. of Applied Math. Statistics. 2002.
- 16. Kumam P, Abubakar AB, Ibrahim AH, Kura HU, Panyanak B, Pakkaranang N. Another hybrid approach for solving monotone operator equations and application to signal processing. Math Methods in App Sciences. 2022;45(12):7897–922.
- 17. Kumam P, Abubakar AB, Malik M, Ibrahim AH, Pakkaranang N, Panyanak B. A hybrid HS-LS conjugate gradient algorithm for unconstrained optimization with applications in motion control and image recovery. Journal of Computational and Applied Mathematics. 2023;433:115304.
- 18. Waziri MY, Yusuf A, Abubakar AB. Improved conjugate gradient method for nonlinear system of equations. Comp Appl Math. 2020;39(4).
- 19. Sun M, Liu J. New hybrid conjugate gradient projection method for the convex constrained equations. Calcolo. 2015;53(3):399–411.
- 20. Yusuf A, Kiri AA, Lawal L, Kiri AI. A hybrid conjugate gradient algorithm for nonlinear system of equations through conjugacy condition. Artif Intell Appl. 2023; 1–8.
- 21. Danmalam KU, Mohammad H, Abubakar AB, Auwal AM. Hybrid algorithm for system of nonlinear monotone equations based on the convex combination of Fletcher-Reeves and a new conjugate residual parameters. Thai Journal of Mathematics. 2021;18(4):2093–106.
- 22. Koorapetse M, Kaelo P. An efficient hybrid conjugate gradient-based projection method for convex constrained nonlinear monotone equations. Journal of Interdisciplinary Mathematics. 2019;22(6):1031–50.
- 23. Li D, Wang S, Li Y, Wu J. A projection-based hybrid PRP-DY type conjugate gradient algorithm for constrained nonlinear equations with applications. Applied Numerical Mathematics. 2024;195:105–25.
- 24. Li D, Wu J, Li Y, Wang S. A modified spectral gradient projection-based algorithm for large-scale constrained nonlinear equations with applications in compressive sensing. Journal of Computational and Applied Mathematics. 2023;424:115006.
- 25. Abubakar AB, Kumam P, Mohammad H, Ibrahim AH, Kiri AI. A hybrid approach for finding approximate solutions to constrained nonlinear monotone operator equations with applications. Applied Numerical Mathematics. 2022;177:79–92.
- 26. Yin J, Jian J, Jiang X, Liu M, Wang L. A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer Algor. 2021;88(1):389–418.
- 27. Hager WW, Zhang H. A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J Optim. 2005;16(1):170–92.
- 28. Yu G, Guan L, Chen W. Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization. Optimization Methods and Software. 2008;23(2):275–93.
- 29. Li M, Feng H. A sufficient descent LS conjugate gradient method for unconstrained optimization problems. Applied Mathematics and Computation. 2011;218(5):1577–86.
- 30. Ibrahima AH, Muangchoob K, Mohamedc NS, Abubakard AB. Derivative-free SMR conjugate gradient method for constraint nonlinear equations. Journal of Mathematics and Computer Science. 2022;24(2):147–64.
- 31. Abubakar AB, Kumam P, Ibrahim AH, Rilwan J. Derivative-free HS-DY-type method for solving nonlinear equations and image restoration. Heliyon. 2020;6(11):e05400. pmid:33294653
- 32. Abubakar AB, Kumam P, Mohammad H, Awwal AM. An efficient conjugate gradient method for convex constrained monotone nonlinear equations with applications. Mathematics. 2019;7(9):767.
- 33. Dolan ED, Moré JJ. Benchmarking optimization software with performance profiles. Mathematical Programming. 2002;91(2):201–13.
- 34. Xiao Y, Wang Q, Hu Q. Non-smooth equations based method for -norm problems with applications to compressed sensing. Nonlinear Analysis: Theory, Methods & Applications. 2011;74(11):3570–7.