https://orcid.org/0000-0002-2836-3239
Figures
Abstract
In this paper, we present a modified nonmonotone line search algorithm that employs a variable parameter to control the degree of nonmonotonicity. This modification enhances both the probability of identifying the global minimum and the rate of convergence. Within the framework of alternating nonnegative least squares (ANLS), we propose a hybrid algorithm that employs either the modified nonmonotone projected Barzilai–Borwein method and the block coordinate descent method to address the subproblems in each iteration. To further accelerate convergence, we integrate a technique that allows for a larger step size. Under mild assumptions, we establish the global convergence of the algorithm. Numerical experiments conducted on both synthetic and real datasets demonstrate that the proposed algorithm is efficient for nonnegative matrix factorization (NMF) and outperforms other state-of-the-art methods.
Citation: Wu J, Li W, Su L, Wang H, Li Y (2026) Efficient hybrid algorithm for nonnegative matrix factorization based on modified nonmonotone linear search. PLoS One 21(7): e0344857. https://doi.org/10.1371/journal.pone.0344857
Editor: Jianchao Bai, Northwestern Polytechnical University, CHINA
Received: November 11, 2025; Accepted: February 26, 2026; Published: July 30, 2026
Copyright: © 2026 Wu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: This study was financially supported by the Shaanxi Provincial Natural Science Foundation Project in the form of a grant (2023-JC-QN-0088) received by JW. This study was also financially supported by the Major Science and Technology Special Project of Xinjiang Uygur Autonomous Region in the form of grants (2024A03007 & 2023A02002) received by LS. This study was also financially supported by the Water Special Project of Xinjiang Uygur Autonomous Region in the form of a grant (2025. B-001) received by LS. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Nonnegative Matrix Factorization (NMF) [1–5] is a prevalent and efficacious technique for the analysis of high-dimensional datasets. It facilitates the automatic extraction of sparse and significant features from a set of nonnegative, redundant data vectors. The standard NMF problem is formulated as follows: given an data matrix V with
and a pre-specified positive integer
, NMF seeks two nonnegative matrices
and
such that
where denotes the approximation error. A common formulation for solving NMF (1) is
where is the Frobenius norm. This model is particularly suitable for the problem (1) when the approximation error follows Gaussian noise, which is a common assumption in many practical applications [6].
The NMF problem (2), initially introduced by Paatero and Tapper [5], has attracted significant attention in recent years owing to its broad applicability in areas such as image processing [7], text mining [8], machine learning [9], and subsystem identification [10].
We now briefly review several well-established approaches for solving the NMF problem (2). Lee and Seung developed a multiplicative update (MU) algorithm [3] and showed its equivalence to a gradient descent method. However, a limitation of the MU algorithm is that it may not converge to a stationary point due to the possibility of division by zero in the update rules [11]. To address this, Pauca et al. [8] introduced a small positive constant into the update rules, leading to improved numerical performance.
The Alternating Nonnegative Least Squares (ANLS) framework [5] provides another major approach, with proven convergence to a stationary point of (11). Within this framework, several methods have been developed, such as the projected gradient method by Lin [12] and a quasi-Newton method by Kim et al. [2]. Other contributions include the alternating projected Barzilai–Borwein methods from Han et al. [13] and a projected Newton method by Gong and Zhang [1].
Despite the variety of methods available for solving NMF problem (2), many suffer from inefficiencies due to challenges in selecting line search step sizes or inverting Hessian matrices, especially for large-scale applications. Recently, several fast algorithms based on proximal point modifications have been proposed, such as NeNMF [14], quadratic regularization projected Barzilai–Borwein [15], and monotone projected Barzilai–Borwein (MPBB) [16]. Guan et al. [14] introduced NeNMF, which leverages Nesterov’s optimal gradient method (OGM) [17] without line search. However, Huang et al. [15] noted that OGM may require a large number of iterations to achieve a given tolerance, limiting the efficiency of NeNMF. The QRPBB method [15] utilizes the Lipschitz constant of the gradient to enhance performance and has been shown to outperform other solvers such as PG [12], APBB2 [13], and NeNMF [14]. Nevertheless, QRPBB involves a computationally expensive nonmonotone line search. To address this, the MPBB method [16] was proposed, which determines the step size without line search. However, our analysis indicates that MPBB still requires many iterations to meet a given tolerance, which can be prohibitive for large-scale problems. These limitations motivate the development of a more efficient algorithm for NMF.
In this paper, we introduce a modified nonmonotone technique based on a variable parameter that controls the degree of nonmonotonicity in the line search rule. We theoretically prove that the modified objective function forms a decreasing sequence, which aids in establishing global convergence under mild conditions. By incorporating this modified nonmonotone line search, we propose an efficient hybrid algorithm that employs either the projected gradient method or the block coordinate descent method to solve subproblems within the ANLS framework. The Barzilai–Borwein step size and a larger step size technique are utilized to accelerate convergence. We establish the global convergence of the proposed method. Numerical experiments on synthetic, image, and text datasets demonstrate the efficiency of our algorithm.
The remainder of this paper is organized as follows. Section 2 introduces the fast hybrid algorithm for NMF and presents its global convergence analysis. Section 3 provides experimental comparisons with other NMF algorithms. Finally, Section 4 concludes the paper.
2 A hybrid method for NMF and its global convergence
In this section, we first propose a modified nonmonotone line search technique, and then develop a fast hybrid method for NMF by incorporating this technique along with a strategy for enabling larger step sizes. We also establish the global convergence of the proposed algorithm.
Note that since the role of W and H is perfectly symmetric for the problem (2), therefore, we focus only on the update of the matrix W. Suppose that the matrices and
are fixed, we only need to solve the following problems:
To simplify the notation, will be denoted simply by f(W) in the following part of this paper.
It is easy to know that the objective function f(W) is convex [18–22] and the gradient of it is Lipschitz continuous by Lemma 1 and 2 in [14]. These properties are very important in our proof, which will be presented in following result.
Lemma 1 [14] The following two statements are valid.
- (i) The objective function f(W) of (3) is convex.
- (ii) The gradient
is Lipschitz continuous with constant .
Since cost function (3) is not strongly convex objective function, and thus one can only expect that the algorithms find a stationary point instead of the global or even local minimizer. To overcome this drawback, similar to [23], in this paper, we consider a proximal point modification version of cost function (3) by linearizing f(W) which is the form of
Obviously, according to (ii) of the Lemma 1, we can deduce that is strictly convex in W for any fixed
. At each iteration, we first calculate a point
by solving the following strongly convex quadratic minimization problem
Due to the strongly convex property of the objective function (4), the problem has a unique closed-form solution
where and the operator P[X] projects all the negative entries of X to zero.
In next section, we will put forward an efficient hybrid strategy to update by using
.
2.1 Main algorithm
In [15], the authors developed a nonmonotone proximal quadratic regularization projected alternating Barzilai-Borwein method for NMF, incorporating a nonmonotone technique and the Barzilai-Borwein step-size [24].
This nonmonotone term, adapted from [25], defines
where M is a predefined constant.
Because the maximum function may discard favorable function values from any iteration, the method’s numerical performance can depend significantly on the choice of M.
To overcome these limitations, Zhang and Hager proposed an alternative nonmonotone technique in [26], where a weighted average of previous function values replaces the maximum function value. This approach relaxes the Armijo-type line search condition as follows:
with
where for
and
. Clearly,
represents a convex combination of the preceding
and
, incorporating the complex terms
and
. This formulation implies that each
implicitly incorporates information from all previously known function values. In practice, however, updating
and
at every iteration becomes computationally burdensome. To address this limitation, the following expression for
[27] was proposed:
using either a fixed or a variable
.
An analysis of (8) reveals that it fails to fully utilize the current well function value. Notably, the nonmonotone strategy exhibits superior performance by incorporating the maximum function—this can be attributed to the fact that the maximum function represents one of the most prominent pieces of information derived from recent successful iterations, a factor that should not be overlooked. Consequently, we define
with , m(0)=0, and
where . As with M in (7), the selection of
critically influences the degree of nonmonotonicity (see [27]). To enhance the efficiency of the nonmonotone line search, Ahookhosh et al. [28] introduced a varying
determined by a simple formula.
Subsequently, Nosratipour et al. [29] argued that should relate to a suitable measure of the distance to the optimal solution, defining it as
However, if the iterative sequence becomes trapped in a narrow curved valley,
may approach zero, forcing
to zero and reducing the nonmonotone line search to the standard Armijo method, which is inefficient due to very short or zigzagging steps. To address this limitation, we propose the following
:
When the function value exhibits a rapid decrease, the absolute difference increases significantly. Accordingly, this induces an elevation in
and enhances the prominence of the nonmonotone strategy. Conversely, as
converges toward the optimal solution,
tends to zero. This subsequent reduction in the absolute difference drives
to approach zero, with the nonmonotone strategy gradually losing its efficacy—thereby ultimately transitioning into a monotone strategy.
We then modify the Armijo-type line search by replacing [27] with
:
Nonmonotone line search methods are known to be efficient for large-scale optimization problems [26–29]. However, a direct application to NMF does not guarantee convergence or the existence of an optimal step size, since the theoretical analysis requires at each iteration (see Lemma 7 and Theorem 1). A key challenge is therefore how to apply the modified nonmonotone line search technique to (4). Numerical experiments indicate that the modified nonmonotone line search performs well provided
and
hold. From these conditions and the definition of
, it follows that
(see Lemma 6). It is therefore reasonable to employ the modified nonmonotone line search for solving (4) when
and
; otherwise, we use the block coordinate descent method [30] to solve the NMF.
Based on this discussion, we propose a hybrid strategy for updating , as detailed in Algorithm 1. The same procedure can be applied to solve for
.
Algorithm 1. An efficient hybrid method for negative matrix factorization (HMNMF)
1. Initialize , M > 0,
,
,
,
,
,
and
. Set t = 1.
2. If , stop.
3. Compute
(14)
4. Compute by (10), and compute
by (9).
5. If , set
and go to Step 6. Otherwise, compute
which is the direction with be the BB stepsize, and let
and
are the smallest nonnegative integer m respectively satisfying
(15)
with and
(16)
where . Calculate
.
6. Calculate and
. If
, set
; otherwise, set
.
7. Set t = t + 1 and go to step 2.
For ease of reference, we define two index sets:
Here, P denotes the index set for the total number of sub-iterations used to solve (4).
Remark 1 Equation (9) indicates that the sequence produced by Algorithm 1 can occasionally increase. This behavior contrasts with the strictly nonincreasing sequences established for the methods in [27,28]. The present nonmonotone condition on
is also simpler to satisfy than its counterpart in those references. Moreover, the incorporation of a nonmonotone
into the development of a new algorithm for solving (4) necessitates a different approach to establishing convergence.
In [31], Nocedal observed that step sizes in nonmonotone line search methods can deviate unpredictably from unity, varying above or below one depending on problem scaling. Following this insight, and analogous to the strategy of employing larger step sizes for updating Lagrange multipliers in [32], we introduce a relaxation factor to update
as
The parameter is empirically selected from the interval (1, 2).
The following lemma 2 is central to our global convergence analysis.
Lemma 2 For all ,
,
- (i)
,
- (ii) Z is a stationary point of (3) if and only if
- (iii)
Proof. The proof of (i) and (ii) are similar to the proof of Lemma 2.1 in [33] and therefore it is omitted here.
- (iii) By use of the Cauchy equality, from (i), we obtain (iii). □
The following lemma is drown from Lemma 3 [15].
Lemma 3 [15] Assume that the sequences and
are generated by Algorithm 1, then we have
To demonstrate that Algorithm 1 is well-defined, we establish the following result.
Lemma 4 For a given iterate and a search direction
at
, there exists a step size
such that for all
, the inequality
holds, where .
Proof. Assume no step size satisfies inequality (19). Then for all sufficiently large positive integers m, we have
By the mean value theorem, there exists such that
which simplifies to
Taking the limit as yields
Given , this implies
, contradicting the condition
. □
The following lemma implies that is monotonically decreasing for all
when
.
Lemma 5 Suppose the sequence for
is generated by Algorithm 1; if
holds for all
, then
.
Proof. For , Lemma 3 implies that
By the definition of , we obtain
The inequality follows from (20). From (21) and the definition of , it follows that
, which establishes the result.
Next, we assume that for all i with , the inequality
holds; then, by the definition of
, it follows that
. If
, the definition of
yields
, which implies
From the definition of together with (22), we obtain
For , the definition of
and Lemma 3 give
Here, the final equality relies on (23). By the definition of , we conclude that
. □
Next, we demonstrate a key property of the modified line search. For a given W0 > 0, the level set is defined as
Lemma 6 Suppose the sequences and
are generated by Algorithm 1; then
, and
is a decreasing sequence.
Proof. We first show that is decreasing by considering two cases.
Case I: For with
, the inequality
together with the definition of
and equation (34) implies
Case II: For , the definition of
gives
If , Algorithm 1 ensures
, and since
by definition, we have
Substituting inequality (26) into (25) yields
where the last step uses the definition of . Hence,
If for
and
, then from Algorithm 1,
Combining this with (22) and (23) gives , and as in the proof of Lemma 5, we obtain
, so
by definition. Otherwise, if
for all
, Lemma 5 again implies
.
The sequence is therefore monotonically decreasing. We next demonstrate that
and
lie within
. Since
holds by definition, we proceed by induction to show that
for all
. Assuming
for
, we verify that
. This result is established by considering the following two cases:
Case I: For , where
, we apply (35) and Lemma 2 to obtain
Combining inequality (30) with Lemma 6 yields
Therefore, the point lies within
.
Case II: When , we use (26), (29), (22), (23) and Lemma 5 to derive
Algorithm 1 gives This equality, combined with (32) and Lemma 6, leads to
Consequently, the point also belongs to
. Thus, the entire sequence
remains within
. Lemma 3 further confirms that
. □
The following corollary establishes the convergence of the sequence .
Corollary 1 Suppose that the level set is bounded; then the sequence
converges.
Proof. From (31), (33), and Lemma 6, it follows that there exists such that for all
,
which implies that the sequence is bounded below. Therefore,
converges. □
We now establish the well-definiteness of the line search.
Theorem 1 Suppose the sequences and
are generated by Algorithm 1. Then Algorithm 1 is well-defined.
Proof. The proof is straightforward for all , so it suffices to show that
and
are well-defined for all . We proceed by induction. The base case t = 0 is easily verified. Now assume that for some
and all i with
, inequalities (34) and (35) hold.
From Algorithm 1, we have . Given that
, it follows that
. For i = t + 1, Lemma 4 guarantees the existence of a step size
such that (19) holds. According to Lemma 3, we derive
The final inequality is equivalent to (34), confirming that (34) is well defined. From (19), we observe that satisfies
This last inequality ensures that (35) is well defined. Consequently, Algorithm 1 is well-defined. □
We now introduce a key property of the sequence , which is essential for establishing global convergence.
Theorem 2 Assume that the sequences and
are generated by Algorithm 1; then,
where and
. Furthermore,
proof From the definition of and Lemma 6, we obtain
By induction,
for . Because
, the desired result (36) follows. We now prove (37). Since f is bounded below and
is a convex combination of certain values of
, the sequence
is also bounded below. Combining the inequalities in (36) with Corollary 1 yields
This completes the proof.
2.2 Convergence analysis
This section analyzes the convergence properties of the proposed algorithm, establishing its global convergence. To prove the main result, we first derive a lower bound for the step length .
Lemma 7 Assume is a step size generated by Algorithm 1; if
is not a stationary point of (3), then for all
, there exists a constant
such that
.
Proof. For the obtained step size , if
does not satisfy (35), that is,
Thus,
Similar to Lemma 4 in [15], we obtain
Inequality (38) is therefore satisfied. To establish global convergence, we now prove the following result.
Lemma 8 Assume that the sequences and
are generated by Algorithm 1. Then
Moreover,
Proof. Algorithm 1 ensures that the step size satisfies for all
. Using Lemma 2(iii), we obtain
The Lipschitz continuity of implies
Therefore, inequality (39) is established.
We now demonstrate the validity of equality (40).
Using Theorem 2, it follows that
Thus
We next establish the following global convergence result for our HMNMF method. □
Theorem 3 Suppose that the level set is bounded; then any accumulation point of the sequence
generated by Algorithm 1 is a stationary point of (3).
Proof. We consider two cases.
Case I: For , Lemma 8 implies that
so any accumulation point of is a stationary point of (3).
Case II: For , the boundedness of
together with Lemma 8 and Lemma 2(ii) shows that
has an accumulation point
that is a stationary point of (3). We now prove Theorem 3 by contradiction. Assume
is not a stationary point of (3). Then Lemma 2(ii) gives
for all
. By continuity and compactness, there exists
such that
for
. Hence, for sufficiently large t,
. Using Lemmas 2, 3, 6 and 7, we derive
which yields
This contradicts the property of the norm, so must be a stationary point of (3).
By Theorem 3, Lemma 2, and Lemma 3, the following key convergence result is established.
Theorem 4 If the level set is bounded, then every accumulation point of the sequence
produced by Algorithm 1 constitutes a stationary point of (3).
3 Numerical experiments
This section presents numerical experiments evaluating our algorithm’s performance against six methods: NeNMF [14], the projected BB method (APBB2 [13]) (The code is available at http://homepages.umflint.edu/~lxhan/software.html), QRPBB [15], MPBB [16], hierarchical alternating least squares (HALS) [34], and the block coordinate descent (BCD) method [30]. We compared these methods using both synthetic datasets and real-world problems from the Yale image database (http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html), the CBCL image database http://cbcl.mit.edu/software-datasets/FaceData2.html, and the Reuters-21578 and TDT2 text corpora (Both Reuters-21578 corpus and TDT-2 corpus in MATLAB format are available at http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html). All code was implemented in MATLAB. The experiments ran on a Lenovo laptop with an Intel Core(TM) i7 CPU at 4.9 GHz and 16 GB of memory, using Windows 10 and MATLAB R2021a.
3.1 Stopping criterion
By the KKT conditions for constrained optimization, (W,H) is stationary point of NMF (2) if and only if it satisfies and
, where
and is defined in the same way. Thus, we use the following stopping criterion as Lin [12] in the numerical experimentation:
where is a tolerance.
3.2 Synthetic data
We first evaluate the performance of HMNMF and the other algorithms on synthetic datasets. The matrix V in this test is exactly low-rank and is constructed as V = LR, where L and R are generated using the MATLAB commands max(0,randn(m,r)) and max(0,randn(r,n)), respectively.
The parameters of our method are set as follows throughout all experiments: ,
,
,
, and
. These settings are consistent with those used in APBB2, QRPBB, and MPBB. The tolerance for all algorithms is set to tol = 10−8. For HMNMF, we choose
and update
according to equation (12). The maximum number of iterations (maxit) is set to 50,000 for all algorithms. All other parameters for APBB2, NeNMF, QRPBB, and MPBB are set to their default values.
For each test dataset with independent Gaussian noise at a signal-to-noise ratio of 60 dB, we use 10 different randomly generated initial points. The average results are summarized in Table 1. Here, “iter” denotes the number of iterations required to satisfy the termination criterion (44), and “niter” represents the total number of sub-iterations for solving subproblems (3) and (4). Performance is measured using the relative error , the final projected gradient norm
, and the CPU time (in seconds). As shown in Table 1, HMNMF achieves the shortest execution time and the smallest number of sub-iterations among all algorithms, particularly for large-scale datasets. These results indicate that HMNMF significantly outperforms the other four solvers across all test cases. This improvement can be attributed to two factors: first, the modified objective function in the nonmonotone line search
may enhance efficiency; second, the introduction of the relaxation factor
in the update rules allows larger step sizes, thereby reducing computational time.
With the close association between HMNMF, QRPBB, and MPBB, and considering HALS and BCD as some of the most efficient NMF methods, we further compare the performance of HMNMF, QRPBB, MPBB, HALS, and BCD. Figs 1–3 present comparisons of these five methods on eight randomly generated datasets with independent Gaussian noise at a signal-to-noise ratio of 30 dB. The algorithms are terminated when the stopping criterion in (44) is satisfied with or when the iteration count exceeds 30. The results clearly demonstrate that HMNMF requires the fewest iterations and the least CPU time to reach the specified tolerance in most cases, especially for high-dimensional datasets. Moreover, for high-dimensional data, the PBB-based methods outperform HALS and BCD, which can be explained by the fact that PBB-based methods implicitly utilize second-order information of the cost function, whereas HALS and BCD rely on first-order approximations to search for stationary points.
3.3 Image data
The Yale face database, created by the Yale Center for Computational Vision and Control, contains 165 grayscale images of 15 individuals, with 11 images per person. The images exhibit variations in lighting conditions (left-light, center-light, right-light), facial expressions (normal, happy, sad, surprised, wink), and the presence or absence of glasses. The data matrix V, with rows representing images, has dimensions 165 × 4096.
The CBCL image database comprises 2,429 facial images, each with 19×19 pixels, resulting in a matrix of size 361×2,429.
For each database, we perform 10 trials with different randomly generated initial points, using in (44) and a maximum of 50,000 iterations for all algorithms. The average results are presented in Table 2. It can be observed that for both databases, HMNMF converges in fewer iterations and less CPU time, while achieving residual values comparable to those of other solvers.
3.4 Text data
The Reuters-21578 corpus includes 21,578 documents distributed across 135 categories. Documents with multiple category labels were discarded, leaving 8,293 documents in 65 categories. After preprocessing, the corpus contains 18,933 distinct terms, represented by a matrix of size .
The TDT2 corpus consists of news articles from sources such as ABC, CNN, VOA, NYT, PRI, and APW from 1998. It contains 11,201 on-topic documents classified into 96 semantic categories. After removing documents appearing in two or more categories and retaining only the largest 30 categories, we obtain 9,394 documents, represented by a matrix of size .
As in the previous subsection, we set , and report average results over 10 random initializations in Table 3. The results show that the CPU time required by HMNMF is significantly less than that of NeNMF, APBB2, QRPBB, and MPBB. Meanwhile, the projected gradient norms obtained by APBB2, QRPBB, MPBB, and HMNMF are similar and smaller than those of NeNMF, which confirms the effectiveness of our proposed HMNMF method.
3.5 Influence of parameters 
and 
In this section, we first intend to investigate what is the impact for our modified nonmonotone line search rules. In particular, how does the definition of in (10) improve the numerical efficiency of algorithms? To do so, we shall compare our
with the other two
, proposed in [28,29]. For simplicity of statement, we introduce the following notations. HMNMF-1:Algorithm 1 with the
defined in [28]; HMNMF-2:Algorithm 1 with the
defined in [29]; HMNMF:Algorithm 1 with the
defined by (12). Fig 4 shows the relative residual value versus the running time on synthetic datas which are the same as those in Section 3.1. The maximum number of iterations (maxit) is set to be 20 for all algorithms, and all of the other parameters for experimental settings here are the same as those in Section 3.1. From Fig 3, it is clear that our
is superior to the other two
in [28,29].
Next we report that how does the adjustability of the relaxation factor s in the update rule of can improve the numerical efficiency of algorithms? In fact, we run the HMNMF with the following different s: s = 0.08, 0.48, 0.88, 1.0, 1.48, 1.88, 1.98 on synthetic datas which are the same as those in Section 3.1. Fig 5 shows the relative residual value versus the running time. The maximum number of iterations (maxit) is set to be 30 for all algorithms, and all of the other parameters for experimental settings here are the same as those in Section 3.1. From Fig 5, we can see that the relaxation factor s fails to accelerate the convergence when s < 1 and increasing constant s significantly accelerates the convergence when 1 < s < 2. As for NMPBB, it is seem that s = 1.88 is much better value than the other values in term of convergence rate, thus we use s = 1.88 for our HMNMF in all experiments.
4 Conclusion
This paper introduces a fast prox-linear variant of a hybrid algorithm for solving NMF within the ANLS framework; the algorithm employs either a modified nonmonotone projected Barzilai-Borwein gradient method or a block coordinate descent method to address the subproblems in each iteration. A modified line search rule incorporating a nonmonotone technique [25,27] is also integrated into the algorithm to enhance its efficiency. Under mild assumptions, the global convergence of the algorithm is established. Numerical experiments demonstrate the effectiveness of the proposed algorithm for NMF.
References
- 1. Gong P, Zhang C. Efficient Nonnegative Matrix Factorization via projected Newton method. Pattern Recognition. 2012;45(9):3557–65.
- 2.
Kim D, Sra S, Dhillon IS. Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem. In: Proceedings of the 2007 SIAM International Conference on Data Mining. 2007. p. 343–54.
- 3. Lee DD, Seung HS. Learning the parts of objects by non-negative matrix factorization. Nature. 1999;401(6755):788–91. pmid:10548103
- 4. Lee DD, Seung HS. Algorithms for non-negative matrix factorization. Adv Neural Inform Process Syst. 2001;13:556–62.
- 5. Paatero P, Tapper U. Positive matrix factorization: A non‐negative factor model with optimal utilization of error estimates of data values. Environmetrics. 1994;5(2):111–26.
- 6. Gillis N. The why and how of nonnegative matrix factorization. Int J Syst Sci. 2014.
- 7.
Guillamet D, Schiele B, Vitrià J. Analyzing non-negative matrix factorization for image classification. In: 16th International Conference on Pattern Recognition (ICPR). 2002. p. 116–9.
- 8.
Pauca VP, Shahnaz F, Berry MW, Plemmons RJ. Text mining using nonnegative matrix factorization. In: Proceedings of the Fourth SIAM International Conference on Data Mining. 2004. p. 452–6.
- 9.
Hualiang Li, Adali T, Wei W, Emge D. Non-Negative Matrix Factorization with Orthogonality Constraints for Chemical Agent Detection in Raman Spectra. In: 2005 IEEE Workshop on Machine Learning for Signal Processing. 2005. p. 253–8.
- 10. Kim PM, Tidor B. Subsystem identification through dimensionality reduction of large-scale gene expression data. Genome Res. 2003;13(7):1706–18. pmid:12840046
- 11. Shahnaz F, Berry MW, Pauca VP, Plemmons RJ. Document clustering using nonnegative matrix factorization. Inform Process Manag. 2006;42(2):373–86.
- 12. Lin C-J. Projected gradient methods for nonnegative matrix factorization. Neural Comput. 2007;19(10):2756–79. pmid:17716011
- 13. Han L, Neumann M, Prasad U. Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. ETNA. 2009;36:54–82.
- 14. Guan N, Tao D, Luo Z, Yuan B. NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization. IEEE Trans Signal Process. 2012;60(6):2882–98.
- 15. Huang Y, Liu H, Zhou S. Quadratic regularization projected Barzilai–Borwein method for nonnegative matrix factorization. Data Min Knowl Disc. 2014;29(6):1665–84.
- 16. Huang Y, Liu H, Zhou S. Quadratic regularization projected Barzilai–Borwein method for nonnegative matrix factorization. Data Min Knowl Disc. 2014;29(6):1665–84.
- 17. Nesterov Y. A method of solving a convex programming problem with convergence rate O(1/k2). Soviet Mathematics Doklady. 1983;27(1):372–6.
- 18. Bai J-C, Bian F-M, Chang X-K, Du L. Accelerated Stochastic Peaceman–Rachford Method for Empirical Risk Minimization. J Oper Res Soc China. 2023;11(4):783–807.
- 19. Bai J, Han D, Sun H, Zhang H. Convergence on a Symmetric Accelerated Stochastic ADMM with Larger Stepsizes. CSIAM-AM. 2022;3(3):448–79.
- 20. Bai J, Hager WW, Zhang H. An inexact accelerated stochastic ADMM for separable convex optimization. Comput Optim Appl. 2022;81(2):479–518.
- 21. Bai J, Li J, Xu F, Zhang H. Generalized symmetric ADMM for separable convex optimization. Comput Optim Appl. 2017;70(1):129–70.
- 22. Bai J, Zhang H, Li J. A parameterized proximal point algorithm for separable convex optimization. Optim Lett. 2017;12(7):1589–608.
- 23. Fukushima M, Mine H. A generalized proximal point algorithm for certain non-convex minimization problems. Int J Syst Sci. 1981;12(8):989–1000.
- 24. Barzilai J, Borwein JM. Two-Point Step Size Gradient Methods. IMA J Numer Anal. 1988;8(1):141–8.
- 25. Grippo L, Lampariello F, Lucidi S. A Nonmonotone Line Search Technique for Newton’s Method. SIAM J Numer Anal. 1986;23(4):707–16.
- 26. Zhang H, Hager WW. A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization. SIAM J Optim. 2004;14(4):1043–56.
- 27. Gu N, Mo J. Incorporating nonmonotone strategies into the trust region method for unconstrained optimization. Comput Math Appl. 2008;55(9):2158–72.
- 28. Ahookhosh M, Amini K, Bahrami S. A class of nonmonotone Armijo-type line search method for unconstrained optimization. Optimization. 2012;61(4):387–404.
- 29. Nosratipour H, Hashemi Borzabadi A, Solaymani Fard O. On the nonmonotonicity degree of nonmonotone line searches. Calcolo. 2017;54(4):1217–42.
- 30. Xu Y, Yin W. A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion. SIAM J Imaging Sci. 2013;6(3):1758–89.
- 31.
Nocedal J. Conjugate gradient methods and nonlinear optimization. In: Adams L, Nazareth JL, eds. Linear and Nonlinear Conjugate Gradient-Related Methods. SIAM; 1996. p. 9–23.
- 32.
Glowinski R. Numerical Methods for Nonlinear Variational Problems. New York, State of New York: Springer-Verlag; 1984.
- 33. Birgin EG, Martínez JM, Raydan M. Nonmonotone Spectral Projected Gradient Methods on Convex Sets. SIAM J Optim. 2000;10(4):1196–211.
- 34.
Cichocki A, Zdunek R, Amari S. Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. Lecture Notes in Computer Science. Springer; 2007. p. 169–76.