Semi-Supervised Projective Non-Negative Matrix Factorization for Cancer Classification

Xiang Zhang; Naiyang Guan; Zhilong Jia; Xiaogang Qiu; Zhigang Luo

doi:10.1371/journal.pone.0138814

Abstract

Advances in DNA microarray technologies have made gene expression profiles a significant candidate in identifying different types of cancers. Traditional learning-based cancer identification methods utilize labeled samples to train a classifier, but they are inconvenient for practical application because labels are quite expensive in the clinical cancer research community. This paper proposes a semi-supervised projective non-negative matrix factorization method (Semi-PNMF) to learn an effective classifier from both labeled and unlabeled samples, thus boosting subsequent cancer classification performance. In particular, Semi-PNMF jointly learns a non-negative subspace from concatenated labeled and unlabeled samples and indicates classes by the positions of the maximum entries of their coefficients. Because Semi-PNMF incorporates statistical information from the large volume of unlabeled samples in the learned subspace, it can learn more representative subspaces and boost classification performance. We developed a multiplicative update rule (MUR) to optimize Semi-PNMF and proved its convergence. The experimental results of cancer classification for two multiclass cancer gene expression profile datasets show that Semi-PNMF outperforms the representative methods.

Citation: Zhang X, Guan N, Jia Z, Qiu X, Luo Z (2015) Semi-Supervised Projective Non-Negative Matrix Factorization for Cancer Classification. PLoS ONE 10(9): e0138814. https://doi.org/10.1371/journal.pone.0138814

Editor: Ramin Homayouni, University of Memphis, UNITED STATES

Received: March 4, 2015; Accepted: September 3, 2015; Published: September 22, 2015

Copyright: © 2015 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files. All relevant data are available on the Zenodo repository, and the access URL is https://zenodo.org/record/21712.

Funding: This work was supported by Ministry of Education of the People’s Republic of China Grant Number 20134307110017 (www.moe.edu.cn), PI: Zhigang Luo and National Natural Science Foundation of China Grant Number 91024030/G03 (http://www.nsfc.gov.cn/), PI: Xiaogang Qiu.

Competing interests: The authors have declared that no competing interests exist.

Introduction

In cancer prognosis and treatment, it is crucial to identify different cancer types and subtypes. Traditional methods often rely on similar morphological appearances but easily induce different clinical courses and responses to therapy because of subjective interpretations and personal experience. This usually results in diagnostic confusion. Fortunately, the emergence of the DNA microarray technique removes this barrier in an objective and systematic manner and has showed great potential in outcome prediction of cancer types in genome-wide scales [1–11].

Numerous learning methods have been developed for cancer classification based on gene expression profiles [1–3]. For instance, Golub et al. [1] used a weighted voting scheme for the molecular classification of acute leukemia. Nguyen et al. [3] incorporated partial least squares (PLS) into the logistic discrimination and quadratic discriminant analysis for tumor classification. However, these methods are not convenient for practical applications because the labeled samples are quite expensive in the clinical cancer research community. To overcome this deficiency, Xu et al. [12] used the semi-supervised Ellipsoid ARTMAP (ssEAM) method for cancer classification. Shi et al. [13] utilized the semi-supervised method termed low density separation (LDS, [14]) to classify different types of cancers. Moreover, Maulik et al. [15] investigated the effectiveness of transductive SVM (TSVM, [16]) in cancer classification. Nevertheless, these algorithmic challenges involve the curse of dimensionality, which indicates that the overwhelming number of measures for gene expression levels contrast with the small number of samples.

This problem often calls for dimension reduction techniques. This paper focuses on non-negative matrix factorization (NMF, [17, 18]) because it is a flexible framework for conducting dimension reduction and performing classification and clustering tasks [19–26]. NMF decomposes a data matrix into the product of two non-negative factors. Due to its effectiveness, NMF and its variants have been applied to analyzing large-scale gene expression datasets [27–29], cancer classification [30, 31] and new class discovery [30]. Brunet et al. [31] originally adopted NMF to uncover molecular meta-patterns by clustering samples of leukemia, medulloblastoma and central nervous system tumors, and indicating that NMF outperforms both hierarchy clustering (HC) and self-organizing map (SOM). However, NMF does not explicitly guarantee the sparseness of the decomposition and violates the uniqueness property. Recent works [32] show that this often degrades the clustering performance. To address this issue, Li et al. [32] proposed local NMF (LNMF) to overcome this deficiency by imposing the sparse constraints over the decomposition. Hoyer et al. proposed sparse NMF (SNMF, [33]) to enforce sparseness in NMF by penalizing the number of non-zero entries of the coefficients rather than the sum of the entries. Furthermore, Gao et al. [34] utilized SNMF to identify the meta-patterns of various cancers for identifying different types of tumors.

Because the aforementioned methods follow regularization theory, they are jointly non-convex and are difficult to optimize. Unlike the above methods, Yuan et al. [35] developed the projective NMF (PNMF) to induce parts-based representation by implicitly imposing the orthogonal constraint over the basis. However, because these methods are unsupervised learning methods that do not take into account labels, their performance in cancer classification can be further improved. In this paper, we propose a semi-supervised projective NMF method (Semi-PNMF) that utilizes both labeled and unlabeled samples to boost classification performance. Particularly, Semi-PNMF learns a non-negative subspace from concatenated labeled and unlabeled samples and predicts classes by the index of the largest entries of their coefficients. Benefiting from the unlabeled data, Semi-PNMF can learn more representative subspaces, which are beneficial for classification tasks. We explored a multiplicative update rule (MUR) to solve Semi-PNMF and proved its convergence. The experimental results of cancer identification for multiclass cancer gene expression profile datasets including GCM [8] and Acute Leukemia [36] datasets show that Semi-PNMF outperforms the representative methods in terms of quantity.

Materials and Methods

Semi-supervised Projective Nonnegative Matrix Factorization

Projective non-negative matrix factorization (PNMF) learns a non-negative projection matrix to project high-dimensional data into the lower-dimensional subspace. Because it can learn parts-based representation, PNMF has been widely applied in pattern recognition [21, 26, 35, 37]. Here, we introduce the other representation form of PNMF that learns the lower-dimensional coefficients of samples to approximate the class indicator for clustering. This is based on the assumption that the basis lies in the subspace spanned by the original samples. Given the data matrix V = [v₁,⋯,v_n]^T ∈ R^{n × m}, where n denotes the number of samples and m their dimensionality, PNMF learns the coefficients H ∈ R^{n × r} to represent original samples, i.e., (1) where ∥•∥_F denotes the matrix Frobenius norm and r the number of clusters.

As in objective (1), it is non-trivial to analyze the convergence in theory because Eq (1) contains a fourth-order term. To remove such a high order term, we first introduce an auxiliary variable, i.e., the cluster centroids, and the equality constraint into Eq (1). Thus, we can obtain (2)

The objective is very similar to BPNMF [26], but we cannot directly apply the optimization algorithm of BPNMF to optimize it especially when additional constraints such as the sparseness constraint and Laplacian regularization are imposed over the coefficients, as these constraints easily induce PNMF to produce the trivial solution. To avoid such a drawback, we propose a semi-supervised PNMF method (Semi-PNMF) by recasting Eq (2) as (3) where α ≥ 0 is a regularization constant and W denotes the non-negative cluster centroid. Model (3) significantly differs from BPNMF because Eq (3) favors the representative capacity of the cluster centroids, while BPNMF focuses on the orthogonality of the non-negative subspace. Thus, Eq (3) induces the sparse coefficients, while BPNMF produces the sparse basis.

According to Eq (3), we can incorporate the local coordinate constraint [38] to improve the representative power of the basis, meanwhile further inducing the sparse coefficients to be true classes. Thus, we recast Eq (3) as the following regularization form: (4) where β trades off the local coordinate regularization and H^ij denotes the i-the row and j-th column element of coefficients H, W^j and Vⁱ, signifying the i-th and j-th row vector of W and V, respectively.

To make full use of partial labeled samples, we propagate the labels of labeled samples to unlabeled ones by minimizing the distance between their coefficients and the corresponding class indicator. Particularly, we require the coefficients of labeled samples to be equivalent with the corresponding class indicator. Consider the first d examples labeled and the rest unlabeled; the data matrix V can be divided into two parts, i.e., . Then, we can obtain the objective function of Semi-PNMF as follows: (5) where Q denotes the partial label matrix wherein Q_ij = 1 if v_i belongs to the j-th class; otherwise, Q_ij = 0. Both H_U and n_U denote the coefficients and number of the unlabeled samples, respectively.

Interestingly, Semi-PNMF has two distinct aspects. First, it replaces the learned coefficients of the labeled samples with the corresponding class indicator. The constraint is so strong that the learned basis completely biases the labeled samples. This might induce the trivial solution to the coefficients of the unlabeled samples. Second, Semi-PNMF completely ignores the representation contribution of the labeled samples. It is so unintelligible that the learned basis only favors the unlabeled samples. It appeared that both aspects contradict each other, but intrinsically, they mutually complement each other in our Semi-PNMF. In essence, the first aspect corresponds to supervised learning, which generates the reasonable solution yet does not ensure it is consistent with the underlying data distribution, while the second one considers data distribution but cannot yield the reasonable solution. Thus, the combination of both aspects can mutually complement each other. Semi-PNMF learns the shared basis by the labeled and unlabeled instances, meanwhile inducing similar instances to have a similar representation, i.e., the coefficients. Because we impose the restriction that coefficients of the labeled samples be their labels as well as the local coordinate constraint over the basis and coefficients, the unlabeled sample coefficients are implicitly as sparse as the label vectors. In this way, Semi-PNMF effectively propagates the labels of labeled samples to the unlabeled ones. Consequently, in cancer classification, it is reasonable that, for each unlabeled sample, we choose the index of the largest entry of its coefficient to predict the classes of this sample once objective (5) yields their coefficients. The above intuition can be further verified by the toy example given in Figs 1 and 2.

Download:

Fig 1. The toy example illustrating (a) the synthetic 3D original data including the labeled and unlabeled samples and the ground-truth labels, (b) the labeled results of unlabeled samples, (c) the learned coefficients of the unlabeled samples, and (d) the learned basis by Semi-PNMF.

In Fig (a), both the square and circle markers signify the unlabeled and labeled samples, respectively, and three different colors stand for three different categories. Fig (b) shows that the unlabeled samples are marked as the ground-truth markers and colors. Figs (c) and (d) shows the coefficients and basis learned by Semi-PNMF, respectively. The index of maximum value of the coefficient for an unlabeled sample appears in red and indicates its class.

https://doi.org/10.1371/journal.pone.0138814.g001

Download:

Fig 2. The toy example illustrating the labeling results obtained from the coefficients when the propagation procedure arrives at the (a) initialization stage, (b) 50-th iteration round, (c) 300-th iteration round, and (d) the resultant convergence (1500-th iteration round), respectively.

https://doi.org/10.1371/journal.pone.0138814.g002

Optimization Algorithm

It is difficult to optimize Eq (5) because it is jointly non-convex with respect to both W and H. Fortunately, it is convex with respect to W and H, respectively. Thus, we can establish the following theorem:

Theorem 1: The objective function (5) is non-increasing under the following multiplicative update rules: (6) , and (7) where ⊗ denotes the element-wise product operator, F_U = diag(sum(H_U)), A = [a,⋯,a] wherein , and B = [b,⋯,b], wherein b = diag(WW^T).

Proof. According to Eq (5), we can obtain the objective with respect to W as follows: (8) where denotes the diagonal matrix whose diagonal elements are the i-th row vector values of V_U.

By Eq (8), we can define the auxiliary function of J(W) as (9)

Obviously, objective (9) has (10)

We can obtain the derivative of Eq (9) as follows: (11) Based on Eq (11), we have (12)

By simple algebra, the formula (6) can be deduced from Eq (12). Likewise, we can obtain the auxiliary function of J(H_U) as follows: (13)

Setting , we have (14)

Thus, according to Eq (14), we also obtain the update rule (7) for H_U.

Moreover, according to Eqs (10), (12) and (14), we have (15)

Based on Eq (15), these update rules always guarantee that the objective function monotonically decreases. Thus, this completes the proof. ■

According to the above theorem, we summarize the multiplicative update rule (MUR) for Semi-PNMF in Algorithm 1.

Algorithm 1 MUR for Semi-PNMF

Input: Examples V ∈ R^{m × n}, penalty parameter α, partial label matrix Q.

Output: H_U.

1: Randomly initialize W⁰ and , and l = 0.

2: repeat

3: Update W_l+1 according to Eq (6).

4: Calculate according to Eq (7).

5: l ← l+1.

6: until {Stopping criterion is satisfied.}

7: .

To reduce the time overhead, Algorithm 1 utilizes the objective relative error as the stopping criterion; in addition, set ɛ to 10⁻⁷ in our experiments. The main time cost of Algorithm 1 lies in line 3 and line 4. Their time complexities are O(r²n+mrn+r²m+rm) and O(mr(n − d)+r²m+rm+r²+r²(n − d)), respectively. Thus, the total time complexity of Algorithm 1 is O(r² n+mrn+mr(n − d)+mrd+r²m+rm+r²+r²(n − d)).

Results

This section conducts a series of experiments on both synthetic and real-world datasets to verify the method proposed in this paper.

Synthetic Dataset

This section generates a small synthetic dataset to clarify the mechanism of Semi-PNMF. The synthetic dataset consists of three categories constructed by the following random samples: and where x ∈ R³, and each of its entry is sampled from the standard uniform distribution U(0,1). For each category, we randomly generated 10 samples, within which three samples were selected as labeled samples and the rest as unlabeled ones. Therefore, the synthetic dataset contains 30 samples in total. For clear illustration, three categories are marked as three different colors, and the labeled and unlabeled samples are distinguished by two shapes.

Fig 1(a) and 1(b) shows the ground truth and resultant labeled results of the unlabeled samples by Semi-PNMF, respectively, while Fig 1(c) and 1(d) displays the learned coefficients of the unlabeled samples and basis. In Fig 1(d), each row of the learned basis has different colors, implying that the basis stands for the centroids of different categories and owns the discriminative representation ability. According to Fig 1(c), each row of the learned coefficients is the lower-dimensional coefficient of the corresponding unlabeled sample. The larger the entry of the coefficient is, the darker its color is. As shown in Fig 1(c), the maximum entry of the coefficient largely exceeds the other entries. All maximum entries make the coefficients take up the diagonal form and imply the cluster memberships of all the samples. Thus, it is reasonable to select the index of the maximum entry of the coefficient as the classes of an unlabeled sample. This verifies our previous intuition. Since all samples shares the common basis, their coefficients become close to each other if they have the same labels. We impose the restriction that the coefficients of labeled samples be equivalent to their label vectors, and thus this also induces the coefficients of the unlabeled to be close to their label vectors. In this way, Semi-PNMF can propagate the labels of the labeled samples to the unlabeled ones. The propagation procedure is illustrated in Fig 2.

GCM Dataset

This experiment merely compares traditional semi-supervised learning methods including low density separation (LDS, [14]), transductive SVM (TSVM, [16]), constrained NMF (CNMF, [24]), soft-constrained NMF (SCNMF, [25]) and Semi-PNMF by separating different types of cancers on the GCM dataset. The GCM dataset [8] contains the expression profiles of 218 tumor samples representing 14 common human cancer classes. It is available on the public website: http://www.broadinstitute.org/cgi-bin/cancer/datasets.cgi, and can also be downloaded from the website: https://zenodo.org/record/21712. According to [8], we combine the training and testing set of this gene expression data into a dataset for cancer classification. Thus, the combined dataset contains 198 samples with 16,063 genes. Table 1 gives a brief description of this dataset. To remove very low noisy values and saturation effects of very high values, we bound the gene expression data into a specific box constraint ranging from 20 to 16,000 units and then exclude those genes whose ratios and absolute variations across samples are under 5 and 500, respectively. Consequently, the resultant expression profile dataset contains the 11,370 genes passing. We compare the effectiveness of Semi-PNMF with LDS, TSVM, CNMF and SCNMF under varying configurations. Both CNMF and SCNMF involve no parameter tuning. For Semi-PNMF, we set two parameters α = 2, and β = 0.0001, respectively. Because these representative methods enable convergence within 1,500 iteration rounds, we set the maximum number of loops to 1,500. For LDS and TSVM, we adopt the parameter settings provided in the source code to obtain the classification results.

Download:

Table 1. Description of the GCM dataset.

https://doi.org/10.1371/journal.pone.0138814.t001

We evaluate the cancer classification by the cross-validation over the whole dataset. This process selects one sample as the unlabeled sample and, meanwhile, learns the prediction model on all the samples for cancer diagnosis. For the unlabeled sample, we choose the index of the largest value of the resultant consensus matrix to predict the classes of this sample. As shown in Figs 3 to 7, the confusion matrix of the predicted results of Semi-PNMF, CNMF, SCNMF, LDS and TSVM are reported in detail. Each column denotes how many the unlabeled samples are assigned to each cancer, while each row signifies the number of the unlabeled samples affiliated to the real tumor type. Each color not only represents a specific cancer type but also highlights the correct prediction results, i.e., the diagonal elements of the confusion matrix.

Download:

Fig 3. Confusion matrix of prediction results using Semi-PNMF, which achieves a total accuracy of 70.71%.