An Ensemble Classifier for Eukaryotic Protein Subcellular Location Prediction Using Gene Ontology Categories and Amino Acid Hydrophobicity

With the rapid increase of protein sequences in the post-genomic age, it is challenging to develop accurate and automated methods for reliably and quickly predicting their subcellular localizations. Till now, many efforts have been tried, but most of which used only a single algorithm. In this paper, we proposed an ensemble classifier of KNN (k-nearest neighbor) and SVM (support vector machine) algorithms to predict the subcellular localization of eukaryotic proteins based on a voting system. The overall prediction accuracies by the one-versus-one strategy are 78.17%, 89.94% and 75.55% for three benchmark datasets of eukaryotic proteins. The improved prediction accuracies reveal that GO annotations and hydrophobicity of amino acids help to predict subcellular locations of eukaryotic proteins.


Introduction
Researches on subcellular location of proteins are important for elucidating their functions involved in various cellular processes, as well as in understanding some disease mechanisms and developing novel drugs. Since experimental determinations of the localization are time-consuming, tedious and costly, especially for the rapid accumulation of protein sequences, it is highly desirable to develop effective computational methods for accurately and quickly predicting their subcellular attributes.
In the past few years, many computational methods have been developed for this purpose [1,2,3,4]. These methods can be divided into two main categories [5]. Methods in the first category are based on the observation that amino acid compositions of extracellular and intracellular proteins are significantly different [6]. Along this line, many computational approaches based on amino acid composition, dipeptide composition [7] and gapped amino acid pairs [8] were proposed. Meanwhile, to incorporate more sequence information, many other features were incorporated, such as amphiphility of amino acids [9], functional domain composition [10], psi-blast profile [11,12] and so on. Methods in the second category are based on a certain sorting signals [13,14], including signal peptides, chloroplast transit peptides and mitochondrial targeting peptides. For example, Emanuelsson et al. [14] provided detailed instructions for the use of SignalP and ChloroP in prediction of cleavage sites for secretory pathway signal peptides and chloroplast transit peptides. However, the reliability of these methods is highly dependent on protein N-terminal sequence assignments, and the molecular mechanisms related to sorting signals are rather complex and not interpreted clearly.
Not only protein sequence information but also prediction algorithms could affect the accuracy of the subcellular localization prediction. So far, many computational techniques, such as the hidden Markov models (HMM) [15,16], neural network [17], Knearest neighbor (KNN) [18] and support vector machine (SVM) [5,19] were introduced for the prediction of protein subcellular localization. However, most of the current predictors are based on a single theory which could have its own inherent defects, so their predictions are not satisfactory. For example, the number of parameters that need to be evaluated in an HMM is large [20]. The neural network can suffer from multiple local minima [21]. Besides, quite a few ensemble classifiers [7,22,23] for prediction of protein subcellular localizations have been proposed. However, many of the ensemble classifiers were actually engineered only by a single algorithm, such as the fuzzy KNN [7], KNN [22], and Bayesian [23]. Other ensemble classifiers, such as CE-PLoc [24] and the KNN-SVM ensemble classifier proposed by Zhang [25], were engineered by different algorithms, mostly including SVM and KNN. Along this line, an ensemble classifier making use of the classical SVM and KNN algorithms was developed in this article to predict subcellular localization of eukaryotic proteins.
We apply our method to three widely used eukaryotic protein datasets. By the jackknife cross-validation test [26,27,28,29], the ensemble classifier shows high accuracies and may play an important complementary role to existing methods.

Datasets
In order to evaluate the performance of the proposed method and compare it with current methods, we introduced three widely used datasets into this study. The first dataset was constructed by Chou [30]. This dataset (denoted as iLoc8897) consists of 8,897 locative protein sequences (7,766 different proteins), which divided into 22 subcellular locations. Among the 7,766 different eukaryotic proteins, 6,687 belong to one subcellular location, 1,029 to two locations, 48 to three locations, and 2 to four locations. None of the proteins has $25% sequence identity to any other in the same subset. The second benchmark dataset was constructed by Park and Kanehisa [8]. This dataset (denoted as Euk7579) contains 7579 proteins, which are divided into 12 subcellular locations. Proteins in this dataset have the pairwised sequence similarity below 80%. The third dataset was constructed by Shen and Chou [31]. This dataset (denoted as Hum3681) consists of 3,681 locative protein sequences (3,106 different human proteins), which are divided into 14 human subcellular locations. Among the 3,106 different proteins, 2,580 belong to one subcellular location, 480 to two locations, 43 to three locations, and 3 to four locations. None of the proteins has $25% sequence identity to any other in the same subcellular location. The detailed information of the three datasets are listed in Table 1.

Gene Ontology
Gene Ontology (GO) is a major bioinformatics initiative. It meets the need for consistent descriptions of gene products in different databases. Gene Ontology database is established on the three criteria: molecular function, cellular component and biological process. It has been developed to manage the overwhelming mass of current biological data from a computational perspective and become a standard tool to annotate gene products for various databases [32,33]. Accordingly, GO annotation has been being used for diverse sequence-based prediction tasks, such as analyzing the pathogenic gene function with human squamous cell cervical carcinoma [34], mapping molecular responses to xenoestrogens [35], predicting the enzymatic attribute of proteins [36], predicting the transcription factor DNA binding preference [37], and predicting the eukaryotic protein subcellular localization [38]. In particular, the growth of Gene Ontology databases has increased the effectiveness of GO-based features [39]. As a result, Gene Ontology could be used to improve the predictive performance of protein subcellular localization [22,40].
We downloaded all GO data at ftp://ftp.ebi.ac.uk/pub/ databases/GO/goa/UNIPROT/(released on March 15, 2010), and searched the GO terms for all the protein entries in the three datasets. We eliminate those proteins, which have no corresponding GO terms and the number (60, 127 and 4 for the iLoc8897, Euk7579 and Hum3681 datasets) are relatively small compared to the total datasets. We consider this would not have a great influence on its final accuracy. After this step, we got a list of GO terms for each protein entry of the three datasets. For example, the human protein entry ''Q9H400'' in the Hum3681 dataset corresponds to four GO numbers, i.e., GO: 0005886, GO: 0006955, GO: 0016020 and GO: 0016021, while the protein As we know, if we want to describe all possible GO terms for a certain dataset, the simplest way to vector represent a protein was using a binary feature component for a protein. We used value 1 if the corresponding GO number appears and value 0 if it does not appear. For example, the human protein entry ''Q8TDM5'' in the Hum3681 dataset corresponds to seven GO numbers in the GO database, i.e., GO: 0001669, GO: 0005515, GO: 0005886, GO: 0007155, GO: 0016020, GO: 0031225 and GO: 0031410, which corresponded to GO_compress: 0000212, GO_compress: 0001037, GO_compress: 0001203, GO_compress: 0001722, GO_compress: 0002543, GO_compress: 0003360, GO_compress: 0003398 in the GO_compress database. So the 212 th , 1037 th , 1203 rd , 1722 nd , 2543 rd , 3360 th , and 3398 th components of the feature vector were assigned the value 1 and the rest 5553{7~5546 components with the value 0. At last, we transformed the GO terms annotated for each human protein into a 5553-dimension input vector.

Amphiphilic pseudo amino acid composition
In a protein, the hydrophobicity and hydrophilicity of the native amino acids play an important part in its folding, interior packing, catalytic mechanism, as well as its interaction with other molecules in the environment [41]. Therefore, the two indices may be used to effectively reflect the subcellular locations of proteins. Both the hydrophobicity and hydrophilicity are introduced in the concept of AmPseAAC. As we know, the concept of AmPseAAC proposed by Chou [22] was widely used by many researchers in improving the prediction quality for protein subcellular localization [42,43]. Following the concept of AmPseAAC, a protein sample could be descripted by a 20z2l dimensional feature vector, where l is equal to L min {1, where L min is the length of the shortest protein sequence in the dataset. The 20z2l dimensional feature vector for a protein comprises 20 features of the conventional amino acid composition (AAC), and the rest 2l components reflect its sequence-order pattern through the amphiphilic feature. The protein representation is called the ''amphiphilic pseudo amino acid composition'' or ''AmPseAAC'' for short. In order to get more local sequence information, we incorporated 400 dipeptide components to the AmPseAAC. Then the new AmPseAAC is constructed and the dimension is increased to 420z2l, which are 420z2|49~518, 420z2|9~438, and 420z2|50~520 for the iLoc8897, Euk7579 and Hum3681 datasets, respectively. Then we combined the new AmPseAAC and Gene Ontology as the features for protein subcellular localization prediction. As a result, the dimensions of the final input feature vectors are 420z2|49z7871~8389, 420z2|9z6533~6971, and 420z2|50z5553~6073 for the iLoc8897, Euk7579 and Hum3681 datasets.

Feature extraction
Due to the limited numbers of learning examples, learning with a small number of features often leads to a better generalization of machine learning algorithms (Occam's razor) [44]. Additionally, with the increase of the dimension of the feature vector, the computational loads for some machine-learning tools, e.g., Support Vector Machine [45] and Neural Network [46], are seriously affected. As a result, we used the ''fselect.py'' in Libsvm software package to reduce the dimensionality. The fselect.py is a simple python script used F-score to select features. After running the python script, one could get an output file called ''.fscore'', in which each feature was given a score to describe the importance of it and all features were sorted by their scores. Then we chose the top features with the highest contribution scores (Figs. 1, 2, and 3).

The KNN-SVM ensemble classifier
A wide variety of machine learning methods have been proposed for predicting protein subcellular localization in recent years [47,48,49,50], such as Markov chain models [51], neural networks [46], K-Nearest Neighborhood (KNN) [18], and Support Vector Machines (SVM) [52,53]. In these methods, KNN and SVM are two popular classifiers in machine learning task. Previous studies presented that each algorithm has its own advantage and the ensemble classifier of different algorithms is the future direction of protein subcellular localization prediction. So, in this paper we proposed an ensemble classifier of KNN and SVM based on one-versus-one strategy and a voting system (Fig. 4). LIBSVM still has a few tunable parameters which affect the accuracy of the subcellular localization prediction and need to be determined. In this article, ''grid.py'' was used in the iLoc8897 dataset to select the parameter c and the regularization parameter C in LIBSVM [24]. Here, the iLoc8897 dataset was selected for optimization of the parameters of the classification models due to the following reasons: (i) compared to the other datasets, this dataset has the largest number of proteins, so it possesses a distinct statistical significance for training; (ii) sequences in this dataset have relatively low pairwise sequence homology; (iii) this dataset covers enough subcellular locations and was widely adopted for evaluating a new proposed method [30,38].
Prediction of protein subcellular localization is a multi-class classification problem. Here, the class number is equal to 22 for iLoc8897 dataset, 12 for Euk7579 dataset and 14 for Hum3681 dataset, respectively. A simple way to deal with the multi-class classification is to reduce the multi-classification to a series of binary classifications. During this study, we adopted the one-versusone method, i.e., 22|21=2~231, 12|11=2~66, and 14|13=2~91 binary classification tasks were constructed for the iLoc8897, Euk7579 and Hum3681 datasets. Compared to the one-versus-one approach, the one-versus-rest strategy has the shortage that the numbers of positive and negative training data points are not symmetric [54]. For each binary classification, the predictor (KNN or SVM) with the higher output accuracy was selected, and the free parameters, i.e., k for KNN and C and c for LIBSVM, are optimized by the iLoc8897 dataset.
Take the Hum3681 dataset as an example. Following the oneversus-one strategy, 14|13=2~91 binary classification tasks were constructed for this dataset. For each binary classification task, the KNN and SVM are used to predict the attribute of each protein.
As a result, we chose the predictor with the higher output accuracy, where the parameters of KNN and SVM were optimized by the iLoc8897 dataset. Then a score function was generated by the KNN-SVM ensemble classifier formed by fusing the 91 individual binary classifiers through a voting system (see Eqs. 1-3). Each protein was assigned to the subcellular location where S 1 ,S 2 , . . . ,S 14 represent the 14 subcellular locations. The voting score for the protein P belonging to class i is defined as where the d function in Eq. 2 is given by Subsequently, the query protein P was assigned to the class that gives the highest score for Eq. 2 of the 91 binary classifiers. We can assume that there are five subsets and 5|(5{1)=2~10 binary classification tasks are constructed. If the predicted classification results for a query protein P with the ten binary classifiers are R(1)~S 2 , R(2)~S 1 , R(3)~S 4 , R(4)~S 5 , R(5)S 2 , R(6)~S 2 , R(7)~S 5 , R(8)~S 3 , R(9)~S 5 , R(10)~S 4 that is, classifiers 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10 assign protein P to subsets 2, 1, 4, 5, 2, 2, 5, 3, 5 and 4, respectively. As a result, the voting scores for protein P are G 1~1 , G 2~3 , G 3~1 , G 4~2 , G 5~3 . Then protein P was predicted to classes 2 and 5, which both give the highest score of G 2~G5~3 .

Assessment of prediction performances
The prediction quality is examined by the jackknife test currently. Three methods, i.e., the jackknife test, sub-sampling test, and independent dataset test are often used for examining the accuracy of a statistical prediction method. The jackknife test is deemed the most objective and rigorous one [55,56].
The accuracy, the overall accuracy, the ''absolute true'' overall accuracy and Matthew's Correlation Coefficient (MCC) [57] for each subcellular location calculated for assessment of the prediction system are formulated as , if all the subcellular locations of the hth protein are exactly predicted without any overprediction or underprediction 0, otherwise where M is the class number, N is the total number of locative proteins, m(i) and m(j) are the numbers of the locative proteins in classes i and j, p n (i) and p n (j) are the numbers of the correctly predicted locative proteins of class i and class j by binary classifier n. V is the so-called ''absolute true'' overall accuracy. D is the number of total proteins investigated. TP i , FP i , TN i , and FN i are the numbers of true positives, false positives, true negatives, and false negatives in class i by the KNN-SVM ensemble classifier, respectively.

Selection of algorithms and parameters
It is important to point out that the best combination of parameters c and C depends on the dimension Dim of the protein top feature vector. In the present work, we select the parameters c and C when parameter Dim varied from 10 to 50. As seen in Table 2, the highest prediction accuracy was 78.01% at c~0:125, C~2 and Dim~45. While the prediction accuracy obtained by KNN changed as parameter k varied from 1 to 9, and the highest prediction accuracy (74.70%) was obtained at k~5 and Dim~45 for the iLoc8897 dataset. Then the same parameters, i.e., c~0:125, C~2, k~5 and Dim~45 were used for all the three datasets.
Because the Hum3681 dataset has 14 subcellular locations, a total of 14|13=2~91 binary classification tasks were constructed. For each one-versus-one classification task, the algorithm (KNN or SVM), which gave a higher prediction accuracy for Eq. 4, was adopt as the final classifier. For example, the 6 th , 21 st , 26 th , 32 nd , 34 th , 42 nd , 43 rd , 76 th , 82 nd , 84 th and 90 th binary classifiers (11 of 91 classifiers) was based on the KNN method, because the accuracy of KNN method was higher than LIBSVM method by jackknife test, while the rest 91{11~80 binary classifiers were based on LIBSVM, because the accuracy of LIBSVM method was higher than KNN method by jackknife test.
In addition, most of the existing methods for predicting protein subcellular localization are limited to a single location. It is instructive to note that the KNN-SVM ensemble classifier can effectively deal with multiple-location proteins as well, that is, the predicted result for a query protein P may be attributed to two or more subcellular locations. For example, the real subcellular locations of the protein entry ''Q05329'' in iLoc8897 dataset are S 2 ,S 12 ,S 21 f g , and the predicted subcellular locations for ''Q05329'' by the KNN-SVM ensemble classifier are also S 2 ,S 12 ,S 21 f g , because S 2 , S 12 , S 21 give the highest score (G 2~G12~G21~2 0) according to Eq. 2.

Comparison with other methods
In order to check the performance of our method, we made comparisons with the following methods: iLoc-Euk [30], Euk-mPLoc 2.0 [38], Hum-mPLoc 2.0 [31], LOCSVMPSI [58], Complexity-based method [59], and the method proposed by Park and Kanehisa [8] which are also based on the Euk7579 dataset. We also compared our method with the KNN binary classifiers, LIBSVM binary calssifiers, and the KNN-SVM ensemble classifier [25]. The comparison is summarized in Tables 3, 4, 5, and 6.
For the iLoc8897 dataset, the absolute true overall accuracy of the current approach is 75.64%, which is 4.37% higher than the iLoc-Euk method, though the overall accuracy is only 0.89% lower than it. In addition, our method achieves the best performances  among the 22 subcellular locations except for the locations of Cytoplasm and Endoplasmic reticulum. Meanwhile, our method also performs better than Euk-mPLoc 2.0 [38] which is also based on the same dataset. For the Euk7579 dataset, the overall accuracy of the current approach is 89.94%, which is also higher than those achieved using the methods listed in Table 4 (from 6.44% to 14.94%). Meanwhile, our method also performs better than some other classifiers such as LOCSVMPSI [58] and complexity-based method [59]. As shown in Table 5, our method also achieves better performances than Hum-mPLoc 2.0. For the Hum3681 dataset, the overall accuracy of the current approach is 75.55%, which is 12.85% higher than the Hum-mPLoc 2.0 method. It is worth noting that all the three datasets (Euk-mPLoc 2.0, iLoc-Euk and Hum-mPLoc 2.0), which also extract sequence features from the Gene Ontology information to represent the query protein, get the comparable accuracies to the present method. This demonstrates that the Gene Ontology information provides a better source of information for the prediction of protein subcellular location. As shown in Table 6, the proposed method, examined by the jackknife test, also performs better than Euk-mPLoc and the KNN-SVM ensemble classifier [25]. For the Euk6181 dataset [60], the overall accuracy of the proposed method is 79.14%, which is 11.74% and 8.64% higher than Euk-mPLoc and the KNN-SVM ensemble classifier respectively [25].
As illustrated by some researchers, protein sequence similarity within the datasets has a significant effect on the prediction performance of protein subcellular location, i.e., accuracies will be overestimated when using high-similarity datasets. To avoid this problem, two low-similarity datasets, i.e., the iLoc8897 dataset and Hum3681 dataset were used to evaluate the performance of our method. The results also show that our method achieves good performances and the prediction accuracies are higher than those achieved using the methods listed in Table 3 and Table 5.

A case study
To evaluate the performance of the proposed method, it was also used to predict the subcellular locations of some proteins used in our laboratory. Take two proteins for example. The first example is fibronectin (FN) [61,62], which is an ''extracell'' protein and abundant in the extracellular matrix and participates in many cellular processes, including osteoblastic differentiation/ mineralization, tissue repair, embryogenesis, cell migration/ adhesion, and blood clotting. The accession number for FN is shown in Table 7. According to our ensemble classifier, this protein was predicted as ''extracell'' protein, which is in accordance with the annotation in Swiss-Prot database. The second is cadherin 11 (CDH 11) [61,62], which is a plasma membrane protein preferentially expressed in osteoblasts. CDH 11 can promote cells to form specialized cell junctions and enhanced crosstalk between adjacent osteocytes. The accession number for CDH 11 is also shown in Table 7. We also predicted it correctly. More examples are list in Table 7. As is shown, 10 of all the 11 proteins are predicted in accordance with the Swiss-Prot annotations by the proposed method. While only 8 of 11 eukaryotic proteins and 2 of 4 human proteins are predicted correctly by iLoc-Euk and Hum-mPLoc2.0 respectively. We also used iLoc-Euk, Hum-mPLoc 2.0 and the proposed method to predict the subcellular locations of some multiplelocation proteins. As can be seen from Table 8, all subcellular locations of the protein Q05329 was correctly identified by the proposed method and iLoc-Euk, but not entirely correctly by Hum-mPLoc 2.0. The second protein P58335 was identified completely correctly by the proposed method, but according to iLoc-Euk and Hum-mPLoc 2.0, it was assigned to only one of its real subcellular locations. The third protein P30622 simultaneously exists at ''Cytoplasm'' and ''Cytoskeleton'' in Swiss-Prot. Both iLoc-Euk and Hum-mPLoc 2.0 only identified one location correctly. Although the proposed method incorrectly predicted P30622 as belonging to ''endosome'', yet it successfully identified two of its subcellular locations.

Conclusions
In this study, a KNN-SVM ensemble classifier by fusing the GO attributes and hydrophobicity features was investigated to predict subcellular location of eukaryotic proteins. Three widely used benchmark datasets were adopted in our work. To improve the prediction quality, the following strategies were applied: (i) representing protein samples by using Gene Ontology could effectively grasp the core features to indicate the subcellular Table 8. Examples to show the predicted results by three predictors on multiple-location proteins.  localization, (ii) adopting the one-versus-one strategy and two most popular classifiers in machine learning task, i.e., LIBSVM and KNN to predict protein subcellular location, (iii) capturing the top features and learning with a small number of features might lead to a better generalization of machine learning algorithms (Occam's razor). In summary, the results of the predictions performed by KNN-SVM ensemble classifier indicate that our method is very promising and may play an important complementary role to existing methods.