Datasets

All datasets used in this study, contain eukaryotic glycoprotein (and glycosites) only. The primary glycoprotein dataset used in this study was extracted from SWISS-PROT June 2011 release by using key name “GLYCOSYLATION” and excluding all “PROBABLE”, “BY SIMILIARITY” and “PREDICTED” entries. This primary redundant datasets (1797 N-linked, 193 O-linked and 14 C-linked proteins) were than subjected to redundancy reduction of different stringency to derive following datasets for further study:

Standard and independent datasets.

Using CD-Hit program with cut-off 40% we got non-redundant standard datasets of 1186 N-, 121 O-, 12 C-linked proteins, where no two proteins are more similar than 40% at the level of (full-length) protein sequence (Figure 1). From these protein sequences overlapping patterns of residue length 21 were generated where glycosylated residue is the central residue of the pattern (Figure 2). Accordingly, we got total 2604 N-linked glycosites (with corresponding 36420 non-glycosylated/negative N-linked sites), 451 O-linked glycosites (with corresponding 9952 non-glycosylated/negative sites) and 48 C-linked glycosites (with corresponding 109 non-glycosylated/negative sites). These patterns were further divided in two parts at a ratio of 4∶1, in order to create standard and independent (blind/unseen) datasets. Thus our standard datsets contain 2083 N-linked glycosites (with corresponding 29136 non-glycosylated/negative N-linked sites), 361 O-linked glycosites (with corresponding 7960 non-glycosylated/negative sites) and 39 C-linked glycosites (with corresponding 87 non-glycosylated/negative sites). Similarly, our independent datasets contain 521 N-linked glycosites (with corresponding 7284 non-glycosylated/negative N-linked sites), 90 O-linked glycosites (with corresponding 1992 non-glycosylated/negative sites) and 9 C-linked glycosites (with corresponding 22 non-glycosylated/negative sites).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Flowchart showing process for creating various datasets used for developing GlycoEP models.

https://doi.org/10.1371/journal.pone.0067008.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. The process of creating of overlapping patterns in a glycoproteins and assigning glycosylated and non-glycosylated patterns.

https://doi.org/10.1371/journal.pone.0067008.g002

Advanced datasets.

Our standard datasets were generated from non-redundant proteins in order to minimize similarity between testing (independent) and training (standard) datasets. As the similarity was removed at the protein level, redundancy still was present in 21 residues long overlapping patterns that were generated from these proteins as explained above as well as in “Patterns and profile construction” section. As to get a robust prediction, a highly non-redundant dataset is always a pre-requisite, we further carried out a redundancy reduction at the pattern level to obtain a highly non-redundant advanced dataset where no two patterns are more similar than 60% [24]. The advance datasets thus contains 2454 N-linked, 235 O-linked, 27 C-linked glycosylated/positive patterns with corresponding non-glycosylated/negative patterns: 24051 for N-linked, 3541 for O-linked and 78 for C-linked respectively (Table 1). These highly non-redundant advanced datasets are used for developing advanced moled for predicted glycosylation (Figure 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Non-redundant glycosylated and non-glycosylated (positive+negative) patterns at different level of similarity cut-off.

https://doi.org/10.1371/journal.pone.0067008.t001

Machine Learning Tools

In the present study, we have attempted to trainWeka classifiers namely, SMO, libsvm, Random Forest, LMT, Bayesian network, naive Bayes and RBF Network as well as SVM. Weka is a widely accepted machine-learning toolkit in bioinformatics implemented in Java [25]. The data for Weka is represented in ARFF (attribute-relation function format) format that consists of a list of all instances, with the attribute values for each instance being separated by commas. The results from the Weka consists of a confusion matrix for both the training and testing set showing the number of instances of each class that have been assigned to each class. Similarly, SVM is based on the structural risk minimization principle from statistical learning theory. In SVM classifier the parameters and kernel functions like linear, polynomial, radial basis function, sigmoid can be adjusted easily [26]. In this study, we have used linear, polynomial and RBF kernel and learning option −g (1.0.0.1,0.01,0.001,0.0001,0.00001), −C parameter (ranges from 1to10) and −j parameter range from 1to 4 and observed that RBF kernel performed better than the others. So finally we have optimized all SVM learning models using RBF kernel [27].

Patterns and Profile Construction

Overlapping patterns of residue length 21 were generated for each glycosite (as shown in Figure 2). The patterns thus generated were termed as positives in cases where central residue is glycosylated and negatives if the same is unglycosylated [12], [28], [29]. To generate a pattern corresponding to the terminal residues in a protein sequence, (L-1)/2 dummy residue "X" is appended at both the termini of the protein (where L is the length of pattern).

Binary Profile of Patterns (BPP)

Fixed length amino acid patterns were converted into binary values [12], [28]. Each residue of patterns was represented by a vector of dimension 21 (e.g. Ala by 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0; Cys by 0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0), which contains 20 amino acids and one dummy amino acid "X".

Composition Profile of Patterns (CPP)

Composition profile of patterns is the fraction of each amino acid in a fixed length sequence pattern. In case of amino acid composition, variable length protein sequences were represented by fixed length patterns of 20 residues.

PSSM Profile of Patterns (PPP)

We have also attempted to use PSI-BLAST generated PSSM profile as an input feature for the training of SVM model. The PSSM for each sequence was generated by PSI-BLAST search against of SWISS-PROT. After three iterations with cut-off E-value of 0.001, it generated a PSSM having the highest score as a part of the prediction process. The PSSM matrix contains 20×M elements, where M is the length of the target sequence. The PSSM contains probability of occurrence of each type of amino acid at each residue position of fixed length sequence patterns.

Secondary Structure Information

Three state secondary structure (Coil, Helix and strand) information that were obtained using PSIPRED [30] were used as input in this study. The PSIPRED provided three secondary structure states for each residue of protein with probability.

Surface Accessibility Information

The accessible surface area (ASA) is the surface area of a protein that is accessible to another protein or ligand. The predicted average accessible surface area values of each amino acid were calculated from sarpred [31].

Based on the understanding that the glycosylation (especially O-glycosylation) may occur preferentially at surface accessible or exposed regions/residues in a protein and that it may also be governed/influenced by certain preferred secondary structural features like beta sheets or turns/loops, we have attempted to include two additional prediction features namely, SS (predicted secondary structure information) and ASA (predicted accessible surface area) in the overall prediction schema. These features have been used as input features in different combinations with BPP and CPP profiles of the sequences and best ones are implemented as an optional method for the users.

Training and Evaluation

In this study, we have used 5-fold cross-validation procedure to train and develop the prediction methods, where five subsets have been constructed randomly from the datasets as described in previous studies [12], [28], [29]. The models have been trained on four sets and the performance is measured on the remaining fifth set. This process is repeated five times in such a way that each set is used once for testing. The final performance is obtained by averaging the performances of all five sets. Standard deviations for iterative five fold training cycles are calculated and reported with corresponding avergae performance of each method for better judgement on performance stability of the method.

The performances of the different methods have been evaluated using parameters namely, sensitivity, specificity, accuracy and MCC (threshold- dependent parameters). Threshold selection is important criteria for checking the consistency of prediction result. In our study, the threshold varied in the range of –1 to +1, normally “0″ was selected as as default threshold that gave balance between sensitivity and specificity. Receiver Operating Characteristic (ROC, using R-package at http://www.r-project.org/) plots (threshold- independent parameter) were drawn between TP rate and FP rate and Area under Curve (AUC values) is calculated, accordingly.

Probability of Amino Acids Around Glycosites

In order to decipher the significant patterns the probability/frequency of specific amino acid residues in and around a glycosite can be derived from weblogos using glycosylated patterns of fixed sequence length. Requirement of a consensus sequence comprising of a tripeptide motif N-X-S/T (where X is any amino acid except Pro) is well established for N-linked glycosylation. Similarly, for C- linked glycosylation a known consensus sequence is W-X-X-W or W-X-X-C or W-X-X-F (where X is any amino acid). However, in case of O-linked glycosites no such defined sequence has thus far been identified in eukaryotic glycoproteins.

The amino acid frequency weblogos for sequences encompassing ten residues on either sides of eukaryotic glycosites in our dataset of 2604 N-linked glycosites (derived from 1186 glycoproteins) suggests that above mentioned consensus sequences are almost always true in case of N-linked and C-linked glycosites. However, exceptions some of which are reported previously are listed in Table S1 where an N-P-S/T, N-X-C and such other rare sequence motifs are found glycosylated at Asn residue. Similarly, weblogos developed from 456 O-linked glycosites (derived from 121 eukaryotic glycoproteins) are also in accordance with known preponderance of Ser and Ala (Alanine) residues around O-glycosites of eukaryotic (mucin type) glycoproteins. Preference for Pro at positions −3, +3 and +1 around glycosylated Ser/Thr (if at position 0) was also observed in our datasets. Additionally, we observed higher frequency of Pro at −6 positions and Gly (Glycine) at +1 position in proteins that are glycosylated at Thr residues (Figure S1, S2, S3). Besides this first we have analysed few glycoproteins to check motif based detection of glycosylated sites in N-linked sequon and we observed that sequon (motif) is easily detected by our method (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The performance of sequon (motifs) detection in N-linked glycosylation using five independent glycoproteins on GlycoEP server.

https://doi.org/10.1371/journal.pone.0067008.t002

Spread of Glycosites Across Protein Lengths

Analysis of 2604 N-linked glycosites in eukaryotic glycoproteins suggest that N-linked glycosites are spread out across the protein sequence length and the probability of N-linked glycosylation was observed poorest in first 5% (N terminal) of sequence length and last 30% of sequence length toward C terminus of protein. Similar analysis of spread of glycosites across O-linked glycoproteins suggested that the most likely position of O-linked glycosylation in eukaryotic glycoproteins is at around N terminal 6–10% of the sequence length followed by another high probability region at around middle 46–50% of the sequence length (Figure S4).

Comparision of Performance of Weka and SVM Classifiers

Using balanced Unique Glycoprotein datasets we have trained several Weka classifiers (as explained above) as well as SVM classifier for the prediction of N-, O- and C- glycosites in eukaryotic protein sequences. The performances of these classifiers using five fold cross validation, were then compared with each other. The detailed results are presented in Table S2, S3, S4, S5, S6, S7, S8, S9, S10. From this comparision, we found overall performance of SVM classifier the best among all tried classifiers, in our hands. Therefore, the further optimization were pursued only with SVM classifier.

Prediction of Glycosites Using SVM on Standard Datasets

Almost all well-known and existing glycosite prediction methods have been developed using only sequence based non-redundant datasets (standard datasets). Therefore, we first developed our models using standard datasets for predicting N- O- and C-linked glycosites and then have compared them with some of these already known prediction programmes (discussed below). Using our models for N- and C-linked glycosites a prediction accuracy of 87.95% and 91.08% was achieved with corresponding MCC of 0.54, and 0.79, respectively and with BPP as input feature. However, O-glycosites’ prediction was best done using CPP based models, where the models achieved an accuracy of 84.08% and an MCC of 0.32. Further inclusion of ASA as an additional input feature with CPP improved O-glycosites’ prediction to an accuracy of 93.12% though the same could not be achieved for N- and C-glycosites (Table S11, S12 S13). Presence of a defined and conserved sequon feature for N-glycosites (but none for O-glycosites) could be one explanation for this result. The ROC plots of the best optimized prediction model using standard datasets are presented in Figure 3. Further, as in most functional residues prediction method there is always a dissimilarity in the ratio of positive and negative instances of the training data, where negative instances are usually much higher than the positive instances. Therefore, in this study we have primarily trained all our methods using such standard datasets (explained under section Datasets). It is well known fact that machine learning techniques learn best on balanced datasets, thus we also trained and tested our models using the balanced datasets containing equal number of glycosylated (positive) and unglycosylated (negative) sites. The performances of these models trained on balanced datasets is shown in following Table S14, S15, S16. In our study, performances of models trained on standard datasets is at par with the performance obtained with related balanced datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Performances of various models on standard datasets in term of ROC, for N-, O- and C-linked glycosites (Panel A, B and C, respectively) in eukaryotic proteins.

https://doi.org/10.1371/journal.pone.0067008.g003

Development and Implementation of Advanced Models

Specificity in glycosylation event is largely governed by the specific sequon/sequence motifs/sequence contexts/sequence patterns. Therefore, we have developed advanced models using a dataset where high redundancy reduction has been executed in the glycosite patterns (sequence windows) to obtain a non-redundant advanced datasets (though smaller than standard dataset) for better class divide between negative and positive patterns. The advantages of this approach has been discussed previously in [24] and the approach has been employed successfully in improving prediction of PTM’s associated with lysine residue in proteins [32]. We developed SVM models on advanced datasets and achieved maximum accuracy value 84.24%, 86.87% and 91.43% for N-, O- and C-linked glycosylation, respectively (with corresponding maximum MCC values of 0.54, 0.20 and 0.78). We have also developed SVM model using balanced patterns of advanced datasets for N-, O- and C-linked glycosylation and achieved maximum accuracy of 92.26%, 62.77% and 79.82% with MCC of 0.85, 0.27 and 0.66, respectively. We term these models as Advanced GlycoEP models for these provide biologically more meaningful and also robust predictions in comparision with standard models and other similar methods discussed above. It was observed that the performances of these advanced models is comparable to standard models (models developed at standard datasets) for N-glycosites’ prediction. This could possibly be explained in conservation of N-glycosite sequon and its strong influence on class divide. However, the predictions of O- and C-glycosites apppeared slightly poor with Advanced GlycoEP models than the performances seen with models trained on standard datasets (Table 3). This can easily be understood in over-optimization of (standard) GlycoEP models for its high similarity and redundancy in overlapping pattern datasets especially for O- and C-glycosites’s standard datasets where as high as 50% of the patterns used in training sets are more similar than 60% (Table 1). Further, decreased size of advanced training datasets post redundancy-reduction at the level of glycosite patterns (almost 50% decrease in the total number of glycosites in standard datasets for O- and C- glycosites, respectively) also contributed in lowering the measured performance of advanced models. However, advanced GlycoEP models are infact more stringently trained on highly non-redundant datasets and therefore, are more robust than standard models in prediction of glycosites. Particularly for the prediction of O- and C- glycosites, advanced models are the most (and probably the only) useful models as advanced datasets employed in training are highly non-redundant therefore, superior to corresponding standard datsets. Further, as advanced GlycoEP models don’t suffer from over-optimization, therefore reports more meaningful. These models have been trained, tested and evaluated using five fold cross validation technique.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. The performance of models developed on advanced datasets for predicting N-linked, O-linked and C-linked glycosites.

https://doi.org/10.1371/journal.pone.0067008.t003

Performance of Standard Models on Independent Datasets

As discussed in [32], the separation of independent (blind/testing) datasets from training data is an important consideration for better PTM predictions. Therefore, as an additional evaluation measure, we sub-divided entire sequence based non-redundant dataset randomly in to standard datasets (trainset) and corresponding independent datasets (blind-test-set: where none of the proteins in independent dataset was used for training dataset). Details and number of glycosites included in each of these independent datasets are listed in Table 4. Finally we have checked performances of standard models on independent datasets. From the results we found and concluded that the performance of our models is equally well on independent datasets (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. The performance of models on an independent datasets, these models were developed on standard datasets.

https://doi.org/10.1371/journal.pone.0067008.t004

Comparison of Standard Models with Existing Methods

As discussed in Introduction section, a good number of computational methods are available for prediction of potential glycosites in eukaryotes. In these, methods like NetOGlyc, EnsembleGly and GPP, have been developed using same glycoprotein datasets as described in previous study, especially for mucin linked O-glycosites prediction [33]. These datasets are not only small but redundant. Our prediction methods have been developed to predict glycosites in similar fashion as in existing approaches, but we have employed larger and a non-redundant datasets that has led to better performances of our models. Among existing methods NetOglyc and NetNglyc are most widely used in prediction of O-linked and N-linked glycosylation, respectively. We have compared prediction results of our best performing SVM models using sequence based non-redundant datasets (standard model) with NetOglyc, NetNglyc, EnsembleGly and Glycosylation Prediction Program (GPP) according to existing comparison approaches. For this comparision purposes we have used the same dataset of 216 glycoproteins as test set that was employed in development of the above-mentioned methods. Standard SVM Models (GlycoEP) developed by us performed better than these programmes and achieved 95.67%, 91.89% and 85.71% accuracy for N-, O- and C-linked glycosite predictions, respectively. Our SVM models could predict Ser and Thr (O-linked glycosylation) with 93.50% and 91.06% accuracy, respectively (Table 5), suggesting an overall better prediction performances of standard GlcoyEP models as well as their applicability to eukaryotic glycosite analysis. In this study, we have compared existing methods with our standard datasets because all existing methods used sequence based non-redundant datasets (standard datasets). Our advanced datasets results are also comparable with these results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparative performances of existing method with our model developed on standard datasets.

https://doi.org/10.1371/journal.pone.0067008.t005

However, as we have discussed already the standard dataset though would qualify as non-redundant dataset at the level of full-length protein sequences, it does carry significant similarity at the overlapping pattern level (Table 1). This may contribute towards certain amount of overprediction in any training method. It would also mean that for a more robust prediction it is desirable to develop methods on a dataset where patterns in training set are highly non-redundant. Therefore, we have developed another set of non-redundant datasets termed advanced datasets as defined already and have developed advanced models using the same.

N-glycosite Prediction Performance of SVM Using N-linked Sequon as a Feature

In general, all existing glycosites’ prediction algorithms are based on the assumption that either the sequence or structural or both contexts around N-glycosylated Asn (present in the sequon N-X-S/T, where X could be any residue but not proline) are different from the non-glycosylated Asn residues in a given glycoprotein sequence. Therefore, most algorithms including GlycoEP have been trained on the datasets where the positive datasets contain N-X-S/T as positive pattern and negative datasets contain any non glycosylated Asn (including the one which is residing within N-X-S/T sequon but not known to be glycosylated). We, in this study have attempted to exploit the contextual difference around glycosylated sequon (as defined above) versus non-glycosylated sequon datasets composed of sequon containing patterns only. However, our algorithms trained on these datasets could not indicate any further improvement in the prediction results. The purpose of improving class divide might have been defeated here if significant number of yet unidentified (experimentally) but positive glycosites patterns are present in negative datasets. To conclude, the approach where sequons are used only in positive dataset is found better than using it in both positive and negative datasets. The result are presented in Table S17.