Data description

The non-redundant labeled training and independent test datasets used by the MoCETSE effector protein classifier were all derived from the multi-class effector protein prediction tool DeepSecE [27]. During the original construction of DeepSecE dataset, the authors further applied CD-HIT (v4.8.1) [29] with a 60% sequence identity threshold to remove redundant and highly homologous sequences both within the training set and between the training and test sets. After obtaining the training dataset, we performed a homology redundancy check between our training dataset and the independent test dataset from Bastion3 [10], CNN-T4SE [14], and Bastion6 [18]. We used CD-HIT with a 90% sequence identity threshold for clustering analysis and removed the overlapping sequences between the training dataset and the benchmark test dataset to ensure the independence of the training dataset. Finally, the training dataset (Dataset S1) contains 1,577 non-effector proteins, 128 T1SEs, 68 T2SEs, 392 T3SEs, 507 T4SEs, and 232 T6SEs, totaling 2,904 samples; the test dataset (Dataset S2) contains 150 non-effector proteins, 20 T1SEs, 10 T2SEs, 30 T3SEs, 30 T4SEs, and 20 T6SEs, totaling 260 samples. The protein sequences in these training and independent test datasets span a wide range of lengths, from fewer than 100 amino acids to over 1,000 amino acids. As shown in S1 Fig, the lengths of these proteins are mainly distributed within the range of 100–600 amino acids.

During the benchmark testing phase, to comprehensively evaluate the performance of MoCETSE against existing mainstream binary and multi-class methods, we selected corresponding benchmark datasets for different types of secreted proteins. For type I secreted proteins, given the lack of publicly available independent test sets, we followed the methodology established in DeepSecE, selecting 20 T1SE samples and 150 non-secreted protein samples from the independent test dataset (Dataset S2) to construct a new benchmark dataset (Dataset S3). For type III, IV, and VI secreted proteins, we directly used the publicly available independent test datasets from Bastion3, CNN-T4SE, and Bastion6, which were designated as Dataset S4, Dataset S5, and Dataset S6, respectively. The samples in these test datasets were all curated by the original authors from primary literature, public databases, or experimental data (encompassing diverse bacterial species and studies). Additionally, the original authors applied CD-HIT with a sequence similarity threshold to remove duplicate or highly homologous samples from both the training and independent test datasets, preventing data leakage and ensuring dataset independence. The detailed specifications of the datasets used in this study are provided in S1 Table, and the distribution of different types of effector and non-effector proteins within the datasets is visually depicted in S2 Fig.

Overview framework of MoCETSE

Our proposed innovative framework, MoCETSE (Fig 1), starts with the input of effector protein sequence data. We leverage ESM-1b [21], a pre-trained protein language model, to encode amino acid sequences into informative feature vectors. Specifically, ESM-1b is constructed by stacking 33 layers of transformer modules, which alternate between self-attention mechanisms and feed-forward connections. Each transformer module contains multiple attention heads capable of capturing sequence information from different perspectives. We limit protein sequences to a maximum of 1,020 amino acids; these are converted into tokens and processed by ESM-1b, resulting in 1,280-dimensional token embeddings. To enhance the model’s learning capacity for protein sequence features and training efficiency, a target preprocessing network (Fig 1B) is utilized. This network reduces the token embedding dimension from 1280 to 256 by leveraging eight convolutional experts that work collaboratively. Subsequently, the features are fed into a transformer module incorporating relative positional encoding (Fig 1C), configured with 4 attention heads and a feed-forward network employing GELU as the activation function [30]. After processing by the transformer module, the obtained feature representations pass through a fully connected layer for linear transformation, mapping them to a dimensional space corresponding to the number of classification categories. Finally, the softmax activation function is used to convert the output values into a probability distribution, ultimately achieving the classification of non-effector proteins and five types of effector proteins (T1SE–T4SE and T6SE):

(1)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Overall workflow of MoCETSE.

(A) MoCETSE uses the pre-trained protein language model ESM-1b, a target preprocessing network, and a transformer module to achieve effective learning of secreted protein features. The snowflake icon on the ESM-1b module indicates that its weights remain frozen during subsequent training. (B) The target preprocessing network is used to refine more critical discriminative features from secreted effector proteins. (C) The relative positional multi-head attention mechanism is employed to enhance the model’s ability to identify key functional motifs in secreted effector proteins.

https://doi.org/10.1371/journal.pcbi.1013397.g001

Where denotes the input protein sequence, and represents the conditional probability of the predicted output given the input sequence , denotes the weight matrix, which maps the model’s output to the final class space, indicates the sequence length, represents the embedding vector of the amino acid residue, PLM represents the pre-trained protein language model, TPN represents the target preprocessing network, and RPET represents the relative position-encoded transformer.

Protein language model

A multitude of studies have confirmed that the feature representations generated by protein language models can effectively capture essential sequence characteristics [31,32]. This advantage has led to their extensive application in predicting protein structures and functional properties. ESM-1b employs an unsupervised learning strategy to learn and extract various biological embedding features of proteins from the UniRef50 database [33], which contains approximately 250 million protein sequences. This database systematically integrates protein sequences from the UniProt Knowledgebase via a clustering algorithm, grouping sequences with at least 50% sequence similarity into the same cluster. The initial weights of ESM-1b were obtained from the pre-trained model parameter repository (https://github.com/facebookresearch/esm) and remained frozen without any fine-tuning during subsequent training.

Target preprocessing network

The target preprocessing network (TPN) adopts a “divide-and-conquer” strategy and is built upon a Mixture-of-Experts (MoE) architecture. Its core function is to preprocess high-dimensional embedding vectors generated by protein language models, facilitating dimensionality reduction while preserving and emphasizing key sequence features. By incorporating a multi-convolutional expert architecture, TPN synchronously captures critical features during the dimensionality reduction process, thereby further enhancing the specificity and representational power of feature extraction. The MoE framework, originally proposed by Jordan and Jacobs in 1991 [34], facilitates efficient modeling of complex data distributions by decomposing intricate tasks into sub-problems handled by specialized expert modules, with their outputs integrated via a gating mechanism. The efficacy of this paradigm has been validated by recent studies such as DeepSeekMoE [35], which achieves reduced parameter counts and competitive performance through refined expert partitioning, and DynamicMoE [36], which balances computational cost and precision using dynamic gating. As illustrated in Fig 1B, TPN in this study comprises three core components: the gating network module [37], the multi-convolutional expert group [38], and the attention module [24].

The gating network module is responsible for dynamically assigning weights to each convolutional expert module. By leveraging the specialized functions of different expert modules, it facilitates task decomposition and mitigates the parameter interference typically associated with global weight sharing in conventional single-network architectures. Specifically, the gating network generates gating weights via a softmax activation function, constructing a probability distribution to adaptively regulate the contribution of each expert. The final output is derived as the weighted sum of the outputs from all experts, where the magnitude of the weights determines the relative influence of each expert. The output of the gating network module, , is defined as follows:

(2)

Where represents the feature vector of raw input data after preprocessing via average pooling and flattening, and represents the learnable weight matrix.

The multi-convolutional expert group module integrates the MoE architecture with convolutional neural networks to construct an efficient and heterogeneous feature extraction system. This module consists of multiple parallel convolutional sub-modules, each serving as an independent expert branch with its own dedicated parameter set. This configuration enables the extraction of features across various spatial scales and dimensions from the input data. Guided by the gating network, which assigns weights to each expert branch, the module executes a weighted fusion of their outputs. This design leverages collaborative multi-expert integration to adaptively combine diverse features, thereby enhancing the TPN’s capacity to model complex data distributions.

For a given input , let represent the output of the gating network, and let represent the output of the convolutional expert network. The output of the multi-convolutional expert group module can be written in the following form:

(3)

To enhance the model’s selective focus on high-value features, the target preprocessing network incorporates a lightweight attention module, whose core function is to achieve feature optimization and selection through dynamic weight adjustment. The module takes the fused features as input, which are synergistically generated by the multi-convolutional expert group and the gating network. Through convolutional operations, the GELU activation function [30], and sigmoid normalization, the module generates an element-wise attention mask . Subsequently, an attention-weighting mechanism is employed to perform element-wise multiplication between and . This process adaptively highlights task-relevant key information while suppressing irrelevant features and noise, thereby improving the network’s robustness in processing complex data. The final output of TPN, , can be defined as follows:

(4)

Relative multi-head attention

The transformer model, proposed by Vaswani et al. (2017) [24], represents a fundamental shift from the conventional recurrent neural networks (RNNs) [39] and convolutional neural networks (CNNs) [40] typically employed in sequence learning tasks. This architecture entirely dispenses with recurrent structures and convolutional operations, instead relying on a parallelized self-attention mechanism to directly capture global dependencies within a sequence. By doing so, it effectively overcomes the inherent limitations of traditional models in long-range modeling and computational parallelization.

However, this architectural design possesses an inherent limitation: due to the permutation-invariant nature of the self-attention mechanism, the model is unable to perceive the positional order of elements without the inclusion of auxiliary positional information. To address this deficiency, attention-based models typically incorporate relative positional encoding or learnable positional embeddings. As illustrated in Fig 1C, this study integrates relative positional encoding [41,42] transformer’s multi-head attention mechanism. This modification enables the MoCETSE model to more accurately capture the relative spatial dependencies between key motifs within effector protein sequences, while simultaneously enhancing the model’s generalization capability across sequences of varying lengths. The mathematical formulation of the relative position-aware scaled dot-product attention is defined as follows:

(5)

Where , , and are the matrices comprising the query, key, and value vectors, respectively; denotes the dimension of the and matrices. Additionally, and refer to the relative positional encoding matrices representing the positional bias and positional correction, respectively.

The relative multi-head attention mechanism incorporates relative positional information into the calculation of attention weights by decoupling content-based and position-based components. This allows the attention scores to simultaneously reflect both the content correlation between the query and key vectors and their relative displacement. The output of the relative multi-head attention mechanism is defined as follows:

(6)

(7)

Where , , , , and are learnable weight matrices for the th head.

Model training

In this study, the MoCETSE model was trained using a five-fold cross-validation strategy. The model was developed using the PyTorch (v1.10.0) framework and executed on an NVIDIA A100 GPU (40 GB) with Python 3.9.7 and CUDA 11.3. Model weights were initialized using the Xavier initialization method, and the Adam optimizer was employed for gradient updates. The initial learning rate was set to 5 × 10^-5 with a weight decay of 4 × 10^-5. The batch size and maximum number of epochs were set to 32 and 30, respectively. To prevent overfitting, an early stopping mechanism with a patience of 5 epochs was implemented by monitoring the F1 score. Within the transformer blocks, the dropout rates for the attention mechanism, multi-head attention layers, and feed-forward networks were configured as 0.05, 0.4, and 0.4, respectively.

Performance assessment

This study adopts a multi-metric approach to evaluate model performance in both cross-validation and independent testing. Specifically, we employed macro-averaged prediction accuracy, F1 score, area under the receiver operating characteristic curve (AUC), and area under the precision-recall curve (AUPRC) for comparison. The macro-averaged prediction accuracy is the arithmetic mean of the accuracies across all classes. This metric assigns equal weight to each class, effectively mitigating bias caused by class imbalance and fairly reflecting the model’s classification ability on minority classes. For the multiclass classification task of predicting five types of secreted effectors, the receiver operating characteristic (ROC) curve provides an intuitive visualization of the model’s ability to distinguish between positive and negative samples across different threshold settings for each category. Additionally, the multi-class confusion matrix provides a detailed view of classification results for each protein category. A higher AUC value (approaching 1) indicates superior discrimination, while a higher AUPRC value reflects a stronger capability to identify positive instances in precision-sensitive contexts. When comparing performance with existing binary and multi-class methods on benchmark datasets, we utilized several standard metrics, including accuracy (ACC), recall (REC), precision (PR), F1 score (F1), and Matthews correlation coefficient (MCC), to ensure a comprehensive assessment.

(8)

(9)

(10)

(11)

(12)

Where TP, TN, FP, and FN represent the numbers of true positives, true negatives, false positives, and false negatives, respectively. All evaluation metrics were calculated using the scikit-learn library in Python.

Benchmark with existing popular methods

To ensure a fair comparison with existing mainstream prediction methods, we selected 11 widely used effector secretion system prediction tools as comparative models and conducted unified evaluations through their official online web servers. The access links to these tools are provided in S2 Table.

In the comparative experiments, we selected benchmark test datasets corresponding to the prediction task types of each method and submitted them to the online web servers of the baseline methods to obtain the prediction results for performance comparison. All methods were evaluated under exactly the same testing conditions. Specifically, for the binary classification tasks, T1SEstacker [9] used Dataset S3, Bastion3 [10], T3SEpp [11] and EP3 [12] used Dataset S4, Bastion4 [13], CNN-T4SE [14], T4SEfinder [15], and T4SEpp [16] used Dataset S5, while Bastion6 [18] used Dataset S6. For the multi-class classification tasks, DeepSecE [27] and BastionX [43] were each evaluated separately on the above benchmark test sets.

UMAP-based embedding feature analysis

To visualize and compare the secreted effector embeddings learned by MoCETSE with the high-dimensional sequence features extracted by the pre-trained ESM-1b model, we employed the Uniform manifold approximation and projection (UMAP) algorithm via the umap-learn Python package for dimensionality reduction [44]. The local neighborhood size and the minimum distance between embedded points were set to 15 and 0.1, respectively. The UMAP experimental scripts are available at https://github.com/YihangLin123/MoCETSE.

Genome-wide effector prediction pipeline

Following the methodological framework of DeepSecE [27], we integrated Macsyfinder [45], a dedicated tool for bacterial secretion system detection, with our MoCETSE model. Specifically, this pipeline first harnesses Macsyfinder to search for structural component proteins of secretion systems based on hidden markov models (HMMs). Subsequently, it precisely identifies secretion system gene clusters by applying predefined spatial colocalization rules, including constraints on the inter-gene distance between adjacent components and the minimum component threshold. Based on these identified secretion system architectures, MoCETSE then scans all Coding Sequences (CDSs) within the bacterial genome, thereby enabling the accurate identification of potential secreted effector proteins.

Three representative Gram-negative bacterial strains, Pseudomonas syringae pv. tomato str. DC3000 (T3SE), Legionella pneumophila subsp. pneumophila str. Philadelphia 1 (T4SE), and Pseudomonas aeruginosa PAO1 (T6SE), were employed to evaluate the genome-wide predictive capability of MoCETSE for secretion system effectors. Recall was calculated as the proportion of experimentally validated effectors detected by MoCETSE:

(13)

Where denotes the number of experimentally validated secreted proteins, and represents the number of overlapping proteins between the experimentally identified and predicted secreted protein sets.

Sequence significance map

Leveraging the relative position multi-head attention mechanism embedded in the transformer module, the MoCETSE model enables quantitative assessment of the functional importance of amino acid residues within protein sequences. Specifically, by extracting attention weights from the multi-head attention matrix, the model transforms complex inter-residue interactions into quantifiable residue contribution scores. During this process, the model first aggregates the outputs from all attention heads and then calculates a global weighted average by incorporating relative positional weights, thereby characterizing the associative influence of specific residues on the entire sequence. The mathematical formulation is defined as follows:

(14)

Where denotes the relative positional weight between amino acids at positions and , and represents the length of the protein sequence. The term refers to the attention weight between positions and , as computed by the attention head. A higher value of indicates a stronger association between the two positions. To visualize the learned positional importance, the Python toolkit Logomaker [46] is utilized to generate the sequence significance map. The map enables the intuitive characterization of key biological motifs within the secretory protein sequences.

UMAP visualization of secretion protein embeddings

To evaluate the discriminative ability of the protein embedding features learned by the model, dimensionality reduction and visualization analysis were performed on the embedding vectors of secretory and non-secretory proteins in the training set (Dataset S1) [44]. UMAP maps the secretion embeddings generated by the MoCETSE model and the high-dimensional features generated by the pre-trained language model ESM-1b into a two-dimensional space, providing an intuitive visualization of the distribution relationships among the samples (Fig 2). The results showed that the embeddings of MoCETSE formed clear clusters in the low-dimensional space: the five different types of effector proteins and non-effector proteins each aggregated into distinct clusters, with clear boundaries between them (Fig 2B). Further observation revealed that samples within the same category were tightly clustered, indicating high consistency and homogeneity of sequence features. In contrast, the clusters of different categories displayed significant spatial separation, demonstrating that MoCETSE effectively distinguished the sequence features of varying protein types during the encoding process. This stands in sharp contrast to the ESM-1b projection, which showed considerable overlap between categories and poorly defined clustering boundaries (Fig 2A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. UMAP visualization of effector and non-effector protein clusters.

(A) UMAP clustering projections of ESM-1b sequence embeddings. (B) UMAP clustering projections of secretion embeddings from MoCETSE. Different colors represent distinct effector types (T1SE–T4SE and T6SE) and non-effector proteins.

https://doi.org/10.1371/journal.pcbi.1013397.g002

Furthermore, the secretion embeddings from DeepSecE (S3 Fig) exhibit noticeable overlaps between certain effector categories, particularly between T3SEs and T4SEs. Their samples show a significant intersection in the UMAP projection, leading to poorly defined cluster boundaries. In contrast, MoCETSE yields more distinct clustering and effectively minimizes such overlaps, demonstrating superior discriminative capacity across multiple effector classes. These results suggest that MoCETSE not only captures category-specific sequence features related to secretion systems but also substantially enhances the fine-grained classification of effector proteins.

Notably, as observed in Figs 2 and S3, the T2SE cluster (orange) partially overlaps with some T4SE samples in the visualization results of both MoCETSE and DeepSecE. This phenomenon can be attributed to the structural and evolutionary homology between the type II secretion system and the type IV pilus (T4P) system [3]. Such intrinsic biological similarities may cause certain T2SEs to be situated in close proximity to T4SEs within the latent embedding space.

Performance evaluation of MoCETSE in effector protein prediction

We evaluated the performance of the MoCETSE model via five-fold cross-validation, where the validation set was partitioned from the training set (Dataset S1). To assess the model’s generalization capability, we conducted additional testing on an independent test dataset (Dataset S2). In the model evaluation, we employed the one-vs-rest strategy and integrated the prediction outputs of the validation dataset during the cross-validation phase to separately plot the ROC curves for the five major types of secreted proteins and non-secreted proteins (Fig 3A). The AUC values for non-effector proteins and the five major secreted effector proteins ranged from 0.930 to 0.992, indicating the model’s excellent classification ability. The multi-class confusion matrix (with percentages rounded to one decimal place) illustrates the model’s prediction sensitivity and misclassification rates across all categories (Fig 3B). MoCETSE achieved a prediction sensitivity of approximately 95.0% for non-effector proteins, demonstrating its effectiveness in reducing the likelihood of false positives. For T1SEs, the sensitivity reached 90.6%, potentially due to their characteristically longer sequence lengths. Conversely, T2SEs exhibited a lower sensitivity of 76.5%, which can be attributed to their limited representation in the training set. Furthermore, we observed relatively higher misclassification rates between T3SE, T4SE, and T6SE; this may stem from their similar secretion mechanisms involving protein translocation across multiple bacterial membranes [47].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Performance evaluation of MoCETSE for secretion system effector protein prediction.

(A to D) Model performance assessed via five-fold cross-validation (A,B) and independent testing (C,D). ROC curves illustrate the classification performance for each effector type. Sensitivity values for each class are indicated along the diagonals of the corresponding confusion matrices. (E–H) Comparison of AUC and AUPRC metrics across different model architectures or training strategies for the five protein categories under five-fold cross-validation (E,F) and independent testing (G,H). Error bars represent the 95% confidence intervals.

https://doi.org/10.1371/journal.pcbi.1013397.g003

On the independent test dataset, MoCETSE exhibited robust generalization performance (Fig 3C), with AUC values for non-effector proteins and the five effector proteins ranging from 0.979 to 0.998. Except for non-secreted effector proteins, the prediction sensitivities for all other effector protein types exceeded 90% (Fig 3D). These results underscore the model’s capability to reliably identify five types of secreted effector proteins while maintaining exceptional robustness and generalization.

Comparison of different foundation methods

In this study, we developed multiple models to conduct a comprehensive performance comparison. Initially, a PSSM-CNN prediction model based on PSSM features was developed using a three-layer convolutional neural network architecture. Subsequently, two pre-trained protein language models, TAPE [48] and ESM-1b [21], were employed as the base feature extraction modules, and a series of derivative models were further constructed based on ESM-1b. To investigate the impact of different training strategies on model performance, two distinct schemes were adopted: Linear Probing and Fine-tuning. Specifically, the Fine-tuning strategy involved unfreezing and optimizing only the final layer of the pre-trained model, whereas the Linear Probing strategy kept all parameters of the pre-trained model frozen and introduced only a linear classifier at the output layer. In addition, using the one-dimensional (1D) embedding features generated by ESM-1b, three classical machine learning algorithms, including the SVM, Random forest, and XGBoost, were employed to build classifiers. Moreover, the DeepSecE [27] model was constructed by integrating ESM-1b with a 1D convolutional layer and a secretion-specific transformer. We evaluated the performance of different models using three metrics: ACC, F1 score, and AUPRC. Experimental results demonstrated that MoCETSE, which incorporates a target preprocessing network and a secretion-specific transformer, significantly outperformed all other comparative models under both cross-validation (Dataset S1) and independent testing (Dataset S2) settings (Table 1, S4 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Model architecture performances across five-fold cross-validation (CV) and independent (Ind) tests.

https://doi.org/10.1371/journal.pcbi.1013397.t001

In cross-validation testing, the PSSM-CNN model achieved an accuracy of 0.799 (95% CI: 0.772–0.826) and an F1 score of 0.712 (95% CI: 0.649–0.774). Incorporating pre-trained protein language models led to clear performance improvements. The TAPEBert-Linear model attained an accuracy of 0.816 (95% CI: 0.781–0.851) and an F1 score of 0.764 (95% CI: 0.686–0.842), while the larger-scale ESM-1b-Linear further improved accuracy to 0.847 (95% CI: 0.803–0.891) and F1 score to 0.771 (95% CI: 0.663–0.879). Notably, the ESM-1b Fine-tuning strategy did not result in a significant performance improvement. Furthermore, the classifiers developed by integrating ESM-1b with three machine learning algorithms further underscore the exceptional feature extraction capabilities of ESM-1b. Specifically, the ESM-1b-XGBoost demonstrated superior performance, achieving an accuracy of 0.858 (95% CI: 0.830–0.886) and an F1 score of 0.786 (95% CI: 0.718–0.854). The DeepSecE model, leveraging a secretion-specific transformer, further enhanced the accuracy to 0.873 (95% CI: 0.835–0.911) and F1 score to 0.843 (95% CI: 0.785–0.901), substantiating the robust feature learning of specialized deep learning modules. Our proposed MoCETSE model outperformed all other models, with an accuracy of 0.878 (95% CI: 0.856–0.897) and an F1 score of 0.850 (95% CI: 0.811–0.889). When evaluated on the independent test set, MoCETSE improved the accuracy from 0.883 (95% CI: 0.836–0.931) to 0.905 (95% CI: 0.861–0.949) and the F1 score from 0.836 (95% CI: 0.782–0.889) to 0.867 (95% CI: 0.823–0.911) compared to DeepSecE, which utilizes a 1D convolutional layer. These results demonstrate that the TPN effectively refines high-dimensional PLM embeddings by filtering redundancy and capturing critical features for effector classification, while simultaneously highlighting the strong generalization capability of the MoCETSE framework.

To further evaluate the generalization capability of the model, 110 effector proteins in the independent test set were stratified into six groups based on their sequence identity relative to the training dataset. Local BLASTP was employed to determine the homology, and sequences were assigned to discrete identity intervals (<20%, 20–30%, 30–40%, 40–50%, 50–60%, and ≥60%) based on the highest identity score achieved under an E-value threshold of ≤0.01. As illustrated in S5 Fig, MoCETSE achieved accuracy scores of 0.874, 0.892, 0.947, 0.970, 0.957, and 0.957 across the six sequence identity intervals, respectively. Notably, MoCETSE secured the top performance in five out of the six groups, with the sole exception being the 50%–60% interval, where its accuracy was slightly lower than that of DeepSecE (0.957 vs. 0.985). In both cross-validation and independent testing, this study compared the AUC and AUPRC metrics of different models for non-secretory proteins and five categories of secretory effector proteins (Fig 3E–H). The results indicated that using a pre-trained protein language model as a sequence feature extractor can improve the models’ classification performance for various types of effector proteins. Overall, MoCETSE performed comparably to DeepSecE in secretory protein recognition tasks and exhibited a slight advantage over other derivative models based on the ESM-1b pre-trained language model.

Finally, we evaluated the effectiveness of the state-of-the-art pre-trained protein language model, ESM-2 [21], within our proposed framework. As illustrated in S3 Table, replacing ESM-1b with ESM-2 did not yield the anticipated performance improvements. This observation is likely attributable to the relatively constrained size of our training dataset (comprising fewer than 3,000 sequences). Although ESM-2 features a more extensive parameter scale and enhanced representative power, its inherent complexity typically necessitates a larger and more diverse data volume to fully manifest its advantages. On a limited dataset, such a high-capacity model is more susceptible to overfitting. In contrast, ESM-1b provides more robust and generalizable feature representations that better align with our current data scale.

Performance comparison against existing popular models

The MoCETSE model aims to predict the secretion system effector proteins (T1SEs–T4SEs and T6SEs) in Gram-negative bacteria. To objectively assess its predictive performance, the present study compares MoCETSE with two mainstream multi-class models, DeepSecE [27] and BastionX [43], as well as nine representative binary classification models, using benchmark datasets (Dataset S3–S6). Specifically, T1SEstacker [9] was used for T1SE prediction; Bastion3 [10], T3SEpp [11], and EP3 [12] for T3SE; Bastion4 [13], CNN-T4SE [14], T4SEfinder [15], and T4SEpp [16] for T4SE; and Bastion6 [18] for T6SE. Evaluation metrics included ACC, REC, PR, F1 score, and MCC.

The experimental results demonstrate that MoCETSE outperforms existing multi-class models in overall performance. For T3SE and T4SE prediction, its performance is comparable to the state-of-the-art binary classification models, while for T1SE and T6SE prediction, MoCETSE exhibits superior performance (S4 Table). In the T1SE classification task (Dataset S3), MoCETSE outperforms T1SEstacker, which relies on amino acid composition without RTX C-terminal motifs (accuracy 98.2% vs 92.9%, F1 score 0.947 vs 0.727, MCC 0.942 vs 0.691) (S6 FigA). For the T3SE classification task (Dataset S4), MoCETSE achieved an accuracy of 91.7%, F1 score of 0.918, and MCC of 0.835. Its performance is comparable to T3SEpp (a method integrating various biological features) (Fig 4A). In the T4SE classification task (Dataset S5), MoCETSE outperforms DeepSecE, which also uses a pre-trained protein language model (accuracy 98.3% vs 97.8%, F1 score 0.949 vs 0.935, MCC 0.939 vs 0.911), and is second only to CNN-T4SE (Fig 4B). For the T6SE classification task (Dataset S6), MoCETSE demonstrated superior performance compared to Bastion6, which relies on PSSM predictors (ACC: 94.3%, F1: 0.946, MCC: 0.892), achieving an accuracy of 98.6%, F1 score of 0.957, and MCC of 0.920 (Fig 4C).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Benchmarking results of MoCETSE against mainstream secretion effector protein prediction models.

(A–C) Performance comparison of MoCETSE with mainstream prediction models on T3SE (A), T4SE (B), and T6SE (C) prediction tasks. (D–F) Comparison of true positive and false positive predictions from different models on the independent test set (Dataset S2), which contains 150 non-effectors, 20 T1SEs, 10 T2SEs, 30 T3SEs, 30 T4SEs, and 20 T6SEs. The heatmaps illustrate the distribution of predicted samples across various effector protein types. (G) Evaluation of the prediction efficiency of each model on the independent test dataset (Dataset S2) containing 260 effector proteins.

https://doi.org/10.1371/journal.pcbi.1013397.g004

Although MoCETSE has not yet surpassed the popular T3SE prediction model Bastion3 and the T4SE prediction model CNN-T4SE in terms of benchmark performance, it demonstrates a significant advantage in minimizing cross-type misclassification on the independent test set (Dataset S2). Binary classifiers often mistakenly predict other types of secretion effector proteins as the target type. In contrast, the MoCETSE model effectively controls the false positive rate when confronted with multiple effector proteins, reducing the likelihood of misclassifying other types of secretion effector proteins (Figs 4D–F,S6C). For instance, although Bastion3 accurately identifies all T3SEs in the independent test dataset, it misclassifies 9 T1SEs and 12 T4SEs as T3SEs (Fig 4D). Similarly, CNN-T4SE misclassifies 10 T1SEs and 4 T3SEs as T4SEs (Fig 4E). Furthermore, we evaluated computational efficiency by comparing the runtime of MoCETSE with other classification methods on Dataset S2. The results indicate that MoCETSE completes predictions in less than one minute, demonstrating superior computational efficiency compared to existing models (Fig 4G).

Ablation experiments

To evaluate the contribution of each core component to the overall performance of the proposed model, we designed and conducted a series of ablation experiments. Using a framework consisting of a 1D convolutional layer (1D Conv) and multi-head attention (MHA) as the baseline architecture, we systematically integrated TPN and relative position encoding multi-head attention (RPE-MHA). The functional efficacy of each module was quantified by evaluating four model configurations across both five-fold cross-validation (CV) and an independent test set (Ind) (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Performance comparison of different model variants in ablation experiments under five-fold cross-validation (CV) and independent (Ind) test settings.

https://doi.org/10.1371/journal.pcbi.1013397.t002

The experimental results demonstrate that MoCETSE, the final model integrating both TPN and RPE-MHA, significantly outperforms the baseline (1D Conv + MHA) across all evaluation metrics. In the five-fold cross-validation, MoCETSE achieved the superior performance with an ACC of 0.878 (95% CI: 0.856–0.897) and an F1 score of 0.850 (95% CI: 0.811–0.889). Notably, on the independent test set, the generalization performance of MoCETSE improved further, reaching an ACC of 0.905 (95% CI: 0.861–0.949) and an F1 score of 0.867 (95% CI: 0.823–0.911), representing a substantial gain over the baseline (ACC: 0.883, F1: 0.836).

Component decomposition analysis revealed that replacing the conventional 1D Conv with TPN, while the MHA remained unchanged, resulted in improvements across all performance metrics. Specifically, the ACC in CV increased from 0.873 to 0.875, and the F1 score improved from 0.843 to 0.847. On the Ind, the ACC rose from 0.883 to 0.893, and the F1 score increased from 0.836 to 0.848. These results indicate that TPN captures the underlying features of target sequences more precisely than 1D Conv, providing a more informative representational basis for subsequent feature fusion and interaction, and thus improving the overall decision-making ability of the model. Furthermore, a significant synergistic effect was observed between RPE-MHA and the different preprocessing modules. Under the 1D Conv-based configuration, the introduction of the RPE mechanism did not yield performance gains. However, when combined with TPN, model performance improved markedly; specifically, the ACC on the independent test set increased from 0.872 to 0.905, and the F1 score rose from 0.831 to 0.867. This comparison indicates that RPE effectively compensates for the lack of relative spatial information in the high-dimensional features extracted by TPN, thereby enabling the model to more accurately capture long-range dependencies within the sequences.

To further verify the robustness of each MoCETSE module across different effector protein classes, especially its generalization ability on sample-scarce categories, we performed a stratified ablation experiment on T1SE, T2SE, T3SE, T4SE, and T6SE using the independent test set (Dataset S2). Our results showed that the component improvements in MoCETSE led to consistent performance gains, both in the T1SE and T2SE categories with relatively small sample sizes and in other categories with more abundant samples (Figs 5A–C, S7). In particular, for T2SE—the category with the highest prediction difficulty—the baseline model (1D-Conv + MHA) achieved an F1 score of only 0.550, suggesting notable limitations in its ability to capture relevant sequence features. After integrating TPN and RPE-MHA modules, MoCETSE yielded performance improvement on this class, with ACC and F1 score reaching 0.961 and 0.636, respectively (S7 FigB). Taken together, these results demonstrate the effectiveness of the proposed improvements in capturing and representing the characteristic features of the target sequences.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Stratified ablation analysis and genome-wide effector protein prediction.

(A–C) Performance comparison of four model configurations across different effector protein classes. The ablation results for (A) T3SEs, (B) T4SEs, and (C) T6SEs on the independent test set are shown. (D–F) Comparison of genome-wide effector protein predictions between MoCETSE and DeepSecE across three representative Gram-negative bacterial strains: (D) Pseudomonas syringae pv. tomato str. DC3000 (T3SE); (E) Legionella pneumophila subsp. pneumophila str. Philadelphia 1 (T4SE); and (F) Pseudomonas aeruginosa PAO1 (T6SE). The Venn diagrams illustrate the overlap between the model predictions and experimentally verified effector proteins (Verified).

https://doi.org/10.1371/journal.pcbi.1013397.g005

Genome-wide prediction of secreted proteins

Genome-wide prediction of secreted proteins is a pivotal strategy for elucidating the pathogenicity, evolutionary dynamics, and distribution patterns within bacterial populations [49]. To evaluate the practical utility and generalization performance of MoCETSE in real-world scenarios, we integrated our model with Macsyfinder [45], a specialized tool for structural identification of secretion systems. This integration established a high-throughput pipeline for the systematic recognition of effector proteins across the genomes of Gram-negative bacteria (detailed in the “Materials and methods” section).

To evaluate the genome-wide prediction capability of MoCETSE for secreted proteins, three representative Gram-negative bacterial strains were selected: Pseudomonas syringae pv. tomato str. DC3000 (T3SE) [National Center for Biotechnology Information (NCBI) accession number: NC_004578.1], Legionella pneumophila subsp. pneumophila str. Philadelphia 1 (T4SE) (NC_002942.5), and Pseudomonas aeruginosa PAO1 (T6SE) (NC_002516.2) (Fig 5D–F, S5 Table). For P. syringae pv. tomato str. DC3000 (T3SE), both MoCETSE and DeepSecE successfully detected 30 experimentally validated effectors, achieving a recall rate of 90.9%.

Notably, MoCETSE identified significantly fewer candidates (273 vs 330), leading to a higher precision (11.0% vs 9.1%) and F1 score (0.196 vs 0.165). These results, combined with a superior AUPRC (0.505 vs 0.412), demonstrates that MoCETSE can enhance prediction specificity without compromising sensitivity. For L. pneumophila subsp. pneumophila str. Philadelphia 1 (T4SE) and P. aeruginosa PAO1 (T6SE), MoCETSE achieved higher recall rates than DeepSecE (96.7% vs 86.3% and 90.0% vs 80.0%, respectively). Although MoCETSE produced more candidates in these two cases and yielded slightly lower precision, its F1 scores remained competitive (0.829 vs 0.827 for T4SE), indicating a robust balance between accuracy and discovery potential. Furthermore, MoCETSE exhibited a higher AUPRC value in the T4SE case, further confirming its superior discriminative power across different thresholds. In practical genomic screening, where minimizing false negatives is often prioritized to avoid missing functional effectors, MoCETSE offers a more comprehensive and reliable solution for identifying experimentally validated secreted proteins in Gram-negative bacteria.

Relative position attention identification sequence motifs

Signal peptides located at the N- or C-terminus of protein sequences are pivotal and widely recognized hallmarks of many secretory proteins, particularly those exported via classical secretion pathways [3]. To evaluate the universality and precision of MoCETSE in identifying functional secretion signals across diverse substrates, we selected representative effector proteins from T1SS, T2SS, T3SS, T4SS, and T6SS. We characterized their secretion motifs through sequence attention weight analysis, which allows for the direct visualization of the model’s focus during feature extraction (detailed in the “Materials and methods” section).

In the analysis of the Escherichia coli T1SS effector HlyA (UniProt accession P08715), MoCETSE not only precisely localized the critical non-cleavable C-terminal secretion signal within the last 20 residues (S8 FigB) but also accurately identified multiple RTX (Repeat in ToXins) motifs characterized by conserved aspartic acid (D) residues distributed between positions 700 and 800 (S8 FigA) [50]. For the Vibrio cholerae serotype O1 T2SS effector CtxA (UniProt accession P11439), the model captured the Sec-dependent signal peptide at the N-terminus (residues 1–18) (S8 FigC); this finding is highly consistent with the established biological feature that CtxA, as a T2SS substrate, must first be translocated to the periplasm via the Sec pathway [51]. Similarly, MoCETSE precisely identified key functional regions of the Salmonella typhimurium T3SS secretory protein SptP (UniProt accession P74873), including its N-terminal secretion signal (residues 1–15) and the chaperone-binding domain (CBD, residues 15–35) associated with specific chaperone binding (Fig 6A) [52]. In the characterization of the Legionella pneumophila subsp.pneumophila str. Philadelphia 1 T4SS effector VpdB (UniProt accession Q9KNE5), the model localized the N-terminal auxiliary initiation sequence (Fig 6B) and further pinpointed the core C-terminal secretion signal at residues 580–600 (Fig 6C). This result provides robust support for the biological mechanism reported in the literature regarding the recognition of C-tail features by the Dot/Icm system for translocation [53]. Regarding the Vibrio cholerae serotype O1 T6SS effector VasX (UniProt accession Q9KNE5), MoCETSE successfully captured sparse conservation patterns within the first 100 N-terminal residues and precisely focused on key sites highly consistent with the MIX (Marker for Type VI effectors) motif, assigning particularly high attention weights to residues D78, Y83, and K89 (S8 FigD). This identification aligns perfectly with the known mechanism of T6SS-mediated protein translocation via specific MIX signals [54]. Collectively, these experimental results demonstrate that MoCETSE can transcend the limitations of sequence position, accurately identifying characteristic motifs of various secretion systems across the entire sequence scale.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Visualization of sequence motifs and attention weight distributions for T3SS and T4SS effector proteins.

(A) Attention weight distribution for the Salmonella typhimurium T3SS effector protein SptP (P74873). The sequence map highlights the high attention weights assigned to the N-terminal secretion signal and the chaperone-binding domain (CBD). (B–C) Attention weight distribution for the Legionella pneumophila subsp. pneumophila str. Philadelphia 1 T4SS effector protein VpdB (Q5ZW60). (B) Highlights the N-terminal initiating auxiliary sequence recognized by the model. (C) Displays the core secretion signal concentrated in the C-terminal region.

https://doi.org/10.1371/journal.pcbi.1013397.g006