2.1.1. Genome assembly and quality filtering.

Viral genomes were assembled using a hybrid strategy combining BWA-MEM v0.7.17 reference-guided assembly (against SARS-CoV-2 reference NC_045512.2, with consensus generation via BCFtools v1.15) and MEGAHIT v1.2.9 de novo assembly (k-mer sizes 21,41,61,81,99). Assemblies were retained only if they met stringent quality criteria: length >29,000 bp, < 1% ambiguous bases (N), coverage ≥100 × , and ≥95% sequence identity to the reference genome. This process yielded 8,000–10,000 high-quality viral genomes.

2.1.2. ORF prediction & filtering.

ORF prediction was performed using Prodigal v2.6.3 in metagenomic mode on assembled contigs. Predicted ORFs were validated against the SARS-CoV-2 reference genome (NC_045512.2) using BLASTN, retaining only those with ≥85% sequence identity, ≥ 80% query coverage, and length between 300–900 bp. This yielded 235,441 high-confidence ORF candidates for downstream analysis.

2.1.3. Clustering & functional annotation.

To reduce redundancy while preserving sequence diversity, the 235,441 validated ORFs were subjected to two-stage clustering using VSEARCH v2.21.1: first at 90% sequence identity, then refined at 95% identity. Cluster representatives were aligned against UniProt/Swiss-Prot (release 2024_01) using DIAMOND v2.0.15 in sensitive mode (--sensitive). The initial alignment yielded 62,689 ORFs with putative UniProt matches. For each match, protein names, functional descriptions, and Gene Ontology (GO) terms were extracted for downstream label assignment.

2.1.4. Functional labeling.

Functional labels (Enzymatic, Structural, Transport, Other) were assigned to the 62,689 UniProt-mapped ORFs using hierarchical keyword matching against a curated dictionary of 847 functional terms derived from UniProt protein descriptions and Gene Ontology annotations. The classification hierarchy prioritized primary molecular functions:

• Enzymatic: Proteins with catalytic activity (GO:0003824), including polymerases, proteases, methyltransferases, and helicases
• Structural: Proteins involved in virion assembly and structural maintenance (GO:0019012), including spike, envelope, membrane, and nucleocapsid proteins
• Transport: Proteins mediating membrane transport and ion channels (GO:0006810, GO:0015267)
• Other: All remaining ORFs, including regulatory proteins, accessory factors, and hypothetical proteins

ORFs with ambiguous or conflicting annotations (e.g., proteins with both enzymatic and structural roles) were manually reviewed by two independent annotators, achieving 94.2% inter-annotator agreement (Cohen’s κ = 0.92). After removing ambiguous cases, this resulted in 53,575 unique labeled ORF sequences with unambiguous functional assignments.The ‘Other’ category encompasses a functionally heterogeneous set of accessory proteins with distinct biological roles, as summarized in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Biological functions and mechanisms of major SARS-CoV-2 accessory proteins in the “Other” category.

https://doi.org/10.1371/journal.pone.0349393.t001

2.1.5. Data augmentation and class balancing.

To mitigate class imbalance and enhance model generalization to natural SARS-CoV-2 diversity, the 53,575 unique labeled ORFs were augmented with sequence variants representing different viral lineages and related coronaviruses:

1. (1). GISAID lineage variants: 11,500 sequences from major SARS-CoV-2 lineages (Alpha: B.1.1.7, Beta: B.1.351, Delta: B.1.617.2, Omicron: B.1.1.529, and reference Wuhan-Hu-1) were selected from GISAID (accessed November 2024). Variants were filtered to retain only those with ≥2% sequence divergence from their corresponding base ORF to ensure meaningful diversity while maintaining functional annotation consistency.
2. (2). Cross-species coronavirus homologs: 936 homologous sequences from related coronaviruses (SARS-CoV, MERS-CoV, HCoV-OC43, HCoV-229E) with 70−75% sequence identity to SARS-CoV-2 proteins were included to improve cross-species generalization and robustness to phylogenetic variation.

This augmentation strategy expanded the dataset from 53,575–66,011 total samples (53,575 base + 11,500 GISAID variants + 936 cross-species homologs), achieving balanced class distribution across all functional categories: Enzymatic (16,503, 25.0%), Structural (16,502, 25.0%), Transport (16,503, 25.0%), and Other (16,503, 25.0%).

Each augmented sample was verified to retain the same functional annotation as its corresponding unique ORF representative through automated sequence alignment (≥85% identity) and manual spot-checking of 500 random samples, ensuring label consistency across variants and homologs.

2.1.6. Dataset partitioning.

The augmented dataset (66,011 samples) was partitioned into training, validation, and test sets using stratified random splitting (scikit-learn v1.2.0, random_state = 42) with proportions of 71.4%, 14.3%, and 14.4%, respectively (Fig 1), ensuring balanced class distribution (~25% per class) in each split. To prevent data leakage and ensure unbiased evaluation, we enforced a critical constraint during splitting: all sequence variants derived from the same unique ORF (i.e., its GISAID lineage variants and cross-species homologs) were assigned to the same partition. Specifically:

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Overview of the SARS-CoV-2 multimodal dataset construction pipeline.

Raw metagenomic reads (SRX28474964) were processed via quality control and hybrid assembly (BWA-MEM + MEGAHIT), followed by ORF prediction (Prodigal) and BLASTN validation to yield 235,441 ORFs. High-confidence, non-redundant ORFs (53,575) were obtained through VSEARCH clustering and DIAMOND mapping to UniProt/Swiss-Prot. Functional categories were assigned via hierarchical keyword matching (Enzymatic, Structural, Transport, and Other), achieving 94.2% expert agreement (κ = 0.92). Data augmentation was performed by incorporating GISAID lineage variants (11,500 sequences from Alpha, Beta, Delta, and Omicron) and cross-species coronavirus homologs (936 sequences), expanding the dataset from 53,575 to 66,011 total samples. The augmented dataset then underwent leakage-aware stratified partitioning: all variants of the same unique ORF were assigned to the same partition (training, validation, or test) to prevent data leakage, resulting in final splits of 47,091 training, 9,417 validation, and 9,503 test samples. Each sample is represented by multi-scale k-mer encodings (4–7-mer) and 768-dimensional semantic annotation embeddings.

https://doi.org/10.1371/journal.pone.0349393.g001

• If a unique ORF was assigned to the training set, all its augmented variants (GISAID and homologs) were exclusively placed in the training set
• Similarly for validation and test sets This “variant grouping” strategy prevents the model from being trained on one variant of an ORF and tested on another highly similar variant (≥85% identity), which would artificially inflate performance metrics. Split integrity was verified using two complementary approaches:
  1. CD-HIT-EST v4.8.1 clustering confirmed no sequences with >85% identity across different splits
  2. BLASTN alignment (E-value < 1e-5) verified no cross-split sequence pairs with >85% identity and >80% coverage Final split sizes: Training (47,091 samples, 71.4%), Validation (9,417 samples, 14.3%), Test (9,503 samples, 14.4%). Manual inspection of 100 random test samples confirmed that none had training set sequences with >85% BLAST identity, validating the leak-proof partitioning.

2.1.7. Task formulation.

We formulate viral ORF functional prediction as a four-class supervised classification task, mapping nucleotide sequences to functional categories {Enzymatic, Structural, Transport, Other}. Input representation: Each sample x consists of two modalities: 1. Sequence modality: A nucleotide sequence bp, encoded as multi-scale k-mer representations (k ∈ {4, 5, 6, 7}) to capture hierarchical sequence patterns [15,16]. 2. Annotation modality: A functional description text a (extracted from UniProt/Swiss-Prot), embedded as a 768-dimensional vector using BioBERT to capture semantic functional information. Output: A functional label y ∈ {Enzymatic, Structural, Transport, Other}. Objective: The model learns a mapping function f: (s, a) → y that jointly leverages sequence patterns and semantic annotations. This multimodal formulation enables systematic evaluation of sequence-only, annotation-only, and integrated learning strategies under a unified experimental framework [1], facilitating direct comparison of different architectural designs and knowledge distillation approaches.

2.2.1. Textual annotation embedding.

We extract curated functional annotations from UniProt, prioritizing manually reviewed Swiss-Prot records when available, including protein names, function descriptions, catalytic activity, subcellular location, and relevant domain/feature notes. These texts are embedded using a pretrained BioBERT model [26], yielding 768-dimensional semantic vectors. A projection layer with ReLU activation aligns the annotation embeddings to the sequence feature space [27].

The hierarchical k-mer resolutions are selected based on their specific biological information density:

• 4-mer: Captures conserved short motifs and local binding signals.
• 5-mer & 6-mer: Reflect local secondary structure-related segments.
• 7-mer: Encodes broader regional contextual features and evolutionary constraints.

2.2.2. Gated multimodal fusion.

To implement the interaction between sequence and annotation modalities, we employ a Gated Multimodal Fusion mechanism.

Let be the multi-scale sequence feature and be the cross-modal interaction vector. The fused output is calculated as:

(1)

where is a learnable gating vector. This design allows the model to dynamically prioritize text embeddings when environmental sequence signals are fragmented or ambiguous.

2.2.3. Triple knowledge distillation.

To compress the model without sacrificing accuracy, we implement a triple knowledge distillation strategy. transferring knowledge from the structure-aware teacher model to the cross-modal student model. This includes the following:

Response-based Distillation: The student learns from the teacher’s softened output distributions using a temperature T = 3.0T, optimized via Kullback–Leibler divergence:

(2)

Feature-based Distillation: Intermediate hidden states from the student are aligned to those from the teacher through a feature adapter layer:

(3)

The Triple Knowledge Distillation facilitates knowledge transfer via a combined objective function :

(4)

Hyperparameters were tuned via grid search on the validation set. The optimal ranges and their biological roles are defined as:

1. (1). , (Task Weight): Controls the supervision from hard labels ,cross-entropy). Grid search range: {0.1, 0.2, 0.3, 0.4, 0.5}; optimal value: α = 0.3.
2. (2). (Response Weight): Governs soft-target distillation with a temperature to capture inter-class correlations. Grid search range: {0.3, 0.4, 0.5, 0.6, 0.7}; optimal value: β = 0.5.
3. (3). γ ∈ [0.1, 0.3], (Feature Weight): Aligns student intermediate hidden states to the teacher’s feature space. Grid search range: {0.1, 0.15, 0.2, 0.25, 0.3}; optimal value: γ = 0.2.
4. (4). δ ∈ [0.1, 0.2], (Attention Weight): Transfers Structure-Aware Attention maps (constrained by structure S from PDB 6VXX) from the teacher to the student, ensuring the efficient student model inherits spatial sensitivity to functional regions. Grid search range: {0.1, 0.12, 0.15, 0.18, 0.2}; optimal value: δ = 0.15.

This strategy effectively improves the student’s discriminative power and generalization ability in multimodal learning.

All experiments were conducted on a single NVIDIA V100 GPU (32 GB VRAM) with FP16 mixed-precision training enabled via PyTorch automatic mixed precision. The AdamW optimizer was used with a learning rate of 3 × 10 ⁻ ⁵, weight decay of 0.01, and batch size of 64, trained for 5 epochs with a cosine annealing learning rate scheduler and 100 warmup steps. Gradient clipping was applied at a maximum norm of 1.0 to stabilize training. The teacher model comprises 4 Transformer layers with hidden size 768 and 8 structure-aware attention heads; the student model comprises 2 Transformer layers with hidden size 384 and 4 cross-modal attention heads. LoRA adapters (rank r = 8, α = 16, dropout = 0.05) were applied to all attention projection layers (query, key, value, and output projections) of the student model exclusively. Knowledge distillation used temperature T = 3.0 with loss weights α = 0.35 (cross-entropy), 1.5 (response-based KD), 0.05 (feature-based), and 2.0 (attention-based).

2.2.4. Ablation study.

To quantitatively evaluate the contribution of each core component in our ViralMultiNet framework, we designed a comprehensive ablation study. The impact of individual modules was isolated by systematically removing or replacing them, resulting in the following ablated configurations for comparison [28–30]:

1. (1). Unimodal Baselines: Models trained using only the sequence modality or only the annotation modality, thereby removing multimodal fusion.
2. (2). Standard Attention Mechanisms: Models where the specialized structure-aware attention (in the teacher) and the Flash Attention-accelerated cross-modal attention (in the student) were replaced with standard self-attention modules.
3. (3). Single-scale Encoding: Models utilizing only a single k-mer length (e.g., 4-mer or 7-mer) instead of the multi-scale representation.
4. (4). Without Knowledge Distillation: A student model trained directly with hard labels, without guidance from the pre-trained teacher model.
5. (5). Without Adversarial Training: Models trained without the embedding-level perturbation used to enhance robustness.

2.2.5. Methods – SOTA baseline comparison.

To rigorously evaluate the performance of our proposed ViralMultiNet, we compared it against several advanced baseline models encompassing both unimodal and multimodal architectures [31,8]. The selected baseline models include:

1. (1). HyenaDNA and GenSLM, which process genomic sequences using convolutional and recurrent neural architectures, respectively;
2. (2). BioBERT, a language model pre-trained on biomedical literature, which we fine-tuned to encode and classify functional annotation texts.
3. (3). DNABERT + CLIP, a multimodal baseline that encodes DNA sequences using DNABERT and aligns them with annotation embeddings through CLIP-style joint training.
4. (4). VirSorter2, a modular viral sequence identification tool that utilizes a collection of customized automatic classifiers and Hidden Markov Models (HMMs) to achieve consistent performance across diverse viral groups. Including this specialized baseline addresses the need to compare our framework against state-of-the-art methods specifically designed for viromics rather than general-purpose genomic models [11].

All models were trained and evaluated under identical dataset and experimental protocols, including consistent data splits (training, validation, and test sets) and evaluation metrics. To ensure a fair comparison, hyperparameters for each baseline model were individually optimized on the validation set. Model performance was assessed using accuracy, macro F1 score, area under the ROC curve (AUC), and recall, providing a comprehensive view of classification capability across varied class distributions [32–34].

2.7.1. Motivation and rationale.

While multi-scale k-mer representations have demonstrated strong performance in viral genome classification, the biological interpretability of such models remains an open question (Table 5). It is unclear whether attention mechanisms operating at different k-mer resolutions capture biologically meaningful signals or merely act as abstract statistical filters. To address this concern, we conducted a structure-aware interpretability analysis systematically mapping high-attention regions derived from multiple k-mer scales onto the three-dimensional architecture of the SARS-CoV-2 Spike protein. This analysis aims to evaluate whether the proposed multi-scale attention framework exhibits coherent and biologically grounded focus across known functional regions, thereby providing mechanistic insight beyond predictive accuracy [37].

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Biological interpretation of multi-scale k-mer attention.

https://doi.org/10.1371/journal.pone.0349393.t005

2.7.2. Multi-scale attention aggregation and region-level mapping strategy.

To enable biologically meaningful interpretation, attention signals extracted from individual k-mer scales (k = 4, 5, 6, and 7) were first aggregated at the region level rather than analyzed at the single-token resolution. Specifically, high-attention k-mers from each scale were projected back onto their corresponding genomic coordinates within the Spike coding region and overlapping k-mer hits were accumulated to generate continuous attention-enriched regions along the Spike sequence (Fig 4). This region-level aggregation mitigates scale-specific noise and allows attention patterns from different k-mer resolutions to be integrated into a unified representation. Importantly [40,41], this strategy preserves the locality of sequence features while enabling cross-scale consistency analysis, thereby facilitating direct comparison with known functional modules of the Spike protein [42–44].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Region-level attention distribution along the SARS-CoV-2 Spike protein derived from aggregated multi-scale k-mer attention scores.

The line plot shows the summed attention importance projected onto Spike residue positions (aa 300–900). Shaded regions indicate key functional domains, including the receptor-binding domain (RBD, residues 319–541), the S1/S2 cleavage region (681–685), and the fusion peptide (FP, residues 816–835). Prominent attention peaks consistently align with these biologically critical regions. Structural mapping onto the Spike protein (PDB ID: 6VXX) further confirms the spatial correspondence between sequence-level attention enrichment and three-dimensional functional modules.

https://doi.org/10.1371/journal.pone.0349393.g004

2.7.3. Structural mapping of attention-enriched regions onto the Spike protein.

To assess whether the aggregated attention signals correspond to biologically meaningful structures, attention-enriched regions identified along the Spike sequence were mapped onto the three-dimensional structure of the Spike protein (Fig 5). Structural visualization was performed using experimentally resolved Spike conformations (PDB ID: 6VXX), focusing on a single protomer to avoid redundancy from the trimeric assembly. Residue indices from the region-level attention profiles were directly aligned to the Spike protein structure, enabling spatial localization of attention-enriched regions in three-dimensional space [45–48]. This structure-aware mapping allows evaluation of whether sequence-level attention signals exhibit spatial coherence and whether they preferentially localize to known functional architectures of the Spike protein [12].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Three-dimensional structural visualization of attention-enriched regions on the SARS-CoV-2 Spike protein.

Residues corresponding to the receptor-binding domain (RBD, 319–541), the S1/S2 cleavage region (681–685), and the fusion peptide (FP, 816–835) are highlighted in red, orange, and purple, respectively, while the remaining regions are shown in grey. This structural annotation corresponds to the region-level attention distribution shown in Fig 4 and demonstrates the spatial localization of sequence-level attention onto known functional modules of the Spike protein. The structure is shown for a single protomer (chain A) from the closed-state Spike conformation (PDB ID: 6VXX).

https://doi.org/10.1371/journal.pone.0349393.g005

2.7.4. Convergence of multi-scale attention on Spike functional modules.

Structural mapping of attention-enriched regions revealed a consistent convergence of multi-scale attention onto three major functional modules of the Spike protein. Across all k-mer resolutions (k = 4–7), high-attention regions prominently localized to the receptor-binding domain (RBD), forming a spatially coherent cluster on the globular head of the Spike protomer. In addition, attention signals were recurrently enriched around the S1/S2 cleavage region and the fusion peptide within the S2 subunit, both of which play critical roles in Spike activation and membrane fusion. Notably, although individual k-mer scales capture sequence patterns at different resolutions, their aggregated attention profiles converge on the same functional architectures in three-dimensional space, indicating a robust and biologically grounded focus of the proposed multi-scale attention framework (Figs 4 and 5).

2.7.5. Biological implications and interpretability of the proposed framework.

The observed convergence of multi-scale attention onto key functional modules of the Spike protein provides strong evidence that the proposed framework captures biologically meaningful representations rather than relying on spurious sequence correlations. By consistently highlighting the receptor-binding domain, the S1/S2 cleavage region, and the fusion peptide [49–51], the model aligns its internal attention mechanisms with established molecular determinants of viral entry and infectivity. This structural interpretability supports the view that multi-scale attention enables the model to integrate local sequence motifs and broader contextual patterns into a coherent functional understanding of viral proteins. Importantly, these findings demonstrate that the proposed classification framework is not a black-box predictor but a structure-aware and biologically grounded model whose decision-making process can be systematically interrogated and validated [52].

2.7.6. Cross-system validation on the RdRp enzymatic machinery.

To further demonstrate the generalizability of our structure-aware attention mechanism beyond structural proteins, we extended the interpretability mapping to the RNA-dependent RNA polymerase (RdRp, NSP12).As shown in Fig 6, the aggregated attention peaks precisely align with critical enzymatic elements, including the nucleoside triphosphate (NTP) entry channel (residues 44–68) and the catalytic Motif A (residues 140–151).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Structural interpretability of region-level attention in RdRp (NSP12).

Top: Region-level attention distribution along the SARS-CoV-2 RNA-dependent RNA polymerase (RdRp, NSP12). Aggregated attention scores derived from the proposed multi-scale model are plotted along the NSP12 amino acid sequence. Shaded regions indicate known functional motifs, including the NTP entry region (residues 44–68, orange), the catalytic Motif A (residues 140–151, red), and the C-terminal thumb domain (residues 769–795, purple). Bottom: Three-dimensional structural mapping of the corresponding high-attention regions on the RdRp structure (PDB: 7BV2). The same regions are highlighted using consistent color coding, demonstrating that sequence-level attention peaks align with established functional and catalytic elements of RdRp. This consistency supports the biological relevance and interpretability of the proposed attention-based framework.

https://doi.org/10.1371/journal.pone.0349393.g006