2.1. Transcriptome conditional encoder

Let denote a normalized drug-induced gene expression profile of K genes, where x_k is the standardized expression value of gene k. The transcriptome conditional encoder is pretrained via self-supervised contrastive learning to extract robust latent representations that remain consistent under diverse perturbations. To explicitly model contextual dependencies among genes, we treat each gene as a token and apply feature-wise attention across all K genes within each sample. Since genes exhibit dependency relationships but no intrinsic sequential order, each one-dimensional gene expression value is first projected into a high-dimensional embedding space through a feed-forward network, and the resulting gene tokens are then processed by a multi-layer attention encoder. In this work, feature-wise attention and the gene-level transformer refer to the same encoder architecture. This design enables the model to capture gene contextual information while mitigating experimental noise.

Two augmented views of x are generated using a stochastic combination of four augmentation operators: feature dropout (D), random scaling (S), adaptive gaussian noise (G_adap), and structured gaussian noise (G_scruct). The augmentation process for the i-th view is defined in Equation (1):

(1)

where each operator is randomly parameterized to create diverse perturbations, and the composition order does not carry specific modeling significance. Specifically, feature dropout randomly masks gene features, random scaling perturbs expression values at the gene level, adaptive Gaussian noise injects feature-wise noise with variances estimated from the training set, and structured Gaussian noise is sampled from a multivariate Gaussian distribution N(0,Σ), where Σ is obtained by scaling and regularizing the empirical covariance matrix of the original data. Efficient sampling is implemented through Cholesky decomposition, and the sampled noise is directly added to the input expression profile.

Each augmented view is then encoded by the gene-level transformer f_θ(·) to produce contextualized gene representations, which are further compressed by a projection head into sample-level latent vectors, as defined in Equation (2):

(2)

where g(·) denotes the projection head and is the compressed transcriptomic representation used for contrastive learning. The model is optimized at the sample level using the InfoNCE loss [37,38]. Specifically, the two augmented views of the same expression profile form a positive pair, while representations from other samples in the same batch serve as negative pairs. The InfoNCE objective is defined in Equation (3):

(3)

where sim(·,·) denotes cosine similarity, τ is the temperature parameter, and B is the batch size. By minimizing this loss, the encoder learns robust and context-aware transcriptomic embeddings that preserve inter-gene dependencies while remaining invariant to noise and augmentation.

After pretraining, the transcriptome conditional encoder is frozen and used to map an input drug-induced expression profile x to a latent conditional vector . This projected representation is then passed to the downstream molecular generation module as the transcriptome-derived condition for candidate molecule generation.

2.2. Molecular generation module

Let denote the target SMILES token sequence of length N, where each s_n ∈V is a token from the SMILES vocabulary V. Given the transcriptome-derived conditional embedding c = g(f_θ(x)) produced by the pretrained and frozen transcriptome conditional encoder, the molecular generation module generates candidate molecules under the guidance of the biological condition. The module is implemented by fine-tuning a pretrained MolT5 model in a conditional generation manner.

To make the transcriptomic condition compatible with the hidden space of MolT5, the conditional embedding c is first transformed into a fixed-length sequence of condition tokens through a feed-forward condition-mapping network, as defined in Equation (4):

(4)

where FFN_cond(·) denotes the condition-mapping feed-forward network, K_C is the number of condition tokens, and d_T is the hidden dimension of MolT5. The resulting condition tokens C are used as the input to the MolT5 encoder. Through encoder self-attention, these tokens are contextualized into condition representations that guide SMILES generation through decoder cross-attention.

At decoding step n, let denote the embedding sequence of the previously generated SMILES tokens s_1,…,s_n-1. The initial encoder and decoder inputs are defined in Equation (5):

(5)

For each encoder layer l, the condition-token representations are updated by the MolT5 encoder block with an inserted lightweight adapter, as shown in Equation (6):

(6)

Let denote the final encoder output after L encoder layers. For each decoder layer l, masked self-attention is first applied to the embeddings of the previously generated SMILES tokens, followed by cross-attention over the encoded transcriptomic condition representations, as defined in Equation (7):

(7)

For clarity, residual connections, layer normalization, and dropout operations are omitted from the equations. Through this encoder-decoder interaction, the decoder attends to both the previously generated SMILES context and the transcriptome-derived condition representations, thereby enabling biologically conditioned molecular generation.

To efficiently leverage the pretrained molecular knowledge in MolT5, lightweight adapter modules are inserted into Transformer blocks, and selective layer unfreezing is applied during fine-tuning. Specifically, most pretrained MolT5 parameters remain frozen, while adapter parameters, the condition-mapping network FFN_cond, and selected upper-layer weights are updated. This strategy reduces the number of trainable parameters while improving adaptation to expression-driven molecule generation.

During training, teacher forcing is applied: when predicting token s_n, the ground-truth preceding sequence s₁,…,s_n-1 is fed into the decoder. The model is optimized using the conditional autoregressive negative log-likelihood, as defined in Equation (8):

(8)

where ψ denotes the trainable parameters of the conditional MolT5 generator, including the adapter modules, the condition-mapping network, and the selectively unfrozen MolT5 parameters. During inference, candidate SMILES sequences are generated autoregressively under the transcriptomic condition using stochastic decoding to improve molecular diversity.

2.3. Training and inference procedure of AET5

Alg. 1 summarizes the training and inference workflow of AET5. The overall framework is trained in two sequential stages. In the first stage, the transcriptome conditional encoder is pretrained using self-supervised contrastive learning on drug-induced expression profiles. For each expression profile x two augmented views are generated and optimized with the InfoNCE loss defined in Equation (3), enabling the encoder to learn robust transcriptome-derived representations. After pretraining, the transcriptome conditional encoder is frozen and used to extract the latent conditional embedding c.

In the second stage, the molecular generation module is fine-tuned using paired expression profiles and SMILES strings. The frozen conditional encoder provides the transcriptomic condition c, which is converted into condition tokens C as described in Equation (4). The condition tokens are fed into the MolT5 encoder, and the MolT5 decoder is trained to generate the corresponding SMILES sequence using teacher forcing. During this stage, the parameters of the transcriptome conditional encoder remain fixed, while the condition-mapping network, adapter modules, and selected MolT5 parameters are updated by minimizing the conditional autoregressive loss in Equation (8).

During inference, a new expression profile is first encoded into c and then transformed into condition tokens C. The fine-tuned MolT5 encoder-decoder generator autoregressively generates candidate SMILES sequences conditioned on C. The decoding process terminates when an end-of-sequence token is produced or the maximum decoding length is reached, and the resulting SMILES strings are collected as candidate molecules.

Alg. 1 Training and Inference Workflow of AET5

 Input: Expression profiles X = (x_i), SMILES strings S = (s_i);

 Output: Generated candidate molecules M;

 Stage 1: Pretrain transcriptome conditional encoder

 for epoch = 1 to E_enc do

  Generate two augmented views for each expression profile;

  Optimize the conditional encoder using the InfoNCE loss in Equation (3);

  Update encoder parameters θ;

 end for

 Freeze the transcriptome conditional encoder;

 Stage 2: Fine-tune molecular generation module

 for epoch = 1 to E_gen do

  Extract latent conditional embeddings c from expression profiles;

  Convert c into condition tokens C as described in Equation (4);

  Fine-tune MolT5 with teacher forcing by minimizing the loss in Equation (8);

  Update generator parameters ψ;

 end for

 Stage 3: Inference

 for each input expression profile x_i do

  Extract latent conditional embedding c_i;

   Convert c_i into condition tokens C_i;

   Generate candidate SMILES sequences conditioned on C_i;

   Add generated molecules to M;

 end for

 return M;

3.1. Datasets

To evaluate the generalization capability and effectiveness of AET5 across different disease-related contexts, we used two categories of gene expression profiles: chemical-induced expression profiles and disease-related expression profiles.

Chemical-induced profiles were obtained from the LINCS L1000 database [39], which contains transcriptional responses of human cell lines under chemical perturbations. In this study, we used chemical-induced expression signatures as the primary transcriptomic dataset for model training. After preprocessing and filtering, chemical-associated expression profiles with matched compound SMILES representations were retained. These profiles characterize cellular transcriptional responses to diverse molecular perturbations and provide paired transcriptome-molecule data for training the expression-conditioned molecular generation model. The input gene features were based on the 978 landmark genes experimentally measured in the L1000 platform. These genes were used because they are directly assayed and therefore provide reliable expression features. In addition, the original L1000 study demonstrated that these 978 landmark genes can be used to infer the expression of more than 10,000 additional genes with good performance, indicating that they capture representative transcriptomic variation. Thus, they provide a compact and informative feature space for modeling drug-induced transcriptional responses.

Disease-specific profiles were collected to evaluate the applicability of AET5 under different disease contexts. For the infectious disease setting, SARS-CoV-2-associated transcriptomic data were obtained from the National Genomics Data Center (NGDC, PRJCA002273) [40] and the Gene Expression Omnibus (GEO, GSE147507) [41]. For the non-infectious disease setting, prostate cancer expression profiles were obtained from the TCGA-PRAD cohort [42]. These disease-related profiles were used as external biological conditions to guide the generation of candidate molecules with potential disease relevance.

For model training and evaluation, the chemical-induced drug perturbation profiles were uniformly divided into training and test sets at a ratio of 9:1. The model was trained only on paired L1000 drug-induced expression profiles and molecular structures. The SARS-CoV-2 and prostate cancer expression profiles were not included in the supervised training set, but were used only as external disease-related conditions in downstream case studies. This design reduces the risk of direct data leakage between model training and disease-specific case-study evaluation.

3.2. Baseline

We compared AET5 with several representative gene expression-guided molecular generation models, including two classical generative baselines and two recently proposed models designed for expression-conditioned molecule generation. Since AET5 focuses on SMILES-based molecular sequence generation, SMILES-based LSTM and VAE models were first adopted as classical baselines. Meanwhile, we also recognized the importance of alternative one-dimensional molecular string representations, particularly SELFIES [43], which has been widely explored for improving the robustness and validity of molecular generation. Therefore, in addition to TransGEM [24], which is inherently trained and generated using SELFIES representations, we further implemented SELFIES-based variants of the LSTM and VAE baselines for a more comprehensive comparison. All selected baselines were adapted to conditional de novo molecular generation from gene expression profiles and evaluated under the same data split and generation protocol.

LSTM [44]: LSTM is a classical recurrent neural network architecture widely used for sequence modeling and SMILES-based molecular generation [12]. In this baseline, gene expression profiles are first projected into a low-dimensional condition vector through a linear layer. This condition vector is then used to condition an LSTM-based autoregressive generator for SMILES- and SELFIES-based molecular generation.

VAE [45]: Variational autoencoders are classical latent-variable generative models that have been widely applied to molecular generation tasks [7]. In this baseline, a conditional VAE is implemented for SMILES- and SELFIES-based molecular generation. Both the encoder and decoder are based on GRU networks [44], and the low-dimensional gene expression condition vector is concatenated with the molecular latent variable before being fed into the decoder.

Gx2Mol [25]: Gx2Mol is another work by the same authors as GxVAEs for gene expression-guided molecular generation. It first maps gene expression profiles into a low-dimensional latent condition space and then employs an LSTM-based molecular generator to produce SMILES sequences conditioned on the expression-derived representation.

TransGEM [24]: TransGEM is a Transformer-based conditional molecular generation model. It first compresses gene expression profiles into latent conditional representations and then uses a Transformer decoder to generate SELFIES sequences autoregressively under the guidance of the expression-derived condition.

GxVAEs [26]: GxVAEs, another work by the same authors as Gx2Mol, employs a dual-VAE architecture for gene expression-guided molecular generation. The first VAE maps gene expression profiles into a low-dimensional latent condition space, while the second VAE generates SMILES molecules conditioned on the expression-derived representation.

SmilesGEN [27]: SmilesGEN uses a dual-VAE framework for expression-guided molecular generation, consisting of a SMILES VAE for molecule modeling and a profile VAE for gene expression encoding. It further models drug-induced phenotypic effects by constraining the latent space so that post-treatment expression minus molecular representation reconstructs the pre-treatment state, and uses the expression-derived latent code to guide SMILES generation.

3.3. Evaluation metrics

To comprehensively evaluate the generated molecules, we grouped the evaluation metrics into two categories: generation quality, structural diversity, and distributional similarity metrics; and drug-likeness, natural-product-likeness, and pharmacokinetic property metrics. The first category includes validity, uniqueness, novelty, internal diversity, SMILES length, and Fréchet ChemNet Distance (FCD), which evaluate whether the model can generate chemically valid, non-redundant, novel, structurally diverse molecules whose distribution is close to that of reference molecules. The second category includes SA, LogP, QED, NP, and ADME-related properties, which assess synthetic feasibility, physicochemical suitability, drug-like character, natural-product-like structural characteristics, and predicted pharmacokinetic behavior. Most molecular properties were computed using RDKit-based cheminformatics pipelines [46], FCD was calculated based on molecular representations extracted by the pretrained ChemNet model [47], while ADME-related properties were predicted using SwissADME [48].

Validity: The proportion of generated SMILES strings that can be parsed into chemically valid molecular structures. This metric reflects whether the model can generate syntactically valid and chemically meaningful molecules.

Uniqueness: The proportion of non-duplicate molecules among the valid generated molecules. This metric evaluates the model’s ability to avoid repetitive generation.

Novelty: The proportion of generated molecules that do not appear in the training set. This metric measures whether the model can generate previously unseen molecular structures rather than simply reproducing training molecules.

InDiv: Internal diversity of the generated molecule set, quantified by the average pairwise dissimilarity of molecular fingerprints. This metric reflects the structural diversity and chemical space coverage of the generated molecules.

FCD: Fréchet ChemNet Distance, which measures the distributional similarity between generated molecules and reference molecules using ChemNet-based molecular representations. Lower FCD values indicate that the generated molecules are closer to the reference molecular distribution.

SA [49]: Synthetic accessibility score, which estimates the ease of chemical synthesis. Lower SA scores indicate that molecules are predicted to be easier to synthesize.

LogP: The octanol-water partition coefficient, which reflects molecular hydrophobicity and influences physicochemical properties such as membrane permeability and solubility.

QED [50]: Quantitative estimate of drug-likeness, which summarizes the overall drug-like character of a molecule based on multiple physicochemical and medicinal chemistry properties.

SMILES length: The number of characters in the generated SMILES string, used as a coarse descriptor of molecular size and sequence complexity.

NP [51]: Natural product-likeness score, which estimates the extent to which a molecule resembles natural products based on fragment-level structural patterns. Higher NP scores indicate stronger natural-product-like characteristics, although this score does not directly indicate biological activity.

ADME-related properties: Predicted absorption, distribution, metabolism, and excretion characteristics, which reflect the pharmacokinetic behavior of candidate molecules. These properties were evaluated using SwissADME [48].

3.4. Evaluation of the transcriptome conditional encoder

To isolate the contribution of the transcriptome conditional encoder from downstream molecular generation, we first evaluated the encoder independently from two complementary perspectives: augmentation robustness and transcriptomic information preservation. Specifically, five augmentation schemes were considered: random dropout, scaling, adaptive Gaussian noise, structural Gaussian noise, and hybrid augmentation. For reconstruction-based evaluation, a three-layer linear network was used to reconstruct the original expression profiles from the compressed representations, and the similarity between the original and reconstructed profiles was used to assess how well the learned representations preserve transcriptomic information.

As shown in Fig 2, different augmentation strategies produced distinct perturbation patterns in the input space. Fig 2b–2f show the UMAP distributions of the encoded original and augmented samples learned under hybrid augmentation, random dropout, scaling, adaptive Gaussian noise, and structural Gaussian noise, respectively. Compared with the input space, the encoded representations exhibit better alignment between original and augmented samples, indicating that the encoder learns augmentation-invariant features.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Evaluation of augmentation strategies in the transcriptome conditional encoder (a) UMAP visualization of the original transcriptomic samples and their augmented counterparts under different augmentation strategies.

(b–f) UMAP visualizations of the encoded representations of the original and augmented samples obtained using encoders trained with hybrid augmentation, random dropout, scaling, adaptive Gaussian noise, and structural Gaussian noise, respectively. (g) Kernel density estimation of the distances between encoded original samples and their augmented counterparts under different augmentation strategies. (h) k-NN coverage of the original samples in the input space under different augmentation strategies, and in the latent space after VAE-based encoding.

https://doi.org/10.1371/journal.pcbi.1014703.g002

We further quantified these effects using the kernel density estimation distributions of distances between encoded original samples and their augmented counterparts, as well as the k-NN coverage in the input space and after VAE-based encoding (Fig 2g and 2h). Smaller and more concentrated distance distributions indicate stronger robustness to augmentation-induced perturbations, whereas higher k-NN coverage reflects better preservation of local neighborhood structure. In Fig 2h, the augmentation-based encoders consistently maintained higher k-NN coverage across different neighborhood sizes, indicating that they better preserved the local structure of the original transcriptomic samples. In contrast, VAE-based encoding resulted in markedly lower k-NN coverage, suggesting a substantial loss of local neighborhood information after compression into the latent space. Among all augmentation strategies, hybrid augmentation achieved the best overall performance, with the closest alignment between original and augmented samples, the most compact distance distribution, and the highest k-NN coverage. Together, the alignment patterns, distance distributions, and k-NN coverage provide complementary evidence that the learned representations are robust to augmentation-induced perturbations while preserving local neighborhood structure more effectively than conventional VAE-based encoding.

Fig 3a and 3b show the expression value distributions before and after reconstruction, together with the relationship between reconstruction mean squared error (MSE) and SMILES length. The reconstructed profiles were highly similar to the original ones, and the reconstruction error remained low across the major SMILES length range, with larger fluctuations mainly observed in regions containing fewer samples. These results suggest that the transcriptome conditional encoder effectively preserves the major information contained in transcriptomic profiles. In addition, the training loss curve in Fig 3c indicates stable convergence of the reconstruction objective.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Reconstruction-based evaluation of the Conditional Encoder representations.

(a) Distributions of expression values before and after reconstruction. (b) Reconstruction mean squared error (MSE) across different SMILES lengths, together with the corresponding SMILES length distribution. (c) Training loss of the transcriptome conditional encoder. (d) Two-dimensional PCA projections of the original and reconstructed expression profiles. (e) Three-dimensional PCA projections of the original and reconstructed expression profiles. (f) Two-dimensional SVD projections of the original and reconstructed expression profiles. (g) Three-dimensional SVD projections of the original and reconstructed expression profiles.

https://doi.org/10.1371/journal.pcbi.1014703.g003

As shown in Fig 3d-3g, principal component analysis (PCA) and singular value decomposition (SVD) of the original and reconstructed profiles revealed substantial overlap and highly consistent marginal distributions, further supporting the effectiveness of the learned representations.

3.5. Evaluation of molecular generation module

To evaluate the performance of the fine-tuned MolT5-based molecular generation module, we first compared AET5 with representative baseline models in terms of generation quality, structural diversity, and distributional similarity. The training loss and corresponding evaluation metrics of the generator module are shown in Fig 4a, indicating that the model was effectively optimized during fine-tuning and achieved stable performance improvement.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Evaluation and visualization of molecular generation results.

(a) Training loss and generation metrics during fine-tuning, including MSE loss, validity, uniqueness, and novelty. Violin plots of the generated molecular property distributions for (b) LogP, (c) NP-likeness score, (d) SA, (e) SMILES length, and (f) QED, compared with the original dataset and baseline models. (g) Number of unique scaffolds generated by different models and representative AET5-generated molecules with distinct molecular scaffolds. (h) Generated molecules with distinct scaffolds, with scaffolds highlighted in blue.

https://doi.org/10.1371/journal.pcbi.1014703.g004

Table 1 compares AET5 with eight baseline models using validity, uniqueness, novelty, internal diversity, and FCD. Under the same dataset and the same training, validation, and test split, AET5 achieved the most balanced overall performance among the compared methods. Specifically, AET5 obtained 100% validity and 100% uniqueness, indicating that the fine-tuned T5 generator can produce chemically valid molecules while effectively avoiding duplicated outputs. AET5 also achieved the highest novelty score of 97.3%, suggesting that it can generate previously unseen molecules rather than simply reproducing structures from the training set. In terms of internal diversity, AET5 reached 0.87, which was comparable to the highest-performing SELFIES-based models and substantially higher than classical SMILES-based LSTM and VAE baselines.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparative evaluation of eight baselines against AET5.

https://doi.org/10.1371/journal.pcbi.1014703.t001

Although SmilesGEN achieved the lowest FCD value, its novelty was only 8.9%, suggesting that its generated molecules may be highly similar to the training distribution or affected by overfitting. In contrast, AET5 maintained a competitive FCD value while preserving high validity, uniqueness, novelty, and internal diversity. Compared with Gx2Mol, AET5 achieved higher validity, uniqueness, novelty, and internal diversity, while also obtaining a lower FCD value, indicating better overall generation quality and stronger distributional consistency with the reference set. In addition, the 100% validity observed in several SELFIES-based models is partly attributable to the inherent robustness of the SELFIES representation. Therefore, validity should be interpreted together with uniqueness, novelty, diversity, and FCD when comparing different molecular representations.

We further examined whether the generated molecules preserved the physicochemical and drug-likeness-related property distributions of the reference molecules. Table 2 summarizes the mean and standard deviation of SMILES length, QED, LogP, SA, and NP for AET5 and the baseline models, whereas Fig 4b-4f shows the corresponding boxplots. Importantly, the statistics in Table 2 were computed from valid generated molecules without penalization, while the boxplots were based on penalized distributions that incorporated invalid or non-novel molecules to provide a stricter assessment of generation quality. Together, these analyses enable evaluation of both central tendency and overall distributional behavior, including medians, interquartile ranges, dispersion, and the impact of low-quality generations.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Mean and standard deviation of AET5 and baseline models for the second-category metrics.

https://doi.org/10.1371/journal.pcbi.1014703.t002

As shown in Table 2 and Fig 4b-4f, AET5 closely matched the original molecular distribution across multiple properties. For SMILES length, AET5 generated molecules with a mean length of 52.43, which was close to the original mean of 49.78 and showed a similar level of dispersion. For QED, AET5 achieved the same mean value as the original dataset, 0.60, indicating that the generated molecules retained a comparable level of drug-likeness. For SA, AET5 obtained a mean score of 3.32, close to the original value of 3.40, suggesting that the generated molecules remained synthetically accessible. For NP, AET5 showed a mean score of -0.72 compared with -0.50 for the original dataset, indicating a moderate shift toward less natural-product-like structures, although the overall distribution remained within a comparable range. Although the LogP distribution of AET5 was slightly shifted toward higher values compared with the original dataset, it remained within a reasonable range and was substantially less distorted than that of VAE, which showed an excessive LogP mean and large variance.

By contrast, several baseline models exhibited clear deviations from the original distribution. TransGEM generated substantially shorter molecules, with a mean SMILES length of 21.60, and showed lower QED, higher SA scores, and a markedly shifted NP distribution, suggesting that its generated molecules were structurally simpler and less aligned with the reference chemical space. VAE produced molecules with excessively long SMILES strings and markedly inflated LogP values, indicating unstable property distributions. LSTM also showed higher molecular length and LogP values than the original dataset. Gx2Mol showed relatively reasonable property distributions, but its generated molecules were slightly longer, had lower QED, and exhibited higher SA scores than those of the original dataset and AET5, indicating a moderate deviation in drug-likeness and synthetic accessibility. Overall, AET5 better preserved the property distribution trends of the reference molecules across SMILES length, QED, LogP, SA, and NP, including peak locations, concentration regions, and dispersion patterns.

We also evaluated scaffold diversity by counting the number of unique scaffolds generated by each model. As shown in Fig 4g, AET5 generated 960 unique scaffolds, closely matching the original dataset with 953 scaffolds and exceeding most baseline models, including Gx2Mol. To further provide a structural-level visualization of this diversity, Fig 4h presents representative AET5-generated molecules with their extracted molecular scaffolds highlighted in blue. These examples show that AET5 can generate molecules containing diverse core scaffolds and substituent patterns, supporting its ability to explore heterogeneous chemical structures under transcriptomic conditions. Although LSTM-based SELFIES generated a larger number of unique scaffolds, scaffold count alone does not guarantee favorable molecular quality and should be interpreted together with validity, novelty, internal diversity, FCD, and property distributions. These results suggest that AET5 can generate structurally diverse molecules while maintaining a property profile close to that of the reference molecules.

To further investigate the contribution of each component in the fine-tuned T5 generation module, we conducted ablation experiments. As shown in Table 3, all ablated variants performed worse than the complete AET5 model, demonstrating that the final performance improvement was not caused by a single isolated component but by the combined effect of the proposed design choices.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Comparative evaluation of ablated variants against AET5.

https://doi.org/10.1371/journal.pcbi.1014703.t003

Removing the embedding component led to clear degradation across all metrics, with validity decreasing from 100% to 94.4%, uniqueness from 100% to 71.2%, and novelty from 97.3% to 72.7%. This result suggests that the embedding design is important for effectively integrating conditional information and guiding molecular generation. Removing pretrained initialization caused an even stronger decline, especially in novelty, which dropped to 48.1%. This indicates that pretrained molecular sequence knowledge is critical for learning valid molecular syntax, structural regularities, and generalizable chemical patterns.

The variants using only one regularization or perturbation strategy also showed limited performance. The “Dropout only”, “Adaptive gauss only”, and “Scaling only” variants achieved relatively high validity but suffered from substantially reduced uniqueness and novelty, indicating that these individual strategies alone were insufficient to support diverse and novel molecule generation. The “Structured gauss only” variant achieved 97.7% validity but showed the lowest uniqueness among the ablated models and the worst FCD value, suggesting that using structured Gaussian perturbation alone may restrict the generated chemical space or introduce distributional mismatch.

In contrast, the complete AET5 model achieved 100% validity, 100% uniqueness, 97.3% novelty, 0.870 internal diversity, and the lowest FCD among all ablated variants. These results demonstrate that the proposed components work synergistically to improve molecular validity, reduce duplication, enhance novelty, maintain structural diversity, and better align the generated molecules with the reference chemical distribution.

To further illustrate the structural relevance of the generated molecules at the individual-sample level, we visualized representative examples by comparing the ground-truth molecules with the corresponding molecules generated by AET5 and selected baseline models under the same transcriptomic conditions (Fig 5). Across these examples, AET5-generated molecules showed closer structural correspondence to the ground-truth molecules, preserving key scaffold-level features, ring systems, linker patterns, and functional group arrangements. In contrast, the baseline models often produced molecules with larger deviations in scaffold topology or substructure organization, although some local chemical motifs were partially retained. These qualitative comparisons provide additional evidence that AET5 can translate transcriptomic conditional information into chemically meaningful molecular structures while maintaining structural diversity rather than simply reproducing the reference molecules.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Qualitative comparison of molecules generated by AET5 and baseline models under the same transcriptomic conditions.

https://doi.org/10.1371/journal.pcbi.1014703.g005

Overall, these comparative, qualitative, and ablation results show that the fine-tuned T5-based molecular generation module can effectively leverage transcriptomic conditional information to generate valid, non-redundant, novel, and structurally diverse molecules. Compared with baseline models, AET5 provides a more favorable balance between novelty, diversity, structural relevance, and distributional consistency, while the ablation study confirms the necessity of its key components for stable and high-quality conditional molecular generation.

3.6. Case study

To further assess the practical utility of AET5 in real-world disease settings, we conducted two computational case studies in mechanistically distinct scenarios: SARS-CoV-2 infection and PC. These analyses were designed to evaluate three aspects of the proposed framework. First, we examined whether AET5 remains applicable under noisy and heterogeneous transcriptomic conditions, which is important for assessing the robustness of the decoupled architecture and the parameter-efficient fine-tuning strategy. Second, we investigated whether the learned disease-conditioned signals can be associated with biologically meaningful loci and candidate targets, thereby providing supportive evidence that the conditional representations may capture disease-related information rather than merely serving as abstract control variables. Third, we tested whether a single expression-conditioned generative framework can be applied across distinct disease contexts and generate candidate molecules with favorable predicted pharmacokinetic properties and plausible target-binding potential.

For the first scenario, we considered SARS-CoV-2 as a representative example of a rapidly emerging infectious disease, where transcriptomic data are often limited, noisy, and collected from heterogeneous sources. The dataset included collective samples from the National Genomics Data Center and individual-level samples from the National Center for Biotechnology Information, which exhibit substantial differences in expression patterns. To preserve biologically relevant information from both sources while encouraging conditional exploration, we adopted a mixed-sample strategy to construct multiple potential expression profiles for molecular generation. Under each disease-specific condition, AET5 generated candidate molecules, which were first subjected to RDKit-based validity checking and canonical SMILES-based deduplication to remove invalid and redundant structures. The remaining valid and non-redundant molecules were further filtered according to predicted synthetic accessibility, retaining candidates with SA score < 4.0 for downstream evaluation. To preliminarily assess their biological relevance, we used the DeepCE model [29], to predict the transcriptomic perturbation profile induced by each retained molecule and calculated its correlation with the corresponding disease-specific expression signature. For each disease condition, molecules showing the strongest negative correlations with the disease-specific profile were selected as prioritized candidates for subsequent docking-based target-binding evaluation and molecular dynamics simulations. The ADME-related properties and drug-likeness of these prioritized candidates were then analyzed using the SwissADME platform and QED estimation. As shown in Fig 6a, 6c, most SARS-CoV-2-conditioned molecules fall within or near the absorbable region of the BOILED-Egg model [52], In addition, the majority of generated molecules exhibit SA scores below 4 and QED values above 0.5, indicating plausible pharmacokinetic properties, favorable predicted synthetic accessibility, and drug-likeness. Representative generated molecules are also shown in Fig 6e together with their SA and QED values, further supporting their favorable predicted synthetic accessibility and drug-likeness. Notably, several generated molecules are located close to known compounds in the TPSA–WLOGP space while exhibiting low structural similarity, suggesting that AET5 can generate novel candidates that preserve desirable pharmacokinetic characteristics without simply reproducing known structures. Taken together, these results suggest that AET5 remains applicable under noisy and heterogeneous transcriptomic conditions, supporting the potential robustness of the proposed framework in small-sample and rapidly evolving disease scenarios.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. ADME property analysis and representative molecules generated under disease-specific conditions.

(a, b) TPSA–WLOGP distributions of generated molecules and known active compounds under SARS-CoV-2 and PC conditions, respectively, projected onto the BOILED-Egg model. Representative generated-known molecule pairs with low structural similarity are shown. (c, d) QED and SA score distributions of valid and non-redundant generated molecules under SARS-CoV-2 and PC conditions, respectively. (e, f) Representative generated molecules selected under SARS-CoV-2 and PC conditions, respectively, with their predicted SA and QED values.

https://doi.org/10.1371/journal.pcbi.1014703.g006

For the second scenario, we selected PC as a representative example of a disease with relatively abundant data but complex pathological mechanisms. In this setting, the key challenge is not only to generate chemically plausible molecules, but also to explore whether disease-related biological signals and potential therapeutic targets can be associated with the generation process. To this end, we extracted attention scores from the molecular generator to quantify the contribution of individual gene loci to token-level molecular decisions. These loci were then cross-referenced with the Open Targets database [53] to identify known proteins associated with high-confidence model-attended genes. Among the 978 loci used in the model, 25 were matched to PC-related targets with confidence scores greater than 0.5, suggesting potential disease relevance. The top 10 loci ranked by attention score are listed in S1 Table. The observed overlap between highly attended loci and known disease-associated targets provides supportive evidence that the disease-conditioned signals learned by AET5 may capture biologically relevant information and may help guide molecular generation toward disease-related cellular mechanisms, rather than acting solely as statistical prompts.

To further assess the potential biological relevance of these loci, we selected targets from the top 50 attention-ranked genes that also had complete structural information, known binding pockets, and available reference molecules for downstream evaluation. Specifically, CHEK2 (PDB ID: 2YCF) and TUBB6 (PDB ID: 6AB2) were chosen as candidate protein targets. CHEK2 [54] is a checkpoint kinase involved in DNA damage response and cell-cycle regulation, and it plays an important role in the proliferation and repair mechanisms of PC cells. TUBB6 [55], a member of the tubulin family, participates in cytoskeletal assembly and dynamics and is closely related to tumor cell migration and division. These proteins were therefore used as biologically plausible targets for subsequent docking-based assessment. In parallel, ADME analysis was also performed for PC-conditioned generated molecules in comparison with known compounds. As shown in Fig 6b, 6d, and 6f, the generated molecules occupy a pharmacokinetic space similar to that of known compounds in the BOILED-Egg model. In Fig 6d, the PC-conditioned molecules are mainly distributed in a low SA-score range, with most candidates showing SA scores below 4, indicating favorable predicted synthetic accessibility. In contrast, their QED values display a relatively broad distribution around 0.5, with candidates present on both sides of this threshold, suggesting variable but overall plausible drug-likeness. Representative candidates with favorable predicted SA and QED values are further shown in Fig 6f, suggesting that the model can generate chemically plausible and pharmacologically relevant candidates under cancer-related expression conditions.

In addition to the two PC-related targets, two key SARS-CoV-2 proteins, Nsp3 [56] (PDB ID: 7LMD) and RNA-dependent RNA polymerase [57] (RdRp, PDB ID: 9I1S), were selected for further validation. Nsp3 is involved in viral polyprotein processing and immune evasion, whereas RdRp is essential for viral RNA replication, making both proteins important antiviral targets. The generated candidate molecules were evaluated by molecular docking using AutoDock Vina [58] and by molecular dynamics simulations using AmberTools20 [59].

Fig 7 shows the binding behavior of the generated molecules and control molecules for the SARS-CoV-2 target 7LMD. Fig 7a and 7b present the 3D binding conformations, showing that AET5 can generate novel molecules that occupy a binding pocket similar to that of the control compound while maintaining clear structural differences. The binding energy comparison in Fig 7c indicates that the generated molecule exhibits a more favorable predicted binding profile than the control molecule. In addition, the time-dependent radius of gyration, hydrogen-bond number, and RMSD curves in Fig 7d-7f suggest that the generated ligand–protein complex maintains competitive conformational compactness and dynamic stability throughout the simulation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Comparison of ligand–protein binding between AET5-generated and control molecules for the 7LMD SARS-CoV-2 target.

(a) 3D interaction and binding site of the generated molecule in the ligand–protein complex. (b) 3D interaction and binding site of the control molecule in the ligand–protein complex. (c) Comparison of binding energies. (d) Time-dependent radius of gyration. (e) Time-dependent number of hydrogen bonds. (f) Time-dependent root mean square deviation (RMSD).

https://doi.org/10.1371/journal.pcbi.1014703.g007

Similar analyses were conducted for additional targets, including 2YCF, 6AB2, and 9I1S, with the corresponding results shown in Fig 8, S3 and S4 Figs Across these targets, the generated molecules showed favorable or comparable predicted binding energies and stable interaction patterns relative to the control compounds, although the detailed dynamic behaviors varied across targets. These results suggest that AET5 does not merely generate chemically valid structures, but can also produce candidate molecules with plausible computational target-binding potential in both infectious disease and cancer settings.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Comparison of ligand–protein binding between AET5-generated and control molecules for the 2YCF PC target.

(a) 3D interaction and binding site of the generated molecule in the ligand–protein complex. (b) 3D interaction and binding site of the control molecule in the ligand–protein complex. (c) Comparison of binding energies. (d) Time-dependent radius of gyration. (e) Time-dependent number of hydrogen bonds. (f) Time-dependent RMSD.

https://doi.org/10.1371/journal.pcbi.1014703.g008

Overall, the two case studies provide complementary computational evidence for the main design goals of AET5. The SARS-CoV-2 results suggest that the framework can remain applicable under limited, noisy, and heterogeneous transcriptomic inputs, supporting the potential robustness of the decoupled architecture and parameter-efficient generator adaptation. The PC results show that the learned disease-conditioned signals can be associated with biologically meaningful loci and known disease-associated targets, providing supportive evidence for the biological relevance of the proposed conditional representation learning strategy. Across both viral infection and cancer scenarios, AET5 generated candidate molecules with favorable predicted pharmacokinetic properties and plausible target-binding behavior in docking and molecular dynamics analyses. These findings highlight the potential generality and practical utility of the proposed expression-conditioned molecular generation framework, while further biochemical or cellular experiments would be required to confirm the therapeutic activity of the generated candidates.