2.1. Feature extraction from cell line

In complex biological systems such as cancer, the behavior of cell lines is influenced by multiple layers of biological information, each of which alone is insufficient to fully explain the drug response of a cell. Therefore, this model utilizes a multi-branch encoder designed to extract a compact and rich representation of the biological state of each cell line. The objective of this encoder is to generate a vector that captures both the molecular and functional aspects of the cell, which can then be used in the subsequent stages of the model to learn drug–cell interactions.

To comprehensively represent the biological status of cancer cells, four types of omics data were used: gene expression, genetic mutations, DNA methylation, and biological pathway scores. Each of these branches captures a specific dimension of cellular biology: the gene expression branch is intended to identify regulatory patterns of genes associated with drug sensitivity, while the mutation branch emphasizes the presence or absence of mutations in key genes. The methylation branch extracts epigenetic information related to gene regulation, and the pathway branch provides a high-level, functional view of intracellular signaling system activity. This diversity of profiles allows the model to simultaneously understand and analyze both low-level molecular data and high-level biological functions. To this end, for each cell , four types of omics data were collected from reputable sources: gene expression , genetic mutations , DNA methylation , and biological pathway activity scores . Pathway scores were calculated using the GSVA method [31] based on the C2 collection of pathways from the MSigDB database [35], utilizing gene expression data. After normalization and alignment with cancer-related genes based on the COSMIC database, these profiles were fed into the neural encoder network.

For each type of data, an independent learning branch is designed to specifically model the unique statistical and semantic structure of that data type. Each branch consists of two fully connected layers, with batch normalization, ReLU activation, and dropout applied between them. This two-layer architecture enables the learning of nonlinear transformations and gradual dimensionality reduction, while also preventing overfitting. In the first stage, the raw vector of each biological data type is projected into an initial compact latent space:

(1)

Subsequently, this representation is mapped into a shared latent space with a fixed dimension, where the vector length is set to l = 128. This value was selected based on empirical testing with different sizes (64, 128, 256) and provides a balanced trade-off between compression and the preservation of biological information:

(2)

In these equations, v ∈ {exp, mut, meth, path} denotes the type of biological profile, and and are the weight matrices of the first and second layers, respectively, whose dimensions are determined according to the input size and the latent space. The four compressed vectors obtained from each branch are then combined in the final stage through concatenation along the feature dimension:

(3)

Here, the symbol ⊕ denotes vector concatenation. The result is a unified 512-dimensional compressed vector representing the biological state of the cell. Instead of using attention-based or weighted fusion, direct concatenation was employed, since the number of biological profiles in this architecture is limited and independent. Initial experiments showed that concatenation provided more stable performance compared to methods that learn fusion weights. This combination is used in subsequent stages of the model particularly in integration with drug features within the Bilinear Attention module. This multi-purpose representation significantly enhances the model’s ability to learn complex drug–cell interactions.

2.2. Feature extraction from drug

Accurate prediction of drug response requires a deep understanding of the chemical structure of drugs. Since drug structures consist of multiple dimensions including topological information, pharmacophoric features, and recurring substructures relying on a single type of representation may lead to the loss of critical information. Therefore, in the MoGraphDRP model, the drug encoder is designed as a dual-branch architecture, comprising a chemical fingerprint branch and a molecular graph branch. Both branches originate from a common input and extract complementary representations of the drug’s structure.

For each drug, the SMILES string is provided as input to the RDKit tool [32]. This tool interprets the chemical structure and generates two independent representations: first, a molecular graph , where each node represents an atom and each edge denotes a chemical bond between atoms. This graph allows for precise modeling of the drug’s topological structure. Second, molecular fingerprint vectors are generated by compressing structural features of the drug into fixed-dimensional, neural-network-compatible vectors. These two types of representations are employed complementarily in the model architecture to capture various aspects of the drug’s chemical structure.

2.2.1. Feature extraction based on fingerprints.

Molecular fingerprints are compact, standardized numerical representations of drug structures that can be efficiently utilized in neural networks. In this model, three types of complementary fingerprints were used to capture diverse structural features. The Morgan fingerprint, with 2048 dimensions, captures atom neighborhood patterns with a fixed radius and is suitable for representing local functional groups. The PubChem fingerprint, with 881 dimensions, is designed based on the presence or absence of predefined chemical substructures. Finally, the ESPF fingerprint, with 2586 dimensions, encodes bio-pharmacophoric features directly from SMILES strings using a structured encoding scheme. All fingerprints were directly extracted from the drug SMILES strings using the RDKit toolkit. Although fingerprints do not provide a holistic structural view, their compactness and computational efficiency make them highly effective for fast and interpretable learning. For each fingerprint type, a dedicated multi-layer encoder was designed to project the numerical data into a more compact latent space. These encoders consist of multiple fully connected layers with ReLU activation and dropout, dynamically configured based on the input dimension. The purpose of this process is to remove noise, extract high-level representations, and align the data with a common learning space. The output of each encoder is a 128-dimensional compressed feature vector.

(4)

Where , and the weight parameters and are learnable. The output of each pathway is compressed to a dimension of l. Then, the three resulting compressed vectors are concatenated with equal weights.

(5)

2.2.2. Feature extraction based on graph network.

Each drug can be modeled as a molecular graph in which atoms are defined as nodes and chemical bonds as edges. This graph is extracted from the SMILES string using RDKit and preserves the topological structure of the drug, whereas fingerprints merely enumerate existing substructures. For this purpose, each drug is represented as an undirected graph , where is the set of nodes (atoms) and is the set of edges (chemical bonds). The features of each node are initialized using a vector that includes the chemical element type, number of bonds, aromaticity, number of hydrogen bonds, and other descriptive properties. To learn the structural representation of these graphs and extract features, a Graph Convolutional Network (GCN) is employed. In this architecture, a GCN is used to capture local features, as illustrated in Fig 2. This network extracts hierarchical local features of the molecule by passing messages between neighboring nodes and applying ReLU and dropout with a rate of 0.4 at each layer. After three rounds of message passing, a mean pooling function is applied across the nodes to generate a fixed-length vector representing the entire graph. The node feature update in each GCN layer is performed as follows:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Architecture of the drug graph module based on GCN.

First, the molecular graph with node feature vectors of dimension 78 is input into a three-layer graph neural network, where each layer employs message averaging, a ReLU activation function, and dropout with a rate of 0.4. Finally, using a mean pooling function, a fixed-length vector of dimension 128 is generated as the representation of the entire graph.

https://doi.org/10.1371/journal.pone.0341458.g002

(6)

After passing through the message-passing layers, the final graph vector is computed using a mean pooling function over all nodes:

(7)

Where T is the total number of graph layers, and represents the final vector representation of the drug. The use of GCN allows the model to learn the global structure of the drug and long-range interactions between atoms an aspect that fingerprints are unable to capture.

2.2.3. Fusion of drug features.

Since the chemical structure of a drug can be described from various dimensions, relying solely on a single representation may overlook critical information. In this architecture, both fingerprints and graphs are used simultaneously. Molecular fingerprints (such as Morgan, PubChem, and ESPF) effectively model aspects of local structure, pharmacological features, and the presence of specific substructures, making them successful in quickly identifying substructures and pharmacophoric features. However, they lack awareness of the overall molecular structure. In contrast, molecular graphs focus on the topological interactions between atoms and the global structure of the molecule, but are computationally intensive and more difficult to interpret directly. To enable the model to obtain a more comprehensive view of the drug molecule, it is essential to integrate these two complementary representations. In the MoGraphDRP architecture, this integration is performed via simple vector concatenation. First, the compressed graph representation of the drug and the fingerprint representation are generated. Then, these two vectors are concatenated along the feature dimension.

(8)

Here, ⊕ denotes vector concatenation, and l is the dimensionality of the compressed graph representation space. The choice of concatenation is motivated by the fact that the number of branches is limited and each provides an independent representation. Based on various ablation studies, learning attention weights or gating at this level adds high complexity without providing significant improvement, and in empirical experiments, simple concatenation has shown more stable and generalizable performance. This fusion method enables the model to simultaneously leverage structural, local, and bio-pharmacological features of the drug. Since none of these representations alone captures all biological aspects of the molecule, their integration can enhance the model’s generalization capability to new and unseen drugs. This dual-branch model, by combining fast, interpretable, and structure-aware representations, offers a comprehensive depiction of the drug molecule while maintaining generalizability to novel data. The final vector serves as the complete drug representation and is used in the next stage for interaction with cellular features.

2.3. Bilinear attention module and drug–cell interaction modeling

One of the fundamental challenges in drug response modeling is understanding the complex, nonlinear, and multi-level interactions between the biological features of cancer cells and the chemical structures of drugs. In many existing architectures, the integration of these two types of information is performed using methods such as concatenation or dot-product attention. However, these approaches typically assume that the relationships between features are direct and linear, and thus are incapable of modeling indirect, combinatorial, and conditional interactions between the two independent spaces (cell and drug). To overcome this limitation, the MoGraphDRP model employs a bilateral attention module called Bilinear Attention, designed to learn independent and conditional feature-level interactions between two vectors. This module enables the model to allow each cellular feature to interact with a specific combination of drug features, and vice versa. Unlike conventional attention mechanisms, which typically operate based on overall similarity or inner product, bilinear attention utilizes a bilinear form, which offers greater capacity for modeling specific combinations of cross-domain features. In this module, the compressed vectors of the drug and the cell are each projected into a shared space via independent linear layers.

(9)

Projection into a shared space is a prerequisite for interaction assessment, as the two input vectors originate from different domains and must reside in a semantically aligned and unified space to enable direct interaction. In the next step, the interaction between these two vectors is modeled using a learnable bilinear weight matrix to compute the bilinear attention mapping:

(10)

In this equation, the scalar value I represents the interaction strength between all features of the drug and the cell, taking into account nonlinear combination weights. To enhance discriminative power and capture various possible interactions, this structure is extended into a Multi-Head Bilinear Attention:

(11)

Where K is the number of attention heads, and each is an independent weight matrix for learning a specific type of interaction. This mechanism allows the model to simultaneously capture multiple distinct patterns of drug–cell associations. To extract an interaction vector from the output of the heads, the values are mapped to a compressed vector using a Fully Connected layer:

(12)

Where is a feature-level drug–cell interaction vector that encapsulates compressed information from all attention heads. This vector serves as a bilinear fusion representation and is used not only for IC50 prediction but also in the subsequent stages of the model (ensemble module). To estimate the IC50 value, the fusion vector produced by the Bilinear Attention module representing the combined features of drug and cell is fed into a multilayer perceptron (MLP). This network consists of two Fully Connected layers in the sequence: FC (128 → 64) → ReLU → Dropout → FC (64 → 1). It’s simple and lightweight structure is designed to prevent overfitting during the regression phase and enables the learning of a nonlinear mapping from the fusion vector space to drug response values:

(13)

Where is the predicted drug response value (IC50) for the given drug–cell pair. In addition to directly predicting the IC50 value, the fusion vector is also independently stored for the reinforcement learning stage and the ensemble model as a feature vector. Thus, the MLP acts as a preliminary predictor, optimized through end-to-end training to provide a raw but effective estimate of drug response, while the fusion output serves as a reinforcement vector for secondary refinement of results. This dual-path design improves accuracy and reduces sensitivity to noise in the final prediction stage. As shown in Fig 3, the Bilinear Attention module is designed to first map both vector spaces into a shared space, then compute multiple bilinear interactions, and finally compress them into a representation usable in the final prediction.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Structure of the Bilinear Attention module in the MoGraphDRP architecture.

First, the compressed vectors of the cell and the drug are mapped into a shared space through linear layers. Then, feature-level interactions between them are learned via multi-head bilinear attention using the matrices. The output of this module consists of two parts: (1) the predicted IC50 value for the drug–cell pair, and (2) the compressed interaction vector , which contains the features extracted by the model. Both outputs are used as inputs for the final stage of the model the ensemble network to enhance prediction accuracy.

https://doi.org/10.1371/journal.pone.0341458.g003

2.4 Ensemble regression head

Despite the high capability of the Bilinear Attention module in modeling drug–cell interactions and producing an initial estimation of drug response, the final prediction accuracy may still be affected by factors such as the complexity of biological relationships, data noise, or the inherent limitations of neural models. Especially in deep architectures with a large number of parameters, the model may fail to correctly adjust residual errors or detect subtle patterns. Therefore, in the final step of the MoGraphDRP architecture, a refinement module based on ensemble learning is designed to enhance the accuracy and robustness of IC50 prediction. At this stage, the XGBoost model is employed as a refinement layer. The choice of XGBoost is motivated by its high capability to model residual errors with complex nonlinear structures and its resilience against overfitting. Unlike models such as Random Forest or AdaBoost, this algorithm provides higher accuracy and better stability in noisy contexts such as biological data by leveraging regularization, second-order gradients, and fine control over model complexity. In contrast to typical ensemble models that replace the main network, XGBoost here plays a complementary role being applied to the outputs of the previous stage to reduce remaining errors. The XGBoost algorithm, through an ensemble of decision trees trained incrementally by following the error gradient, is highly effective at identifying residual patterns not captured by the main model. Each tree in this algorithm is of the CART (Classification and Regression Tree) type, where node splits are made to maximize the reduction in the loss function. These trees are capable of modeling nonlinear relationships and conditional interactions among features. The input to this module is a combination of two key components of the model: 1) the compressed drug–cell interaction vector , 2) the initial IC50 prediction by the MLP network. These two are concatenated to form the final vector :

(14)

The final model benefits from the combination of two types of knowledge: deep feature interaction extracted by the neural network, and statistical knowledge extracted by the boosting ensemble (tree-based residual modeling). The XGBoost model is applied to this training vector, using an ensemble of decision trees that incrementally reduce the model error based on the gradient. This model attempts to correct the residual between the actual and the current predicted value. At each iteration t, a weighting function a decision tree is added to the model so that the prediction function is updated as follows:

(15)

The training process is carried out by minimizing the total loss function, which includes the prediction error and a regularization term to prevent overfitting:

(16)

In this equation, l is the regression loss function, which in the MoGraphDRP is computed using MSE, and Ω is the model complexity term. The additive structure of XGBoost is designed to apply the most accurate correction to the residual error at each iteration by utilizing both the first- and second-order gradients. In the present implementation, the number of trees is set to T = 100, the maximum depth of each tree is 6, the learning rate is 0.05, and the subsample ratio is 0.8. These settings are chosen to balance prediction accuracy and overfitting prevention. After training, the final model output is obtained by summing the initial predictions and the corrections made by the trees as follows:

(16)

The value is considered the final, refined prediction of IC50 for the drug–cell pair (). As shown in Fig 2, the outputs of the Bilinear module, including and , are concatenated and fed into the ensemble model. The ensemble module, as the final stage of the model, serves as a refinement step to enhance the accuracy and stability of IC50 prediction. This two-stage design combining the representational power of deep learning in the Bilinear stage with the corrective accuracy of tree-based models in the ensemble stage results in improved accuracy, reduced prediction variance, and enhanced robustness of the model under diverse and noisy data conditions.

2.5. Implementation and learning procedure

The MoGraphDRP model was implemented using the open-source libraries PyTorch 2.0.1 and DGL 1.1.2 in the Python programming language. All experiments were conducted on a system equipped with an NVIDIA RTX 3090 GPU (24GB) and the Ubuntu 20.04 operating system. To ensure reproducibility of the results, the initial seed was set to 42, and all operations were executed on the GPU indexed as cuda:0.

The dataset was split into 80% for training, 10% for validation, and 10% for testing. This split was performed randomly while maintaining the distribution of IC50 values across each subset. The main network configurations, layers, and optimization parameters are detailed in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Implementation details of the MoGraphDRP method.

https://doi.org/10.1371/journal.pone.0341458.t001

To ensure learning stability and normalization of the internal feature distributions, batch normalization was applied in most encoder layers. All key model hyperparameters including the dropout rate, embedding dimension, batch size, learning rate, the number of trees in the ensemble model, and the strategy for integrating multi-omics and drug features were selected empirically through multiple rounds of experiments. For each parameter, several different values and settings were tested, and only those configurations that consistently produced the best validation performance and overall model stability were ultimately used and reported here. In the main model training phase, the Mean Squared Error (MSE) loss function was used for regression. This function is defined as follows:

(17)

Where is the predicted value and is the true drug response value (IC50) for the cell–drug pair, and N is the number of samples in each batch. To compute the final loss, the errors of all samples are summed and then averaged, which is considered the final value of the loss function. This method, known as mean reduction, ensures that each sample contributes equally to the weight update process, while maintaining the model’s sensitivity to large errors an important aspect in noisy biological data. In the ensemble model, the default Squared Error Loss function was used along with regularization to control model complexity and prevent overfitting.

3.1. Materials and dataset construction

In this study, version 2 of the GDSC (Genomics of Drug Sensitivity in Cancer) database was used as the primary source for collecting drug response data [36]. This database contains sensitivity measurements such as IC50 values for hundreds of drug–cell line combinations. Initially, the raw dataset included 286 drugs and 969 cancer cell lines. For each pair, the IC50 value was collected on a logarithmic scale. To obtain the molecular features of the cells, the CCLE (Cancer Cell Line Encyclopedia) database [6] was utilized, which provides multi-omics data including gene expression, genetic mutations, and DNA methylation for over 1,000 cancer cell lines. We extracted and integrated the gene expression, mutation, and methylation data for cell lines overlapping with those in the GDSC dataset. To eliminate irrelevant features and reduce noise, only genes listed in the COSMIC (Catalogue of Somatic Mutations in Cancer) database [37] were selected. Genes appearing in the omics data but absent from the COSMIC list were removed. For the mutation data, all mutation types related to a given gene were aggregated to reduce redundancy. As a result of this feature selection process, the input vector dimensions for each omics type were specifically determined: 714 genes were selected for gene expression data, 715 genes for the mutation vector, and DNA methylation was represented using 603 genes. To extract structural features of drugs, we referred to the PubChem database [8]. SMILES strings for each drug were retrieved and converted into molecular graphs. Additionally, drug fingerprints were obtained from Morgan, ESPF, and PubChem sources. During the data cleaning phase, drugs with invalid PubChem IDs or incomplete data were removed. For drugs with multiple identifiers in the GDSC, the version providing the highest IC50 coverage was selected. After the above processing, 536 cell lines and 169 drugs remained, forming 81,467 drug–cell line combinations. This dataset was considered the benchmark dataset. Out of 90,584 possible combinations (536 × 169), approximately 9.6% of the IC50 values were unavailable, and during dataset construction, only those combinations with missing IC50 values were excluded. In the next step, the processed data were used to train the proposed deep learning model. Fig 4 presents a step-by-step diagram of the data preparation process.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Schematic of the data preparation process.

https://doi.org/10.1371/journal.pone.0341458.g004

The drug response data were obtained from the GDSC2 database and the cellular biological data were extracted from CCLE and aligned with each other. To evaluate the model’s performance on unseen data, the available IC50 values from the CCLE database were also utilized. By excluding combinations already present in the benchmark set, an independent test set was created, consisting of 184 drugs and 8,909 IC50 values for 551 cancer cell lines.

3.2. Performance evaluation metrics

To evaluate the accuracy of the model in predicting the drug response of cells to various drugs, a set of standard regression metrics was employed. Each of these metrics captures a specific aspect of model performance and collectively provides a comprehensive view of its accuracy and generalizability. Mean Squared Error (MSE) was used as the model’s primary loss function and measures the average squared deviation of the predictions from the actual values. Root Mean Squared Error (RMSE) was applied to assess the absolute prediction error in the original output unit, and Mean Absolute Error (MAE) evaluates the average absolute distance between predicted and true values. Additionally, the Coefficient of Determination (R²) indicates the proportion of variance in the actual values explained by the model. Pearson Correlation Coefficient (PCC) assesses the linear correlation between predicted and actual values, while Spearman Correlation Coefficient (SCC), a non-parametric metric, measures rank-based correlation.

(18)

(19)

(20)

(21)

(22)

(23)

3.3. Performance comparison of MoGraphDRP Method and existing methods

To assess the accuracy of the MoGraphDRP model, its performance was compared with six advanced models: tCNNs [28], DeepCDR [22], Precily [29], DeepTTA [23], GADRP [30], and BANDRP [26]. These models were previously developed for predicting drug response in cell lines and employed various methods for feature extraction and data integration. All models were trained on a unified benchmark dataset, using an identical data split of 80% for training, 10% for validation, and 10% for testing. The comparison results are reported in Table 2. The evaluation of these results indicates that the MoGraphDRP model outperformed all other approaches across all five primary performance metrics. In particular, the RMSE was reduced to 0.6622, representing a substantial improvement over BANDRP, which achieved an RMSE of 0.9305. Additionally, the MAE was also reduced to 0.4884 in the MoGraphDRP model. In terms of correlation metrics, the MoGraphDRP method achieved the highest concordance with actual values, reaching PCC = 0.9689 and SCC = 0.9524. These results demonstrate that the model’s predictions are not only statistically more accurate but also better aligned with the trends observed in the true data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Performance comparison of the MoGraphDRP method with other state-of-the-art approaches on the benchmark dataset.

https://doi.org/10.1371/journal.pone.0341458.t002

The superiority of the MoGraphDRP model can be attributed to its multi-layered and comprehensive design. Firstly, the simultaneous utilization of four cellular multi-omics data sources gene expression, genetic mutations, DNA methylation, and biological pathway enrichment scores provides the model with a broad and multifaceted view of each cell line’s biological state. On the drug side, the integration of two complementary approaches molecular fingerprints and molecular graph structure ensures that drug features are described from multiple perspectives. This combination of low- and high-level representations prevents feature omission and results in a more complete representation of drugs. Moreover, the design of the Bilinear Attention module plays a critical role in accurately modeling drug–cell interactions. Unlike conventional methods that rely on simple feature fusion, this module enables the model to learn bidirectional, feature-level, and nonlinear dependencies between the two independent feature spaces.

Ultimately, one of the main contributors to the improved performance of the model is the incorporation of a refinement stage based on XGBoost at the end of the architecture. In preliminary experiments, the model lacked sufficient accuracy without this component. However, with the addition of this layer, predictions were made more robust against residual errors, and the model’s performance stability was significantly enhanced. This hybrid design leveraging the feature extraction power of deep networks alongside the precision of boosting algorithms has proven particularly effective under complex and noisy biological data conditions.

To better illustrate the performance differences, Fig 5a and 5b are presented. As shown in Fig 5a, the MoGraphDRP model outperformed the compared methods in terms of RMSE and MAE. This superiority is especially evident in the RMSE metric, which was reduced to 0.6622. In Fig 5b, the values of PCC, SCC, and R² for various models are compared. The MoGraphDRP model clearly surpassed all others across all three metrics, indicating that its outputs not only closely match the true values but also align well with the reference trends and distributions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. a. Comparison of the performance of different models based on MAE and RMSE metrics on the benchmark dataset.

The MoGraphDRP model achieved lower error values in both metrics compared to other methods, indicating higher accuracy in predicting IC50 values. b. Performance comparison of different models based on SCC, PCC, and R² metrics shows that the MoGraphDRP model not only achieved the highest linear correlation coefficient (PCC = 0.9689) but also recorded the highest explained variance (R² = 0.9388) among all methods.

https://doi.org/10.1371/journal.pone.0341458.g005

To further examine the learning dynamics of the MoGraphDRP model, the values of the loss function and the Pearson Correlation Coefficient (PCC) were recorded and analyzed throughout the training process. As shown in Fig 6, the loss values for both the training and validation sets steadily decreased over time, and from approximately epoch 50 onward, a stable and nearly uniform decline was observed. This trend indicates proper convergence of the model without signs of overfitting. Moreover, Fig 7 illustrates the progression of the PCC score on the validation set. The gradual increase in this coefficient, reaching above 0.96 by the end of training, reflects the model’s improved alignment with the actual values and its effective learning of complex drug–cell interaction patterns. This improvement corresponds to a 28.8% reduction in RMSE and more than 27% reduction in MAE compared to BANDRP, which served as the strongest baseline model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Loss curve over 200 epochs.

https://doi.org/10.1371/journal.pone.0341458.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Pearson Correlation Coefficient (PCC) trend on the validation set during training.

https://doi.org/10.1371/journal.pone.0341458.g007

3.4. Prediction of unknown drug responses

At this stage, the developed model after being trained on the available IC50 values from the benchmark dataset was utilized to estimate unknown drug responses. Out of 90,584 potential drug–cell line combinations, 81,467 combinations with experimentally measured IC50 values were used as training data. Following training, the model was able to impute 9,117 missing IC50 values. These predictions were organized into a cell–drug response matrix, enabling the ranking of drugs based on their predicted anticancer activity. For this purpose, the mean predicted IC50 value for each drug was computed, and the drugs were ranked accordingly. Fludarabine was identified as the most potent drug candidate. This compound, a nucleoside analog of adenosine, is intracellularly converted into its active triphosphate form and inhibits key enzymes involved in DNA synthesis, such as DNA polymerase and ribonucleotide reductase. As a result, it induces cell cycle arrest and triggers apoptosis in tumor cells [38]. Fludarabine is primarily used in the treatment of hematological malignancies, such as chronic lymphocytic leukemia (CLL). The consistently low predicted IC50 values for this drug across various cell lines support its strong potential as an effective anticancer agent [39]. In contrast, cimetidine was identified as the most resistant drug. Cimetidine is a histamine H2 receptor antagonist commonly used for treating acid-related gastrointestinal conditions [40]. Since this drug does not directly participate in pathways associated with cell division or tumor cell death, its high IC50 value in the predictions appears reasonable [41]. To assess the model’s ability to distinguish between active and inactive drugs in the absence of experimental data, the distribution of predicted IC50 values for the top 10 sensitive and top 10 resistant drugs is illustrated in Fig 8. As observed, the sensitive drugs exhibit a lower and more compact IC50 distribution, while the resistant drugs have a higher and more separable distribution. This supports the model’s capability to accurately predict responses even without empirical data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. The distribution of predicted IC50 values for the top 10 sensitive and top 10 resistant drugs is shown.

This pattern demonstrates the model’s capability to distinguish between active and inactive compounds, even in the absence of direct experimental data.

https://doi.org/10.1371/journal.pone.0341458.g008

The conducted analyses demonstrate that the model is not only effective in imputing missing values but also capable of identifying genuinely anticancer compounds among unknown drugs even when such compounds were not present during training. Fig 9 provides an overview of the drug response (IC50) matrix, visually illustrating both the structure of the data and the model’s ability to complete missing entries.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Heatmap of a subset of the predicted IC50 matrix for 30 drugs and 30 cell lines.

The color variation indicates the level of cell sensitivity to different drugs. Darker colors represent stronger responses (lower IC50 values).

https://doi.org/10.1371/journal.pone.0341458.g009

3.5. Feature importance analysis

To better understand the behavior of the ensemble model and identify which features contributed most to the final IC50 predictions, the feature importance scores of the XGBoost model were analyzed. The importance of each feature was measured using the F score, which indicates the number of times a feature was used to split nodes in the decision trees. In other words, a higher F score reflects a greater influence of that feature on the model’s decision-making process. As shown in Fig 10, certain dimensions of the Bilinear Attention output vector (the f features), as well as the initial predicted IC50 value (predicted_ic50), had the highest importance in the ensemble model. This demonstrates that both the nonlinear drug–cell interactions modeled by the Bilinear Attention module and the direct output of the deep model play a significant role in improving the prediction accuracy. The feature importance scores for the top 20 features in the XGBoost ensemble model are presented in the figure.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Feature importance scores (F score) of the XGBoost ensemble model.

Features fN represent dimensions of the Bilinear Attention output vector, and predicted IC50 is the preliminary deep model prediction.

https://doi.org/10.1371/journal.pone.0341458.g010

4.1. Independent test evaluation

To evaluate the generalization capability of the MoGraphDRP model on unseen data, the final experiment was conducted on an independent test set from the CCLE database. To ensure complete independence of this set from the training data, all drug–cell line pairs that were also present in the benchmark dataset (GDSC) were removed. The final set consisted of 8,909 IC50 values for 184 drugs and 551 cancer cell lines, none of which were used during the training, validation, or internal testing phases of the model. The results of this experiment are presented in Table 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Comparison of MoGraphDRP performance on the independent test set before and after ensemble.

https://doi.org/10.1371/journal.pone.0341458.t007

As shown by the results in Table 7, adding the XGBoost-based ensemble module led to a significant improvement across all performance metrics. Specifically, the RMSE decreased by approximately 19.7%, while the correlation indices PCC and R² increased noticeably. This enhancement was achieved despite the CCLE dataset being entirely unseen during the model training process. Therefore, the obtained results indicate the high generalization capability of the MoGraphDRP model in cross-dataset scenarios. Moreover, the ensemble module’s role in increasing robustness and reducing systematic errors is clearly validated by this evaluation.