Benchmark dataset

Comparative Toxicgenomics Database (CTD) [36] is an authoritative public bioinformatics platform that plays a crucial role in uncovering the potential mechanisms linking environmental chemicals to human diseases. By systematically integrating experimentally validated chemical–disease associations and leveraging bioinformatics prediction methods, CTD provides valuable support for environmental health research. Notably, its collection of literature-confirmed drug–disease interaction data is especially valuable for association prediction studies. DrugBank [37], a globally recognized drug knowledgebase, not only offers comprehensive records of molecular properties but also provides a detailed network of drug actions, including drug targets, metabolic enzymes, and drug–drug interactions. It serves as an essential reference for drug development and clinical pharmacology. MeSH (Medical Subject Heading) [38] thesaurus system, developed by the U.S. National Library of Medicine (NLM), employs a strictly standardized hierarchical classification system. It provides professional terminology standards for indexing and retrieving biomedical literature, significantly enhancing the accuracy and efficiency of medical information retrieval.

The dataset used in this study was derived from the work of Zhang et al. [39]. To mitigate the impact of data sparsity on prediction outcomes, we selected drug and disease entries from the CTD database that each had more than 10 associated relationships. After this rigorous screening, the final dataset was constructed, containing 269 drugs, 598 diseases, and their 18,416 verified positive associations. Additionally, the following key data sources were systematically integrated: 1) SMILES molecular fingerprint data for 269 drugs, extracted from the DrugBank database. 2) Hierarchical MeSH tree codes for 598 diseases, used for computing semantic similarities between disease terms.

Following the principle of theoretical association space integrity, we explicitly define the full combinatorial space of potential drug–disease pairs as 160,862. Among these, 18,416 represent confirmed positive associations, accounting for 11.4% of the entire space. The remaining 142,446 unverified pairs form the original negative sample pool. Recognizing the possibility that some unverified pairs may represent undiscovered true associations, we avoid naively treating them as true negatives. To ensure both class balance and biological plausibility, we adopt the KSU algorithm to select 18,416 unverified drug–disease pairs from the negative sample pool. This results in a balanced training set with a 1:1 ratio of positive and negative examples. This strategy not only maintains the theoretical space’s integrity and alleviates class imbalance, but also avoids the oversimplified assumption of non-association for unverified pairs—thus providing a scientifically sound foundation for model training.

Feature extraction methods

In drug–disease association prediction tasks, feature extraction plays a pivotal role in model construction, aiming to transform heterogeneous biomedical entities into low-dimensional, dense, and computationally tractable vector representations [40]. Traditional methods, which rely on hand-crafted descriptors (such as molecular fingerprints or disease codes), suffer from limitations like being high-dimensional and sparse, a lack of semantic information, and poor transferability. With the advancement of deep learning, distributed representation learning has emerged as a mainstream paradigm. Through large-scale unsupervised pretraining, this approach effectively captures the underlying structural relationships and semantic similarities between biomedical entities. Such methods adhere to two fundamental principles: they must fully preserve the topological relationships of the original data (e.g., the substructural connectivity of drug molecules and the hierarchy of the disease MeSH tree), and they must ensure that semantically similar entities are placed in close proximity in the vector space (i.e., they receive adjacent embeddings). In this context, MeSHHeading2vec serves as a typical representative of this paradigm for disease representation. For drug representation, a hybrid strategy integrating Mol2vec and K-BERT is adopted. This strategy fuses information derived from chemical graph structures (via Mol2vec) and knowledge-based SMILES sequences (via K-BERT), aiming to construct a comprehensive and generalized feature representation capable of robustly supporting downstream prediction tasks.

Drug feature extraction based on Mol2vec and K-BERT

This study leverages the complementary feature learning frameworks of Mol2vec and K-BERT to achieve a unified representation of drug molecules, capturing both local structural details and global semantic information.

We employ Mol2vec [41] to extract feature representations from drug SMILES sequences. This technique transforms molecular structures into chemically meaningful feature vectors through an innovative molecular vectorization pipeline. Inspired by the Word2Vec framework in natural language processing, Mol2vec deconstructs molecular topology and analyzes local atomic environments to construct a chemically relevant “molecular vocabulary.” It then applies the Skip-gram algorithm to learn contextual relationships between substructures, enabling efficient encoding of molecular features. The implementation process consists of three key steps:

1. Molecular structure parsing and substructure generation: The Morgan fingerprint algorithm is used to extract local structural features centered on each atom within a defined radius (radius = 1). Each atomic environment is represented by a unique substructure identifier (e.g., “C-1” denotes a carbon atom in a specific local context). The entire molecule is thus converted into a sequence of substructures, such as [‘C-1’,’O-3’,’N-5’,...].
2. Substructure vector space modeling: Based on a large-scale molecular corpus, a Word2Vec model is trained on a large-scale molecular dataset using the Skip-gram algorithm (with a context window of 10). It learns vector representations by modeling the contextual relationships between substructures, resulting in a fixed-length vector for each unique substructure.
3. Molecular vector aggregation: For each molecule, a molecule-level vector is generated by calculating a weighted average of the vectors for all of its constituent substructures. This process results in a final 300-dimensional global feature vector that represents the drug’s overall chemical characteristics. This feature vector implicitly encodes information such as functional groups and local chemical environments through the contextual relationships of its substructures. This approach avoids the need for hand-crafted descriptors (like LogP or molecular weight). Moreover, the pre-trained model offers high transferability to tasks with smaller datasets and allows for the analysis of molecular features through their constituent substructure vectors. These advantages significantly enhance both the representational power for drug structures and the effectiveness of predictive models.

Concurrently, the knowledge-based BERT model, K-BERT [42], is introduced to enhance global molecular semantic representation. Diverging from the initial multi-task design, during the actual pre-training phase, the model takes tokenized SMILES embeddings as input. It utilizes a Transformer encoder to learn sequence representations and employs a fully connected layer to predict predefined molecular fingerprints. This mechanism enables the model to directly learn the mapping between structural patterns and holistic chemical semantics. Pre-trained on approximately 1.8 million molecules from the ChEMBL database, K-BERT employs a six-layer Transformer architecture (hidden dimension = 768, 12 attention heads). The resulting embeddings (K-BERT-FP) effectively complement traditional fingerprints by capturing high-order characteristics—such as molecular size and chirality—that are often difficult for conventional descriptors to cover. Finally, the local substructure embeddings from Mol2vec are concatenated with the global molecular embeddings from K-BERT to form the definitive drug feature vector. This fused representation not only preserves fine-grained topological information but also encapsulates global semantics and fingerprint-level chemical features, providing a comprehensive, transferable, and interpretable input representation for downstream drug–disease association prediction.

Disease feature extraction based on MeSHHeading2vec

DeepWalk [43] graph embedding algorithm is employed to extract features from the Directed Acyclic Graph (DAG) structure of diseases, in order to fully capture their hierarchical dependencies and global semantic structure within the Medical Subject Headings (MeSH) system. Specifically, for each disease node D, based on its MeSH TreeNumber code, a hierarchical backtracking strategy is applied: by progressively truncating the tail segments of the TreeNumber, a DAG structure containing all ancestor nodes of disease D is systematically constructed. This hierarchy-based decomposition approach not only accurately depicts the inheritance relationships among disease concepts but also effectively captures potential semantic association networks. At the algorithmic level, DeepWalk uses a carefully designed random walk strategy combined with neural network modeling to transform complex graph structural information into distributed representations in a low-dimensional vector space. The process consists of two main stages:

1. (1) Random Walk Generator: Define the disease graph as G= (V, E), where V is the set of all disease nodes, and E represents the edges between nodes. Starting from each disease node , DeepWalk performs several truncated random walks to generate node sequences of length T: (1)

Where is uniformly sampled from the set of neighbors N() of node , with the transition probability defined as:

(2)

1. (2) Node Embedding Learning: For each random walk sequence w, the Skip-Gram model is used to optimize the following objective function: (3)

This process maps each disease node into a dense vector of dimension d = 64, such that topologically similar nodes (e.g., direct parent-child diseases or those sharing common ancestors) are distributed closely in the embedding space. The model learns low-dimensional representations for each disease node by predicting the context neighbors of the current node. As a result, structurally similar or hierarchically related disease nodes are mapped to nearby regions in the vector space. In a DAG structure, diseases are not only connected via direct parent-child relationships but may also exhibit complex higher-order semantic dependencies through multiple paths. DeepWalk preserves both local adjacency information and global topological structure during the embedding process, making it particularly well-suited for knowledge graphs like MeSH, which contain strong hierarchical semantics. Ultimately, the model encodes each disease node into a 64-dimensional embedding vector, which serves as the disease semantic feature input for the downstream drug-disease association prediction task.

K-means similarity undersampling algorithm based on adaptive multi-distance metric mechanism

In datasets for drug-disease association analysis, a significant class imbalance problem exists between positive and negative samples, with the ratio of associated to unassociated pairs reaching as high as 1:7. This imbalance in data distribution can lead traditional classification algorithms to become overly biased toward the majority class, thereby neglecting minority class samples that may possess critical research value. To address this critical challenge, this study innovatively proposed an improved KSU algorithm framework: AMDKSU (AMDKSU, K-means similarity undersampling algorithm based on adaptive multi-distance metric mechanism). By integrating an adaptive multi-distance metric mechanism with a dynamic group sampling technique, this algorithm effectively enhances the robustness of sample balancing and strengthens the discriminative power of the training data’s features. This lays a solid foundation for the subsequent construction of a high-precision drug-disease association prediction model.

KSU undersampling is an efficient sample selection technique whose core idea is to filter out the most representative samples using distance metrics. The method first employs K-means clustering to partition the majority class samples into k clusters. Subsequently, it rearranges sample pairs within each cluster based on their distance and dissimilarity matrices. During the implementation, the system identifies the N pairs of samples with the highest similarity and randomly removes one sample from each pair. This innovative process not only effectively preserves the distributional characteristics of the majority class but also significantly balances the sample size disparity between classes, thereby enhancing the classification model’s performance. The pseudocode of the algorithm is presented as follows.

Algorithm 1. KSU Undersampling Algorithm.

To better select representative samples, we introduces a grouped sampling strategy. Specifically, majority class samples are grouped based on drug molecular structures, and KSU undersampling is independently applied within each group to ensure local balance, thereby achieving global control over sample size. Concurrently, the approach adaptively utilizes five distinct distance metrics: Euclidean, Hamming, Chebyshev, Manhattan, and Minkowski. Their definitions and formulas are presented in Equations (4)–(8), respectively:

(4)

Euclidean Distance is Suitable for numerical, continuous feature spaces. it measures the straight-line distance between two points.

(5)

Hamming Distance is applicable to discrete or binary feature spaces. it counts the number of positions at which the corresponding elements of two vectors are different. I (⋅) is the indicator function, which equals 1 if the condition is true, and 0 otherwise.

(6)

Chebyshev Distance measures the maximum difference between two vectors along any coordinate dimension, making it suitable for scenarios where extreme values are significant.

(7)

Manhattan Distance is also known as the city block distance or L1 distance. It measures the sum of the absolute differences of the coordinates of two points.

(8)

Minkowski Distance is generalized form of distance metrics. The parameter p controls the calculation, allowing it to represent several classic distances.

Key feature extraction based on F-TEST

The F-test was selected as the feature selection method in this study because it provides a simple yet powerful statistical criterion for measuring the linear dependency between continuous features and categorical outcomes. Compared with more complex wrapper or embedded methods, the F-test offers high computational efficiency and strong robustness, making it particularly suitable for high-dimensional biological data. Moreover, as a classical and interpretable statistical approach, it enables transparent evaluation of feature importance, aligning with our goal of building an efficient and explainable prediction framework. F-test is employed to perform feature selection on drug features. As a filter method based on statistical inference, the F-test is essentially an analysis of variance (ANOVA) technique used to evaluate the strength of linear correlation between individual continuous features and categorical variables. As illustrated in Fig 2, the F-distribution shows the probability density functions corresponding to different F-values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Density function of the F-distribution.

https://doi.org/10.1371/journal.pcbi.1013947.g002

During the feature selection process, the F-test compares the variance of different classes across each feature dimension to assess the relationship between each feature and the target variable (e.g., drug–disease association). Specifically, the F-value is defined as the ratio of between-group variance to within-group variance, reflecting whether the feature exhibits statistically significant differences across categories. The basic formula is given in Equation (9).

(9)

Between-group variance reflects the degree of mean difference across categories, while the within-group variance captures the variability of samples within each category. A higher F-value indicates that the feature shows stronger discriminative power among different classes and exerts a greater influence on the target variable. Therefore, such features are considered important and retained. Through F-test, key drug features highly correlated with the target variable can be selected, leading to an optimized model structure with improved prediction accuracy and generalization capability.

Construction of the ensemble model

To effectively capture the potential complex association patterns between drugs and diseases, this study proposes a classification framework based on ensemble learning, which integrates three classical four-based models: Extreme Gradient Boosting (XGBoost), Decision Tree, Random Forest and HyperFast. The objective of this ensemble strategy is to combine the structural advantages of different models to improve the overall prediction accuracy and generalization capability.

Extreme Gradient Boosting (XGBoost)

XGBoost (Extreme Gradient Boosting) is an advanced implementation of the Gradient Boosting Decision Tree (GBDT) framework, widely applied to classification and regression tasks on structured data due to its high efficiency, flexibility, and excellent scalability. By iteratively minimizing a loss function and introducing a regularization term, XGBoost effectively controls model complexity and reduces the risk of overfitting. This process allows it to leverage precise feature split gains to deeply explore non-linear relationships and complex interaction patterns among high-dimensional features. In this study, XGBoost takes the feature vectors of drugs and diseases as input. Leveraging its tree structure’s automatic splitting mechanism, it deeply investigates the complex connections among high-dimensional features, thereby implicitly identifying the most critical ones. Thanks to its strong non-linear modeling capabilities and its native mechanism for handling missing values, XGBoost demonstrates excellent performance and robustness when modeling the intricate interactions between drugs and diseases.

Decision tree

The Decision Tree, characterized by its intuitive structure and high interpretability, is widely applied to both classification and regression tasks. It captures relationships between features by recursively partitioning the dataset into subsets, building a tree-like model according to criteria such as Gini impurity or information gain. This makes it suitable for solving a wide range of structured data problems. In this study, a Decision Tree with fixed hyperparameter settings is employed to jointly model the 300-dimensional drug features and 64-dimensional disease features. The model creates a series of decision rules in the feature space based on feature splits, automatically learning the relationships between drug and disease features. It selects the optimal feature and performs node splitting according to the Gini impurity. After training, the Decision Tree can make step-by-step decisions based on the input drug-disease feature vector, ultimately assigning a corresponding class label. It also provides a complete tree structure including tree depth, number of leaf nodes, and feature importance, serving as a foundation for subsequent interpretability analysis.

Random forest

Random Forest is a classic Bagging-based ensemble learning method, which constructs multiple independent decision trees and aggregates their predictions to maintain high classification accuracy while effectively suppressing overfitting. By leveraging the Bagging technique, Random Forest performs random sampling with replacement on the training dataset to build diverse decision trees, thereby enhancing generalization performance. During the construction of each tree, Random Forest randomly selects a subset of features at each node split, ensuring independence among trees. The final prediction is made by majority voting or averaging across all trees. In this study, the Random Forest model is implemented by randomly sampling with replacement from the original dataset to create multiple training sets, ensuring data diversity across trees. At each split node, a random subset of features is considered for splitting, which effectively reduces the risk of overfitting and improves the model’s generalization ability. By capturing various data patterns and complex nonlinear relationships, Random Forest enhances the classification performance through the ensemble of multiple tree predictions [44].

HyperFast

HyperFast [45] is constructed based on a hierarchical mapping mechanism involving a Hypernetwork and a Main Network. The overall framework consists of three core modules: “Feature Preprocessing and Mapping,” “Hypernetwork Weight Generation,” and “Main Network Classification Inference,” forming an efficient one-shot generative classification system. The specific architecture is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The framework of HyperFast.

https://doi.org/10.1371/journal.pcbi.1013947.g003

In the feature preprocessing phase, the input sample matrix is denoted as:

(10)

where n is the number of samples and d is the feature dimension. The model first performs a standardization mapping on the input:

(11)

where and are the mean and standard deviation of the feature dimensions, respectively. Subsequently, HyperFast introduces random feature transformation to approximate nonlinear kernel mapping. Through a Gaussian random matrix and a bias term b, the model maps the input to a high-dimensional random feature space:

(12)

where is a nonlinear activation function (such as ReLU). This process mathematically approximates the implicit projection of an arc-cosine kernel. To further enhance feature compactness and stability, the model performs Principal Component Analysis (PCA) on the random feature space, retaining the top k principal components:

(13)

where is the projection matrix obtained from the eigendecomposition of the feature covariance matrix. This step effectively reduces feature redundancy and noise interference while preserving principal information.

In the weight generation phase, the hypernetwork receives the support set representation and its labels , generating all parameters for the main network via a single forward pass:

(14)

where represents the set of hierarchical weights for the main network. The hypernetwork adopts a layer-wise weight generation strategy: for the first L-1 layers, it utilizes global average pooling for feature statistic aggregation; for the classification layer (the L-th layer),it combines class-wise mean pooling and bias adjustment to generate class-discriminative vectors. This mechanism enables the model to generate task-specific parameters directly based on the feature distribution of support samples, achieving rapid modeling without iterative training.

The classification inference phase is performed by the main network , which is structured as a three-layer Multi-Layer Perceptron (MLP) and introduces residual connections between layers to improve gradient stability and representational capacity. For query set samples , the model performs forward propagation to obtain class predictions:

(15)

and obtains the posterior probability of the target class via the Softmax function:

(16)

The entire inference process is completed under the static weights generated by the hypernetwork, without parameter updates or gradient backpropagation, thereby significantly enhancing inference efficiency and task adaptability. In summary, the HyperFast classifier achieves an end-to-end mapping from input features to prediction results through random feature kernel mapping, hierarchical weight generation, and lightweight forward inference. Its core innovation lies in replacing the traditional training process with a hypernetwork, realizing efficient non-iterative classification via one-shot weight generation, endowing the model with outstanding generalization performance and computational scalability across different tasks.

To fully leverage the complementary advantages of different models in feature expression and generalization capability, this study adopts a two-layer Stacking ensemble strategy to realize multi-model fusion. This method introduces four types of heterogeneous learners at the Level-0 layer—XGBoost, Decision Tree, Random Forest, and HyperFast—and obtains Out-of-Fold (OOF) prediction results through $K$-fold cross-validation to provide robust inputs for upper-level learning. Subsequently, the Level-1 layer employs Logistic Regression as the meta-model, using the OOF prediction probability matrix as features to learn the optimal combination of outputs from different base models, thereby statistically achieving adaptive weighting and nonlinear fusion. Overall, this Stacking strategy learns the complementary relationships of different models at the meta-level, making the fused model superior to any single model in both prediction accuracy and stability. Experimental results indicate that this method can effectively capture multimodal structural features in drug–disease association tasks, providing an efficient and generalizable systemic solution. Model parameter configurations are shown in Table 1. It is worth noting that as a pre-trained hypernetwork, HyperFast’s parameters are frozen during the meta-training phase, relying on rapid weight generation to achieve task adaptability, requiring no gradient updates or hyperparameter tuning during integration.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Parameter settings of individual models.

https://doi.org/10.1371/journal.pcbi.1013947.t001

Evaluation metrics

The predictive performance of the model was evaluated using several standard metrics: Accuracy (ACC), Precision (Pre), Sensitivity (SN), Specificity (SP), Matthews Correlation Coefficient (MCC), Area Under the Receiver Operating Characteristic Curve (AUC), and Area Under the Precision-Recall Curve (AUPR) [46–50]. The mathematical definitions for these metrics are provided in Equations (17)–(23) [51]. Specifically, in the context of drug-disease association prediction, a sample is defined as a positive sample if it represents a verified association; otherwise, it is classified as a negative sample.

(17)

(18)

(19)

(20)

(21)

TP represents the number of verified associations correctly predicted as positive; TN is the number of non-associations correctly predicted as negative; FP is the number of non-associations incorrectly predicted as positive; and FN is the number of verified associations incorrectly predicted as negative.

(22)

The Receiver Operating Characteristic (ROC) curve is an essential tool for evaluating the discriminative ability of predictive models. Its horizontal axis represents sensitivity, while the vertical axis corresponds to specificity. By plotting this curve, the model’s performance across different threshold values can be visualized. The area under the ROC curve (AUC) serves as a core indicator, ranging between 0 and 1. A higher AUC value indicates superior model discrimination. Specifically, an AUC close to 1 suggests near-perfect separation between positive and negative samples, while an AUC of 0.5 implies performance equivalent to random guessing.

(23)

The Area Under the Precision-Recall Curve (AUPR) is another important evaluation metric in machine learning. It is calculated by integrating the area under the Precision-Recall curve, providing a comprehensive measure of the classifier’s performance across different decision thresholds. The value of AUPR also ranges from 0 to 1, with a value closer to 1 indicating superior discrimination ability and predictive stability. Compared to other metrics, AUPR is particularly well-suited for scenarios with imbalanced class distributions, as it more accurately evaluates a model’s effectiveness in identifying the minority class.

Effectiveness of drug feature extraction

Accurate drug feature extraction serves as the cornerstone of successful Drug-Disease Association (DDA) prediction. High-quality embeddings enable learning algorithms to discriminate subtle variations in molecular function, enhance generalization to novel compounds, and ultimately improve prediction reliability. To rigorously validate the rationale underpinning our selected feature extraction framework, we conducted a comparative analysis involving five distinct drug embedding strategies. The results are presented in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of molecular embeddings.

https://doi.org/10.1371/journal.pcbi.1013947.t002

Our experimental results indicate that while single-feature methods such as Mol2vec, K-BERT, GIN, and ChemBERTa-2 each capture distinct aspects of molecular information, the variances in their performance are only moderate. In sharp contrast, our proposed fusion strategy (Mol2vec + K-BERT) demonstrates a significant performance leap, achieving the highest scores across all eight evaluation metrics, with an AUC of 0.9725 and an F1-score of 0.9305. This superiority validates the complementarity hypothesis of our study: Mol2vec explicitly encodes the frequency of local chemical substructures, whereas K-BERT injects global semantics and domain knowledge. By integrating these two dimensions, the model overcomes the limitations inherent in unimodal approaches, thereby providing a robust and comprehensive input for downstream predictors.

Effectiveness of undersampling algorithms

Due to the inherent class imbalance in drug–disease association datasets, the scarcity of positive samples often leads to models that underestimate their misclassification risk. This results in reduced sensitivity and inflated specificity. To mitigate this bias, this study introduces undersampling algorithms to balance the distribution of positive and negative samples. The KSU algorithm, in particular, effectively reduces the number of samples in the majority class, creating a more balanced dataset for model training, as visually demonstrated in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Comparison of data distribution before and after applying KSU undersampling.

https://doi.org/10.1371/journal.pcbi.1013947.g004

By comparing the AUC performance of several algorithms (including KSU, NCR, NearMiss, OSS, and Random Undersampling), the KSU algorithm was ultimately selected. Moreover, we further optimized its sample selection strategy by tailoring the distance metrics according to different model architectures. The experiments cover a wide range of machine learning models (such as XGBoost, Random Forest, Naive Bayes, etc.) as well as deep learning models (including DNN, BiLSTM, GRU, etc.). The AUC performance of each model under different undersampling algorithms is summarized in Table 3. For this initial benchmark comparison, the KSU algorithm utilized the standard Euclidean Distance as its metric to ensure a fair and consistent baseline.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. AUC comparison of different models using various undersampling algorithms (KSU with Euclidean Distance).

https://doi.org/10.1371/journal.pcbi.1013947.t003

The results indicate that the KSU undersampling algorithm achieved the optimal performance across all models. Specifically, XGBoost reached an AUC of 0.9623, RandomForest achieved 0.9208, DecisionTree also reached 0.9208, and DNN attained an AUC of 0.9189. To further optimize the sample selection strategy and investigate the interaction between model architectures and distance metrics, we evaluated the performance of six distinct classifiers—XGBoost, Decision Tree, Random Forest, DNN, TextRCNN, and RNN—using five different distance formulas within the K-means undersampling (KSU) framework. The comparative AUC scores are presented in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of AUC scores for different models using various distance formulas.

https://doi.org/10.1371/journal.pcbi.1013947.t004

The results reveal a clear architectural divergence regarding metric sensitivity. Tree-based models (XGBoost, Decision Tree, Random Forest) consistently achieved optimal performance with Hamming Distance, likely due to its alignment with the discrete nature of molecular fingerprints. In contrast, sequence-based and deep models favored spatial metrics: notably, the RNN peaked using Chebyshev Distance (AUC 0.8007) but performed poorly with Hamming (0.7358), while TextRCNN preferred Minkowski Distance (0.9101). Despite these variations, the combination of XGBoost and Hamming Distance yielded the superior overall AUC (0.9696). Consequently, Hamming Distance was prioritized for the final prediction framework. Furthermore, to quantify the joint effect of undersampling and distance metrics, the performance of the ensemble model before and after sampling was compared, as shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of model performance before and after training set sampling and with different distance formulas.

https://doi.org/10.1371/journal.pcbi.1013947.t005

As indicated by the data in the table, the model’s performance improved significantly across multiple metrics after sampling. Prior to sampling, the model exhibited an Accuracy of 0.8913, an AUC of 0.7593, an F1-score of 0.1489, an MCC of 0.2193, and a Sensitivity of just 0.0832. These results indicate a substantial bias on the class-imbalanced dataset and a particularly weak ability to identify positive class samples. After applying undersampling with the baseline Euclidean Distance, the model’s performance improved significantly across all metrics. Notably, when the optimal Hamming Distance identified in Table 5 was applied, performance was further boosted, with sensitivity increasing to 0.9053, the F1 score reaching 0.9374, and the MCC improving to 0.8811. These results indicate substantial enhancement in the model’s ability to predict both positive and negative classes, especially in identifying positive samples (sensitivity) and achieving overall balanced performance (F1 score). Therefore, both undersampling and the selection of an appropriate distance metric play a critical role in boosting model performance, particularly in handling imbalanced datasets.

Effectiveness of F-TEST feature selection

To evaluate the effectiveness of the F-test feature selection method in this study, we performed dimensionality reduction on the original features and plotted the distribution of F-values for different dimensions, as shown in Fig 5. The figure displays the frequency distribution (on a logarithmic scale) of F-values for the complete feature set (n = 300) as well as for feature subsets reduced to 140 and 160 dimensions. It can be observed that as the number of features decreases, the distribution of F-values shifts toward higher-value regions. In particular, the proportion of high F-value features (e.g., F > 100) significantly increases in the k = 140 and k = 160 subsets, indicating that the retained features exhibit stronger discriminative power between classes. Meanwhile, the number of low F-value features drops sharply, suggesting that redundant or noisy features were effectively removed. Additionally, the statistical information in the legend confirms that the discriminative capability of the feature sets was enhanced through dimensionality reduction. The average F-value of the full feature set was 45.7, which increased to 87.4 when the top 140 features were selected and remained high at 79.7 for 160 features. These results demonstrate that the F-test not only enables effective dimensionality reduction but also serves as a robust feature selection method, contributing to improved model training efficiency and predictive performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison of F values for different numbers of features.

https://doi.org/10.1371/journal.pcbi.1013947.g005

To further assess the effectiveness of the F-test in improving model training efficiency, we conducted a comparative analysis of training time under different drug feature dimensions, as illustrated in Fig 6. When the number of features was reduced to 140 and 160, the training time decreased by 3.82 seconds and 3.39 seconds, respectively, indicating a significant improvement in computational performance due to dimensionality reduction. At the same time, model accuracy remained stable or even slightly improved on these feature subsets, suggesting that dimensionality reduction did not impair predictive performance but rather enhanced the model’s generalization capability. Therefore, the F-test method effectively compresses the high-dimensional drug feature space in this task, selecting a subset of features with stronger discriminatory power. This approach substantially reduces computational overhead while maintaining predictive performance, proving it to be an efficient and robust feature selection strategy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Comparison of model training time under different feature dimensions.

https://doi.org/10.1371/journal.pcbi.1013947.g006

Effectiveness of the ensemble model

This study evaluated the performance of 15 mainstream machine learning and deep learning classification models through a comparative experiment to systematically select base models. Recognizing that the performance of each model is highly dependent on the selected parameters, hyperparameter tuning was conducted via grid search to identify the optimal parameter subset for each model. To avoid overfitting and ensure the reliability of the results, ten-fold cross-validation was used in conjunction with grid search. Based on the results obtained from the grid search, a model with the optimal parameter set was constructed for each algorithm. Subsequently, the best model was selected from this set based on its predictive performance on the validation set. The experimental results are shown in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison results of models.

https://doi.org/10.1371/journal.pcbi.1013947.t006

Based on the experimental results after hyperparameter optimization via ten-fold cross-validation and grid search, quantitative analysis indicates that XGBoost, HyperFast, Decision Tree, and Random Forest form a distinct “first tier” across key performance metrics, significantly outperforming other baseline models. This conclusion is strongly supported by the data: XGBoost leads in all evaluation metrics, achieving the best performance particularly in AUC (0.9623), AUPR (0.9710), and F1-score (0.9050). HyperFast, as a novel meta-learning model, demonstrates exceptional competitiveness, ranking second in both AUC (0.9401) and AUPR (0.9479), surpassing all traditional models except XGBoost. Meanwhile, Decision Tree (F1: 0.8818) and Random Forest (F1: 0.8735) closely follow in accuracy and F1 scores, ranking among the top performers.

To obtain the most robust predictive performance, this study does not rely on a single model but instead selects these four models—which possess advantageous and highly complementary performance and mechanisms—to jointly construct the Level-0 base model layer of the Stacking ensemble.Specifically: XGBoost (AUC 0.9623), as a representative of gradient boosting, provides extremely high prediction accuracy and overfitting control by integrating weak learners; HyperFast (AUC 0.9401) represents a novel meta-learning paradigm, where its “one-shot inference modeling” mechanism offers a unique perspective for model combination, demonstrating superior performance in key metrics; Random Forest (AUC 0.9208) adopts a Bagging strategy, enhancing model stability and generalization capability through the introduction of randomness; and Decision Tree (AUC 0.9208) provides transparent decision paths via Gini impurity splitting rules, enhancing model interpretability.

The ROC curves for each model and the final fusion model are shown in Fig 7. As seen in the figure, the AUC values of all four single base models (XGBoost, HyperFast, Decision Tree, Random Forest) are lower than that of the final fusion model. This indicates that the Level-1 meta-model (Logistic Regression) effectively learns and integrates the strengths of these four heterogeneous base learners, mitigating potential bias and variance issues inherent in single models. By intelligently weighting the predicted probabilities from the Level-0 models, the fusion model significantly improves classification accuracy and stability. When handling tasks such as drug–disease association prediction—which are high-dimensional, complex, and potentially subject to data imbalance or noise—the fusion method demonstrates stronger generalization capability and reduces the risk of overfitting, thus yielding an AUC value higher than any single model. This result powerfully validates the effectiveness of the proposed four-model fusion strategy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. ROC curves of the individual models and the ensemble model.

https://doi.org/10.1371/journal.pcbi.1013947.g007

Analysis of model complexity and performance trade-off

While pursuing high prediction accuracy, the computational complexity of a model is a critical factor determining its practicality and scalability. The final prediction model adopted in this study is based on a weighted Ensemble Model, combining the complementary advantages of four heterogeneous base learners: XGBoost, RandomForest, DecisionTree, and HyperFast. Therefore, quantitatively analyzing the overall computational cost of this ensemble model and comparing it with its independent components is of great significance for model selection under different computational environments. This section provides a comprehensive evaluation of the performance and complexity of each model based on training duration and peak memory usage. It should be noted that the total training time of the ensemble model is calculated as the sum of the training durations of its constituent base learners. Relevant results are presented in Table 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Comprehensive comparison of performance and complexity between the final ensemble model and its components.

https://doi.org/10.1371/journal.pcbi.1013947.t007

As indicated by the data analysis in Table 7, FKSUDDAPre achieved highly competitive predictive performance (AUC = 0.9725), attributable to its integration of base models with diverse mechanisms. However, this performance improvement is accompanied by increased computational costs. The total training duration (109.20 seconds) and peak memory usage (64.02 MB) of the ensemble model are primarily dominated by the HyperFast component (99.33 seconds, 64.02 MB). Although HyperFast, as a meta-learning model, introduces higher resource overhead, its unique “one-shot inference” mechanism provides the ensemble framework with a perspective distinct from traditional tree models, enhancing the model’s generalization potential under complex distributions.

In contrast, the traditional tree model components demonstrated extremely high computational efficiency: RandomForest emerged as the preferred choice for rapid iteration with the shortest training time of 2.11 seconds; XGBoost stood out most prominently among the standalone models, not only boasting fast computation (2.86 seconds) but also performing on par with, or even slightly better than, the ensemble model in terms of AUC (0.9730). This indicates that in scenarios where resources are severely constrained and deploying the full ensemble model is unfeasible, XGBoost serves as the optimal standalone alternative.

Comparison with other advanced methods

To further evaluate the effectiveness of the proposed method, we conducted a comparative study on the same dataset with several state-of-the-art approaches, including HINGRL [28](2021), CMFMTL [18](2020), HDGAT [52](2024), GRLGB [53](2023), MFFGCN [8](2023), and AMDGT [30](2023). All of these methods utilized publicly available datasets as reported in their original publications.

Fig 8 presents the comparison of the average AUC scores achieved by each method on the B dataset. The results clearly show that our method outperformed all other approaches, achieving an average AUC of 0.9725. Specifically, it surpassed the comparative methods by 8.9%, 8.27%, 10.83%, 5.24%, 10.63%, and 3.88%, respectively. These findings strongly demonstrate the superior performance and robustness of our proposed approach.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Comparison of AUC values between FKSUDDAPre and other existing drug–disease interaction prediction methods.

https://doi.org/10.1371/journal.pcbi.1013947.g008

LIME interpretability analysis

Although deep learning models demonstrate exceptional performance on complex tasks, their nonlinear structures and high-dimensional parameter spaces make their prediction mechanisms difficult to interpret directly. To enhance the transparency of the model’s decision-making process, this study adopted LIME (Local Interpretable Model-agnostic Explanations) as a tool for local interpretability analysis [54]. The fundamental idea of LIME is to generate a series of perturbed samples in the neighborhood of an original input sample. It then obtains the predictions for these perturbed samples from the black-box model and uses them to fit a weighted linear model. This process approximates the model’s true behavior in the local space, explaining the marginal impact of features on an individual prediction. This method is model-agnostic, offering broad applicability and strong human readability.

Fig 9 presents a LIME-based feature importance visualization for a specific test sample, aiming to illustrate the local influence of input features on the model output during prediction. The horizontal axis represents the influence on prediction, while the vertical axis lists the top 25 key features sorted by importance, each annotated with its corresponding threshold condition (e.g., “feature_44_y ≤ -0.30”). This effectively approximates the model’s local decision rule. The color coding employs a gradient heatmap, where deep green indicates that a feature’s current value makes a significant positive contribution to the positive class prediction. In contrast, deep red signifies a strong inhibitory effect (negative contribution) on the positive class prediction. Lighter shades indicate a weaker influence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. LIME-based feature importance analysis.

https://doi.org/10.1371/journal.pcbi.1013947.g009

From the figure, features such as feature_44_y ≤ -0.30, feature_63_y ≤ -0.40, and feature_18_y ≤ -0.33 appear in dark green, indicating that they are major positive drivers for the model’s prediction of the positive class in this sample. Conversely, feature_34_y > 0.32, feature_33_y > 0.37, and feature_41_y > 0.20 are shown in dark red, suggesting they strongly suppress the model’s positive class prediction, functioning as negative indicators in the local decision logic. Each feature’s logical expression includes a specific threshold boundary, offering insight into the local decision boundary around the sample. This enables the behavior of the complex model near this sample point to be approximated in a rule-based manner, improving interpretability.

Case study

To evaluate the practical applicability of the proposed model (FKSUDDAPre), we conducted an in-depth case study on two representative neurodegenerative diseases: Alzheimer’s Disease (AD) and Parkinson’s Disease (PD). All known drug-disease associations in the dataset were used as training data, and the model was trained to predict the association probabilities for all unknown drug-disease pairs. Subsequently, drugs were ranked based on the predicted probabilities, and the top 10 candidate drugs were selected for each disease.

Alzheimer’s Disease is the most common form of progressive dementia in the elderly and represents a severe neurodegenerative condition. Table 8 presents the top 10 candidate drugs predicted by the model for AD treatment. Among them, 8 drugs have been validated by existing literature, indicating the model’s strong practical value. For example, Tolbutamide, a KATP modulator, was found to provide significant protection against Aβ-induced memory defects. Albendazole, an FDA-approved anti-parasitic drug, has been shown in recent studies to reduce levels of AD-associated Tau protein in cells. Furthermore, Felodipine, an L-type calcium channel blocker, can induce autophagy and clear various aggregation-prone proteins related to neurodegenerative diseases.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. The top 10 potential drugs identified by for Alzheimer’s disease.

https://doi.org/10.1371/journal.pcbi.1013947.t008

Parkinson’s Disease (PD) is the second most common neurodegenerative disorder after Alzheimer’s Disease. Table 9 lists the top 10 predicted candidate drugs for PD generated by the model. Among them, 6 drugs have been previously reported in relevant literature, further demonstrating the model’s effectiveness in identifying potential therapeutic agents. For example, Caffeine is supported by numerous epidemiological studies linking its intake to a reduced risk of developing PD. Albendazole has demonstrated neuroprotective effects in PD models by activating key pathways like Nurr1 and suppressing neuroinflammation. Furthermore, Chlorpropamide has been shown to inhibit the fibrillization of alpha-synuclein, a key pathological process in PD.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. The top 10 potential drugs identified by for Parkinson’s disease.

https://doi.org/10.1371/journal.pcbi.1013947.t009

These findings highlight the practical applicability and clinical relevance of the FKSUDDAPre model in drug repurposing for neurodegenerative diseases. By accurately prioritizing candidate drugs with known or emerging therapeutic relevance, the model can serve as a valuable decision-support tool for clinicians and biomedical researchers. For Alzheimer’s Disease and Parkinson’s Disease—both lacking effective disease-modifying treatments—FKSUDDAPre offers a systematic approach to rapidly identify repurposable compounds from existing drug libraries, potentially accelerating the drug discovery process and reducing clinical trial costs. Moreover, the inclusion of non-traditional drug classes (e.g., antihistamines and chemotherapeutic agents) among the top candidates may stimulate new clinical hypotheses and broaden the scope of pharmacological intervention strategies. Future work integrating this predictive framework with clinical trial design and patient stratification may further enhance its translational impact.