Datasets overview and preprocessing

This study was designed to systematically compare two widely used RNA-seq differential expression tools, edgeR and DESeq2, using a diverse collection of publicly available bulk RNA-seq datasets. The selected datasets span viral infection, bacterial infection, and chronic fibrotic disease contexts, including SARS-CoV-2, RSVB, EBOV, Mpox, bacterial pneumonia, influenza, and idiopathic pulmonary fibrosis (IPF). These datasets also include both human and nonhuman primate samples, which enabled evaluation across different biological systems and experimental designs. Each dataset included clearly defined control and treated, infected, stimulated, or diseased sample groups.

Datasets were organized according to the main objective of each analysis stage. First, the RSVB dataset (GSE196134) was used for sample-size sensitivity and outlier-perturbation analyses because it provided a balanced design with 45 unstimulated control samples and 45 RSVB-stimulated samples. Second, five datasets were used to assess the classification performance of tool-specific significant genes within the discovery dataset: Mpox (GSE234118), EBOV (GSE115785), bacterial pneumonia and influenza (GSE161731), IPF (GSE231693), and SARS-CoV-2 (PMC8202013). Third, all biological contrasts used for within-dataset DEG comparison were also used for pathway-level concordance analysis. Finally, four independent SARS-CoV-2 datasets, PMC8202013, GSE152418, GSE161731, and GSE171110, were used for cross-study validation of tool-specific gene sets.

For each dataset, the available gene-count matrix was downloaded from the corresponding public repository and matched with the accompanying metadata. Metadata fields describing infection status, stimulation condition, disease status, or time point were used to assign samples to control and treated groups. Samples with ambiguous or inconsistent metadata annotations were excluded. Analyses were restricted to protein-coding genes, and duplicated or unsupported gene symbols were removed during formatting. After preprocessing, each matrix was organized with genes as rows and samples as columns. No additional normalization was applied before differential expression analysis, because normalization was performed within the tool-specific edgeR and DESeq2 workflows. Table 1 summarizes the datasets used in this study, their sample sources, control and treated groups, and their role in the comparative analysis.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Summary of RNA-seq datasets used in this study. Each dataset was selected to enable systematic comparison of differential gene expression results from edgeR and DESeq2 under well-defined baseline and treated, infected, stimulated, or diseased groups.

https://doi.org/10.1371/journal.pone.0353788.t001

Tool configuration and execution strategy

To simulate real-world application and usability, we ran both edgeR and DESeq2 using their recommended and widely adopted default pipelines. These reflect the typical usage patterns of practitioners who perform standard RNA-Seq differential expression analysis using default guidance from respective documentation and vignettes. The following summarizes how each major methodological component was implemented in edgeR (v4.4.2) and DESeq2 (v1.46.0):

• Model Framework: In edgeR, a negative binomial GLM is fitted using quasi-likelihood methods. Differential expression (DE) analysis is performed using glmQLFit() followed by glmQLFTest(). In DESeq2, a negative binomial GLM is fitted using DESeq(), and hypothesis testing is conducted using the Wald test.
• Normalization: edgeR performs normalization using the calcNormFactors() function, which implements the Trimmed Mean of M-values (TMM) method. DESeq2 uses the median-of-ratios method implemented in estimateSizeFactors().
• Dispersion Estimation: edgeR estimates common, trended, and tagwise dispersions using estimateDisp(). DESeq2 uses estimateDispersions() to compute gene-wise and fitted dispersions.
• Statistical Test and Output: edgeR applies the GLM quasi-likelihood F-test via glmQLFTest(). In DESeq2, the DESeq() function fits the negative binomial GLM and, under the default setting, performs Wald tests for the model coefficients; likelihood ratio tests are also available as an alternative option. The results() function was then used to extract the requested coefficient estimates, Wald test p-values, adjusted p-values, and log₂ fold changes.

Sensitivity to sample size and robustness to outliers

The first phase of our comparative evaluation aimed to assess how edgeR and DESeq2 respond to sample size variation and the presence of outliers, both common challenges in real-world RNA-Seq experiments. To perform this, we selected the RSVB dataset (GSE196134), which contains 90 total samples: 45 control (unstimulated) and 45 RSVB-infected (stimulated). This balanced and large dataset enabled controlled subsampling and simulation of outlier scenarios.

We first applied both tools to the full dataset, contrasting the RSVB-infected group against the control group. Subsequently, we generated three subsampled datasets, each with 20, 10, and 5 samples per group, respectively. Each subsampled dataset was randomly drawn from the original 45 samples per group. This design allowed us to evaluate the sensitivity of each method to reductions in sample size. For each dataset version (full and subsampled), we applied the same analysis pipeline using edgeR and DESeq2. DEGs were defined as those with a Bonferroni-adjusted 0.05 [36] and an absolute log₂ fold change . This stringent threshold was chosen because applying an FDR cutoff resulted in an excessively large number of differentially expressed genes for both tools, which hindered meaningful comparison. Using Bonferroni-adjusted p-values allowed for a more conservative and balanced comparison, which better suited to our goal of evaluating differences between the two methods. To quantify consistency and directional agreement between tools, we introduced the following metric:

Definition: Let A and B be two sets of differentially expressed genes. The ordered pair (A, B) is interpreted as comparing the set A against the reference set B. The Directional Overlap, or , is defined as the proportion of elements in B that are also found in A, given by

(1)

which quantifies the extent to which the reference set B is recovered by the comparison set A. Notably, in general due to its asymmetry. For the sample-size analysis, we computed directional overlap between the DEG sets identified by edgeR and DESeq2 within each subsampling replicate. For n = 5, n = 10, and n = 20 samples per group, directional overlap values were summarized as mean ± standard deviation across 20 independent subsampling replicates. For the full dataset (n = 45 samples per group), directional overlap was computed once using the complete dataset.

We also evaluated the tools’ robustness to outliers by introducing controlled sample swaps between the treatment and control groups. Specifically, we generated synthetic outliers by swapping 1–5 samples between groups. For instance, in the 1-swap scenario, one control sample was swapped with one RSVB-infected sample, thus injecting one outlier into each group. This process was repeated for 2–5 swaps. To ensure statistical robustness and mitigate the effect of random variation in swap choices, each swap level (1–5) was independently repeated 20 times. Given the original dataset’s size (45 per group), the proportion of swaps remained appropriate.

To assess changes introduced by these simulated outliers, we computed the Jaccard Index [37] between the original DEG set (from the full, unperturbed dataset) and the DEG set after introducing outliers:

(2)

where, A represents the set of significant genes identified after injecting outliers, and B is the original DEG set without outliers. Jaccard similarity provides a symmetric measure of agreement and reflects how much the gene sets overlap before and after contamination with outliers.

Classification performance of uniquely identified gene sets within the discovery dataset

In the second phase of our analysis, we systematically compared edgeR and DESeq2 across multiple datasets to evaluate the concordance and discrepancy in their differential expression results. To evaluate the biological signal of tool-specific genes, we applied this analysis to five representative datasets covering diverse biological conditions: Mpox (GSE234118), EBOV (GSE115785), Bacterial and Influenza infection (GSE161731), Idiopathic Pulmonary Fibrosis (IPF, GSE231693), and SARS-CoV-2 (PMC8202013).

For each dataset, we quantified the number of significant genes identified by each tool using a stringent threshold: Bonferroni-adjusted and . We then evaluated concordance and divergence between edgeR and DESeq2 by reporting the number of genes uniquely identified by each tool, the number of shared significant genes, and the direction-specific Jaccard indices (Equation 2) for upregulated and downregulated gene sets, where A and B represent the respective sets from edgeR and DESeq2. To further examine agreement between tools, we calculated Pearson [38] and Spearman [39] correlation coefficients for the log₂ fold changes and Bonferroni-adjusted p-values among the common significant genes.

Finally, to assess how effective each tool is in identifying biologically meaningful significant genes, we focused specifically on the genes uniquely identified by each method. In the absence of an external ground truth for differentially expressed genes, one way to evaluate the “trueness” of these genes is to assess their ability to differentiate between biological groups, in this case, control and treated samples.

For each dataset, we extracted the genes uniquely identified as significant by each tool. Raw count values for these genes were log-transformed using without any normalization to preserve the original scale and avoid introducing method-dependent biases. Principal Component Analysis (PCA) [40] was then applied to reduce dimensionality while retaining key variance components. We retained only the first two principal components (PC1 and PC2), which capture the dominant structure in the expression space of the selected genes and help mitigate overfitting while maintaining interpretability.

PC1 and PC2 were used as predictors in a logistic regression classifier [41] trained to distinguish control from treated samples. Standard classification metrics, accuracy, precision, recall, and F1 score [42], were computed to evaluate how well the tool-specific significant genes separated the biological conditions. A higher classification performance suggests that the genes uniquely identified by the corresponding tool are more likely to reflect meaningful biological signal.

To further benchmark and summarize tool performance across all datasets and conditions, we adopted the Dolan-More performance profiling method [43]. This technique evaluates each method’s relative performance consistency across datasets using the F1 score as the basis for comparison.

Let denote the F1 score of method a on dataset i, and let be the highest F1 score obtained on dataset i across methods. The performance ratio for each method is defined as:

with the convention that if . A smaller indicates better performance, with signifying that the method achieved the best score on that dataset. The Dolan-More profile for method a is then the cumulative distribution function:

where n is the total number of datasets. This function quantifies the fraction of datasets where a given method’s performance is within a factor of the best-performing method. A Dolan-More curve that rises quickly and reaches earlier indicates a more consistently high-performing method. In our context, this analysis provides a global, dataset-agnostic perspective that complements pairwise comparisons and highlights each tool’s overall reliability and robustness in identifying biologically meaningful gene sets.

Pathway enrichment concordance analysis

To determine whether gene-level differences between edgeR and DESeq2 translated into differences in downstream biological interpretation, we performed a pathway enrichment concordance analysis using the significant DEG sets identified by each tool. This analysis was added because differences in the number or identity of significant DEGs may not necessarily lead to divergent pathway-level conclusions.

For each dataset contrast and each tool, we used the significant DEG set defined by the same threshold applied in the corresponding DEG analysis. We then performed pathway over-representation analysis using two curated pathway collections: the MSigDB Hallmark gene sets, which summarize coherent biological processes with reduced redundancy [44,45], and KEGG pathways, which provide manually curated pathway maps for biological systems and molecular interactions [46,47].

For each pathway collection, enrichment was tested using the hypergeometric test. The background was defined as the union of genes represented in the corresponding pathway collection. Pathway-level p-values were adjusted using the Benjamini-Hochberg procedure, and pathways with FDR < 0.05 were considered significantly enriched. For each dataset contrast, enriched pathways were classified as edgeR-only, shared by both tools, or DESeq2-only.

To quantify pathway-level agreement between tools, we computed the Jaccard similarity between the enriched pathway sets from edgeR and DESeq2:

where and denote the sets of significantly enriched pathways derived from edgeR and DESeq2 DEG sets, respectively. Higher values indicate greater pathway-level concordance between tools.

Generalizability of tool-specific gene sets across independent studies

As the final stage of our analysis, we evaluated the generalizability of the significant genes uniquely identified by each tool. While the previous classification-based analysis assessed how well tool-specific genes separated samples within the same dataset in which they were discovered, this section extends the evaluation to a more rigorous cross-dataset framework. Specifically, we asked: to what extent are the unique genes identified by each method in some datasets transferable and predictive in unseen datasets? This helps assess whether the tool-specific genes capture biologically consistent signals that generalize across studies.

We focused this evaluation on four independent bulk RNA-Seq datasets of SARS-CoV-2 infection: GSE152418, GSE161731, GSE171110, and PMC8202013. To ensure consistent gene coverage across datasets, we first aligned all datasets to a common gene set by restricting our analysis to genes shared across all four datasets. For each tool (edgeR and DESeq2), we identified significant genes in each dataset using an FDR threshold of 0.05 and an absolute log₂ fold change . Unlike the previous section, where we applied the more conservative Bonferroni correction due to reliance on a single dataset, we used the FDR approach here because this analysis required reproducibility across multiple independent datasets. This cross-study design helps reduce the influence of dataset-specific findings and provides a more appropriate framework for evaluating transferable gene signatures.

We implemented a leave-one-dataset-out cross-study validation strategy using the four SARS-CoV-2 datasets. In each fold, three datasets were used as training datasets and the remaining dataset was used as the held-out test dataset. This procedure was repeated four times, so that each dataset served once as the held-out test dataset. The four folds were: (1) GSE152418 held out, with GSE161731, GSE171110, and PMC8202013 used for training; (2) GSE161731 held out, with GSE152418, GSE171110, and PMC8202013 used for training; (3) GSE171110 held out, with GSE152418, GSE161731, and PMC8202013 used for training; and (4) PMC8202013 held out, with GSE152418, GSE161731, and GSE171110 used for training.

Within each fold, significant genes were first identified separately for each tool in each of the three training datasets. Let , , and denote the significant genes identified by edgeR in the three training datasets, and let , , and denote the corresponding significant genes identified by DESeq2. We first defined the reproducible training gene set for each tool as the intersection of significant genes across the three training datasets:

This intersection represented the genes reproducibly identified by each tool across independent studies. After defining these reproducible training gene sets, we removed genes shared between edgeR and DESeq2, because the purpose of this analysis was to evaluate tool-specific signals rather than genes identified by both methods. Thus, the edgeR-specific and DESeq2-specific training signatures were defined as:

These tool-specific unique gene sets were then evaluated in the held-out dataset that was not used during gene selection. This design separates gene discovery from testing and evaluates whether genes uniquely and reproducibly identified by each tool in three independent studies can classify samples in an unseen fourth study.

To evaluate the predictive power of these tool-specific unique gene sets, we extracted raw expression values (log-transformed using , without normalization) for the unique genes from the test dataset. As before, we performed PCA and retained the first two principal components (PC1 and PC2), which capture the dominant variation within each gene set. These components were used to classify control versus SARS-CoV-2–positive samples via a logistic regression model. Classification performance was quantified using metrics including area under the ROC curve (AUC) [48], precision, and recall.

Note that this approach differs from the within-dataset classification described in the previous subsection. In that approach, genes were selected and evaluated on the same dataset, potentially capturing dataset-specific variance or artifacts. In contrast, the current design separates gene discovery (training) and evaluation (testing) across distinct datasets, which provides a more stringent test of biological generalizability and reproducibility. This mirrors standard machine learning principles by assessing how well tool-specific discoveries made during training generalize to unseen, external data.

Sensitivity to sample size and robustness to outliers

Fig 1 panels (a–h) summarize the DEG discovery performance of edgeR and DESeq2 as sample size increases from 5 to 45 samples per group using the RSVB dataset. Panels (a–d) show results for edgeR, and panels (e–h) show results for DESeq2. For the reduced sample-size settings (n = 5, n = 10, and n = 20 per group), the displayed bar plots show representative subsampled datasets, while the directional overlap analysis in panel (i) summarizes 20 independent random subsampling replicates. In the representative subsamples shown, DESeq2 identified more DEGs than edgeR at n = 5 (484 versus 148), n = 10 (985 versus 644), and n = 20 (1,640 versus 1,554). In the full dataset (n = 45 per group), edgeR identified 2,009 DEGs and DESeq2 identified 1,963 DEGs. Thus, the difference between tools was most pronounced at smaller sample sizes and decreased as sample size increased.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Sensitivity of edgeR and DESeq2 to sample size and outliers.

Panels (a–d) show DEG counts and top significant genes for edgeR across 5, 10, 20, and 45 samples per group. Panels (e–h) show the same for DESeq2. For reduced sample sizes (n = 5, n = 10, and n = 20 per group), panels (a–c) and (e–g) show representative subsampled datasets, while panel (i) summarizes directional overlap across 20 independent random subsampling replicates at each reduced sample size. Panel (i) shows mean ± standard deviation directional overlap between DEG sets identified by the two tools; the full dataset (n = 45 per group) was analyzed once. Panel (j) shows DEG set stability under sample swapping using Jaccard similarity, with 20 independent replicates for each swap level from 1 to 5.

https://doi.org/10.1371/journal.pone.0353788.g001

Panel (i) quantifies directional overlap (DO) between DEGs identified by the two tools at each sample size. For n = 5, n = 10, and n = 20 per group, DO values were computed separately for each of the 20 independent subsampling replicates and are reported as mean ± standard deviation. At n = 5, , indicating that edgeR recovered a smaller fraction of DESeq2’s DEG set at the smallest sample size, while , indicating that DESeq2 recovered all edgeR DEGs across replicates. At n = 10, the corresponding values were and , and at n = 20, they were and . In the full dataset (n = 45 per group), directional overlap was nearly symmetric, with and . These replicated subsampling results confirm that agreement between tools is lowest at small sample size and increases as sample size grows.

Panel (j) evaluates the tools’ robustness to sample contamination by computing the Jaccard similarity between DEG sets derived from the original full dataset and those obtained after introducing 1–5 randomly swapped sample pairs between the control and RSVB groups. Each swap level was repeated 20 independent times. Both tools showed a monotonic decline in Jaccard similarity as the number of swapped samples increased, indicating reduced DEG-set stability with increasing contamination. Across swap levels, edgeR and DESeq2 showed broadly similar behavior, with DESeq2 showing slightly higher median Jaccard similarity at several swap levels. These results indicate that both tools are sensitive to increasing sample-label perturbation, while the magnitude of the difference between tools under this perturbation is modest.

Classification performance of uniquely identified gene sets within the discovery dataset

Fig 2 summarizes the comparison of edgeR and DESeq2 across 13 biological contrasts spanning viral, bacterial, and fibrotic conditions. Each panel quantifies different aspects of agreement and divergence in DEG calls between the two tools. Panel (a) displays the number of genes uniquely identified as significantly upregulated or downregulated by each tool, log₂-transformed for scale. Across most datasets, DESeq2 identified a substantially larger number of unique upregulated genes, with pronounced differences observed in GSE115785 — EBOV DPI 7 and GSE115785 — EBOV NEC (e.g., 908 and 1,439 uniquely upregulated genes, respectively, compared to 107 and 37 for edgeR). The pattern was even more marked for downregulated genes: in GSE234118 — Mpox DPI 3 and GSE234118 — Mpox DPI 7, DESeq2 identified 56 and 253 uniquely downregulated genes respectively, while edgeR identified only 1 and 5. However, for EBOV contrasts (DPI 7 and NEC), edgeR identified far more uniquely downregulated genes (667 and 762) compared to DESeq2 (28 and 12), suggesting some context-specific reversal in sensitivity.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison of edgeR and DESeq2 across multiple biological contrasts.

Panel (a) shows the log₂-scaled number of uniquely identified upregulated and downregulated genes by each tool across 13 contrasts spanning viral, bacterial, and fibrotic conditions. Panel (b) displays the Jaccard index for upregulated and downregulated gene sets, indicating overlap between tools. Panel (c) shows Pearson and Spearman correlation coefficients computed for Bonferroni-adjusted p-values among common significant genes. Lines connecting points are included only as visual guides to aid comparison across contrasts and do not represent a continuous trend, interpolation, or fitted model. Dataset labels are shown as accession ID followed by biological contrast; infection timepoints are shown as DPI or NEC where applicable. The two influenza contrasts are labeled separately as PMC8202013 — Influenza and GSE161731 — Influenza.

https://doi.org/10.1371/journal.pone.0353788.g002

Panel (b) illustrates the Jaccard index between upregulated and downregulated gene sets from both tools across all datasets. The Jaccard index for upregulated genes was generally high (often exceeding 0.8), reflecting strong overlap between the two tools for the most transcriptionally active genes. However, downregulated gene sets showed much weaker agreement, with Jaccard indices ranging from 0.10 to 0.87. This discrepancy suggests that downregulated genes are less consistently detected across tools, likely due to lower signal strength or tool-specific modeling differences in shrinkage and dispersion estimation.

Panel (c) shows correlation coefficients (Pearson and Spearman) for Bonferroni-adjusted p-values among the common significant genes between tools. While Spearman correlations were consistently high (typically >0.75), indicating agreement in rank ordering, Pearson correlations were much lower, often below 0.4, and in one case (EBOV-DPI 3) dropped as low as 0.027. This indicates a non-linear relationship in adjusted significance levels despite overall agreement in gene ranking, and reinforces that tool-specific modeling may affect statistical inference even when fold-change estimates are aligned.

Fig 3 evaluates the biological relevance of tool-specific significant genes identified by edgeR and DESeq2 across 13 datasets via classification. Classification performance was assessed using precision, recall, and F1 score, shown respectively in panels (a), (b), and (c). Panel (a) shows that both tools achieved consistently high precision across datasets, often exceeding 0.9. edgeR outperformed DESeq2 in 7 of the 13 contrasts, achieving perfect or near-perfect precision (1.000 or >0.98) in GSE234118 — Mpox DPI 7, GSE234118 — Mpox DPI 10, GSE234118 — Mpox DPI 14, and all GSE115785 — EBOV contrasts. In comparison, DESeq2 had higher precision in GSE234118 — Mpox DPI 3, GSE231693 — IPF, and GSE161731 — Influenza. Both tools performed comparably well in PMC8202013 — SARS-CoV-2, GSE161731 — Bacterial, and the influenza contrasts.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Classification performance of uniquely identified genes from edgeR and DESeq2.

Each gene set was evaluated using a logistic regression classifier trained on PC1 and PC2 from log-transformed expression values. Panel (a) shows precision, (b) shows recall, and (c) shows F1 score for each dataset. Higher values indicate greater biological separability of control versus treated samples. Panel (d) shows the Dolan-More profile of both tools based on F1 scores, summarizing overall method robustness across all datasets.

https://doi.org/10.1371/journal.pone.0353788.g003

Panel (b) presents recall values, showing that edgeR exhibited greater consistency across contrasts, with perfect recall (1.000) in 7 contrasts, particularly in GSE115785 — EBOV and inflammatory disease comparisons. DESeq2 showed lower recall in GSE234118 — Mpox DPI 7 (0.722) and PMC8202013 — SARS-CoV-2 (0.903), though it matched edgeR in several other contrasts. Panel (c) summarizes classification performance via F1 score, capturing the trade-off between precision and recall. edgeR outperformed DESeq2 in 9 out of 13 contrasts, achieving high F1 scores in GSE234118 — Mpox DPI 7 (0.971), GSE115785 — EBOV DPI 3 (0.800), and GSE231693 — IPF (0.976). DESeq2, while yielding generally strong results, showed more variability with notably lower F1 scores in GSE234118 — Mpox DPI 7 (0.788) and EBOV DPI 3 (0.870). Overall, edgeR provided more robust gene sets for classifying biological conditions.

Panel (d) shows the Dolan-More performance profile for both tools, providing a global benchmark of method consistency across all datasets. This method quantifies, for each tool, the fraction of datasets where its F1 score is within a factor of the best-performing method. edgeR achieved the highest F1 score in 10 of 13 datasets (), with an average performance ratio of 1.02. In contrast, DESeq2 was optimal in 6 datasets (), with a slightly higher average ratio of 1.05. The Dolan-More curves illustrate that edgeR reaches faster, indicating greater consistency and reliability across a range of biological contrasts.

Pathway-level concordance of significant DEG sets

We next evaluated whether differences in significant DEG sets between edgeR and DESeq2 led to differences in downstream pathway enrichment results. For each dataset contrast, significant DEGs from each tool were analyzed using Hallmark and KEGG pathway collections. Enriched pathways were classified as edgeR-only, shared by both tools, or DESeq2-only, and pathway-level Jaccard similarity was used to quantify agreement between tools.

Fig 4 summarizes enriched pathway counts across all evaluated contrasts. Hallmark enrichment results showed substantial pathway-level agreement in several contrasts, including cases where most enriched pathways were shared between edgeR and DESeq2. However, selected contrasts showed lower Jaccard similarity and a larger number of tool-specific enriched pathways. KEGG enrichment showed a similar pattern: several contrasts produced highly concordant pathway-level results, while others retained clear edgeR-only or DESeq2-only pathway enrichment. These results indicate that pathway-level conclusions can remain concordant even when DEG counts differ, but downstream pathway interpretation can still be tool-dependent in some contrasts.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Pathway-level concordance between edgeR and DESeq2 using significant DEG sets.

Panel (a) shows Hallmark pathway enrichment results, and panel (b) shows KEGG pathway enrichment results. For each dataset contrast, enriched pathways were classified as edgeR-only, shared by both tools, or DESeq2-only using FDR < 0.05. The value above each bar indicates the Jaccard similarity between the enriched pathway sets from the two tools. Higher Jaccard values indicate stronger pathway-level concordance.

https://doi.org/10.1371/journal.pone.0353788.g004

To better visualize the contrasts with the largest pathway-level differences between tools, we selected representative comparisons with the strongest divergence in enriched pathway composition (Fig 5). This analysis highlights pathways that were uniquely enriched by edgeR, shared by both tools, or uniquely enriched by DESeq2. For both Hallmark and KEGG collections, the selected contrasts showed a mixture of shared and tool-specific pathway signals. Shared pathways often captured broad biological programs, whereas tool-specific pathways reflected differences in the significant genes prioritized by each method.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Selected pathway-level differences between edgeR and DESeq2.

Panel (a) shows selected Hallmark contrasts, and panel (b) shows selected KEGG contrasts with the largest pathway-level differences between the tools. Bubble color indicates whether an enriched pathway was identified only by edgeR, shared by both tools, or identified only by DESeq2. Bubble size represents enrichment strength, quantified as .

https://doi.org/10.1371/journal.pone.0353788.g005

Generalizability of tool-specific gene sets across independent studies

We next evaluated whether tool-specific significant genes discovered in a subset of SARS-CoV-2 datasets could generalize to an unseen independent dataset. This analysis used four independent bulk RNA-seq datasets of SARS-CoV-2 infection and followed a leave-one-dataset-out validation design. In each fold, three datasets were used to identify reproducible tool-specific training signatures, and the remaining dataset was held out for validation. The procedure was repeated four times, so that GSE152418, GSE161731, GSE171110, and PMC8202013 each served once as the held-out test dataset.

Fig 6 summarizes the training gene-selection procedure used in each fold. In panel (a), reproducible training genes were defined as genes that were significant in all three training datasets for a given tool using FDR < 0.05 and . When GSE152418 was held out, edgeR and DESeq2 identified 459 and 425 reproducible training genes, respectively. When GSE161731 was held out, the corresponding counts were 462 and 461; when GSE171110 was held out, they were 319 and 281; and when PMC8202013 was held out, they were 376 and 328.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Training gene-selection procedure for leave-one-dataset-out cross-study validation.

Panel (a) shows the number of reproducible training genes identified by edgeR and DESeq2 in each fold. A gene was considered reproducible for a given tool if it was significant in all three training datasets using FDR < 0.05 and . Panels (b–e) show the decomposition of these reproducible training genes into edgeR-unique genes, genes shared by both tools, and DESeq2-unique genes for each held-out dataset. Unique gene counts are shown outside the Venn diagrams next to the corresponding tool, while shared gene counts are shown in the overlap. These tool-specific unique gene sets were subsequently evaluated in the corresponding held-out dataset.

https://doi.org/10.1371/journal.pone.0353788.g006

Panels (b–e) show how these reproducible training genes were divided into genes unique to edgeR, genes shared by both tools, and genes unique to DESeq2. Across all four folds, most reproducible training genes were shared between the two tools. However, each fold also retained tool-specific signatures. For the GSE152418, GSE161731, GSE171110, and PMC8202013 held-out folds, the numbers of edgeR-unique genes were 65, 21, 52, and 65, respectively, while the corresponding numbers of DESeq2-unique genes were 31, 20, 14, and 17. The numbers of shared reproducible genes were 394, 441, 267, and 311 across these folds.

After defining the reproducible tool-specific signatures shown in Figs 6, 7 evaluates whether these signatures remained predictive in the corresponding held-out datasets. This analysis assessed whether genes uniquely and reproducibly identified by each tool captured biological signals that generalized beyond the datasets used for gene selection.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Cross-study generalizability of uniquely significant genes from edgeR and DESeq2.

(a) Mean ROC curves across four independent SARS-CoV-2 datasets, with shaded regions representing ±1 standard deviation of the true positive rate (TPR) at each false positive rate (FPR). Metrics in the legend summarize mean AUC, accuracy, precision, and recall with standard deviation. (b) PCA-based classification using DESeq2-specific genes from training datasets applied to test set GSE152418, yielding moderate separation (AUC = 0.783). (c) Corresponding classification using edgeR-specific genes for the same test set, yielding perfect separation (AUC = 1.000).

https://doi.org/10.1371/journal.pone.0353788.g007

Panel (a) presents the average ROC curves with shaded bands indicating ±1 standard deviation of the true positive rate (TPR) across folds. edgeR achieved a mean AUC of , accuracy of , precision of , and recall of . In contrast, DESeq2 yielded lower performance with a mean AUC of , accuracy of , precision of , and recall of .

Panels (b) and (c) show representative classification performance on the test dataset GSE152418. For DESeq2-specific genes (panel b), the model yielded AUC = 0.783, with accuracy, precision, and recall all equal to 0.75. In contrast, using edgeR-specific genes (panel c) resulted in perfect separation, with AUC, accuracy, precision, and recall all equal to 1.000.

Sensitivity to sample size and robustness to outliers

The semi-simulated experiments presented in Fig 1 highlight distinct performance profiles of edgeR and DESeq2 with respect to sample size and robustness to outliers. DESeq2 consistently identified more DEGs than edgeR across all tested sample sizes, with particularly notable differences at low n. This behavior may reflect DESeq2’s use of empirical Bayes shrinkage and regularization, which improves variance estimation under limited replication. While such conservatism helps stabilize inference, it also introduces a more inclusive DEG set in small-n settings, potentially increasing the risk of false positives.

As sample size increased, the DEG sets produced by both tools converged in both size and content. Directional overlap analysis demonstrated that edgeR recovered a growing proportion of DESeq2-identified genes as statistical power increased. By n = 45, both tools achieved over 95% mutual overlap, suggesting that differences in underlying models and statistical testing become less consequential when replication is sufficient. These findings suggest that, for well-powered studies, both tools offer highly comparable DEG discovery performance.

Robustness analysis under sample swapping (Fig 1j) revealed that both tools degrade predictably in the presence of controlled contamination, with Jaccard similarity decreasing as the number of outlier samples increased. However, DESeq2 maintained slightly higher mean similarity scores and lower variability across replicates at all levels of perturbation. This suggests that DESeq2 offers marginally greater robustness to moderate outlier effects.

Together, these findings indicate that DESeq2 is more sensitive to detecting significant genes at small sample sizes, though whether this increased sensitivity reflects true biological signal or introduces additional noise is a question discussed in the next two sections. As sample size increases, both tools become more consistent, with convergence in both DEG count and content.

Classification performance of uniquely identified gene sets within the discovery dataset

The classification-based analysis presented in Figs 2 and 3 offers a performance-driven assessment of the biological validity of genes uniquely identified by edgeR and DESeq2. By training logistic regression models using PC1 and PC2 derived from the expression of uniquely significant genes, we evaluated each tool’s ability to recover gene sets that effectively discriminate between control and treated samples. This framework serves as a proxy for evaluating the “trueness” of significant genes in the absence of an external ground truth.

While both tools demonstrated high precision across most datasets (Fig 3a), edgeR consistently achieved equal or superior precision in more than half of the contrasts. This indicates that, despite identifying fewer unique DEGs than DESeq2, the genes it does report tend to be more predictive and less noisy. The recall results in Fig 3b further reinforce this observation, with edgeR exhibiting stronger sensitivity in multiple datasets, especially in EBOV and inflammatory disease contrasts.

The F1 score results shown in Fig 3c provide an integrated view of classification performance, balancing both precision and recall. edgeR outperformed DESeq2 in 9 out of 13 contrasts, achieving notably high scores in GSE234118 — Mpox DPI 7 (0.971) and EBOV DPI 3 (0.800), and GSE231693 — IPF (0.976). In contrast, DESeq2 showed larger fluctuations in F1 performance, with reduced scores particularly in datasets where it identified many unique genes that ultimately contributed less to sample separability.

These trends suggest that some of the uniquely identified genes from DESeq2, especially in low-sample or highly variable settings, may include false positives. This finding aligns with earlier observations (Fig 1) that DESeq2 is more inclusive in small-n contexts, potentially increasing sensitivity at the expense of specificity.

The Dolan-More profile in Fig 3d further highlights the comparative consistency of each method across diverse datasets. edgeR attained the highest F1 score in a majority of contrasts and exhibited more stable performance overall, reinforcing its reliability in identifying gene sets with meaningful classification potential.

In summary, while DESeq2 tends to identify more significant genes, edgeR provides gene sets that are, on average, more predictive of biological condition when evaluated via classification. This suggests that increased sensitivity in DEG calling, especially in smaller datasets, may not always correspond to biological relevance, and highlights the importance of downstream validation when interpreting differential expression results.

Pathway enrichment concordance between tools

The pathway enrichment concordance analysis provides an additional view of how tool choice affects downstream interpretation (Figs 5 and 4). Although the DEG-level analyses showed that edgeR and DESeq2 can differ in the number and identity of significant genes, the pathway-level results showed that many biological interpretations remained concordant between tools. This suggests that, in several contrasts, the dominant pathway-level signals are robust to differences in DEG calling.

However, pathway-level agreement was not uniform across all contrasts. Some comparisons showed clear tool-specific enriched pathways, indicating that differences in significant DEG sets can affect downstream pathway interpretation. This is important because pathway analysis is frequently used to summarize DEG results into biological themes. Therefore, even when two tools identify broadly similar biological signals, tool-specific DEGs can still alter which pathways are emphasized.

These findings refine the interpretation of the DEG-level results. Tool choice can influence downstream enrichment results, but the extent of this influence depends on the dataset, the biological contrast, and the pathway collection used. Thus, pathway-level concordance should be evaluated alongside DEG counts, classification performance, and cross-study validation when comparing differential expression tools.

Generalizability of tool-specific gene sets across independent studies

The cross-study validation results presented in Figs 6 and 7 offer compelling evidence that the biological signal captured by tool-specific gene sets differs in generalizability and predictive power. Although both edgeR and DESeq2 are widely accepted for differential gene expression analysis, the present results highlight clear differences in the robustness of the gene sets they identify when transferred to independent datasets.

One of the most important findings is that edgeR-specific genes yielded significantly higher classification performance than those uniquely identified by DESeq2. This was consistent across multiple folds of leave-one-out cross-study validation and was further exemplified by the perfect classification obtained in the representative GSE152418 test case. These results indicate that edgeR not only identifies fewer unique genes but that these genes are more likely to represent consistent, transferable biological signals rather than study-specific artifacts.

In contrast, the performance drop observed with DESeq2-specific genes suggests that some of its identified features may be more reflective of noise or dataset-specific variance. This aligns with previous observations from our semi-simulation and classification experiments, where DESeq2 exhibited increased sensitivity but also greater variability in performance. It is plausible that DESeq2’s regularized dispersion estimation and more inclusive thresholding increase the likelihood of capturing subtle but less reproducible patterns, particularly in smaller or noisier datasets.

The contrast between the two tools in this context reveals a fundamental trade-off. DESeq2 appears more permissive and sensitive, which may be advantageous for exploratory analyses or hypothesis generation but potentially at the cost of precision and reproducibility. On the other hand, edgeR employs stricter criteria that may reduce the total number of reported DEGs but improve their specificity and cross-dataset stability. In the context of biomarker discovery or translational applications where reproducibility is paramount, the conservative profile of edgeR may be preferable.

Furthermore, these findings reinforce the importance of integrating downstream classification or validation frameworks into DEG analysis pipelines. The number of DEGs alone is not a sufficient metric for evaluating tool performance; rather, the ability of these genes to generalize across biological contexts and datasets is a more meaningful benchmark. The consistent superiority of edgeR in our cross-study framework underscores its capacity to identify gene sets that are not only statistically significant but biologically informative and generalizable.

In sum, while both tools are capable of capturing relevant gene expression changes, edgeR provides more reliable and transferable gene sets, particularly in applications demanding high reproducibility. These results advocate for tool selection to be guided not only by statistical properties but also by the intended downstream use of the identified genes.

Integration with prior benchmarks and implications for tool choice

Taken together, our results extend previous benchmarking studies by showing how statistical differences between edgeR and DESeq2 translate into downstream biological utility. Prior work has already shown that RNA-seq DGE tools can differ in DEG yield, type I error behavior, robustness to perturbation, and sensitivity to sample size [20–22,24–26]. The main contribution of the present study is that we evaluated what happens after tool-specific genes are identified: whether those genes separate biological groups, whether they lead to concordant pathway-level interpretations, and whether they generalize across independent datasets. This downstream evaluation is important because a DEG list can appear statistically significant yet still be less useful for classification, pathway interpretation, or transfer to an external cohort.

The sample-size analysis provides one point of agreement with previous work and one important extension. Consistent with earlier studies, we found that tool agreement was lowest at smaller sample sizes and increased as replication improved. However, the direction of the difference in our data was informative: DESeq2 generally identified more DEGs than edgeR at smaller sample sizes, whereas the two tools became highly concordant in the full RSVB dataset. This suggests that, when sample size is adequate and the biological signal is strong, the practical difference between the two tools may become smaller. In contrast, when sample size is limited, tool-specific modeling choices can have a larger effect on the resulting DEG list. This finding is consistent with the broader literature, but our analysis further shows that early differences in DEG yield can carry into downstream classification and external validation.

The classification results provide a functional perspective on the false-positive concern raised in previous studies. We did not directly estimate gene-level FDR using permutation or known ground truth, so our results should not be interpreted as proving that a specific set of DESeq2-specific genes are false positives. Instead, our analysis asks a complementary question: when each tool identifies genes that the other tool does not, which set better captures biological separation? Across 13 contrasts, edgeR-specific genes more often produced higher F1 scores and more consistent classification performance. This pattern suggests that the larger DESeq2-specific gene sets observed in some contrasts may include genes with weaker biological separability, whereas the more restricted edgeR-specific sets more often captured signal that distinguished the relevant biological groups. This result is consistent with prior reports that larger DEG lists may include less stable or less reliable signals, but it adds an explicit classification-based evaluation that most earlier studies did not include.

The pathway enrichment analysis adds a second layer of interpretation. Previous studies have shown that tool-specific DEG lists can produce different functional conclusions, and Li et al. showed that spurious DEG calls can even generate biologically plausible pathway enrichment [24]. In the present study, many contrasts showed strong pathway-level agreement between edgeR and DESeq2, indicating that major biological programs can remain stable even when DEG lists are not identical. At the same time, selected contrasts showed tool-specific enriched pathways, which means that downstream interpretation can still depend on the DGE tool. This distinction is important: tool choice does not always change the biological conclusion, but it can change which pathways are emphasized, especially when tool-specific genes drive enrichment results.

The cross-study SARS-CoV-2 validation provides the strongest practical distinction between the two tools in this study. Genes uniquely and reproducibly identified by edgeR across training datasets produced higher AUC, precision, and recall in held-out datasets than genes uniquely and reproducibly identified by DESeq2. This result is not simply another measure of DEG overlap or statistical significance; it directly evaluates transferability across independent studies. In this sense, our findings extend previous benchmarking work by showing that tool-specific gene sets can differ in external predictive value. For applications such as biomarker discovery, signature development, or translational studies, this type of external performance may be more informative than DEG count alone.

Overall, the combined evidence suggests a practical interpretation of edgeR and DESeq2. DESeq2 may be useful for broad exploratory discovery because it often identifies a larger candidate set, especially in smaller-sample settings. However, larger DEG yield should be followed by additional checks, including pathway concordance, sensitivity to outliers, and validation in independent data. edgeR, in our analyses, produced tool-specific gene sets that were more often predictive and transferable, which makes it attractive when reproducibility and downstream validation are priorities. This does not mean that edgeR is universally superior or that DESeq2 is unreliable. Prior studies show that both tools can be affected by dataset structure, outliers, filtering, and model assumptions. Rather, the practical message is that DESeq2 may be better suited for sensitivity-oriented candidate generation, while edgeR may be preferable when the primary goal is a smaller, more reproducible gene set with stronger downstream performance.

A useful strategy is therefore to treat shared genes and shared pathways from both tools as a robust core signal, while interpreting tool-specific genes according to the goal of the study. If the goal is hypothesis generation, tool-specific genes from either method may be worth exploring. If the goal is reproducible classification or biomarker development, tool-specific genes should be evaluated using downstream separability and external validation, as done here. This study therefore supports a broader view of DGE benchmarking: the best tool is not necessarily the one that identifies the most genes, but the one whose genes remain stable, interpretable, and useful beyond the discovery dataset.