Fig 1.
Properties of the null distribution of CBEA under the global null simulations.
Panel (B) presents kurtosis and skewness of CBEA scores while panel (A) presents the goodness of fit (as Kolmogorov-Smirnov D statistic) for mixture normal and normal distributions. Panel (C) is a density plot of the shape of the null distribution. Results indicated the necessity of estimating an empirical null and demonstrating that the mixture distribution was the better fit compared to the basic normal.
Fig 2.
Random taxa set analyses for inference at the sample level of CBEA under different parametric assumptions compared against a Wilcoxon rank-sum test.
Type I error (y-axis) was evaluated by generating random sets of different sizes (x-axis) (500 replications per size) and computing the fraction of samples in which the set was found to be significantly enriched at α = 0.05. Error bars represent the mean type I error ± sample standard error computed across 500 replications of the experiment. Only the unadjusted CBEA with the mixture normal distribution and the Wilcoxon rank sum test were able to control for type I error at 0.05. All approaches are invariant to set sizes.
Fig 3.
Random sample label (A) and random set (B) analyses for population level inference. (A) Type I error (x-axis) was estimated as the overall fraction of sets found to be enriched α = 0.05 using randomly generated sample labels (500 permutations). Error bars represent the mean type I error ± sample standard error. (B) Proportion of significant sets (y-axis) using 100 randomly generated sets of different set sizes (x-axis). Confidence intervals computed using Agresti-Couli method for binomial proportions. For sample label permutation (A), all CBEA approaches were able to control for type I error but not for corncob and DESeq2. For random set analyses (B), all approaches demonstrated similar rates of accepting significant sets and were invariant to overall set size.
Fig 4.
Statistical power (A) and score rankings (B) to assess phenotype relevance. (A) Power (x-axis) was estimated as the overall fraction of aerobic microbes found to be enriched in supragingival samples at α = 0.05. 95% confidence intervals were computed using the Agresti-Couli approach for binomial proportions. (B) Score rankings were evaluated by comparing computed scores against true values using AUROC (x-axis). DeLong 95% confidence intervals for AUROC were computed.
Fig 5.
Statistical power to assess phenotype relevance of inference tasks at the population level.
Power (x-axis) was estimated as the overall fraction of sets representing genera that are aerobic or anaerobic microbes found to be differentially enriched across sample type (supragingival or subgingival). 95% confidence intervals were computed using the Agresti-Couli approach for binomial proportions.
Fig 6.
Predictive performance of a naive random forest model trained on CBEA, ssGSEA, and GSVA generated scores, as well as the standard CLR approach on predicting patients with inflammatory bowel disease versus controls using genus level taxonomic profiles.
The data sets used span both 16S rRNA gene sequencing (Gevers et al. [57]) and whole-genome shotgun sequencing (Nielsen et al. [56]). CBEA performs better than GSVA and ssGSEA but not as well as CLR, with the exception of the whole genome sequencing data set.