Rejection-ABC.

Rejection-Approximate Bayesian Computation [23] is a simple SBI method. It includes the following steps: i) Sample parameter proposals from the prior distribution, θ_i∼p(θ). ii) Simulate synthetic data, x_i ~ f(θ_i), where f is a simulator that generates data from the intractable likelihood, p(x|θ). iii) Compute summary statistics for the synthetic data, s(x_i), and empirical data, s(x_o). iv) Accept proposal θ_i if d(s(x_o),s(x_i)) ≤ ε, where d(⋅,⋅) is a dissimilarity function and ε is a dissimilarity threshold. v) Repeat until enough proposals are accepted. With sufficient summary statistics and ε → 0, the distribution of accepted parameters converges to p(θ|x_o). This approach can be easily parallelized but can suffer from low acceptance rates (and therefore low computational efficiency) when the prior and posterior distributions have a small overlap, when the number of parameters is large, and when ε is small.

Neural Posterior Estimation (NPE).

A prominent example of SBI with neural density estimation is Neural Posterior Estimation [25,26]. NPE directly approximates the posterior density of model parameters conditioned on an observation, p(θ|x) (S1A Fig). The neural conditional density estimator q_φ(θ|x) (parameterized by φ) is trained to approximate p(θ|x): a training set of pairs (θ_i, x_i) is generated by sampling parameters from the prior, θ_i ~ p(θ), and simulating observations, x_i ~ f(θ_i); then gradient-based optimization is used to find φ* that maximizes Σ_ilog(q_φ(θ_i|x_i)). This yields an amortized posterior estimator, q_φ*(θ|x), which, when conditioned on a specific observation x_o, provides q_φ*(θ|x_o)≈p(θ|x_o). Importantly, once trained, evaluating the posterior estimator q_φ* on a new observation x_o does not require additional training or simulation, in contrast to sampling methods such as rejection-ABC, which require additional simulations to infer the posterior distribution for every new observation. The neural density estimator can be implemented by a normalizing flow, which provides a general way of constructing flexible probability distributions over continuous random variables [44]. Our previous studies [29,45] demonstrated that NPE infers more accurate and confident posterior distributions from evolutionary experimental data than popular sampling-based Bayesian inference methods (rejection-ABC and SMC-ABC [46]).

Neural Posterior Score Estimation (NPSE).

Recent studies on SBI from independently and identically distributed (i.i.d.) observations have introduced score-based models for improved accuracy of SBI [39–41]. While their results are promising, we did not observe any advantage in using NPSE over NPE for our empirical data (S6 Fig). We therefore proceeded with NPE as the neural SBI method for individual posterior estimation. For tasks where NPSE is better at individual posterior estimation, our method can be applied using NPSE as the individual posterior estimator, or by leveraging NPSE’s i.i.d. formulation [39,40].

Standard collective posterior distribution.

Assuming that multiple replicates {x_i}ⁿ_i=1 are independent conditioned on the model parameters, θ, that is, p(x_i,x_j|θ) = p(x_i|θ)p(x_j|θ), we can compute the collective posterior distribution of model parameters conditioned on all replicates, p(θ|x₁,…,x_n), from individual posterior distributions, p(θ|x_i), and the shared prior distribution, p(θ), by applying Bayes’ theorem twice (a full derivation is available in S1 Text),

(1)

Robust posterior distribution.

Posterior distribution approximation in SBI, while accurate in the high posterior density region (HDR), tends to be uncalibrated in low-density regions, sometimes producing erratic posterior density values (S3 Table). To mitigate this, we introduce a robust individual posterior distribution,

(2)

We combine these robust individual posterior distributions into a robust collective posterior distribution,

(3)

For ε → 0, we have p_ε(θ|x_i)→p(θ|x_i) and p_ε(θ| x₁,…,x_n)→p(θ|x₁,…,x_n). High values of ε will focus only on high-density regions of the individual posterior distributions, overfitting to a given replicate set. An optimal choice of ε balances robustness to outliers with accurate parameter inference (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Collective posterior inference.

(A) First, we are given a set of independent empirical replicates (e.g., time series; blue), possibly including outliers. (B) A density estimator infers replicate-specific posterior distributions (blue). (C) The naïve solution would combine the individual posterior distributions to form a collective posterior conditioned on the full replicate set (grey). (D) Instead, we use the ε-flooring to form a robust collective posterior distribution (red). (E) Finally, posterior predictive checks assess whether collective-posterior predictions (red) reproduce the empirical distribution of time series across replicates (blue), including between-replicate variability.

https://doi.org/10.1371/journal.pcbi.1014534.g001

Sampling from the robust collective posterior distribution.

To sample from the collective posterior (Eq. 3), we implemented and compared three distinct sampling strategies. Based on the trade-off between computational efficiency and approximation accuracy, Sampling Importance Resampling (SIR) was selected as the primary method for the results presented in this study. The other strategies are detailed in the supporting information (S1 Text).

Sampling Importance Resampling (SIR).

To efficiently approximate the collective posterior without relying on sequential chains (i.e., MCMC), we used an SIR scheme [47]. We used the prior distribution p(θ) as the proposal distribution to ensure robust coverage of the parameter space while not relying on the sampling efficiency of the individual posterior distribution. Importance weights were calculated as the ratio of the unnormalized robust collective posterior density to the prior density. To counteract the artificial variance reduction inherent in the product-of-experts aggregation, where multiplying independent densities shrinks the variance, we used an adaptive temperature scaling to the importance weights. This temperature was determined via binary search to maintain a target effective sample size (ESS), preventing weight collapse as the number of replicates increased. Then, at resampling, we applied Gaussian jittering to the results. This vectorized approach enabled rapid, high-throughput posterior approximation. While practical and accurate, our SIR scheme is designed for high-performance approximation rather than the asymptotic sampling guarantees of MCMC.

Synthetic evolutionary data.

Consider a Wright–Fisher model of a haploid population of fixed size N = 10⁴ with three bi-allelic loci. The eight genotypes are represented by a bit-string (b₁,b₂,b₃)∈{0,1}³, with the population initialized as all wild type, (0,0,0). Genotype fitness is multiplicative: the presence of a mutant allele 1 contributes a fitness factor 1 + s_i, so the fitness of genotype g = (b₁,b₂,b₃) is ∏³_i=1[(1 + s_i)^bi].

Mutation is independent across loci. By default, we allow only forward mutation 0 → 1 with per-locus rates μ₁, μ₂, μ₃, so that mutation is applied by an 8 × 8 transition matrix M, where M[g,h] is the probability of mutation from genotype g to genotype h; this probability factorizes across loci by the mutation rates. Genetic drift arises from multinomial sampling given the expected genotype frequencies after mutation and selection.

We simulate the model for 1,000 generations, and we record allele (rather than genotype) frequencies every 100 generations. The simulator returns the time points and three allele-frequency time series as a single 30-dimensional vector, which is used as a single replicate.

For testing, we generate a dataset of 200 observations, each consisting of 10 replicates (Fig 2). The intrinsic stochasticity of this model produces substantial variance among replicates. However, to better resemble real datasets, we further contaminate the data: of the 10 replicates, seven replicates correspond to simulations using parameters perturbed by additive Gaussian noise sampled from N(0, 0.1), mimicking biological variability through a hierarchical process. The remaining three are simulations of the same parameters, perturbed by a large additive Gaussian noise sampled from N(0,1), which introduces outlier observations to the dataset. Finally, to simulate measurement error, we perturb each observation with heteroscedastic Gaussian noise, where the standard deviation is set to 10% of the observed value (σ = 0.1 ⋅ x_o). Overall, this design aims to mimic empirical datasets, which are inherently noisy due to both stochastic and measurement variability. We found that introducing these perturbations produces datasets that closely resemble experimental observations in their variability and structure (S4 Fig).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. The collective posterior outperforms NPE + PIE on noisy evolutionary simulations.

(A-C) Synthetic replicate sets. Each test instance comprises n = 10 Wright-Fisher replicate time series simulated with shared parameters, θ (A, Clean). To introduce between-replicate heterogeneity, we generate 7 replicates using parameters perturbed by moderate Gaussian noise and 3 replicates using parameters perturbed by larger Gaussian noise (B, Hierarchical). Finally, we add measurement noise to the time series (C, Noisy). (D) Coverage. For each nominal confidence level α, we compute the empirical coverage Pr[θ_i ∈ HDR_α] of per-parameter highest-density regions (ideal: diagonal). The collective posterior achieved consistently better coverage (closer to the diagonal) than NPE + PIE, indicating less overconfidence. (E-J) Accuracy. Distributions of per-parameter estimation errors (defined as the log-ratio of the posterior mean and true value). Across parameters, the collective posterior is more tightly centered around zero, indicating more accurate inference.

https://doi.org/10.1371/journal.pcbi.1014534.g002

Empirical evolutionary datasets.

We consider our recent study [42] in which we inferred the parameters of a Wright-Fisher model from experimental evolutionary data. The data are time series of the proportions of cells with more than one copy of the GAP1 gene in haploid yeast populations (4 different genotypes, 27 time series in total) in a glutamine-limited environment over 116 generations. By inserting a green fluorescent protein (GFP) next to GAP1, the CNV proportion was monitored throughout the experiment using flow cytometry [48]. The inference task involves estimating three parameters: the selection coefficient s of GAP1 duplications, its formation rate δ, and its initial undetectable frequency φ, which is non-zero due to cells with non-GFP GAP1 duplications in the founder population. The evolutionary model includes four genotypes: WT, with an initial proportion X⁰_WT = 1-φ and fitness of w_WT = 1; C^-, representing cells with GAP1 duplications that formed before the fluorescent reporter insertion (thus undetectable throughout the experiment), with an initial proportion X⁰_C- = φ and fitness of w_C-  = 1 + s; C⁺, representing cells with GAP1 duplications that formed after reporter insertion, with an initial proportion X⁰_C+ = 0 and fitness w_C+ = 1 + s; and B, representing other beneficial genotypes, with initial proportion X⁰_B = 0 and fitness w_B = 1.001. At generation , mutations occur with rates δ from WT to C⁺ and μ = 10^-5 from WT to B, so that the change in the genotype frequencies is described by

Then, genotype frequencies X_i’ change following their associated fitness w_i,

Finally, frequencies change due to random genetic drift,

The effective population size in the chemostats was estimated to be N_e = 3.3 × 10⁸ [29]. Here, we use a lower value of N_e = 10⁷ for training and testing the neural density estimator to increase the variance between observations, posing a more difficult inference challenge. Our main analyses focus on the empirical evolutionary data, and we also benchmarked performance on synthetic datasets by simulating 500 observations, each consisting of 10 replicates (see SI for results).

Swamp sparrow song dataset.

Lachlan et al., 2018 [5] analyzed song patterns from six geographically distinct swamp sparrows (Melospiza georgiana) populations to infer the parameters of a cultural evolutionary model of syllables transmission: μ, mutation rate per learning event; ν, variance of demonstrator/tutor bias; p_att, proportion of syllable types that are attractive to learn; α, conformity bias; and N_t, number of potential demonstrators. Posterior inference was performed using Population Monte Carlo ABC (PMC-ABC). The authors computed 13 summary statistics per population, applied partial least squares (PLS) regression for dimension reduction, and then ran PMC-ABC on the resulting PLS features with decreasing acceptance threshold.

Using the published simulations output, we constructed population-specific posteriors by selecting, for each population, the 100 simulations most similar to the empirical PLS features (Euclidean distance) and fitting a kernel density estimator (KDE) to corresponding accepted parameter values. We then combined the six KDE-based posteriors into a collective posterior distribution using our method. For comparison, we also reconstructed the single posterior corresponding to conditioning on the mean empirical PLS feature vector across populations, as in Lachlan et al. [5]. Conditioning on the mean feature vector is a common ABC strategy for multi-replicate data, but it removes information about between-replicate variation.

SBI benchmarks.

We also test our method on two common simulation-based inference benchmarks [27]: Gaussian Linear Uniform (GLU), where the prior θ ~ U[-1,1]¹⁰ and data points are drawn from a multivariate normal distribution such that y ~ MVN(θ, Σ = 0.1⊙I); and Simple Likelihood Complex Posterior (SLCP), which presents a more challenging simulation-based inference task. The prior is θ ~ U[-3,3]⁵, and the observations are sets of four two-dimensional points sampled from a normal distribution whose mean and variance are nonlinear functions of θ. The resulting posterior has four symmetrical modalities and vertical cut-offs, making parameter inference particularly challenging.

In previous studies, experimental limitations in biological settings restricted the number of empirical replicates to between three and eight [5,29,42,43,45]. However, serial dilution plates are typically designed to facilitate experiments with up to 12–96 replicates. Therefore, we chose n = 10 replicates per observation set as a balanced compromise.

Evaluation metrics with synthetic data.

We assess the estimation accuracy of the inferred posterior by computing the log-ratio of the posterior mean and the true parameter value. To evaluate estimation uncertainty, we use coverage [49], defined as the probability that the true parameter is within an HDR of level α, P(θ_true∈HDR_α), for α = 0.1, 0.2, …, 0.9, 0.95. A perfectly calibrated posterior would obtain P(θ_true∈HDR_α)=α. While recent studies focus on the local classifier 2-sample test [50], we believe that the marginal metrics of accuracy and coverage better reflect the computational biologist’s needs from SBI. First, biological research is typically parameter-centric. Marginal metrics allow us to disentangle performance across distinct biological processes, verifying that key parameters are estimated reliably even if high-order correlations are imperfect. Global metrics like LC2ST compress this complexity into a single score, obscuring the specific sources of error. Second, calculating LC2ST requires training a separate classifier for every test observation, which becomes computationally prohibitive when validating performance across hundreds or thousands of synthetic multi-dimensional observations. In contrast, marginal accuracy and coverage can be efficiently computed, enabling us to perform high-throughput validation.

Evaluation metrics with empirical data.

To assess whether each method produced calibrated predictions for the empirical trajectories, we computed its posterior predictive coverage [51]. For each dataset and inference method, we drew parameter samples from the inferred posterior and simulated frequency trajectories with these parameter samples. At each nominal coverage level α∈{0.1, 0.2,…, 0.9, 0.95}, we formed central posterior predictive intervals by taking the (1-α)/2 and (1 + α)/2 quantiles of the simulated trajectories at each generation. We then computed the fraction of empirical observations that fell within these intervals. A calibrated posterior predictive distribution should have empirical coverage close to the nominal level. We summarized calibration error as the mean absolute deviation between empirical and nominal coverage across all nominal levels. We plotted 1-MAE as a coverage score, where larger values indicate better predictive calibration. To quantify whether posterior estimates were stable, we used an inference cycle [52,53]: first, we inferred the posterior distribution conditioned on the empirical data (5–8 replicates). Then, we used 10 samples from the posterior distribution to simulate a synthetic dataset of 10 replicates. We inferred a posterior distribution conditioned on the synthetic dataset and compared it to the original posterior (conditioned on the empirical data). For each parameter, we computed the overlapping coefficient (Weitzman’s Δ, or OVL) [54] between the marginal posterior distributions, OVL(p, q) = ∫min[p(θ), q(θ)] dθ, where p is the original posterior marginal and q is the inference-cycle posterior marginal. The OVL ranges from 0 to 1, with 0 indicating no overlap and 1 indicating identical marginal distributions. In practice, we estimated the OVL from posterior samples by binning both sample sets on the same parameter range, normalizing the bin counts to probability masses, and summing the minimum probability mass in each bin. We report the mean OVL across parameters as a metric for inference-cycle stability, where larger values indicate greater agreement between the original and cycle posteriors.

Hyperparameter tuning.

Our implementation of the robust collective posterior distribution with SIR sampling includes a single hyperparameter: ε, the lower bound density of the robust individual posterior distribution, p_ε(θ|x_i). Based on our experiments, as long as the simulator and inference method do not change, the same value of ε remains useful, i.e., leads to both accurate inference and robustness to outliers. To avoid arbitrary selection, we derived a global and adaptive data-driven heuristics based on the dynamic range of the neural density estimator. First, we defined the baseline uninformative posterior density by drawing independent pairs of parameters and observations (θ,x) from the prior, p(θ), and the simulator, p(x|θ), respectively, and computing their posterior densities, p(θ|x), with the neural density estimator. A high (e.g., 95th) percentile of these densities represents the highest density the estimator assigns to a random pair (θ,x). Setting ε to the chosen percentile imposes a lower bound on the posterior density, ensuring that outlier observations are assigned a minimal weight in the collective posterior. While this heuristic shifts the hyperparameter selection from a raw density value to a percentile, the latter is more interpretable and stable across different tasks, as it scales naturally with the estimator’s dynamic posterior density range. This prevents outliers from disproportionately collapsing the collective posterior. We applied this approach to the synthetic benchmarks presented in the Results section. Second, in empirical applications, measurement noise and model misspecification (the simulation-to-reality gap) often cause the estimator to assign systematically lower posterior densities to empirical data compared to synthetic training data. To account for this shift, we propose an alternative estimation method. Here, ε is calculated specifically for each replicate set {x_i}_i by evaluating the posterior densities of a batch of parameters {θ_j}_j sampled from the prior, p(θ), conditioned on the replicates, {p(θ_j|x_i)}_i,j. ε is then set to a chosen percentile of these posterior densities. Crucially, this estimation of ε remains independent of the number of replicates N, because it is based on the assigned posterior densities given to individual observations. This calibrates the minimum posterior density to the specific experimental conditions, ensuring robustness even when the overall posterior densities are shifted due to model misspecification.