2.1 A Mixture of Transition Kernels

The factorization in Eq (1) reveals that by first sampling a parameter value θ* ∼ π(θ) and then data x* ∼ π(x ∣ θ*), with probability (density) π(x*) we have generated θ* from the proposal π(θ ∣ x*). With the acceptance probabilities ϕ defined as in Eq (2), we have that each generated proposal π(θ ∣ x*) corresponds to a unique Metropolis transition kernel, i.e., a transition probability distribution with density: for which the posterior π(θ ∣ x) is the invariant distribution: Since each Metropolis kernel in the SVE algorithm has the same invariant distribution, so does their mixture [4, 17]: where the integration is over all possible generated datasets Ω_x*. This formulation shows that the inefficiency of the SVE algorithm is caused by the generation of kernels that have a low probability of making a move, i.e., kernels corresponding to statistics t(x*) that are far from the observed statistic t(x) (c.f. Eq (3)).

To illustrate, consider a simple example from educational measurement; the Rasch model [27]. The Rasch model describes the distribution of a pupil’s item responses as a function of an ability parameter θ, and a difficulty parameter δ for each of k items: where X = 1 denotes a correct response and X = 0 an incorrect response. The test score x₊ = ∑_i x_i is sufficient for θ such that its posterior depends on the data only through the test score [28], and the mixture ranges over the k + 1 possible test scores : The mixing distribution (i.e., test score distribution) π(x₊) is shown in Fig 1 for a test consisting of k = 20 items, and confirms that it gives much weight to kernels corresponding to values of that are far from the observed value t(x) = x₊. For instance, for a pupil with test score t(x) = x₊ = 9, say, the probability to generate a kernel corresponding to values , with , is approximately 35% in this example (see S1 Script).

3.1 Oversampling for Single Parameter Updates

We wish to assign more weight to kernels with a high probability of accepting a move, while preserving the correct invariant distribution π(θ ∣ x). A simple way to achieve this is to generate m ≥ 1 proposed points and then select the one that yielded a sufficient statistic t(x*) most similar to t(x) (c.f. Eq (3)), where m = 1 results in the original SVE algorithm. Just as in the original SVE algorithm we have that the posterior distribution π(θ ∣ x*) is the invariant distribution. Fig 2 illustrates the improvement of this procedure in our Rasch model example for a test score x₊ = 9; improving the probability of directly generating from the target (where ) from 0.1 to about 0.4 with m = 5 samples and about 0.8 with m = 20 samples. Even when no direct sample was produced, the proposal distributions became increasingly more similar to the target distribution, thus increasing the overall probability of making a move.

In the application above, we have used functions t() of the observed x and simulated data x* to select a proposal distribution (i.e., a transition kernel). In practice, however, we might have more information available that informs the selection of good proposal distributions, and one can use functions f() that incorporate this information. In the next section we illustrate such a function that incorporates, for instance, covariates that are used in the statistical model. Observe that we do not use the current state of the parameter, θ′, or the proposed point, θ*, to select a proposal distribution. This ensures that the posterior distribution π(θ ∣ x) remains the correct invariant distribution of the Markov chain.

Fig 2 reveals that using the sufficient statistic we are able to select statistically more efficient proposals as m increases. This follows from inspecting the acceptance probability in Eq (3), and observing that the statistically more efficient proposals are those for which |t(x*) − t(x)| is at a minimum, and furthermore that the minimum min_m {|t(x*) − t(x)|} over m proposals is non-increasing with m, i.e., It is important to note that the m proposals can be generated in parallel so that the oversampling of proposals need not increase the computational burden. However, only one of the m proposals is subsequently accepted by the Markov chain. As we will see next, all generated proposals can be put to good use when simultaneously sampling from more than one target distribution.

3.2 Matching for Multiple Parameter Updates

With the commonly assumed conditional independence of observations in hierarchical models, we have independent posterior distributions for each of n random effects (or latent variables) [28]: Since proposals are also independently generated, it is convenient to consider the joint application of n independent SVE kernels: where the original SVE algorithm has due to independence. We wish to modify π(x*) such that each component assigns more weight to kernels with a high probability of accepting a move.

Similar to our oversampling procedure we can generate m ≥ 1 proposals and assign each of the generated proposals to a target distribution. Here, we choose the number of generated proposals to be equal to the number of target distributions, which implies that we simply rearrange the m = n generated proposals. We wish to rearrange the proposals such that each of the n kernels has a high probability of accepting the proposed point; i.e., match proposals to targets such that for each target distribution the proposal statistic t(x*) is close to the observed statistic. However, even for relatively small values of n it becomes expensive to search the n! possible arrangements for the statistically most efficient one, which suggests to use a simple rule to automatically choose an arrangement given a generated dataset x*.

We illustrate such a simple rule with sampling from the posterior distribution of n ability parameters in the Rasch model; We order the posterior distributions based on the corresponding test scores; those corresponding to a low test score are placed first whereas those corresponding to a high test score are placed last. Next, we generate m = n proposal distributions which are ordered in the same way as the target distributions; those corresponding to a low generated test score are placed first and those corresponding to a high generated test score are placed last. It is clear that the first proposal is likely to be a good proposal for the first target distribution, the second proposal for the second target distribution, and so on. That this procedure improves the statistical efficiency is shown in Fig 3. The solid horizontal line in Fig 3 shows the average acceptance rate of the original SVE algorithm applied separately to each of the n latent variables, the efficiency of which is independent of n, and the points refer to the acceptance rate using our kernel matching procedure. Even with as little as n = 25 latent variables there is a substantial improvement to the statistical efficiency when the proposals are matched, with an average acceptance rate of 29% in the original SVE algorithm and 67% when the proposals are matched. Moreover, Fig 3 reveals that the statistical efficiency continues to improve as n increases.

Fig 3 reveals that if t(x) is sufficient for θ, we have a good way to match proposals to targets and as n becomes sufficiently large, each kernel tends to make a move such that we sample approximately i.i.d. from each of the n posteriors. We note that the proposals can be generated and subsequently accepted in parallel. The only non-parallizable part of the procedure is in matching the proposals, although one can find clever ways to do this. Sorting the statistics t(x*) (posterior distributions) is of an order of complexity that is often usually n log n but at most n², which compares favorably to the order of complexity n! that is needed to find the statistically most efficient rearrangement.

Sampling-unit specific prior distributions π_p(θ) are easily handled by incorporating the information encoded in the prior distributions, such as covariates, in matching the proposals. Since this information is also encoded in the mixing probabilities, it is available for matching proposals to target distributions. The only difference is that when a point drawn from π_q(θ) is proposed to a posterior with prior density π_p(θ), p ≠ q, the prior distributions do not cancel from the expression in Eq (3), and we accept θ* with probability: ensuring that π(θ ∣ x) ∝ π(x ∣ θ)π_p(θ) remains the invariant. For most prior distributions, the ratio of prior distributions is easily computed as many parts cancel in the expression.

To illustrate, consider a latent regression model in which each of n abilities θ is assigned a unique prior distribution: where y_p constitutes a covariate vector for person p and β a vector of regression weights. Assuming that a point drawn from π_q(θ) is proposed to a posterior with prior density π_p(θ), p ≠ q, we consequently accept θ* with probability: which also reveals that it is opportune to use the statistic to match proposals to targets.

3.3 A Rejection Algorithm

When t(X*) is a discrete random variable, a simple Rejection procedure is to generate proposals until an exact match t(x*) = t(x) is produced [2]. The matching of kernels points to an extension of the Rejection algorithm for sampling from n ≥ 1 posteriors. Consider sampling from the posterior distribution of n ability parameters in the Rasch model using a common prior. There are at most k + 1 different posterior distributions to sample from; one for every possible test score. Let denote the number of observations for a test score x₊. We generate a proposal corresponding to a test score ; if we retain the proposed point and decrease by one, otherwise we reject the proposed point. This procedure is repeated until for each possible test score, after which the generated values can be assigned to the target distributions. Note that this simplifies to the original rejection algorithm when .

Fig 4 shows that recycling the otherwise wasted proposals can significantly improve the computational efficiency (reduce the order of complexity). The solid horizontal line shows the average number of proposals that need to be generated when the original rejection algorithm is applied separately to each of n latent variables, the efficiency of which is independent of n, and the points show the average number of proposals that need to be generated when the proposals are recycled. Most significant is the reduction of the computational expense when samples are required from increasingly larger numbers of target distributions. When n becomes sufficiently large, only n proposals need to be generated to sample once from each of the n target distributions.

3.4 Binning the Statistics

The rejection algorithm is unsuited for applications in which t(X) is a continuous random variable or a discrete random variable with many possible realizations. Even though repeated sampling does not guarantee an exact match, oversampling revealed that we do continue to produce better proposals. In general, a good proposal is one for which the statistic t(x*) is “sufficiently close” to t(x); i.e., t(x*) is in some range , with a > 0. This suggests that we generate proposals until , with the value of a controlling the quality of our proposals.

To illustrate, consider a simple extension of our Rasch model known as the two-parameter logistic model. The two-parameter logistic model describes the distribution of a pupil’s item responses as a function of the ability parameter θ and an item discrimination α_i and difficulty δ_i for each of k items: where the weighted test score t(x) = ∑_i α_i x_i is sufficient for θ. Since the discrimination parameters α_i are real-valued (typically positive) we have that t(X) is a discrete random variable with 2^k possible realizations, one for every possible vector of item scores.

We consider sampling from a posterior π(θ ∣ t(x)) for a weighted test score t(x) ≈ 19 (x₊ = 9) based on a k = 20 item test with discriminations α_i that are sampled uniformly between 0 and 4. Fig 5 reveals that generating proposals until t(x*) falls in a bin increases the quality of proposals as a function of a (using a ∈ {∞, 5, 3, 2}); improving the overall acceptance rate from approximately 17% for a = ∞ (the original SVE algorithm) to approximately 74% for a = 5.

The idea of generating proposals until the statistic t(x*) falls within a certain range is closely related to the idea behind the ABC-rejection algorithm, where one simply accepts a proposed point when [21, 22]. When a is “sufficiently small” the proposed point θ* will be drawn from a posterior distribution π(θ ∣ t(x*)) that is “close” to the target distribution π(θ ∣ t(x)). For the SVE approach a need not be “sufficiently small” as the Metropolis kernel ensures that the correct posterior distribution π(θ ∣ x) is the invariant distribution. It is clear that decreasing the value of a implies higher acceptance rates, but also that more samples are required to produce a value t(x*) in , on average. However, when there are multiple target posterior distributions one could bin the observed statistics into m non-overlapping ranges, and apply recycling to the m bins to reduce the number of samples that need to be produced.

3.5 A Data Augmentation Procedure

Matching and oversampling use auxiliary- or sufficient statistics t(x*) to choose more efficient proposals. Marsman, Maris, Bechger and Glas [29] generalized this approach by making clever use of the augmented variables that are often used to sample from π(x ∣ θ). For example, logistic random variables are commonly used to sample item scores in the Rasch model: (4) Although Gibbs samplers are frequently used to sample from the augmented posteriors π(z, θ ∣ x), such procedures tend to converge slowly. The solution to this problem is to only use the augmented variables to generate proposals from a slightly different model that, when used in an independence chain (i.e., the SVE algorithm), does not suffer from this slow convergence.

Consider the distribution of w = {z₁, …, z_k, θ}; the joint distribution of the augmented variables and the parameter. We have that the conditional distribution corresponds to a unique posterior distribution , where x* is a function of w* defined through relations as Eq (4) (i.e., if , and 0 otherwise). Note that this relation can also be used to define posteriors for each of the k remaining conditional distributions . The difference between the posterior and is that w_k+1 = θ is used as the augmented variable in and w_i = z_i is the proposed point, whereas w_k+1 = θ is the proposed point in and w_i = z_i is used as the augmented variable to generate . This means that is a posterior distribution that corresponds to a slightly different model that uses the same model components, but where one of the likelihood functions switched places with the prior distribution (see Ref. [29] for more details). What makes this approach interesting is that from generating a single data vector we obtain k + 1 proposal distributions and we can choose the statistically most efficient one.

Fig 6 illustrates the approach with sampling from posterior distributions π(θ ∣ x₊) in the Rasch model using as the proposal distribution (right panel). Note that the procedure is statistically efficient and further improves when more observations become available; even though the posteriors become more concentrated. Also shown in Fig 6 is the original SVE approach (left panel), which becomes less efficient as the number of observations increases.

The procedure applies also to Logistic regression models, and, when the augmented variables have a non-logistic distribution, for instance a normal distribution, we obtain other Bernoulli regression models, such as probit regression [30]. Marsman et al. [29] used the procedure to estimate Ising network models using a full-data-information procedure, utilizing a latent variable expression of the Ising model, where the conditional distribution π(x ∣ θ) was found to be a multidimensional two-parameter logistic model [31].

Exponential Family models are closed under conditioning, that is, π(x ∣ θ, x ∈ ω ⊂ Ω_x) is also in the Exponential Family. In this manner, we find that generating responses to two Rasch items corresponds to a three category Partial Credit Model [32] whenever and the procedure similarly extends to such situations. In principle, this procedure can be used to generate other models, such as multinomial logit models [33], and extends the framework of Marsman et al. [29] to Potts network models.