Data model.

We start by describing the basic model for those states for which we have raw data on the 8 km grid, and in Model for township data we describe the extension of the model to accommodate data aggregated at the township level.

The statistical model treats the observations as coming from a multinomial distribution with a (latent) vector of proportions for each grid cell, where y_i is the vector of counts for the P taxa at the ith cell, n_i is the number of trees counted in the cell, and θ(s_i) is the vector of unknown proportions for those taxa at that cell. Note that we use a standard multinomial distribution without overdispersion, because the set of trees in the dataset is roughly uniformly sampled across the cells or townships [6].

The proportions, θ_p(s_i), p = 1, …, P, are modeled spatially by a set of P Gaussian spatial processes, one per taxon, α_p(s_i), p = 1, …, P. This collection of processes defines a multivariate spatial process for composition. The α_p(s) processes are defined on the 8 km grid, α_p = {α_p(s₁), …, α_p(s_m)} for the m grid cells. In Latent variable model we introduce a multinomial probit model that relates the α_p(s) processes to the proportion processes, θ_p(s), via the introduction of latent variables, with an implicit sum-to-one constraint, . A multinomial probit model is similar to logistic regression, used for modeling a binary outcome based on an underlying probability the outcome will occur, but generalizes to modeling a categorical variable based on probabilities for each category.

The critical component of the statistical model is the representation of α_p(s) as a spatial process. A spatial process is a statistical representation that models spatially-correlated values. It provides a prior structure that serves to smooth across noise in the observations and allows for prediction at locations based on information from nearby locations, including interpolation to locations with no data. Apart from the sum-to-one constraint, the taxa are considered to be independent in the prior. We did not want to impose any structure that ties the different taxa together, as any correlation will likely vary across space.

In the next section, we consider two spatial models to define the structure of the α_p(s) processes, a standard conditional autoregressive model [20] and a Gaussian Markov random field (MRF) approximation to a Gaussian process with Matérn covariance [21]. These models are specific statistical formulations of spatial processes that represent spatial correlation by defining neighborhoods around each location that are used to help inform predictions at the location.

Spatial process models.

MRF models represent the neighborhood information by working directly with the precision matrix (the inverse of the covariance matrix) of the values of the spatial process, so calculation of the prior density of α_p is computationally simple [22]. However, in situations in which the likelihood is not normal, such as our multinomial likelihood, it can be difficult to set up effective MCMC algorithms that are able to move in the high-dimensional space of α_p. The latent variable representation helps to alleviate this problem. Next we describe two alternative spatial models that we considered; in Model comparison, we evaluate the models on held-out data to choose between the two.

Standard conditional autoregressive models.

Our first model is a standard conditional autoregressive (CAR) model; technical details can be found in [20]. We use a standard form of this model that treats the four cardinal neighbors of each grid cell as the neighbors of the grid cell. The corresponding precision matrix has diagonal elements, Q_ii, equal to the number of neighbors for the ith area (i.e., four except for cells on the boundary of the domain), while Q_ik = −1 (the negative of a weight of one) when areas i and k are neighbors and Q_ik = 0 when they are not. This gives the following model for the values of α_p(s_i) collected as a vector across all of the grid cells, i = 1, …, m: The use of the generalized inverse notation indicates that Q is not full-rank, but is of rank m − 1; this gives an improper prior on an implicit overall mean for the process values. Note that we specify an explicit mean of zero because a non-zero mean would not be identifiable in light of the implicit mean. This specification is called an intrinsic conditional autoregression (ICAR) and we can write Q = D − C where C is the m × m adjacency matrix defining the neighborhood relation of the locations; that is C_ik = 1 if locations i and k are neighbors and zero otherwise. The matrix D is an m × m diagonal matrix containing the row sums of matrix C as the diagonal entries,

We refer to this as the CAR model.

Gaussian process approximation.

Gaussian processes (GP) are also standard models for spatial processes [20]. GP models are computationally challenging for large datasets because of computational manipulations involving large covariance matrices. Given this, [21] proposed a new framework for using Gaussian MRFs (GMRFs) as approximations to GPs, based on the use of stochastic partial differential equations (SPDEs).

Gaussian processes are generally constructed using one of a number of correlation functions that define how the strength of correlation between the values of the process at two locations decays as a function of the distance between the locations. We consider Gaussian processes in the commonly-used Matérn class, using the following parameterization of the Matérn correlation function, where d is Euclidean distance, ρ is the spatial range parameter, and is the modified Bessel function of the second kind, whose order is the smoothness (differentiability) parameter, ν > 0. ν = 0.5 gives the exponential covariance. For any pair of locations, R(d) defines the correlation of the process, (i.e., α_p(s) in our context), as a function of the distance between the locations. Considering all pairs of locations, this defines a correlation matrix for all locations of interest.

The approach of [21] allows us to consider MRF approximations to the Matérn -based GP for ν = 1 and ν = 2. Our second spatial model is this Lindgren approximation for Matérn -based GPs with ν = 1. To implement this Lindgren model, one modifies the Q matrix defined previously as follows based on the technical specification of the precision matrix provided in [21]. Let . The diagonal elements of Q are 4 + a². The entries corresponding to cardinal neighbors are −2a. Those for diagonal neighbors are 2, and those for 2nd-order cardinal neighbors are 1. This extends the neighborhood structure relative to the CAR model and parameterizes it as a function of ρ.

The primary difference between the CAR and Lindgren models is that the Lindgren model provides an additional degree of freedom by estimating ρ. In particular ρ allows us to estimate the locality of the spatial smoothing. As ρ decreases, the model uses increasingly localized data to estimate the compositional proportions at a given location, effectively averaging the empirical proportions over smaller neighborhoods. In general, the [21] model will generally provide for a smoother estimate than the CAR model [23].

To ensure that the σ² parameter is mathematically equivalent between the two models, we reparameterize, producing our second model:

We refer to this model as the SPDE model.

Prior distributions.

The ICAR specification contains a set of hyperparameters p = 1, …P. Following [24] we use a uniform distribution on each σ_p parameter, with upper bound of 1000. For the SPDE model we also have hyperparameters {μ_p}, which we give flat, non-informative priors (truncated at ±10), and {ρ_p}, which we give uniform priors on the interval (0.1, exp(5)). These various hyperparameters are unknown parameters that control the spatial structure of the two spatial models and are estimated from both the data and the prior distributions just specified based on the Bayesian approach.

Latent variable model.

It is well-known that devising an effective MCMC algorithm for models with latent Gaussian process(es) and a non-Gaussian likelihood is difficult [22, 25, 26]. To develop an algorithm, we make use of a latent variable representation for the multinomial probit model [27]. The representation introduces latent variables that allow one to develop an MCMC sampling strategy that takes advantage of closed-form full conditional distributions (so-called Gibbs sampling steps) for α_p.

Suppose that compositional counts are available at a number of locations. At location i, a sample of size n_i observations is collected, and each observation (i.e., each tree) can be classified into P distinct categories. For a given tree j at location i, let Y_ij denote the response variable indicating the category. Let Y_ij be associated with P latent variables W_ij1, …, W_ijP such that Y_ij = p if and only if ; in other words, the maximum of the set of latent variables determines the category of observation j at location i. The final piece of the latent variable representation is the relationship between the W variables and the α_p(s) processes. We have that independently for all of the W_ijp values. Consider the following example with two locations that are neighbors and P = 2 categories. Each tree j at location i is associated with two variables W_ij1 and W_ij2, governed by the latent variables α₁(s_i) and α₂(s_i), respectively. Suppose that α₁(s_i)>α₂(s_i) for a given location i. Then this model implies that any tree j is more likely to be labeled 1 than 2 at location i. The difference between α₁(s_i) and α₂(s_i) explains the difference in probability of categories 1 and 2 at location i, and the similarity between α_p(s₁) and α_p(s₂) explains the correlation between the probabilities at locations 1 and 2 for category p.

Model for township data.

We developed an extension of the model described in previous sections to account for data at a different aggregation than our core 8 km grid. This extension introduces a new set of latent variables, one per tree, that indicate the grid cells in which the trees are located and that can be sampled within the MCMC as additional unknown parameters. Specifically, c_tj is the latent “membership” variable for tree j in township t, t = 1, …, T. The prior for c_tj is a discrete distribution that puts mass, ψ_ti, i = 1, …, m, proportional to the areal overlap between the township in which the tree is located and the m grid cells, giving independently across all trees. Because the townships overlap a limited number of grid cells, most of the ψ_t1, …, ψ_tm values are zero.

Using the latent variable representation, we have that W_tjp ∼ N(α_p(s_ctj), 1) for tree j in township t. In updating the other parameters in the model during the MCMC (specifically the α values), we condition on the current values, {c_tj}, which provides a “soft” (i.e., probabilistic) assignment of trees to grid cells that respects both the known township in which the trees occurred and the uncertainty in which grid cells the trees occurred.

Note that this prior represents the location of each tree in a township as being independent of the other trees; this is somewhat unrealistic because it does not represent our knowledge that the trees in a township would be distributed somewhat regularly across the area of the township because the witness trees were used to indicate property boundaries.

Computation.

The [27] representation is convenient for MCMC sampling, particularly in this high-dimensional spatial context, as it allows us to draw from the posterior conditional distributions of the W_ijp variables (these distributions are truncated normal) in closed form and to draw the entire vector of latent process values for each taxon, α_p, as a single sample that respects the spatial dependence structure for each taxon.

While the latent variable representation provides great advantages in the MCMC sampling for each α_p compared to joint Metropolis updates or updating each location individually, there is still strong dependence between the hyperparameters, and the latent process values (as well as between the latent process values and the latent W_ijp variables). To address the first, we developed a “cross-level” joint updating strategy for the CAR model in which we propose ϕ_p = σ_p, p = 1, …, P, (and for the SPDE model, ϕ_p ∈ {μ_p, (σ_p, ρ_p)}) via a Metropolis-style random walk and then given the proposed value, , propose α_p from its full conditional distribution given and the latent W_p variables, where W_p is the vector of all W_ijp values for taxon p: W_p = {W_ijp}, i = 1, …, m;j = 1, …, n_i. This is equivalent to sampling from the marginalized (with respect to α_p) distribution of ϕ_p conditional on W_p. For these various joint samples of hyperparameters and α_p, we use adaptive Metropolis sampling [28].

The full description of the MCMC sampling steps is provided in Appendix. In addition, in the latent variable representation, θ_p(s) never appears explicitly and cannot be calculated in close form. Instead we use Monte Carlo integration over W_ijp, p = 1, …, P to estimate θ_p(s_i), also described in Appendix.

The model is implemented in R [29] with core computational calculations coded in C++ using the Rcpp package [30]. We also make extensive use of sparse matrix representations and algorithms, using the spam package in R [31]. All code is available on Github, including pre- and post-processing code, at https://github.com/PalEON-Project/composition.

Design.

We compared the CAR and SPDE models by holding out data from the fitting process and assessing the fit of the model on the held-out data. We used two experiments with held-out data:

1. The first experiment used a subregion containing most of Minnesota and a small amount of western Wisconsin, defined to be the cells whose x-coordinate was less than 300,000 m (this defines a north-south line that approximately goes through Duluth, Minnesota) and hereafter referred to as the “Minnesota subregion”. We chose this subregion for evaluation because of its high data density, allowing us to experiment with the effects of increasing data sparsity on model performance. We held out all data from 95% of the cells in this Minnesota subregion, with cells selected at random. This was meant to assess the ability of the model to interpolate from a sparse set of cells/townships and mimics the limited data in Illinois and Indiana.
2. We held out 5% of the trees from all of the trees in the dataset for the midwestern subdomain (leaving aside the held-out Minnesota subregion cells). This was meant to assess the ability of the model to estimate the composition in cells in which data were available.

Finally, in a separate sensitivity analysis we instead left out 80% of the cells in Minnesota subregion at random. This variation on the first experiment above was meant to indicate whether our model comparison conclusions would be robust as the digitization process for Illinois and Indiana progresses and provides us with increasingly dense data.

There has been extensive work in the statistical literature on good metrics to use to compare the predictive ability of models; these metrics are referred to as scoring rules. A general conclusion from this work is that predictive distributions should maximize sharpness subject to calibration. That is, the predictive distribution should be as narrow as possible while being calibrated such that the observations are consistent with the distribution [32]. When thinking in terms of prediction intervals as summaries of the predictive distribution, we seek intervals that are as narrow as possible while still covering the truth the expected proportion (e.g., 95% for a 95% prediction interval) of the time.

Following the suggestions in [32], we considered the following metrics: Brier score, log predictive density, mean square prediction error, mean absolute error, and coverage and length of prediction intervals. Further details on each are given below. For experiment 1, we define Y_i = {Y_i1, …, Y_iP} as the count of all trees in held-out cell i and for experiment 2, Y_i is the count of held-out individual trees in the cell, while y_ijp is an indicator variable taking value either 0 or 1 depending on whether the jth held-out tree in the ith cell is of taxon p. is the empirical proportion in category p for the n_i held-out trees in cell i. We calculated each of the metrics in two ways. First, we used the posterior mean composition estimates (as an evaluation of our core predictions), with being the posterior mean. Second, we averaged the metric over the posterior samples (as an evaluation of our full data product, including uncertainty), taking to be an individual MCMC sample and then averaging the metric over all the posterior samples.

1. Brier score: [32] suggest this metric, which has been in use for decades. For multi-category as opposed to binary outcomes, this takes the form where is the total number of held-out trees for a given experiment and j indexes across held-out trees in cell i.
2. Log predictive density: This metric takes the log of the probability density of held-out observations under the fitted model, , summing on the log scale across all of the held-out data.
   While in principle, this metric should be optimal [33], it is very sensitive to small predictions near zero [32]. Even worse, our Monte Carlo estimation of θ used 10000 samples, so in some cases . When a tree is present in a cell but its corresponding proportion is 0, this gives a log density of −∞, preventing use of the metric. As an informal solution to this we set in such cases, but given these issues we treat the log predictive density as a secondary measure.
3. (Experiment 1 only) Weighted root mean square prediction error (RMSPE), and mean absolute error (MAE) These metrics calculate the error of the estimated proportions relative to the empirical proportions based on the held-out trees, averaging over cells and taxa. We weight by the number of held-out trees in each cell to account for the greater variability in the empirical proportions in locations with few held-out trees.
4. (Experiment 1 only) Coverage and length of 95% prediction intervals for Y_ip. We considered only cells with at least 50 trees to focus our assessment on cases where empirical proportions were reasonably certain and avoid being strongly influenced by predictive inference for cells where observational variability dominates.

Note that all of the metrics except coverage and interval length can be applied to individual posterior samples and therefore allow us to estimate the posterior probability that one model has a lower (better) value of the metric than the other model by simply calculating the proportion of samples for which the model has a lower value of the metric. Also note that in addition to allowing comparison between models the MAE and RMSPE metrics allow one to assess absolute performance of each model in predicting composition.

In our initial exploratory fitting, we noticed that the SPDE model produced boundary effects in the predicted composition near the edges of the convex hull of the observations. To attempt to alleviate this, we added a buffer zone with a width of six grid cells around our entire original domain, but note that the boundary effects were still evident even after inclusion of the buffer. For the model comparison, we included this buffer for both the SPDE and CAR models.

We ran each model for 150,000 iterations. After discarding 25,000 iterations for burn-in, we retained a posterior sample of 250 subsampled iterations—we use a subsample instead of the full 125,000 post-burn-in iterations to reduce post-processing computations and storage needs.

Results.

Here we summarize the results of our analyses that inform the choice between the CAR and SPDE models.

For Experiment 1 (full cells held out) for cells in the Minnesota subregion held out of the fitting process, the CAR model outperforms the SPDE model based on the posterior distribution over the predictive metric values (Table 1). For the posterior mean predictions, the SPDE model appears to outperform the CAR model to a lesser degree, but we do not have any uncertainty estimates for this comparison. Coverage and interval lengths are similar between the two models (Table 2). From a practical perspective, based on the difference in mean absolute error, the differences between the models are small (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Predictive ability based on several predictive metric criteria for the CAR and SPDE spatial models when holding out 95% of entire cells of data in Minnesota.

https://doi.org/10.1371/journal.pone.0150087.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Coverage and length of prediction intervals for the CAR and SPDE spatial models when holding out 95% of entire cells of data in Minnesota.

https://doi.org/10.1371/journal.pone.0150087.t002

The results for the variation on Experiment 1 in which the proportion of cells that are held out decreases from 95% to 80% show that the SPDE model generally outperforms the CAR model, but again differences from a practical perspective, based on mean absolute error, are limited (Tables 3 and 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Predictive ability based on several predictive metric criteria for the CAR and SPDE spatial models when holding out 80% of entire cells of data in Minnesota.

https://doi.org/10.1371/journal.pone.0150087.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Coverage and length of prediction intervals for the CAR and SPDE spatial models when holding out 80% of entire cells of data in Minnesota.

https://doi.org/10.1371/journal.pone.0150087.t004

In Experiment 2 (individual trees held out), we have evidence (posterior probability of 0.93) that the SPDE model is better based on the Brier score, but the Brier score values for the two models are numerically almost the same (Table 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Predictive ability based on several predictive metric criteria for the CAR and SPDE spatial models when holding out 5% of trees.

https://doi.org/10.1371/journal.pone.0150087.t005

The differences between models are not consistent across the various comparisons, so there is not a clear choice. In our final data product we use the CAR model, for three reasons. First, the CAR model has modestly better performance when data are sparse, as is still the case for Illinois and Indiana. Second, the model is simpler and easier to explain, and computations can be done more quickly. Third, predictions from the SPDE model showed boundary effects, with some taxa showing non-negligible posterior mean values at the edges of the domain, well away from where the taxa were present in the empirical data. This included non-negligible values within (but near the edge of) the convex hull of locations with data.