Introduction

The ability to predict crop yield in response to climate variability at regional scale is crucial for developing agricultural policies, forecasting and analyzing global trade trends, and identifying effective adaptation strategies to climate change. Crop yield variability due to extreme weather events such as drought and high temperatures remains a major concern for farmers, governments, and markets, reinforcing the need for accurate and timely predictions of crop yield in an uncertain climate. In particular, year to year temperature fluctuations have been identified as the main source of uncertainty in crop yield predictions [1] while rapidly increasing global population is making food security an imminent challenge for many governments [2].

Two commonly used approaches for predicting crop yield responses to climate variability include process-based modeling and statistical modeling. Process-based crop models are powerful tools for crop yield predictions, particularly at the field scale, because they simulate physiological processes of crop growth and development in response to environmental conditions and management practices. However, due to intensive data and calibration requirements of process-based crop models, the use of detailed process-based crop models for timely predictions of crop yield at regional or global scale remains challenging [3]. On the other hand, statistical modeling estimates direct relationships between predictor variables (e.g., climate and soil factors) and crop yield in a given data set without considering the underlying processes in crop physiology and ecology. Statistical models can provide simple but reasonable predictions, provided that sufficient and reliable data were used for model training and that the predictions are made within the boundaries of training data. Statistical models are less dependent on field calibration data and may provide commonly used performance assessment measures useful for uncertainty analyses at regional scale [3]. Simple or multiple linear regressions (MLR) are commonly used statistical models for predicting crop yield [4, 5].

Statistical modeling methods based on machine-learning algorithms can provide alternatives to traditional regression approaches and overcome some of their limitations. These machine-learning techniques belong to the class of algorithmic approaches, which assume the data mechanism as a ‘black box’ or ‘gray box’. Machine-learning techniques have been used increasingly in recent years as niche-based classification modeling tools for predicting species habitat suitability in response to climate change [6, 7]. Random Forests (RF) is a non-parametric advanced classification and regression tree (CART) analysis method that has been adopted widely in many scientific fields, including for predicting suitable habitat of various plant and animal species [8–11] and gene expression interpretation [12–14]. A majority of applications of RF have been focused on its utility as a classification tool with only limited studies exploring its regression capabilities for predicting ecosystem or crop productivity [15–17]. Several studies have pointed to a number of promising advantages as well as drawbacks of RF as a regression tool over traditional regression models [18–20]. To date, RF regression applications in the fields of agronomy and crop science remain scarce, with few exceptions [15].

The objective of this study was to evaluate the performance of RF regression using MLR as a benchmark for global and regional crop yield predictions for three staple crops: wheat (Triticum aestivum), maize (Zea mays), and potato (Solanum tuberosum). Specifically, we applied RF to predict: 1) grain yield of wheat at the global scale that included all regions designated as mega-environments (ME) defined by the International Maize and Wheat Improvement Center (CIMMYT) (Fig 1A), 2) county-level maize grain yield in the United States for a 30 year period (Fig 1A), and 3) yield of potato tuber and silage maize in the Northeastern Seaboard Region (NESR) consisting of thirteen states from Maine to Virginia of the United States (Fig 1B).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

Study regions: global wheat mega-environments (A), US maize producing counties (B), and northeastern seaboard region (NESR) (C). All 12 wheat mega-environments are shown with different colors (A). Maize grain yield by the US counties in 2013 surveyed by USDA-NASS is visualized using different shades with darker shades representing higher yields (B). The NESR includes 433 counties of Connecticut, Delaware, Maine, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont, Virginia, and West Virginia. The red dots indicate the location of the data points, where weather stations exist. Point type data was used for this region (C).

https://doi.org/10.1371/journal.pone.0156571.g001

Materials and Methods

The RF models were trained to predict crop yield using multiple biophysical variables as predictors (Table 1). Environmental variables included climate, soil, photoperiod, water, and fertilization data. The same data were used for training MLR models for benchmarking purposes. The RF algorithm intrinsically set aside partial data for its own internal validation, called out-of-bag (OOB) data. However, to ensure a fair and conservative comparisons between RF and MLR, we used only a random half of each dataset (i.e., wheat, maize grain, potato, silage maize) for training (‘training dataset’) both RF and MLR models. The other half that was not used for training was then used as the ‘test dataset’ to validate and compare performances between the RF and MLR models. This process ensured that identical data points were available for training and independent testing using the data points not included in training.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The predictors used for random forests (RF) and multiple linear regression models.

Rank corresponds to a variable importance measure determined by RF models for each dataset.

https://doi.org/10.1371/journal.pone.0156571.t001

Model performance evaluation and inter-comparison

Four methods were used to evaluate and compare the model performances: 1) root mean square error (RMSE, Eq [1]), 2) Nash-Sutcliffe model efficiency (EF, Eq [2]), 3) index of agreement known as Willmott’s d (d, Eq [3]) [36], and 4) observed vs. predicted plots. These are measures commonly used for agricultural systems and crop models [37, 38].

[1][2][3]

Where, y_i represents the observations in the test data sets, ŷ_i the predictions, and is the observation mean in each test dataset. Briefly, RMSE is a measure of deviations between the observations and model predictions. EF is a model skill score in comparison with the observation mean; a perfect model will produce EF = 1 while a model with the same predictability as the mean will have EF = 0. A value of EF < 0 is also possible with no lower boundary [37, 39]. Willmott’s d value varies from 0.0 (poor model) to 1.0 (perfect model) [36, 37]. In addition, an observed vs. predicted plot was made to visualize the model performance. A simple linear regression line was drawn over the plot to compare the accuracy of the model predictions. All performance measures (i.e., RMSE, EF, d, and observed vs. predicted comparisons) were assessed using the model predictions made for test datasets set aside for validation and evaluation purposes only. That is, no training data were included for assessing model performance.

Results

Global wheat yield predictions

RF successfully predicted global scale wheat yield when compared against the test data that had not been included in model training. The RF model explained 96% of yield variance with a good agreement between the predicted and observed values in the test data (Table 2; Fig 2A). The RMSE of the model was 0.32 tons/ha, which is 11.9% of the average observed yield, EF was 0.96, and d was 0.99. The performance of a MLR benchmark was less satisfactory: RMSE = 1.32 tons/ha which is 49.2% of the average observed yield, EF = 0.31, and d = 0.68) (Table 2). The comparison between the observed and predicted (Fig 2A) indicates that RF model predictions corresponded highly with the observations with a slope of 0.94 and Pearson’s r = 0.98. Overall, RF clearly outperformed MLR in all measures of model performance for global wheat yield predictions (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Random Forests model performance for test datasets.

Observed vs. predicted plots are shown for four case studies: (A) global wheat grain yield, (B) US maize grain yield over 30 years, (C) potato wet tuber yield in northeastern seaboard region (NESR), and (D) maize silage yield in NESR The dashed lines indicate 1:1 relation and the solid line represents linear regression between the observations and predictions made for test datasets. The linear regression equation for the solid line is provided along with RMSE, EF, d, and Pearson’s r.

https://doi.org/10.1371/journal.pone.0156571.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Random Forest (RF) and multiple linear regression (MLR) model performance evaluation statistics.

See text for detailed description and equation of each statistic shown in the table.

https://doi.org/10.1371/journal.pone.0156571.t002

Variable importance measures of the RF model revealed that NFERT was the most influential variable followed by EVA, PREC49, and DAYL (Table 1). Climate variables were ranked lower in their relative variable importance. The partial dependence plot for NFERT indicates that global wheat yield begins to saturate at N fertilization rates between 150 and 200 kg/ha when the effects of other factors were excluded (Fig 3A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Partial dependence plots for the top ranked predictor variable from variable importance measures of Random Forests models.

(A) N fertilization rate (NFERT) in global wheat grain yield predictions, (B) year (YR) in the 30-year US maize grain yields, (C) Latitude (lat) for potato wet tuber yields in northeastern seaboard region (NESR), and (D) lat for maize silage yield in NESR. The Y-axis of each plot indicates the average of all of the possible model predictions for the X predictor value. The X-axis hash marks indicate deciles.

https://doi.org/10.1371/journal.pone.0156571.g003

Discussion

Our results illustrate that RF regression is highly effective for global and regional scale crop yield predictions. The RF models outperformed MLR models in all crops and scales we tested. Although RF has been used widely as a classification algorithm for species distribution and habitat suitability modeling in ecological studies in recent years [6, 11] to date few studies have explored its regression capabilities for crop yield or primary productivity studies in agriculture and ecology. Our results demonstrate that RF regression has many merits that are desirable for predicting complex crop responses in agricultural systems.

The RF algorithm intrinsically separates a random subset of data for performance testing from calibration data by using only the remaining set of data for model training. Therefore, splitting data for training and testing purposes is likely a redundant procedure when applying RF for crop yield modeling and its performance may increase as more data are included for training. For example, the RF model performance improved when we increased the size of training data used for potato tuber and maize silage predictions in NESR whereas MLR model performance did not improve (data not shown). Nevertheless, we separated training and test datasets in order to ensure that the same data points were used for training and testing between RF and MLR models. In practice, including more data for model training is likely to improve the predictability of RF regression.

Overall the RF model performed best (e.g., EF = 0.96) for predicting out of sample global wheat grain yield than any other cases we tested. However, the RF model performance was also highly satisfactory in other applications when applied for predicting US maize grain yields over 30 years or in NESR with much smaller datasets to train the model (Table 2; Fig 2). We note that the high performance for global wheat yield may be a result, in part, due to spatial autocorrelation in yields between grid cells belonging to identical political units; this feature of the gridded yield data limits evaluation for both RF and MLR.

In addition to its predictive capability, RF can also provide useful information about the variable importance and dependence. The variable importance rank and the partial impact of the variable on the response can be evaluated for the purpose of systems analysis [12, 40–42]. We used a commonly used variable importance measure: mean decrease accuracy (%IncMSE in r randomForest output) to identify the most influential variables determining crop yield in the regions we tested (Table 1). The partial dependence plots are useful for assessing the relationship between each predictor and the response variable [43, 44]. The variable importance measures and partial dependence plots revealed unique responses with respect to crops and regions. For example, variable importance output and the partial dependence plots show that NFERT was the top predictor of global wheat yield and US maize yield with a saturating response to increasing fertilizer (Fig 3). For 30-year US maize yield predictions, the model performance improved substantially when YR was included in the predictors; EF was 0.76 with YR and 0.64 without YR included. The temporal yield trends due to technological changes such as genetic improvements, agronomic intensification, pesticides and herbicides uses, and management practices have been well-documented [26, 45] and the RF model identified this trend as the most influential variable that was responsible for over 2 tons/ha increase over the 30 year period in determining maize grain yield in the entire US (Fig 3B). With the NESR cases, latitude (lat) was found to be the most influential variable to determine both potato tuber and maize silage yield (Fig 3C and 3D). Clay content was ranked as the second most important variable for maize silage yield in NESR as yield decayed with increasing clay content (data not shown) indicating drainage may be a key factor to achieving high silage corn yield in the NESR. The partial dependence plots for other variables showed relatively small effects with complex patterns that may not be readily explained by known physiological or agronomic responses indicating limitations of the analyses solely based on RF model outcomes.

Our results showcase the utilities of RF regression for crop yield predictions. Multiple advantages may exist for using RF regression over other approaches including traditional MLR when predicting crop yield responses. First, RF regression models have been shown to outperform traditional MLR models in explaining the variability in data [46] and our results strongly support that this is the case for crop yield predictions. Note that there could be many ways to improve MLR model performance such as including additional predictor variables. For example, there is evidence that the inclusion of extreme temperatures could further improve the performance of MLR models [47–49]. We did not include additional variables only to MLR models to maintain the predictors between RF and MLR comparable. The high performance of RF is likely more evident when the response is a result of complex interactions between multiple predictors as in crop systems where interactions among biophysical, ecological, physiological, and management factors can complicate modeling. Second, the RF regression has advantages when predictor or explanatory variables are highly correlated (e.g., temperature derived variables) [19, 20]. Many predictors of crop production, such as climate, management, and soil, are often highly correlated with and within each other and may have multi-collinearity. Variable collinearity can be a critical problem in traditional regression models that are derived from linear regression. RF uses the single best variable when it splits responses at each node of decision trees and averages the predictions of the trees in the forest to make a multi-dimensional step function. This means that even if multiple variables are correlated and drive the response similarly, only one of them can affect the RF regression model at a time. Third, RF algorithms can use multiple types of predictors in a model more easily than traditional multiple linear or non-linear regressions can [20]. Available data for crop yield prediction vary by type and collection methods. For example, temperature and precipitation data are quantitative and continuous while soil orders and crop cultivars are categorical. Sometimes, continuous data recorded is in a categorical form, such as high, medium, low content of soil organic matter. RF is an ensemble of decision trees, which consist of binary nodes that split the response. At every RF node, any type of splitting of predictor variables, such as continuous and categorical, is evaluated and selected for the split under the same standard: how well the given variable can split the response.

Limitations of the RF regression for crop yield modeling are also identified. As found with partial dependence measures of low importance variables (data not shown), the behavior of the RF model may be less intuitive to interpret than traditional regression models because its algorithm consists of an ensemble of a large number of decision trees that may not be fully described mechanistically. In addition, the RF algorithm may overfit data [18, 50]. In general, statistical models have the potential of overfitting issues within the range of training data. With RF, only the values included in training data are used for splitting regression trees and thus lumping the predictions for the conditions outside the range of training data. This means making predictions beyond the training data range is impractical because of inherent inability to extrapolate to data dimensions (e.g., future climate) where no training has been done. For predicting crop yield in the future scenarios, this limitation of RF regression could be critical, since at least some part of the current crop field is expected to have new and more extreme environmental conditions in the future that did not exist in the past and present data domains. In practice, the model training dataset should be selected to cover as wide a range as possible for most critical predictor variables. An approach to overcome the limitations associated with predicting novel extremes may be to train RF to learn the novel responses by coupling it with process- or MLR models that are capable of modeling crop responses in extreme conditions where observational data are sparse.

The balance of the variable variance distribution also needs to be considered in RF modeling. It is known that if the number of positive and negative response in the model training dataset is unbalanced, RF classification models overestimate the higher numbered responses [18]. The splitting and averaging algorithm of RF regression may result in underestimation of the marginal responses in the range where data points are scarce in extreme ends (see Fig 2). This response distribution balance can affect RF based crop yield predictions. Although a sufficient sample size can minimize this problem, an RF model’s predictive power for a future period can be compromised if the response domain is not conserved in the future scenarios. All of the RF models in this study showed an overall trend of underestimation above the mean and overestimation below the mean (Fig 2) because the marginal yield points were either underestimated or overestimated. Having a sufficiently large training dataset with balanced predictor variance distributions is likely to help minimize this issue; this is illustrated in the global wheat case where both accuracy and precision were improved compared to the other cases (Fig 2).

Conclusions

This study evaluated the efficacy of RF regression using MLR as a benchmark to model complex yield responses of wheat, grain maize, potato, and silage maize at global and regional scales. The RF algorithm has many advantages to regress complex crop systems, but is not yet being widely used in this field. We demonstrated that RF provides superior performance in predicting yield of all crops and regions tested. The result of this study shows strong potential for the implementation of an RF algorithm as an alternative statistical modeling method for crop yield predictions. It should be noted that RF has the risk of overfitting data for the conditions where training data were concentrated while its accuracy can diminish where training data were sparse. Similarly, applying RF regression to extrapolate outside training data dimensions should be avoided. In summary, our results support that RF regression can be an effective tool for predicting crop yield at the global and regional scale with cautious selection of a training dataset that includes a wide range of predictor variability.

Acknowledgments

We thank Elliot Blasich and Marian Hsieh for their assistance with this project. This study was supported by a Cooperative Research Program for Agricultural Science and Technology Development (Project No. PJ01000707), Rural Development Administration, Republic of Korea (SHK; KMS). Additional support was provided in part by a Specific Cooperative Agreement: 58-1265-1-074 between University of Washington and USDA-ARS (SHK; VRR), the USDA-ARS Headquarters Postdoctoral Research Associate Program (DHF), the USDA-NIFA-AFRI Grant no. 2011-68004-30057: Enhancing Food Security of Underserved Populations in the Northeast through Sustainable Regional Food Systems (DHF), the USDA AFRI fellowship 2016-67012-25208 (NDM), the NSF Hydrological Sciences grant 1521210 (NDM), and the Packard Foundation (EEB).