https://orcid.org/0000-0003-3622-8237
Figures
Abstract
Suitable combinations of observed datasets for estimating crop model parameters can reduce the computational cost while ensuring accuracy. This study aims to explore the quantitative influence of different combinations of the observed phenological stages on estimation of cultivar-specific parameters (CPSs). We used the CROPGRO-Soybean phenological model (CSPM) as a case study in combination with the Generalized Likelihood Uncertainty Estimation (GLUE) method. Different combinations of four observed phenological stages, including initial flowering, initial pod, initial grain, and initial maturity stages for five soybean cultivars from Exp. 1 and Exp. 3 described in Table 2 are respectively used to calibrate the CSPs. The CSPM, driven by the optimized CSPs, is then evaluated against two independent phenological datasets from Exp. 2 and Exp. 4 described in Table 2. Root means square error (RMSE) (mean absolute error (MAE), coefficient of determination (R2), and Nash Sutcliffe model efficiency (NSE)) are 15.50 (14.63, 0.96, 0.42), 4.76 (3.92, 0.97, 0.95), 4.69 (3.72, 0.98, 0.95), 3.91 (3.40, 0.99, 0.96) and 12.54 (11.67, 0.95, 0.60), 5.07 (4.61, 0.98, 0.93), 4.97 (4.28, 0.97, 0.94), 4.58 (4.02, 0.98, 0.95) for using one, two, three, and four observed phenological stages in the CSPs estimation. The evaluation results suggest that RMSE and MAE decrease, and R2 and NSE increase with the increase in the number of observed phenological stages used for parameter calibration. However, there is no significant reduction in the RMSEs (MAEs, NSEs) using two, three, and four observed stages. Relatively reliable optimized CSPs for CSMP are obtained by using at least two observed phenological stages balancing calibration effect and computational cost. These findings provide new insight into parameter estimation of crop models.
Citation: Zhang Y, Zhang Y, Jiang H, Tang L, Liu X, Cao W, et al. (2024) Effects of different observed datasets on the calibration of crop model parameters with GLUE: A case study using the CROPGRO-Soybean phenological model. PLoS ONE 19(6): e0302098. https://doi.org/10.1371/journal.pone.0302098
Editor: Dafeng Hui, Tennessee State University, UNITED STATES
Received: January 1, 2024; Accepted: March 26, 2024; Published: June 13, 2024
Copyright: © 2024 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files
Funding: The work is supported by the Shandong Provincial Natural Science Foundation of China (ZR2021MC044, ZR2021MF085, ZR2019PF023, ZR2022QF046, ZR2023QC116), National Natural Science Foundation of China (31872847, 62006174, 62302338), and the Higher Educational Science and Technology Program of Shandong Province in China (J17KB183). The above funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. No authors received a salary from any of the above funders.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Crop model is a crucial and powerful tool to reveal the impact of growth conditions change on crop production [1–5]. Crop model uncertainties can be attributed to three main sources, namely model inputs, structures, and parameters [6–9]. Uncertainty in parameters is a crucial factor in model prediction and has attracted significant attention in reducing uncertainty and improving prediction [10–12]. Cultivar-specific parameters (CSPs) are commonly used in crop models to quantify the developmental characteristics that distinguish crop varieties, cultivars, or hybrids. However, since CSPs are unknown and difficult to measure directly in field conditions, estimation of CSPs is of great importance for obtaining reliable model predictions [13]. A natural question is: can more calibration data lead to better results in model parameter estimation and improved model performance? This question is vital for optimizing model parameter estimation, since more optimization data incurs more labor and resources.
Several automatic optimization methods have been developed to calibrate crop model parameters [4,14]. Generalized Likelihood Uncertainty Estimation (GLUE) is a popular Bayesian statistical method for simultaneously calibrating model parameters and estimating model uncertainty [4,15]. He et al. [16] employed GLUE method to estimate the genetic and soil parameters of the CERES-Maize model, achieving accurate prediction of sweet corn production. Jin et al. [17] used GLUE and a formal Bayesian method to optimize the parameters in a simple conceptual model of water balance, and then compared and analyzed the differences between the two methods. Marin et al. [18] utilized GLUE method to optimize ten parameters of the DSSAT-CaneGro model, thereby enhancing its prediction accuracy. With the utilization of GLUE, Pathak et al. [19] estimated the genetic parameters of the cotton model, resulting in improved model performance. Some scholars also employed GLUE to estimate parameters of the crop-hydrology model for enhancing its prediction performance [20]. Sun et al. [21] utilized the GLUE method to evaluate the uncertainty and sensitivity of 25 parameters related to nutrient transport, soil properties, and crop genetics in the agricultural-hydrological model of RZWQM-DSSAT. GLUE has been playing a significant role in calibrating crop model parameters. The incorporation of GLUE into the Decision Support Systems for the Agro-technology Transfer (DSSAT) model has facilitated the estimation of parameters for various crops [13]. The estimation effect of GLUE on model parameters relies on the careful selection and combination of likelihood functions [22]. Recently, GLUE is combined with the parallel processing technique to calibrate the crop model parameters more efficiently and in a shorter timeframe [23]. Most previous studies on GLUE have primarily focused on parameter calibration for various crop models and applied to assess and quantify model uncertainties [24,25], but the quantitative influence of the observed data on model parameter estimation and model performance by using GLUE remains largely unexplored.
In addition, crop phenology controls the life cycle of crops and the partitioning of assimilates between crop organs [26]. Accurate prediction of crop phenology is essential for assessing crop responses to environmental changes and developing management strategies [27–29]. In recent years, CROPGRO-Soybean model has been a commonly used tool for determining the optimal number of experiments [30] and of site × year × planting dates combinations [31] that ensures reliable model calibration and minimizes prediction errors. As a case study of the CROPGRO-Soybean phenological model (CSPM), the objectives of this study are (1) to calibrate the CSPs of CSPM by exploiting different combinations of the four observed phenological stages with the GLUE method; (2) to evaluate and compare the quantitative influence among different combinations of observed phenological stages in estimating CSPs. The results would help to balance the relationships among the amount of observed data, the effect, and the resource in estimating crop parameters.
2. Materials and methods
2.1 Brief introduction to CROPGRO-Soybean model
The CROPGRO-Soybean model is a process-based dynamic and generic crop model that simulates carbon, water, and nitrogen balances for the soybean plant and soil [32,33]. It includes 15 cultivar-specific parameters that characterize phenology and vegetative and reproductive growth [34,35]. The model operates on a daily time step. It has been embedded in DSSAT and widely used in agriculture production and management [36].
Four critical phenological stages, including initial flowering (FS), initial pod (PS), initial grain (GS), and initial maturity (MS), are simulated in CSPM with the CSPs optimized by GLUE. Three coefficients related to photoperiod sensitivity (CSDL, PPSEN, and R1PPO), and four coefficients related to photothermal duration of life phases (EM-FL, FL-SH, FL-SD, SD-PM) in CSPM are usually calibrated at the cultivar level [37]. These seven coefficients are associated with the simulation of the four phenological stages. The value ranges typical for these parameters are displayed in Table 1 [38].
2.2 Experimental design
Four field experiments (Exp. 1, 2, 3 and 4) were respectively conducted to collect soybean phenological data on five high quality and high yield soybean varieties from the Yangtze-Huai Soybean Breeding Program (Table 2). Row planting was conducted with a length of 2 m, a row spacing of 0.5 m, a planting depth of 3 cm, and with three replicates. In each field experiment, four phenological data of initial flowering (FS), initial pod (PS), initial grain (GS), and initial maturity stage (MS) were recorded according to phenological classification of Fehr et al. [39].
In the field experiments, no obvious light, temperature, water, nutrition, pest, or disease stresses were observed during the crop growth seasons, and the yield of each soybean cultivar is within the normal range. Daily meteorological data were obtained from the meteorological information center of the State Meteorological Administration of China [28]. Actual sowing depth, meteorological data and latitude are used in CSPM. Default values in CSPM for parameters other than CSPs were used.
2.3 The research framework and GLUE algorithm
The framework for estimating model parameters is demonstrated in Fig 1. The steps for using the GLUE method by Beven and Binley [40] is summarized as follows:
- Step 1: The prior distributions of parameters to be optimized are assumed to be uniform within the ranges of parameter values shown in Table 1. Initial populations composed of parameter combinations are randomly generated with the uniform distribution.
- Step 2: Each parameter combination is used to drive the CSPM for Monte-Carlo simulation, and the simulation results are employed to compute the related parameters.
- Step 3: Eq (1) is chosen as the likelihood function in our work.
Where θi is the ith parameter combination composed of CSDL, PPSEN, R1PPO, EM-FL, FL-SH, FL-SD, and SD-PM. S(θi) is the simulated values for the four phenological stages (FS, PS, GS, and MS) calculated using θi at time t. and
are the observed and mean observed value of the jth (j = 1, 2, 3, 4) phenological stage at time t, respectively.
is the likelihood function value corresponding to the jth (j = 1, 2, 3, 4) phenological stage. Abs() is the absolute of a numerical value.
- Step 4: the combination likelihood function Lcj(θi|Oj) corresponding to the the jth (j = 1, 2, 3, 4) phenological stage is calculated by Eq (2)
where Oj is the observed value of the jth (j = 1, 2, 3, 4) phenological stage. Not is the number of observation moments, Not = 1 in this paper.
- Step 5: The validity of parameter values in crop models is often determined by the prediction accuracy of multiple phenological stages, Eq (3) is employed to calculate the likelihood function of a certain parameter combination.
Where LD(θi|O) denotes the value of likelihood function corresponding to multiple phenological stages. Nps is the number of observed phenological stages. O is the observed dataset of phenological stages.
- Step 6: Eq (3) is used to compute the likelihood function of every parameter combination. The threshold value of the likelihood function is utilized to find out the parameter combinations with “positive effect”. The threshold value is set 0.90, and the parameter combinations corresponding to the likelihood function value greater than 0.90 are considered as having “positive effect” on the simulation results of CSPM. Eq (4) is utilized to obtain the posterior probability density of parameter combination.
where P(θi) is the posterior probability density of θi. Nfi is number of parameter combinations filtered with the threshold value of 0.90.
- Step 7: On the basis of posterior probability density of each parameter combination, Eqs (5) and (6) are used to respectively compute the mean and standard deviation of each variable in the parameter combination.
where θim is the mth parameter in θi. is the mean value of posterior distribution of θim.
- Step 8: The array composed of
(m = 1,…,7) is considered as the optimal solution of parameter combination for each calibration. The rest of the notation is the same as above.
Parameter setting: During model parameter calibration using GLUE, the number of initial populations is assigned as 20000 and the number of generations is set to 1.
2.4 The evaluation criteria
Root means square error (RMSE), mean absolute error (MAE), coefficient of determination (R2), and Nash Sutcliffe model efficiency (NSE) are used to evaluate the differences between simulated and measured values.
Where, Ntol is the number of the observed or simulated values. Oi and Si are the observed value and the simulated value, respectively. The lower the value of RMSE (MAE) the better the model performance [41–43].
Where RSS and TSS denote the residual sum of squares and the total sum of squares, respectively. R2 has a range between 0 and 1. The values of R2 = 1 can be interpreted as “very good” about model performance, values from 0.8 to 1 as “good”, from 0.6 to 0.8 as “satisfactory”, and less than 0.6 as “weak” [41].
Where, Ō is the means of observed value. NSE ranges between 0 and 1, the values ranging between 0.75 and 1.0 are considered as “very good” regarding the performance of the model, between 0.65 and 0.75 as “good”, between 0.5 and 0.65 as “satisfactory”, and between 0.4 and 0.5 as “acceptable” [41,43]. The rest of the symbols is the same as above. The definitions of all symbols are given in Table 3.
2.5 Calibration and Evaluation for CSPM
Datasets from Exp. 1 and 3 are used for CSPs estimation and model calibration. Datasets from Exp. 2 and 4 are used for model evaluation (Table 4). CSPs estimation and model calibration for each combination are repeated three times to obtain three sets of optimized CSPs. It is ensured that there are no significant differences among the three sets (P>0.05). The CSPM is then run with these three sets of optimized CSPs to obtain three sets of simulated values of the four phenological stages (FS, PS, GS, and MS). Since there are no significant differences among the three sets (P>0.05), mean value from the three sets of optimized CSPs is considered as the optimum value of the CSPs.
3. Results
The RMSE, MAE, R2, and NSE values for each combination and cultivar are shown in Table 5. Average RMSE(MAE, R2, NSE) values are 15.50 (14.63, 0.96, 0.42), 4.76 (3.92, 0.97, 0.95), 4.69 (3.72, 0.98, 0.95), and 3.91 (3.40, 0.99, 0.96) for the calibration datasets from Exp. 1 and validation datasets from Exp. 2; and 12.54 (11.67, 0.95, 0.60), 5.07 (4.61, 0.98, 0.93), 4.97 (4.28, 0.97, 0.94), and 4.58 (4.02, 0.98, 0.95) for the calibration datasets from Exp. 3 and validation datasets from Exp. 4. The simulated and measured values are displayed in Figs 2 and 3. Based on results in Figs 4 and 5, it can be observed that the RMSE (MAE) value decreases, and R2 (NSE) increases with an increase in the number of observed phenological stages used for parameter calibration.
Where, R2 of Exp. 1 denotes the R2 is obtained based on the calibration data from Exp. 1. Other symbols follow this pattern.
There are four, six, four and one class combinations of calibration data containing one, two, three, four observed phenological datasets. For the four-class data combination scenario, RMSE (MAE, R2, NSE) falls within the range of [14.05, 16.74] ([13.58, 16.35], [0.96, 0.97], [0.35, 0.54]), [4.23, 5.52] ([2.93, 4.75], [0.97, 0.98], [0.93, 0.96]), [4.43, 4.94] ([3.38, 4.08], [0.98, 0.99], [0.94, 0.95]), and [3.91, 3.91] ([3.40, 3.40], [0.99, 0.99], [0.96, 0.96]) for the calibration datasets and evaluation datasets respectively from Exp. 1 and Exp. 2, and [10.33, 14.16] ([9.61, 13.23], [0.94, 0.97], [0.49, 0.73]), [4.52, 5.85]([4.12, 5.53], [0.97, 0.99], [0.91, 0.95]), [4.49, 5.47]([3.67, 4.83], [0.97, 0.99], [0.92, 0.95]), [4.58, 4.58] ([4.02, 4.02], [0.98, 0.98], [0.95, 0.95]) for those from Exp. 3 and Exp. 4. No other obvious difference of calibration effects is found for the different combinations with the same number of observed phenological stages.
4. Discussions
Several recent studies emphasize the importance of quantifying model uncertainty components in crop models [8,44,45]. The contribution of parameters to model uncertainty has been studied and compared in relation to the structures of model inputs [9,46,47]. Parameter uncertainty arises from the uncertainty of model parameter values [48]. This uncertainty can be attributed to the data used for calibration, the limited amount of data available, and the calibration process itself [4]. Therefore, quantifying the impact of different calibration data on estimating crop parameters is vital to minimize uncertainty and improve model performance. But research in this area is largely unexplored. This work aims to fill the knowledge gap. we collect the phenological data of five soybean cultivars from four field experiments to calibrate model parameters using the GLUE method and evaluate the CSPM performance. We utilize fifteen different combinations of the four observed phenological stages for each cultivar to calibrate the CSPs of CSPM. By comparing and analyzing the calibration effects between different combinations, we can identify the rules for balancing calibration effect and computational cost in estimating crop parameters. Identifying the minimum number of observed datasets needed for model calibration will allow rational use of available resources and increase the efficiency on agriculture research.
The evaluation results show that using a larger number of observed stages for model calibration result in the lowest RMSE (MAE) and largest R2 (NSE). Combinations of observed data using one, two, three, and four phenological stages to calibrate model can provide R2 ranging from 0.95 to 0.99, with “good” effect regarding the performance of the model; those using two, three, and four phenological stages provide NSE ranging from 0.91 to 0.96, with “very good” effect; but those using one phenological stage just provide NSE less than 0.73 (Table 5). R2 is insensitive as an evaluation parameter in this work. The above analysis shows that the combinations using two, three, and four phenological stages provide similar results in RMSE (MAE, NSE) with no significant differences among them in this work. Similar results were found in other studies: generally, with the number of experiments used to calibrate the model increases, the RRMSE becomes smaller, and the best cost-benefit during the calibration was two experiments [28]. This phenomenon suggests that one set of phenological stage alone cannot adequately represent the genetic characteristics of a crop cultivar. Rather, a comprehensive collection of phenological stages can provide a better reflection of the cultivar’s genetic traits. The results indicate relatively reliable CSPs is obtained by using two or three observed phenological stages, taking into consideration both model performance and computation cost. These results would provide a variety of options of observed data in the parameter calibration of crop phenological model according to the actual conditions with the good calibration effect, which can reduce the limitations of only using the data of flowering and physiological maturity stages in previous researches [27,28]. The quantitative relationship between calibration performance and the number of datasets holds for other crops remains to be further investigated.
The major contributions and novelty of this study are summarized as follows: (1) The quantitative influences of different combinations of four observed phenological stages in soybean on calibrating the CSPs of CSPM are explored using GLUE method. (2) Comparing and analyzing the calibration effects between different combinations using different evaluation indicators of RMSE, MAE, R2, and NSE, can help us to identify the rules for balancing calibration effect and computational cost in estimating crop parameters. (3) The results would provide decision to obtain the suitable observed data according to the actual conditions and reduce the limitations of previous researches.
5. Conclusions
The GLUE method is used in this study to explore the quantitative influences of different combinations of observed phenological stages on calibrating the CSPs. Fifteen combinations of observed phenological stages from each soybean cultivar are used to estimate the CSPs of CSPM. Furthermore, independently measured data are employed to evaluate the CSPM model (Table 4). The evaluation results indicate that as the number of observed phenological data in calibration increases, the calibration effect also increases (Table 5 and Figs 4 and 5), with minor differences among the calibration combinations of two, three, and four observed phenological data. Relatively reliable CSPs for the CSPM considering the cost-benefit relation are obtained using at least two observed phenological stages.
These results can serve as guidance for studying crop model calibration, which balances model accuracy and ground-truth data collection. For future research, more observed phenological data of soybean plants need to be collected under various conditions, such as different planting dates, climate regions, nutrient levels, and water availability, in order to further verify our conclusions and to discover new rules for estimating crop model parameters.
Acknowledgments
The authors thank Dr. Yubin Yang of Texas A&M University System and Prof. Xiong You of Nanjing Agriculture University for reviewing an earlier draft of the manuscript.
References
- 1. Yao F., Xu Y., Lin E., Yokozawa M., Zhang J., 2007. Assessing the impacts of climate change on rice yields in the main rice areas of China. Clim. Change 80, 395–409.
- 2. Michel L., Makowski D., 2013. Comparison of statistical models for analyzing wheat yield time series. PLoS ONE 8, e78615. pmid:24205280
- 3. Reinmuth E., Parker P., Aurbacher J., Högy P., Dabbert S., 2017. Modeling perceptions of climatic risk in crop production. PLoS ONE 12, e0181954. pmid:28763471
- 4. Chapagain R., Remenyi T.A., Harris R.M.B., et al., 2022. Decomposing crop model uncertainty: A systematic review. Field Crop Res. 279, 108448.
- 5. Vishwakarma D.K., Pandey K., Kaur A., et al., 2022. Methods to estimate evapotranspiration in humid and subtropical climate conditions. Agr. Water Manage. 261, 107378.
- 6. Iizumi T., Yokozawa M., Nishimori M., 2009. Parameter estimation and uncertainty analysis of a large-scale crop model for paddy rice: application of a Bayesian approach. Agr. Forest. Meteorol. 149 (2), 333–348.
- 7. Wallach D., Thorburn P., Asseng S., Challinor A.J., Ewert F., Jones J.W., et al, 2016. Estimating model prediction error: should you treat predictions as fixed or random? Environ. Model. Softw. 84, 529–539.
- 8. Wallach D., Thorburn P.J., 2017. Estimating uncertainty in crop model predictions: current situation and future prospects. Eur. J. Agron. 88, A1–A7.
- 9. Tao F., Rötter R.P., Palosuo T., Díaz-Ambrona C.G.H., Schulman A.H., 2018. Contribution of crop model structure, parameters and climate projections to uncertainty in climate change impact assessments (eng). Glob. Change Biol. 24, 1291–1307.
- 10. Kersebaum K.C., Boote K.J., Jorgenson J.S., et al., 2015. Analysis and classification of data sets for calibration and validation of agro-ecosystem models. Environ. Modell. Softw. 72, 402–417.
- 11. Wallach D., Nissanka S.P., Karunaratne A.S., Weerakoon W.M.W., Thorburn P.J., Boote K.J., et al, 2017. Accounting for both parameter and model structure uncertainty in crop model predictions of phenology: A case study on rice. Eur. J. Agron. 88, 53–62.
- 12. Gao Y., Wallach D., Liu B., Dingkuhn M., Boote K.J., Singh U., et al, 2020. Comparison of three calibration methods for modeling rice phenology. Agr. Forest. Meteorol. 280, 107785.
- 13. Jones J.W., He J., Boote K.J., Wilkens P., Porter C.H., Hu Z., et al, 2011. Estimating DSSAT cropping system cultivar-specific parameters using bayesian techniques. Methods of Introducing System Models into Agricultural Research 365–394.
- 14. Ma H.J., Malone R.W., Jiang T.C., Yao N., Chen S., Song L.B., et al, 2020. Estimating crop genetic parameters for DSSAT with modified PEST software. Eur. J. Agron. 115, 126017.
- 15. Li Z., He J., Xu X., Jin X., Huang W., Clark B., et al, 2018. Estimating genetic parameters of DSSAT-CERES model with the GLUE method for winter wheat (Triticum aestivum L.) production. Comput. Electron. Agric. 154, 213–221.
- 16. He J., Dukes M.D., Jones J.W., Graham W.D., Judge J., 2009. Applying Glue for estimating ceres-maize genetic and soil parameters for sweet corn production. T Asabe 52 (6), 1907–1921.
- 17. Jin X.L, Xu C.Y, Zhang Q., Singh V.P., 2010. Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model. J. Hydrol., 383,147–155.
- 18. Marin F.R., Jones J.W., Frederick R., 2011. Parameterization and evaluation of predictions of DSSAT/CANEGRO for Brazilian sugarcane. Agron. J., 103: 304–315.
- 19. Pathak T.B., Jones J.W., Fraisse C.W., et al, 2012. Uncertainty analysis and parameter estimation for the CSM-CROPGRO-Cotton model. Agron. J. 104(5), 1363–1373.
- 20. Houska T., Multsch S., Kraft P., et al., 2014. Monte Carlo-based calibration and uncertainty analysis of a coupled plant growth and hydrological model. Biogeosciences, 11(7):19509–19540.
- 21. Sun M., Zhang X.L., Huo Z.L, Feng S.Y., Huang G.H, Mao X.M., 2016. Uncertainty and sensitivity assessments of an agricultural–hydrological model (RZWQM2) using the GLUE method. J. Hydrol. 534, 19–30.
- 22. He J., Jones J.W., Graham W.D., Dukes M.D., 2010. Influence of likelihood function choice for estimating crop model parameters using the generalized likelihood uncertainty estimation method. Agric. Syst. 103 (5), 256–264.
- 23. Hyun S., Park J.Y., Kim J., Fleisher D.H., Kim K.S., 2022. GLUEOS: A high performance computing system based on the orchestration of containers for the GLUE parameter calibration of a crop growth model. Comput. Electron. Agr. 197, 106906.
- 24. Seifi A., Ehteram M., Nayebloei F., Soroush F., Gharabaghi B., Haghighi A.T., 2021. GLUE uncertainty analysis of hybrid models for predicting hourly soil temperature and application wavelet coherence analysis for correlation with meteorological variables. Soft Comput. 25, 10723–10748.
- 25. Elbeltagi A., Seifi A., Ehteram M., Zerouali B., Vishwakarma D.K., Pandey K., 2023. GLUE analysis of meteorological-based crop coefficient predictions to derive the explicit equation. Neural Comput. Appl. 35, 14799–14824.
- 26. Menzel A., Sparks T.H., Estrella N., et al., 2006. European phenological response to climate change matches the warming pattern. Global Change Biol. 12,1969–1976.
- 27. He D., Wang E., Wang J., Lilley J., Luo Z., Pan X., et al, 2017. Uncertainty in canola phenology modelling induced by cultivar parameterization and its impact on simulated yield. Agr. Forest. Meteorol. 232, 163–175.
- 28. Liu L., Wallach D., Li J., Liu B., Zhang L., Tang L., et al, 2018. Uncertainty in wheat phenology simulation induced by cultivar parameterization under climate warming. Eur. J. Agron. 94, 46–53.
- 29. Xu J., Yan W., Zhao T., Jiang H., 2021. Genetic analysis of cultivar parameters of CROPGRO soybean flowering simulation model. J. Nanjing Agr. Univ. 44, 778–788.
- 30. Fensterseifer C.A., Streck N.A., Baigorria G.A., Timilsina A.P., Zanon A.J., Cera J.C., et al, 2017. On the number of experiments required to calibrate a cultivar in a crop model: The case of CROPGRO-soybean. Field Crop. Res 204, 146–152.
- 31. Salmerón M., Purcell L.C., 2016. Simplifying the prediction of phenology with the DSSAT-CROPGRO-soybean model based on relative maturity group and determinacy. Agr. Syst. 148, 178–187.
- 32.
Hoogenboom G., Jones J.W., Wilkens P.W., Batchelor W.D., Bowen W.T., Hunt L.A., et al, 1994. Crop models. In: Tsuji G.Y., Uehara G., Balas S. (Eds.), DSSAT V3. University of Hawaii, Honolulu, Hawaii, pp. 95–244.
- 33.
Boote K.J., Jones J.W., Hoogenboom G., Wilkerson G.G., 1997. Evaluation of the CROPGRO-soybean model over a wide range of experiments. In: Kropff M.J., Teng P.S., Agarwal P.K., Bouma J., Bouman B.A.M., Jones J.W., van Laar H.H. (Eds.), Systems approaches for sustainable agricultural development: Applications of systems approaches at the field level. Kluwer Academic Publishers, Boston, pp. 113–133.
- 34.
Boote K.J., Jones J.W., Hoogenboom G., 1998a. Simulation of crop growth: CROPGRO model. p. 651–692. In R.M. Peart and R.B. Curry (ed.) Agricultural systems modeling and simulation. Marcel Dekker, New York.
- 35.
Boote K.J., Jones J.W., Hoogenboom G., Pickering N.B., 1998b. The CROPGRO model for grain legumes. p. 99–128. In G.Y. Tsujiet al. (ed.) Understanding options for agricultural production. Kluwer Academic Publ., Dordrecht, the Netherlands.
- 36. Jones J.W., Hoogenboom G., Porter C.H, Boote K.J., Batchelor W.D., Hunt L.A., et al, 2003. The DSSAT cropping system model. Eur. J. Agron. 18, 235–265.
- 37. Boote K.J., Kropff M.J., Bindraban P.S., 2001. Physiology and modelling of traits in crop plants: implications for genetic improvement. Agric. Syst. 70, 395–420.
- 38. Boote K.J., Jones J.W., Batchelor W.D., Nafziger E.D., Myers O., 2003. Genetic coefficients in the CropGro–soybean model: links to field performance and genomics. Agron. J. 95, 32–51.
- 39. Fehr W.R., Caviness C.E., Burmood D.T., et al., 1971. Stage of development descriptions for soybeans, Glycine max (L.) Merrill. Crop Sci.11(6): 929–931.
- 40. Beven K.J., Binley A.M., 1992. The future of distributed models: model calibration and uncertainty prediction. Hydrol. Process. 6, 279–298.
- 41. Vishwakarma D.K., Kumar R., Abed S.A., et al., 2023. Modeling of soil moisture movement and wetting behavior under point‑source trickle irrigation. Sci. Rep. 13, 14981. pmid:37696862
- 42. Markuna S., Kumar P., Ali R., et al., 2023. Application of Innovative Machine Learning Techniques for Long-Term Rainfall Prediction. Pure Appl. Geophys. 180, 335–363.
- 43. Mirzania E., Vishwakarma D.K., Bui Q.A.T., Band S.S., Reza Dehghani R., 2023. A novel hybrid AIG‑SVR model for estimating daily reference evapotranspiration. Arab. J. Geosci. 16, 301.
- 44. Asseng S., Ewert F., Rosenzweig C., et al., 2013. Uncertainty in simulating wheat yields under climate change. Nat. Clim. Change 3, 827–832.
- 45. Ewert F., Rötter R.P., Bindi M., et al., 2015. Crop modelling for integrated assessment of risk to food production from climate change. Environ. Model. Softw. 72, 287–303.
- 46. Wallach D., Goffinet B., Bergez J.-E., Debaeke P., Leenhardt D., Aubertot J. N., 2002. The effect of parameter uncertainty on a model with adjusted parameters. Agronomie 22, 159–170.
- 47. Tao F., Zhang Z., 2013. Climate change, high-temperature stress, rice productivity, and water use in eastern China: a new super ensemble-based probabilistic projection. J. Appl. Meteorol. Climatol. 52, 531–551.
- 48. Ramirez-Villegas J., Koehler A.-K., Challinor A.J., 2017. Assessing uncertainty and complexity in regional-scale crop model simulations. Eur. J. Agron. 88, 84–95.