Panel data regression with fixed effects

Our goal is to obtain empirical evidence of the influence of climate variables on a target economic variable . All variables are indexed by two dimensions: One is spatial (here: countries), denoted by . The other one is temporal (here: years), denoted by , where is a set of consecutive time points. Thus, the data forms a panel and panel data analysis methods can be applied. We do not assume that the panel is full, i.e., we may only observe certain region–time combinations for some index set . Denote the total number of observations as .

To achieve our goal, we apply a linear regression model of the form

(1)

Here are features derived from the climate variables. These can, e.g., be transformations like or ; or time lags like , , ...; or interaction terms like . The terms are fixed effect variables, e.g., dummy variables for a given time point or time trends for a given region . The parameters , are unknown. They differ in that we are interested in statistical inference on , whereas are nuisance parameters that model unobserved effects that we want to control for but have no further interest in. The errors are modeled as centered random variables, which are normally distributed or fulfill moment conditions that allow to use asymptotic normality in later analysis. In matrix notation, we have

(2)

where

(3)

(4)

(5)

Aside from statistical inference on , we are also interested in variable selection to identify models with high predictive power for unseen observations. While our primary metric is predictive accuracy, this framework supports causal interpretation due to the strict exogeneity of weather shocks: short-term economic fluctuations do not influence concurrent climatic conditions. Consequently, a model that robustly predicts out-of-sample is plausibly capturing stable structural relationships rather than spurious correlations, thereby validating the causal links necessary for future projections. In order to find such a model, we need the set of selected features to be as small as possible while explaining as much as possible of the variation of the target . Model selection is explored in Step 3: Model Selection via Out-of-Sample Test.

Year fixed effects cannot be estimated for unseen years, and time trends extrapolated forward are not guaranteed to remain predictive. With such time controls, the model is not intended to predict the absolute value of ; the goal is to identify the time-persistent influence of the climate features on .

Data

Our primary data source is the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP). Specifically, we utilize climate variables from the GSWP3-W5E5 dataset [26], the tropical cyclone data [27], as well as GDP (PPP) [28] and country-level population data [29]. In addition, we incorporate gridded population data from the History Database of the Global Environment (HYDE), version 3.3 [30]. To assess the robustness of our findings with respect to economic data sources, we also employ World Development Indicator GDP data [31]. All datasets are restricted to the period 1960–2018, and our analysis covers a total of 203 countries.

Climate econometric model

Our target variable in the regression is the economic growth rate , where is the gross domestic product per capita in country s and year t either using the ISIMIP or the World Bank data.

Most of the predictor variables we use, are based on the 11 climate variables given in Table 1. All variables are first given as daily gridded () values. For each grid cell, they are aggregated to yearly values using the mean, standard deviation, minimum, and maximum. This results in 44 variables indexed by grid cell and year. We have additional 12 variables derived from and (Table 1) by calculating the deviation from a rolling window of 11, 31, and 91 days (, , , , , ) and the rate of extreme events larger than , , of the whole observation time distribution (, , , , , ). All yearly gridded values are aggregated to country means using population as grid weights. Furthermore, we add the number of people affected by tropical cyclones of at least 34 knots, 48 knots and 64 knots (, , ) to our mix of variables. We arrive at 44 + 12 + 3 = 59 variables.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Climate-related base variables from GSWP3-W5E5 and their descriptions.

https://doi.org/10.1371/journal.pclm.0000962.t001

For each of the 59 base variables, we create linear and squared terms and up to 5 lags. This leaves us at predictors to choose from. We do not include lagged growth-rate terms on the right-hand side, following the standard specification in the climate-econometrics literature [4,3]. Instead, we rely on weather variables and time-dependent controls to capture dynamic effects.

The country and year fixed effects together with the time-trend controls absorb a large part of the systematic economic variation in growth—persistent cross-country differences in development and growth regimes, globally common shocks, and slowly evolving country-specific trajectories—so that climate is evaluated against the residual, higher-frequency variation. Supporting a causal interpretation of the influence of climate on GDP would rest on the assumption that year-to-year climate anomalies are exogenous to contemporaneous economic activity. This assumption is plausible in our setting: the long-run channel through which economic activity affects climate via emissions operates at low frequency and is absorbed by our nonparametric time-trend controls.

We have all variables in a base form () and a difference form (), leading to two different kinds of regression models. We follow the terminology of Newell et al. [9] and distinguish a growth and a level regression, where the labels refer to whether the climate level is related to GDP growth or to the GDP level. In the growth regression, the climate predictors enter in original (level) form and relate directly to the growth-rate target . In the level regression, the predictors enter in differenced form ; since the target is also a difference, this is the differenced form of a relationship between the climate level and the GDP level , which is the source of the “level” label, even though the climate level itself does not appear explicitly in the estimated equation. We do not mix original and differenced predictors within the same model and do not delve into the interpretation of the two modeling strategies here. Instead, we refer readers to Newell et al. [9] for a detailed discussion and present results for both approaches.

Summary statistics of all variables are given in Section A.1 in S1 Appendix. We verify in Section A.2 in S1 Appendix that the GDP growth-rate target and the differenced climate variables are stationary, and that the original climate variables show no detectable non-stationarity once our sufficient time-trend controls are included. Both modeling strategies therefore yield balanced regressions, so the choice between them is not pinned down by stationarity considerations and is best interpreted in economic terms, as discussed in Newell et al. [9].

Outliers

Outliers in the data can substantially distort results, potentially overshadowing true effects or introducing spurious findings that lack causal basis. Broadly speaking, outliers are data points that appear highly atypical or unexpected relative to the statistical model. We categorize outliers into two primary types here: mistakes and nuisance events.

Mistakes refer to incorrect data points, often due to issues such as instrument failure or transcription errors. As an example, version 2 of the DOSE data set [32] (MCC-PIK Database Of Sub-national Economic Output) showed a decline in economic production per capita in Madrid, Spain, of almost 90% from 1994 to 1995. In version 4, there is instead an increase by 5%. The steep decline in the old version might be due to a wrongly placed decimal point. See Table K and L in S1 Appendix for more details on this example. Another mistake is the in Oman in 1965 in the World Bank data, where it is reported to grow by a factor of 10⁶ (by many orders of magnitude the largest value in the dataset and clearly wrong). We reported this issue on February 14, 2025, which led to a correction of this mistake in a subsequent version of the data.

In contrast, nuisance events are data points that accurately reflect reality but arise from rare, extreme non-climate events, which are not captured by the model and yet can be understood through external knowledge sources. For example, the ISIMIP economic data shows a decline of economic production per capita in Iraq in the year 1991 by 65%. This is an extreme event in the data compared to 90% of the data having growth values between -5.0% and 7.8%. Furthermore, it can be attributed to a non-climate event: the Gulf War. See Table M in S1 Appendix for more details on this example.

Off-distribution data

Another type of data that can disrupt a statistical analysis is Off-Distribution Data, which refers to data that does not originate from the distribution under study. This type of data often enters a dataset through processes like imputation, where missing values are filled with proxy estimates derived from other available information instead of direct measurements. While such data can be valuable for maintaining dataset completeness, it may also substantially skew the results of a statistical analysis. Unlike outliers, Off-Distribution data points typically have values that align with the general range of the target distribution, making them less conspicuous. However, they often exhibit distinct statistical properties, such as reduced variance or artificially inflated correlations with other variables. These discrepancies can introduce bias or lead to invalid conclusions in the analysis.

For instance, consider the ISIMIP GDP data for post-Soviet states, which spans from 1960 to 2021, including periods before these states existed independently of the Soviet Union. In the dataset, economic growth data for the years 1960–1985 is imputed with a linear method, see Fig A in S1 Appendix. This imputation introduces an artifact where the correlation coefficient between any two post-Soviet states for the economic growth rate equals 1. Consequently, there is no independent statistical information about economic changes of individual states during this period beyond what is reflected in the aggregated output of the Soviet Union. See Finding and Treating Off-Distribution Data for a more comprehensive correlation analysis.

Example of the impact of outliers

As an example, we use the following model to find the economic impact of tropical cyclone and changes in mean annual surface temperature:

(6)

where, for region s and year t, , is the mean (population weighted) annual near-surface air temperature (tas), and is the mean ratio of people affected by a tropical cyclone (TC) with ℓ lag years.

Our initial results indicate that lags of tropical cyclone events are significant, while temperature predictors are not. Through influence analysis, we identify the 2002 observation from the U.S. Virgin Islands (VIR) as exerting the greatest influence on the lag terms with a growth value of +48.7%, which ranks as the 11th highest out of 10752 observations. This value seems to be an artifact of the dataset; likely a change in source data between 2001 and 2002 (the World Bank GDP time series for U.S. Virgin Islands starts at 2002) rather than a real event: According to the US Virgin Islands Bureau of Economic Research, there was some weakening in the performance of the US Virgin Islands economy during 2002, see https://usviber.org/wp-content/uploads/2022/07/EconReviewJan2003.pdf. Removing this single influential observation changes the statistical significance of the lagged effects strongly, and with all outliers and off-distribution data excluded, surface air temperature and its quadratic term become significant. See Table 2 for details.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Coefficients and p-values for applying model (6) to different subsets of the ISIMIP dataset. Raw is the whole dataset; for Clean 1, VIR 2002 is removed; for Clean All, the cleaned economic data as described in A Cleaned Dataset is used.

https://doi.org/10.1371/journal.pclm.0000962.t002

Treating outliers

A straightforward approach to detecting outliers involves inspecting the largest and smallest values of each variable; unusually high or low values may indicate potential outliers. In connection with further knowledge about the data, we can identify outliers. For example Table 3 shows the largest and smallest GDP growth values in the ISIMIP dataset. The largest decline in GDP is observed in Iraq in 1991. This extreme value can be attributed to the Gulf War and thus be counted as an outlier (it is extreme and not climate related).

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. For the ISIMIP economic data, the most extreme changes in as identified by . For a translation of the ISO country code to a country name, see Tables N to P in S1 Appendix. Less than 1% of all 15,189 growth values in the dataset are below −12% and less than 1% are above 14%.

https://doi.org/10.1371/journal.pclm.0000962.t003

In this study, we manually review potential outliers characterized by extreme growth events and investigate their causes through online searches. If a non-climate-related explanation for the extreme growth is identified, we remove the corresponding data point from the dataset. The resulting dataset is described in A Cleaned Dataset.

It is crucial to emphasize that we do not begin by performing a regression analysis to identify points with high residuals or those that might contradict our hypothesis. Such an approach could risk (either intentionally or unintentionally) introducing p-hacking. Instead, we base the exclusion of data points on external, independent information, ensuring that the removal decision is not influenced by its effect on the regression results.

Alternative treatments of outliers

Additionally, external datasets can be utilized to supplement information when a given data point is deemed unreliable. For instance, one might use a conflict dataset, such as the Uppsala Conflict Data Program [33], to exclude data points associated with armed conflicts of a certain scale. While this approach has the merits of a more formal and less laborious procedure than checking extreme values by hand, it is not without limitations. In particular, the time frame or the magnitude of the conflict and the economic signal too often do not align. Overall, although such datasets can provide valuable insights, the results lack sufficient precision, leading us to forgo this approach.

Another alternative approach is robust regression, which automatically down-weights data points that deviate substantially from the model. However, this method has a key drawback: extreme weather events, which often produce such outlier data points, are precisely the phenomena we aim to analyze. Excluding or down-weighting these points would mean losing valuable information, even if retaining them increases variance and uncertainty in the results. Additionally, applying robust regression to a large dataset with many predictors can be computationally expensive and numerically unstable. See Section C in S1 Appendix for an example.

Instead of removing outliers, one could introduce a dummy variable, such as one indicating the presence of armed conflict. However, because the economic consequences of armed conflict vary substantially across cases, this approach would not fully address the outlier characteristics of the affected data points.

Alternatively, one might choose to ignore outliers altogether, assuming they are neither numerous nor severe enough to substantially influence the results. However, this approach is only valid if an analysis demonstrates that the impact of outliers is indeed negligible. As shown above, outliers can have a substantial effect on the results, making this a risky assumption.

Finding and treating off-distribution data

We focus on identifying off-distribution data in the economic target variable of the ISIMIP dataset. To achieve this, we perform two key analyses: a correlation-based clustering of time series to detect unusual patterns and a search for constant segments within the data. Based on these analyses, we identify and remove off-distribution observations.

Specifically, we compute the correlation matrix between each pair of economic time series of the form , , where is the economic growth rate for country s in year t. Using these correlations, we perform hierarchical clustering, specifically the unweighted pair group method with arithmetic mean (UPGMA), to identify clusters of highly correlated countries.

Since imputation is often applied to segments of time series data, we compute correlations for each time interval within the dataset’s full time range, ensuring a minimum interval length of 7 years () to obtain reliable results. Additionally, to minimize false positives due to spurious correlations, we only consider clusters with extremely high mean correlations. The resulting clusters, with mean correlations close to 1, are shown in Table Q in S1 Appendix.

The ISIMIP dataset also includes time series with segments of exactly constant economic output, which are likely the result of imputation. These segments, having zero variance, produce undefined correlations with other variables, making correlation analysis inapplicable. To address this, we identify all segments where the time series remains exactly constant for at least 3 years (zero growth over at least 2 years) and mark them as off-distribution. These segments are detailed in Table R in S1 Appendix.

Using these findings, along with additional historical context, we identify and remove observations classified as off-distribution, as described in A Cleaned Dataset.

A cleaned dataset

To obtain a cleaned economic dataset from the original ISIMIP data, we have marked: 182 observations as (non-climate related) outliers by checking the most extreme growth events individually; 2260 observations as imputed because of highly correlated time series segments; 1046 observations as imputed because of constant time series segments. Additionally, we remove post-Soviet and post-Yugoslavia observations up to 1996 (756 observations), as they are partially imputed and partially reflect political changes so that we deem them uninformative on climate events. From the original 15189 values, we remove 3422 observations in total, which is less than the sum of the number of elements in the previously mentioned categories as those intersect. Thus, the final dataset has 11767 values.

The power of Frisch–Waugh–Lovell

The number of control parameters for may be large while the total number of observations may be too small to accurately estimate their values. Fortunately, the Frisch–Waugh–Lovell (FWL) theorem demonstrates that this limitation has minimal impact on the accuracy of the estimator of the parameters of interest, .

Recall the general regression setup introduced in Panel Data Regression with Fixed Effects. By the FWL theorem, the least squares estimate of the parameters of interest in the model is equivalent to the least squares estimate of in the transformed model

(8)

where the transformed variables and are obtained by removing the components explainable by . Specifically, and , with M being the linear projection onto the complement of the column space of :

(9)

where is the identity matrix.

This result not only allows an accurate estimation of without precise estimation of , but also offers substantial computational advantages when analyzing panel data with many control variables. Given the raw data , we can streamline the computation by first creating the features to analyze. Next, we apply a control variable design and transform the target and features to obtain the controlled dataset . This preprocessing step decouples the estimation of from the high-dimensional controls, substantially reducing computational complexity.

This approach is especially advantageous when the dimension q of is large relative to the dimension p of . For instance, in the example described in (6), we have 225 countries and 61 years, requiring q = 733 control parameters if quadratic time trends are included. In contrast, the parameter of interest, , has dimension p = 8. While q may still be manageable for a single regression, scenarios involving bootstrap or cross-validation (as discussed in Step 3: Model Selection via Out-of-Sample Test) often require thousands of regressions with the same controls. In such cases, the two-step procedure enabled by the FWL theorem provides immense computational benefits. Furthermore, covariance matrices needed for confidence intervals and hypothesis tests can be computed in the dimensions of the transformed model instead of dimensions of the original model equation (if small sample corrections are adjusted correctly).

Local time trends

To account for the long-term time trend of each country, the function in (7) must be specified. A common approach is to model as a polynomial in t, with the degree typically ranging from 0 (constant, capturing only the country-specific mean growth) to 3 (cubic, allowing for up to two changes in direction). While low-degree polynomials capture only simple trends, higher degrees may inadvertently remove meaningful information that should be attributed to the parameters of interest.

However, polynomial time trends have notable drawbacks. First, practitioners must choose the polynomial degree, and there is no universally accepted rule for making this decision. This may (involuntarily) lead to p-hacking when the choice of polynomial degree is influenced by the desired claim of the research article. Second, even the best-fitting polynomial may fail to adequately approximate the time trend for all regions. Third, polynomials can also introduce artificial long-distance correlations, which may distort the results (see Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Effect of different control variable designs on temporal correlation of residual economic growth after controlling.

Each black dot is an empirical correlation value between two different years calculated from the ISIMIP economic data. The gray curve shows a Loess smoother of these correlations. For each fixed temporal difference, we test whether the correlations values for pairs of years with this given difference are significantly different from zero via a conservative permutation test, see Section D in S1 Appendix. Furthermore, we apply the Benjamini-Hochberg correction (with false discovery rate level of 0.05) to account for the multiple testing issue. Blue and red triangles show correlations significantly different from zero, whereas green squares indicate that non-zero correlation cannot be detected. MAC is the mean absolute correlation. Without controls (None), we observe a high positive correlation for more than 20 years. If only year fixed effects are applied, the result is exactly the same as this only changes the means of the correlated variables. All polynomial time trends are able to reduce the positive correlation of small temporal difference, but introduce negative correlations of temporal differences between 3 and 14 (and more) years. Both local control designs remove correlations in later years. Only (mostly negative) correlations remain for temporal differences of six to seven years. Overall the Kernel-based method seems to show the best results.

https://doi.org/10.1371/journal.pclm.0000962.g001

Rather than relying on polynomials, we propose a more flexible nonparametric approach that directly embodies the idea of a typical value for a given time period—longer than a single time point but shorter than the entire time interval. Specifically, we use a local linear estimator for the time trend. We refer to this method as Window and Kernel, respectively, depending on whether all observations are weighted equally or according to a quadratic kernel. These local linear methods resemble a rolling window approach (local constant) but offer statistical advantages, particularly near the edges of the time interval [34]. An alternative nonparametric time trend approach using basis functions instead of local regression can be found in Kneip et al. [35].

As with the rolling window method, the local linear estimator requires a parameter to define how much of the surrounding data influences the estimate at each time point. This parameter, known as the bandwidth, determines the threshold for considering neighboring time points when calculating the trend. To make the method fully automatic, we propose using leave-one-out cross-validation with the economic response variable as target to select the optimal bandwidth. The automatically selected values are such that 10–12 years before and after a given year are used to calculate the local time trend.

Figs 1 and 2 contrast the temporal and spatial correlations, respectively, resulting from different control variable designs. The local linear kernel approach yields the least systematic temporal and spatial correlations and does not introduce spurious correlation artifacts over extended time periods. Furthermore, using the automatic bandwidth choice, we remove further control design decisions ensuring objectivity of results.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Effect of different control variable designs on spatial correlation of residual economic growth.

Plotting details and significance testing are the same as in Fig 1, except that the horizontal axis shows the distance between country centroids. Significance tests are performed on 40 bins of spatial distances with roughly equal numbers of pairs. Results: Without controls (None), we observe a high positive correlation for any spatial distance. As soon as year fixed effects are applied, only low distance positive correlations remain, but some negative correlations for larger distances get newly introduced. Adding polynomial or local time trends to the controls reduces the negative correlations. From this perspective, local time trends or polynomial time trends of degree at least 1 should be preferred.

https://doi.org/10.1371/journal.pclm.0000962.g002

Correlations and standard errors

Our investigation of correlations focuses primarily on the economic growth variable and its residual values after controlling for global shocks and regional time trends. This approach is a justified heuristic approximation: after controlling, the estimated residuals are nearly identical to the residual of a full regression model with climate variables. This holds because, in our application, no climate variable explains a large portion of the variation in economic growth. Consequently, a preliminary correlation analysis for the climate variables is not crucial. Nonetheless, should we identify a highly predictive variable, it would be appropriate to redo the analysis of correlations based on the residuals from the final regression model.

Methods

The controlled data is split into training (pre-2007) and testing (2007 onward) subsets, yielding 41 years of training and 11 years of testing data. To assess robustness, we also evaluate alternative splits using 1997 and 2002 as threshold years. We avoid shorter test periods, as they reduce the reliability of evaluation statistics. The split at a threshold year maintains spatial dependencies and reduces the influence of temporal auto-correlation. Furthermore, it exactly mimics the main use case of the model results in which economic damages are projected into the future.

This design reduces the influence of temporal auto-correlation but does not eliminate it: after residualizing against the controls, the data may retain some short-range dynamic dependence, and the earliest test years in particular share information with the adjacent training years. Such residual dependence would tend to make the test error slightly optimistic. We do not introduce a temporal gap (“padding”) between the training and test sets to suppress this further, for two reasons. First, with only about 50 annual observations per country, the test set must remain large enough to detect an effect in this highly noisy setting; padding would consume scarce observations. Second, padding introduces an additional hyperparameter with its own trade-offs. We retain the simpler split because our central results are negative: most models show little to no predictive power even in this potentially favorable setting, so any optimistic bias from residual dependence works against our conclusions rather than inflating them.

Most selection methods involve tuning hyperparameters, which we optimize via five-fold cross-validation on the training set. Folds are composed of consecutive years (for the 2007 train-test split: 1966–1974, 1975–1982, 1983–1990, 1991–1998, 1999–2006), ensuring that all data from a given year remains in the same fold. This design again respects both spatial correlation and reduces the influence of temporal auto-correlation.

We emphasize that cross-validation here serves only to tune hyperparameters; our reported conclusions rest on the held-out test set, evaluated in temporal order. The fold structure thus affects only model tuning and not the validity of the final out-of-sample assessment.

Once optimal hyperparameters are identified, each model is retrained on the full training set and used to predict the (controlled) economic response in the test set. We assess the predictive pseudo-significance of each model by benchmarking its performance against a random prediction independent of any predictor variables and modeled by an isotropic Gaussian random vector with optimally chosen variance. See Section F in S1 Appendix for details.

To assess the stability of predictor selection, we employ a block bootstrap approach with yearly blocks, resampling the training data 1000 times. Each resample includes a number of years equal to the original training period, drawn with replacement. When a year is drawn multiple times, all data from this year appears multiple times in the resampled dataset. For each bootstrapped sample, we reapply the selection procedure (but not the hyperparameter tuning) and track how often the originally selected predictors are re-selected. This re-selection rate serves as an indicator of the robustness of the variable selection process.

Note that all of these calculations happen on the controlled data. This is necessary, as we cannot estimate the controls of the test data using the training data, and all analysis has to be carried out orthogonally to the controls. We expand on this issue and compare our framework with that of Newell et al. [9] in Section G in S1 Appendix.

LASSO, ridge, and elastic net.

The LASSO and Ridge Regression penalize the absolute values of the regression coefficients so that small absolute values are preferred. For a hyperparameter , the respective coefficient estimates can be written as

(10)(11)

The penalties allow these methods to be applied in settings with a large number of coefficients compared to the sample size. Meaningful estimates are possible even in settings with more coefficients than observations. The different penalties between LASSO (L¹) and Ridge (L²) yield different coefficient estimates: LASSO prefers to set many coefficients to exactly 0 (thus implying a selection of variables), whereas Ridge typically does not estimate any coefficient to be exactly 0 but all coefficients will be shrunken sufficiently close to 0. Note that both methods tend to estimate coefficients with lower absolute values than OLS, i.e., they imply a form of shrinkage. The Elastic Net combines both LASSO and Ridge Regression:

(12)

The penalty parameter and the trade-off parameter for ElasticNet are chosen using 5-fold cross validation on the training set with the folds consisting of all data from mutually exclusive consecutive years. Ensuring promotes ElasticNet to set some coefficients to exactly 0 as happens in LASSO implying variable selection.

Test error

All experiments.

First, we consider the mean squared error (MSE) on the test set and its pseudo-significance (see Section F in S1 Appendix) for our six methods: Ridge Regression, LASSO, Elastic Net, Pair Selection, Forward Selection and Fordge. Apart from different methods, we also compare raw vs cleaned data, ISIMIP vs World Bank GDP, 8 different control designs, level vs growth regression, and three different options for the threshold year that splits our data into train and test set. In total, this makes 1152 experiments.

For binary attributes the results are summarized in Table 4. Generally, the cleaned data is clearly easier to predict (in 197 cases with 337 draws and 42 opposite cases). This is to be expected as outliers are difficult or even impossible to forecast. The World Bank data seems somewhat more predictable than the ISIMIP data (in 136 cases with 369 draws and 71 opposite cases), but the difference is not large. Level and growth regression have similar predictability (359 draws, 93 cases where level is more predictable, 124 growth).

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Summary of experiments along different attributes. We evaluate the binary attributes preprocessing, regression design, and GDP data source by comparing the p-value on the test set for both instances of the attribute and counting how often each instance of the respective attribute has a lower value than the other instance. The high number of draws (both versions perform equally) is due to the p-value being set to 1 if the method’s test error is larger than the variance of the test data, i.e., the test error of the trivial constant prediction.

https://doi.org/10.1371/journal.pclm.0000962.t004

Predictive power slightly improves when increasing the train-test split year from 1997 to 2007: The mean test error ranks among the three threshold choices are 2.07 (split in 1997), 1.97 (split in 2002), 1.95 (split in 2007), respectively. Furthermore, the number of experiments with any predictive power (test error lower than for the trivial model without predictors) increases with later split year. Out of 384 experiments nonzero predictive power is shown in 82 (split in 1997), 96 (split in 2002), and 112 (split in 2007) experiments, respectively. Both results seem to be a consequence of better estimates due to more training data for later split years.

The average predictability of data with different control does not differ a lot with Window having the highest predictability. Note that high predictability is not necessarily a quality criterion for the control design.

The test error results for different model selection methods are collected in Section H in S1 Appendix, which we further summarize in Table 5. Ridge regression shows the highest predictive power followed by Elastic Net and LASSO, while Pair Selection, Fordge and Forward Selection show worse performances.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Summary of performance of different methods according to their test error rank (1 to 6) averaged over different experimental settings. All comparisons are made from all 1152 experiments. For the main comparisons we restrict to cleaned data, kernel controls, and 2007 as the split year and vary only between level and growth regression designs and ISIMIP and World Bank as GDP data sources.

https://doi.org/10.1371/journal.pclm.0000962.t005

Selected variables

To evaluate which among the 708 variables in our regression are most important, we focus on the main experiments (cleaned data, Kernel controls) considering level and growth regression and ISIMIP as well as World Bank data sources for GDP. Furthermore, as we here do not need a test set for this evaluation, we use all data for training, i.e., until 2018, to get the most accurate results. We compare the four out of six model selection methods that explicitly select variables (Forward, Pair, LASSO, Elastic). The results are summarized in Section I in S1 Appendix.

There is only a single variable that gets chosen systematically (except by Forward on growth regression, which does not select any variable): The square of the maximal specific humidity of the previous year, (or in the level regression case). This variable also has the highest bootstrap re-selection rate among all chosen variables in all settings. Reasons for the relatively high predictive power could be the correlation of high specific humidity to heat stress, storms, and flooding, as well as correlation of low specific humidity with droughts, water shortages, wild fires, respiratory issues and airborne pollutants. Moreover, the squared term may capture the non-linear influence arising from threshold effects where impacts on human health or agriculture escalate disproportionately.

No other variable is chosen robustly. In particular, classical variables such as , , , are never chosen by any algorithm but some transformations of and do show up for some settings without strong consistency.

Shrinkage

LASSO, Ridge Regression, and Elastic Net typically perform better than the other three methods and Fordge is often slightly better than Forward Selection. This suggests that penalized regression in general has an advantage in our setting. To investigate this hypothesis further, we select specific variables either classical , , , or preferred by our variables selection methods (, ) and use OLS as well as LASSO and Ridge fits for these variables. 6 shows the results. In all cases the shrinkage of coefficients implied by the penalty of LASSO and Ridge helps to achieve lower test errors.

Stationarity

In our regression analysis, we model the proportional influence of shocks in the climate variables on the economic growth rate to be constant over time and space (temporal and spatial stationarity). This, of course, is an approximation. Even if this does not perfectly reflect the real world, such a model may still be useful, as it allows us to estimate an average effect. But this approximative nature of our analysis has to be kept in mind when interpreting the results or using them for projections into the future.

To ensure validity of future projections based on such regression results, we do require temporal stationarity: The weather–economy relationship should be stable over the observation time to reasonably assume that this relationship also exists in the future. We investigate this property for models involving the classical temperature parabola [4] and models involving the humidity-variable favored by our variable selection analysis. With an F-test, we check whether the data before a threshold year exhibits a similar regression coefficient to the data thereafter, see Fig 3. Most models show significantly different effects in the first time period compared to the later time period, if the threshold year is chosen between 2000 and 2010 indicating a change point within these years. Only the pure specific humidity model (huss A) for level regression is stable over the whole training period. Adding maximal wind surface speeds to the level regression humidity model (huss B), seems to exhibit a change point before 1980 but stability thereafter. These two models show the greatest temporal stationarity, which is consistent with their low test errors in Table 6.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. For a selected number of different sets of predictor variables, we calculate the OLS, LASSO and Ridge fit, using 5-fold cross validation to determine the penalty parameter for the latter two. We use the cleaned ISIMIP data with Kernel controls and split it into train and test data in 2007. The columns MSE% show the respective change in percent in mean squared error on the test set compared to the trivial null-model. The columns vs OLS show the change of MSE% from OLS to LASSO and Ridge, respectively.

https://doi.org/10.1371/journal.pclm.0000962.t006

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. p-values of F-test for stationarity of effects.

We split the data (Kernel controls, cleaned ISIMIP economic data) into two parts: up until the year indicated on the horizontal axis and afterward. Then, for both parts, we fit the OLS models indicated by the color of the displayed curves, see Table 6. We test the Null-hypothesis of equal coefficient vectors for both parts using an F-test for linear hypothesis and the Driscoll-Kraay covariance matrix with 8 lag years. The p-value of the test is shown via the vertical axis, which is on a log-scale and cut at 10^-4. Using different symbols, we indicate different significance levels (their threshold is displayed by black horizontal lines). We only show threshold years from 1977 to 2007, even though the data ranges from 1967 to 2018, to ensure that there is enough data on each side of the threshold year to make the results reliable. Note that there is a shift in the source data of ISIMIP between 1978 and 1979, which may explain the low p-values for some models around that time.

https://doi.org/10.1371/journal.pclm.0000962.g003

Investigating the change of coefficients in the temperature parabola models further reveals that the quadratic influence of temperature on the economic growth seems to strongly reduce after 1992, see Fig B in S1 Appendix. Thus, when measuring predictive power with a test set of some years after 1993, the quadratic relationship learned from previous years is not predictive. This instability is reason for concern when projecting economic influence of temperature to the future. Coefficient values of the level regression humidity model (huss A), seem slightly more stable, in particular when estimated over time periods of at least 15 years.

Spatial stationarity appears less critical than temporal stationarity, as we are typically projecting outcomes for existing countries rather than entirely unseen ones. If the goal is merely to estimate the average effect of weather on the economy, spatial stationarity is not required. However, to make country-specific projections, spatial stationarity either must be demonstrated or it must be clearly stated that the projections do not reflect country-specific economic structures. Instead, they represent average outcomes for a hypothetical country with similar climatic conditions but globally averaged economic characteristics—ignoring factors such as the level of industrialization or the share of agriculture in economic output of the specific country.

It is also important to note that calculating global GDP impacts represents a form of averaging over countries, though with a different weighting scheme than that used in model estimation. While the model gives equal weight to each country—regardless of economic size, population, or geography—global damage estimates are typically dominated by the losses of wealthier countries due to their larger contributions to global GDP.