We want to start our response to the comment above by pointing out that we neither think that the study of Bentley et al. 2014 is uninteresting, nor that the theoretical argumentation presented in the paper is implausible. Our critic concerns the fact that in the Bentley et al. 2014 paper, the authors use standard statistical models that can be used for cross-sectional data but that do not account for the within-series error-dependency that is typical for the analysis of time series data. This is not just an opinion that comes from reading the Bentley et al. 2014 paper, but is based on a re-analysis of the original data that was provided for us directly by Bentley et al. 2014. We personally communicated the results of our re-analysis and discussed potential alternative methods (e.g. changes instead of levels) with Bentley et al. before we wrote this paper.

Let us now demonstrate why we think that we did not “misrepresent” their study in our paper.
First of all, one can argue that even if the comment of Bentley, Ormerod, Lampos & Acerbi was correct, this would not imply that we have “misrepresented” the Bentley et al. 2014 study, because in the comment the authors actually claim that “using several additional statistical tests […] the variables are indeed non-stationary in level form, but are stationary in first-difference form”. However, in the original Bentley et al. 2014 study, the variables are treated as non-stationary in the level form. In our opinion, this alone should lead to a correction of the Bentley et al. 2014 study, given the fact that, as we argue in our paper, this transformation also re-formulates the research question, and, as Bentley et al. 2014 demonstrate in their comment above, affects all presented results, e.g. the model fit and levels of statistical significance. However, as we will demonstrate in what follows, the general model presented in the comment above actually points in the opposite direction: personal communication with one of the authors of the comment (P. Ormerod) revealed that the original LM (“literary misery”) index was accidentally reversed in the opposite direction for the general model presented in the comment above. This implies that instead of a positive correlation, the authors of the comment present evidence for a negative correlation of year-to-year changes of (lagged values of) EM (“economic misery”) on year-to-year changes of LM. Of course, this has major consequences for the Bentley et al. 2014 study that – in our opinion – need to be addressed by the authors of the paper.

As written above, in the original Bentley et al. 2014 study, all variables are (implicitly) treated as non-stationary. If we regress the level of LM on the level of EM (“economic misery”) as Bentley et al. 2014 did, we indeed find a correlation of r = .25 (at p < .05). However, as Durbin’s alternative test for autocorrelation shows, there is strong reason to reject the null hypothesis of no serial correlation for all lags from 1 up to 4 at all standard levels of significance (all p-values < .0001). Similar observations can be made, if we regress LM (i) onto the seventh lag of EM or (ii) onto the 11-year moving average of EM as constructed by Bentley et al. 2014, in all cases up to a lag of 4, there is a strong indication of temporal autocorrelation (all p-values < .0001). Again, we believe that this problem in the original Bentley et al. 2014 study needs to (and can) be addressed. This is not a matter of debate; it is a matter of statistical necessity that has substantial consequences: for example, a regression with Newey-West standard errors that are robust to arbitrary autocorrelation up to lag 10 reveals that if we control for serial correlation, the effect of EM on LM is rendered insignificant (p = 0.161). Or, if we do the opposite of Bentley et al. 2014 and regress EM onto the 20th lag of LM, again with Newey-West standard errors, we find that LM significantly (at p < .01) predicts EM with a negative correlation between both series of r = -0.392. To sort out that apparent inconsistency in the argumentation of Bentley et al. 2014, more formal tests (e.g. for Granger causality) would be needed, because as the authors themselves point out in the comment above: “Indeed, had it been the other way around, this would have falsified our hypothesis…“

In addition, the authors of the comment present a new general model where they look at year-to-year changes instead of levels to analyze the potential influence of EM on LM. We re-analyzed this model and obtained:

diff(LM(t)) = - 0.00982 - 0.197* diff(LM(t-1)) - 0.00871*diff(EM(t-2)) - 0.0134*diff(EM(t-11))

This model is similar to the equation presented in the comment above but with opposite signs (standard errors and the adjusted R2 value are identical). As written above, this is due to the fact that the original LM index was accidentally reversed in the opposite direction for the general model presented in the comment above. Thus, while the original Bentley et al. 2014 paper is flawed due to autocorrelated errors that violate statistical key assumption, the re-analysis in the comment above points towards an effect that is directly the opposite of what the authors actually claim: according to the analysis in the comment above POSITIVE changes of LM from last year to this year are the result of NEGATIVE changes of lagged values of the EM and vice versa. As this observation strongly contradicts the original Bentley et al. 2014 paper, we again believe that this inconsistency needs to be addressed by the authors.

All in all, we do not have the impression that the comment above weakens in any way the argumentation laid out in our paper. On the contrary, we think that it demonstrates again why it is important to correctly account for the structure of temporal data.

Alexander Koplenig & Carolin Müller-Spitzer


A Stata do-file to reproduce all results presented in this response is available upon request from AK.