## Reader Comments

Post a new comment on this article

#### Post Your Discussion Comment

Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:

- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language

#### Thank You!

Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.

close### Response by Bentley, Ormerod, Lampos, Acerbi

####
Posted by RABentley
on
**
13 Mar 2016 at 13:07 GMT **

This paper provides some useful general advice about how to examine correlations between time series. It makes the well known point that any two time series that are trending in the same direction over time will yield levels that correlate with each other. If sea level rises over the century and cereal prices rise over the century, then of course sea level and cereal prices will correlate on a bivariate plot.

Our study (Bentley et al. 2014), however, which is cited as part of the authors' list of `cautionary tales', was nothing like this. We first developed the theoretical expectation of finding a positive correlation between mood, as expressed in books, and economic conditions in the recent past. Our work was theoretically grounded: we were not simply data mining and searching for what might prove to be arbitrary correlations. We expected to find a positive correlation between `literary misery' and recent values of `economic misery', and we found it.

We also showed that this was true for two independent pairs of time series -- English-language books and the American economic misery index, and German-language books with the German economic misery index (which is quite different from the American). Using these two different pairs of time series, we found that in each case the best correlation was between the literary misery index and the 11-year and 10-year moving average of the economic misery index, respectively.

We have demonstrated the robustness of the literary misery index elsewhere (Acerbi, et al. 2013, 2013b), but this comment gives us a chance to elaborate on the statistics of the relationship reported between literary misery (LM) and economic misery (EM) indices for the United States (Bentley et al. 2104). Here we can confirm, using several additional statistical tests, that the variables are indeed non-stationary in level form, but are stationary in first-difference form. Rather than a moving average of EM (which implicitly assumes that the coefficients on each of the lags is the same), we estimated a general model in which LM(t) was regressed on several lags of itself, and the contemporaneous EM(t) and one through 11 lags of this variable (all variables in first difference form). We subsequently eliminated one by one the insignificant variables, and obtained as a final equation, sample period 1901-2000:

diff(LM(t)) = 0.0098 ? 0.197* diff(LM(t-1)) + 0.0087*diff(EM(t-2)) + 0.0133*diff(EM(t-11))

(0.017) (0.093) (0.004) (0.004)

which yields an adjusted R-squared of 0.128. The figures in brackets above are standard errors.

Several widely used econometric tests of specification on the residuals indicate that the equation is well specified. The Durbin Watson statistic is 2.065 (0.634), where the figure in parentheses is the p-value at which the null hypothesis is rejected. The Breusch-Godfrey test, for serial correlation of order up to and including 4, is 4.15 (0.386). The Ramsey RESET(2,93) specification test yields 0.545 (0.581). The Kolmogorov-Smirnov test, that of the null hypothesis that the residuals are normally distributed, is 0.063 (0.806).

The correlation of changes (not levels) between these two sets of time series is indisputable. This leaves some of the qualitative critiques of the article, which raises the question as to how carefully the critics read our study. There is the obvious point that changes in N-grams do not reflect the present population (`babies do not write books'), which is the point made by the title of our article, i.e. that books average the previous decade of economics. Indeed, had it been the other way around, this would have falsified our hypothesis that the moving average of a causal driver (economy) predict changes in a delayed effect (authors). There are other hypotheses, of course, which Bentley et al. (2014) discuss, but as we pointed out, it is quite plausible that economics would subtly affect the `mood' of authors who grew up in a certain economic climate, as has been shown, for example, in qualitative studies of children of the Depression (e.g. Elder 1974). **References**

Acerbi A, Lampos V, Garnett P, Bentley RA (2013) The expression of emotions in 20th century books. *PLoS ONE* 8 (3) e59030.

Acerbi A, Lampos V, Bentley RA (2013b) Robustness of emotion extraction from 20th century English books. IEEE BigData 2013 Proceedings.

Bentley RA, Acerbi A, Omerod P, Lampos V (2014) Books average previous decade of economic misery. *PLoS ONE* 9(1): e83147.

Elder GH (1974) *Children of the Great Depression.* Westview Press.

**No competing interests declared.**

### RE: Response by Bentley, Ormerod, Lampos, Acerbi

####
AKoplenig
replied to
RABentley
on
**
23 Mar 2016 at 10:58 GMT **

We want to start our response to the comment above by pointing out that we neither think that the study of Bentley et al. 2014 is uninteresting, nor that the theoretical argumentation presented in the paper is implausible. Our critic concerns the fact that in the Bentley et al. 2014 paper, the authors use standard statistical models that can be used for cross-sectional data but that do not account for the within-series error-dependency that is typical for the analysis of time series data. This is not just an opinion that comes from reading the Bentley et al. 2014 paper, but is based on a re-analysis of the original data that was provided for us directly by Bentley et al. 2014. We personally communicated the results of our re-analysis and discussed potential alternative methods (e.g. changes instead of levels) with Bentley et al. before we wrote this paper.

Let us now demonstrate why we think that we did not “misrepresent” their study in our paper.

First of all, one can argue that even if the comment of Bentley, Ormerod, Lampos & Acerbi was correct, this would not imply that we have “misrepresented” the Bentley et al. 2014 study, because in the comment the authors actually claim that “using several additional statistical tests […] the variables are indeed non-stationary in level form, but are stationary in first-difference form”. However, in the original Bentley et al. 2014 study, the variables are treated as non-stationary in the level form. In our opinion, this alone should lead to a correction of the Bentley et al. 2014 study, given the fact that, as we argue in our paper, this transformation also re-formulates the research question, and, as Bentley et al. 2014 demonstrate in their comment above, affects all presented results, e.g. the model fit and levels of statistical significance. However, as we will demonstrate in what follows, the general model presented in the comment above actually points in the opposite direction: personal communication with one of the authors of the comment (P. Ormerod) revealed that the original LM (“literary misery”) index was accidentally reversed in the opposite direction for the general model presented in the comment above. This implies that instead of a positive correlation, the authors of the comment present evidence for a negative correlation of year-to-year changes of (lagged values of) EM (“economic misery”) on year-to-year changes of LM. Of course, this has major consequences for the Bentley et al. 2014 study that – in our opinion – need to be addressed by the authors of the paper.

As written above, in the original Bentley et al. 2014 study, all variables are (implicitly) treated as non-stationary. If we regress the level of LM on the level of EM (“economic misery”) as Bentley et al. 2014 did, we indeed find a correlation of r = .25 (at p < .05). However, as Durbin’s alternative test for autocorrelation shows, there is strong reason to reject the null hypothesis of no serial correlation for all lags from 1 up to 4 at all standard levels of significance (all p-values < .0001). Similar observations can be made, if we regress LM (i) onto the seventh lag of EM or (ii) onto the 11-year moving average of EM as constructed by Bentley et al. 2014, in all cases up to a lag of 4, there is a strong indication of temporal autocorrelation (all p-values < .0001). Again, we believe that this problem in the original Bentley et al. 2014 study needs to (and can) be addressed. This is not a matter of debate; it is a matter of statistical necessity that has substantial consequences: for example, a regression with Newey-West standard errors that are robust to arbitrary autocorrelation up to lag 10 reveals that if we control for serial correlation, the effect of EM on LM is rendered insignificant (p = 0.161). Or, if we do the opposite of Bentley et al. 2014 and regress EM onto the 20th lag of LM, again with Newey-West standard errors, we find that LM significantly (at p < .01) predicts EM with a negative correlation between both series of r = -0.392. To sort out that apparent inconsistency in the argumentation of Bentley et al. 2014, more formal tests (e.g. for Granger causality) would be needed, because as the authors themselves point out in the comment above: “Indeed, had it been the other way around, this would have falsified our hypothesis…“

In addition, the authors of the comment present a new general model where they look at year-to-year changes instead of levels to analyze the potential influence of EM on LM. We re-analyzed this model and obtained:

diff(LM(t)) = - 0.00982 - 0.197* diff(LM(t-1)) - 0.00871*diff(EM(t-2)) - 0.0134*diff(EM(t-11))

This model is similar to the equation presented in the comment above but with opposite signs (standard errors and the adjusted R2 value are identical). As written above, this is due to the fact that the original LM index was accidentally reversed in the opposite direction for the general model presented in the comment above. Thus, while the original Bentley et al. 2014 paper is flawed due to autocorrelated errors that violate statistical key assumption, the re-analysis in the comment above points towards an effect that is directly the opposite of what the authors actually claim: according to the analysis in the comment above POSITIVE changes of LM from last year to this year are the result of NEGATIVE changes of lagged values of the EM and vice versa. As this observation strongly contradicts the original Bentley et al. 2014 paper, we again believe that this inconsistency needs to be addressed by the authors.

All in all, we do not have the impression that the comment above weakens in any way the argumentation laid out in our paper. On the contrary, we think that it demonstrates again why it is important to correctly account for the structure of temporal data.

Alexander Koplenig & Carolin Müller-Spitzer

A Stata do-file to reproduce all results presented in this response is available upon request from AK.

**No competing interests declared.**

### RE: RE: Response by Bentley, Ormerod, Lampos, Acerbi

####
albertoacerbi
replied to
AKoplenig
on
**
08 Apr 2016 at 14:50 GMT **

First of all I would like to thank Alexander Koplenig and Carolin Müller-Spitzer for the time spent to accurately check the analysis presented in Bentley et al. 2014, and in the comment above.

They are indeed correct in noting that, in the comment, we used the reversed LM index, so that the statistics we present there are not relevant for the 2014 paper. I still believe that the results we presented in 2014 paper are theoretically important, and they shed light on an interesting phenomenon – i.e. the correlation between economic and literary “mood” – which has empirical reality.

The further analysis performed by Koplenig & Müller-Spitzer on our data is an excellent example of post-publication peer-review, and I hope that our paper, together with this dialogue, will be useful for other scientists working on similar topics.

Please notice this comment represent my (AA) personal opinion, and it should not be intended as an answer from the Bentley et al. 2014 authors.

**No competing interests declared.**