Fig 1.
Example of a headline variation experiment pair and derived features.
The a variant (top) had a higher clickthrough rate (12.3%) than the b variant (bottom; 5.5%). The a variant contains a definite and an indefinite article, a negative emotion word, and a third-person singular pronoun, whereas the b variant contains a positive emotion word, a third-person singular pronoun, and a second-person pronoun. Character count and Flesch reading-ease score are also shown.
Table 1.
Design table.
Table 2.
Hypothesis word dictionaries.
Fig 2.
Main regression analysis on the upworthy archive confirmatory and exploratory datasets.
Error bars visualize 95% (dark bars) and 99% (light bars) confidence intervals on the logistic regression coefficients.
Table 3.
Main analysis regression with exploratory dataset.
Logit regression analysis for confirmatory data.
Table 4.
Outcomes of the tested linguistic hypotheses, depending on the sign of the corresponding coefficient and its p-value.
The entries on the diagonal represent aligned interpretations on confirmatory and exploratory datasets (H1, H2a, H2b, H3, H4, H5b, H6a, H7, and H8a). Off-diagonal entries represent discordant results on confirmatory and exploratory datasets (H5a, H6b, and H8b).
Fig 3.
Accuracy (on the y-axis), for ranges of difference in click-through rate between the compared headlines (on the x-axis), split into quintiles.
Fig 4.
Top 15 topics by frequency among the confirmatory headlines. The frequency is measured as the fraction of headlines labelled with the respective Empath topic.
Table 5.
For the positive and negative emotion categories, Linguistic Inquiry and Word Count (LIWC) words most frequently categorized as either positive or negative emotion in the confirmatory dataset.
Fig 5.
Correlations between linguistic features.
Pearson correlation coefficient between values of linguistic features. All correlations are small, ranging between -0.13 and +0.12.
Table 6.
Linguistic variables and the frequency in which they are manipulated among the comparison pairs, in the exploratory dataset, and in the confirmatory dataset.