Ten Simple (Empirical) Rules for Writing Science

“. . .though a Philosopher need not be sollicitous that his style should delight its Reader with his Floridnesse, yet I think he may very well be allow’d to take a Care that it disgust not his Reader by its Flatness, especially when he does not so much deliver Experiments or explicate them, as make Reflections or Discourses on them; for on such Occasions he may be allow’d the liberty of recreating his Reader and himself, and manifesting that he declin’d the Ornaments of Language, not out of Necessity, but Discretion. . .”—Robert Boyle, Proemial Essay [1].

we recorded the number of citations, authors, references, words in the abstract, as well 13 as the year of publication, the journal title, the discipline, the full set of keywords, and, 14 naturally, the abstract. The disciplines were chosen so that biology was represented by 15 three closely-related fields (Ecology, Evolution, Genetics), and the "outgroup" 16 contained a wide variety of fields. Some journals belong to multiple disciplines (e.g.,

17
the journal "Evolution" is considered in Ecology, in Evolution, and in Genetics).

18
To make sure that all records were complete, and that the abstracts were correctly The goal of the analysis is to ascertain the effect of a particular abstract feature, x, on 28 the number of citations an article receives. To this end, we want to account for factors 29 that are likely to influence citation counts, such as the journal where the article has 30 been published, the age of the article, its number of authors and number of references.

31
First, instead of modeling citation counts, we chose log(citations + 1) as our 32 response variable. In this way, given that citations to articles of a given age tend to 33 follow a log-normal distribution [1], we should recover approximately a normal 34 distribution for each journal-year combination. In Fig. 1, we show that a normal 35 distribution well-approximates the log(citations + 1), especially for the older articles. Physics all journals seem to have adopted a similar length requirement, the distributions for several other disciplines display multi-modality, due to the fact that different journals have different requirements. Notably, all disciplines contain outlier articles with extremely lengthy abstracts, often exceeding 1000 words (e.g., Psychology: [2] > 1600 words, Ecology: [3] ≈ 1500 words).
With this notation in place, we can write the linear model: where α is a common intercept, β j(i)y(i) specifies the effect of journal-year 51 combination, γ measures the effect of having a number of authors that is larger than 52 the mean for the journal, δ the effect of having more references than what is typical 53 for the journal, and ζ measures the effect of having a certain feature of the abstract, x, 54 with values that are above the mean for the journal. The residuals are stored in ǫ i .

55
Note that ζ measures the effect of being one standard deviation above the mean for 56 the journal. Suppose that article a has a feature x (e.g., number of words) taking 57 exactly the value of the mean for the corresponding journal. Then z(x) a = 0. Article b 58 has the same features as a, besides having x exactly one standard deviation above the 59 mean. Thus, z(x) b = 1. The difference log(citations + 1) b − log(citations + 1) a = ζ. is the percentage of citations gained or lost due to having feature x one standard 63 deviation above the mean. 64 We ran a different regression (using the package biglm of the statistical software R) 65 for each discipline, and then repeated the analysis at the journal level. Basically, we 66 are interested in the sign and magnitude of ζ for each feature of the abstract x and 67 each discipline. For simplicity, we tested each feature of the abstract separately, rather 68 than trying to model them all together. Notice that many features are correlated (e.g., 69 it is difficult to write an abstract with many sentences but few words), so that 70 correlated features will tend to return similar effects.

71
Because we are testing multiple hypotheses using the same data set, we used the 72 Bonferroni correction when determining whether ζ is significantly different from 0. We 73 used a desired significance level of 0.01 when analyzing disciplines (for which we have  R4. We tagged all verbs using nltk, and computed the fraction (present + gerund) / 90 (present + gerund + past + past participle).

R6.
We counted how many words in the abstract were also keywords (when 94 keywords were reported; otherwise we set this value to not available).

95
R7. We set the variable to 1 whenever the abstract contained at least a word 96 signaling novelty (R7a) or importance (R7b) and to zero otherwise.

R9.
We computed the proportion of words in the abstracts that were in a dictionary 100 of "hedge words".