https://orcid.org/0000-0002-8386-5397
Figures
Abstract
The title of an article is the main entrance for reading the full article. The aim of our work therefore is to examine differences of title content and form between original research articles and its changes over time. Using PubMed we examined title properties of 500 randomly chosen original research articles published in the general major medical journals BMJ, JAMA, Lancet, NEJM and PLOS Medicine between 2011 and 2020. Articles were manually evaluated with two independent raters. To analyze differences between journals and changes over time, we performed random effect meta-analyses and logistic regression models. Mentioning of results, providing any quantitative or semi-quantitative information, using a declarative title, a dash or a question mark were rarely used in the title in all considered journals. The use of a subtitle, methods-related items, such as mentioning of methods, clinical context or treatment increased over time (all p < 0.05), while the use of phrasal tiles decreased over time (p = 0.044). Not a single NEJM title contained a study name, while the Lancet had the highest usage of it (45%). The use of study names increased over time (per year odds ratio: 1.13 (95% CI: [1.03‒1.24]), p = 0.008). Investigating title content and form was time-consuming because some criteria could only be adequately evaluated by hand. Title content changed over time and differed substantially between the five major medical journals. Authors are advised to carefully study titles of journal articles in their target journal prior to manuscript submission.
Citation: Heßler N, Ziegler A (2023) Content and form of original research articles in general major medical journals. PLoS ONE 18(6): e0287677. https://doi.org/10.1371/journal.pone.0287677
Editor: Boyen Huang, University of Minnesota School of Dentistry, UNITED STATES
Received: March 10, 2023; Accepted: June 10, 2023; Published: June 28, 2023
Copyright: © 2023 Heßler, Ziegler. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: There are no patents, products in development or marketed products to declare. AZ is a licensed Tim Albert trainer. Tim Albert trainings deal with advising people how to write medical papers. AZ has held several courses in the past based on Albert’s concept. This does not alter our adherence to PLOS ONE policies on sharing data and material.
Introduction
Researchers have the duty to make the results of their research on human subjects publicly available according to the declaration of Helsinki [1], and many recommendations for the reporting of studies have been developed. An overview on these reporting guidelines is provided by the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network, which aims to tackle the problems of poor reporting [2]. One consequence of systematic reporting is that many scientific articles are organized in the same way [3, 4], and they generally follow the IMRAD structure, which stands for Introduction, Methods, Results, And Discussion. The IMRAD structure is also standard for the writing of abstracts. It is therefore of interest to researchers how they can individualize their reports to increase the citation counts, which is one important measure for career advancement [5].
Approximately 30 factors affecting citation frequency have already been identified [6–9]. While journal- and author related factors are generally not modifiable, some article-specific factors are subject to active modification by the authors. Especially the title has been proposed as a modifiable component of a research article [9–11]. Researchers should use titles that accurately reflect the content of their work and allow others easily to find and re-use their research [12]. Most research has focused on the form of article titles because these analyses could be performed automatically and are not very time-consuming [9, 13, 14].
While the article content has been studied well both in features, such as tense, voice and personal pronouns, and in the IMRAD sections between different research disciplines, title content has received less attention, and the main focus was title length [15, 16]. One reason could be the lack of automated internet searches until approximately 25 years ago. For example, PubMed was first released in 1996, Web of Science is online since 1997 and Google Scholar started not earlier than in 2004. With the advent of automated internet-based searches the importance of the title has changed, and it is now the “billboard” of a research article [17]. Another reason could be that these evaluations have to be made manually, and they are thus time-consuming [18]. An additional time-consuming factor could be that guidelines such as the Standards for Reporting of Diagnostic Accuracy (STARD) statement [19] strongly recommend that at least two observers should do an independent evaluation where applicable.
Most articles investigating the form of the title compared whether the title was a full sentence [20], descriptive, indicative, or a question [18, 21], or whether the title included non-alphanumeric characters, such as a colon or dash [22]. Very few publications looked at other title components of a research article. Specifically, Kerans, Marshall [23] compared the frequency of Methods mentioning or Results mentioning for the general major medical journals, specifically the New England Journal of Medicine (NEJM), the BMJ, the Journal of the American Medical Association (JAMA), and the Lancet by analyzing the first approximately 60 articles published either in 2015 or 2017 in each of the journals. Both articles investigated only a few months from a single publication year per journal. The development of title content over time was thus not considered.
The aim of our work therefore was to examine properties of title content for original research articles published in one of the five major clinical journals (BMJ, JAMA, Lancet, NEJM, and PLOS Medicine (PLOS)) over the 10-year period from 2011 until 2020. Specifically, we aimed at identifying differences between the five journals and changes over time regarding title content and title form. We also compared our findings to those of Kerans et al. [15, 23].
Materials and methods
Search in Medline and Web of Science
The search strategy has been described in detail elsewhere [9]. In brief, we first extracted all original research articles finally published between 2011 and 2020 in the five major clinical journals BMJ, JAMA, Lancet, NEJM, and PLOS. The restriction to the publication year 2011 allows for proper comparisons between journals because PLOS was reshaped in 2009 [24].
The variables PubMed identifier (PMID), journal name, article title, author names, publication year, citation, PubMed Central identifier (PMCID) and digital object identifier (DOI) were extracted from the Medline search. From the Web of Science, we reduced available information to journal name, article title, PMID, abstract for the identification of original research articles, DOI and publication date. Both PMID and DOI were used to merge articles identified in Medline (n = 8396) and the Web of Science (n = 10267). Articles being listed with an abstract remained in the data set, while articles only listed in the Web of Science were excluded. Articles being only downloaded in the Medline files were checked whether they were indeed original research articles. If not, they were excluded as well. After data cleaning, a set of 8096 articles was available.
Evaluation of title content and form
To investigate title content and form, we randomly selected 500 original articles from the years 2011 to 2020. The random selection was done with stratification by journal and year so that ten original articles per year (100 articles per journal) were randomly chosen. To avoid a priori information on the specific journal article, only the title and the PMID were presented in the database. In addition, the order of the 500 articles was randomized prior to evaluation. All article titles were evaluated manually by two raters/authors. Both raters performed a training and independently evaluated 25 randomly selected journal articles—five per journal—prior to the evaluation of the 500 articles. These training articles were excluded from the main evaluation. Conflicts in ratings were solved by agreement.
Items for title content and form are displayed in Table 1 and were inspired by other works [15, 25, 26]. One reviewer asked for the discoverability in each of the title items, therefore, we provided two examples of article titles with the result of our evaluation in Table 1.
The first block of Table 1 reports results on title content. Title content was divided into the topics Methods and Results. The former is concerned with the mentioning of methods in the title, such as the study design or a novel technique used in the paper [15]. Other elements from the methods concern the mentioning of a patient population, the geography, the clinical context, an intervention, and the use of study names in the title. The latter examines results mentioned in the manuscript. The first question was whether results were stated in the title at all. More detailed were the questions whether quantitative information or semiquantitative or ordinal information was provided [26]. It was also noted whether the title reported on a relation between two or more variables [26].
The second block of Table 1 is related to the form of a title divided into the topics Methods, and Conclusion/Discussion. The use of abbreviations, dashes and subtitles was investigated for the Methods. The three single items for Conclusion/Discussion were whether the title was declarative, phrasal, or formulated as a question.
Recently, we performed an analysis after an automatic search for country and city mentioning in the title by the use of the R package maps [9], and we did not expect substantial differences to our hand search.
Sample size considerations
The main aim of our work was to investigate trends over time by a regression model. In general, regression models have a sufficient sample for a single independent variable, such as time, if n ≈ 50 [27, 28]. Specifically, for a weak effect size of R2 = 0.14 [29], the required sample size is 51. In case of a weak effect size of Cohen’s f [29] with f2 = R2 / (1 –R2) = 0.14, the required sample size is 403 to achieve a power of 80%. A sample size of 500 as used in our work yields a power of 87.75% at a significance level of 5%.
Statistics
Descriptive statistics for the specified title properties, i.e., absolute and relative frequencies were reported for each journal over time, refraining of descriptive p-values for investigating journal differences. Fisher’s exact tests were performed at a significance level of 5% to compare the findings of this study with those of Kerans et al. [15, 23] regarding methods mentioning, patient population, geography, clinical context, and treatment. Corresponding 95% confidence intervals (CI) were provided. Furthermore, overall tests were performed to compare frequencies of these items between all journals. Bias-corrected Cramérs V effect measures were estimated with corresponding parametric bootstrapped CIs. The DerSimonian and Laird [30] (DSL) approach was used to perform random effect (RE) meta-analyses, which allows for variability in the variables of interest properties between journals and over time. The logit transformation was used for estimating the pooled proportions [31], and standard errors were not back-transformed.
The effect of time regarding the specific title properties was investigated by logistic regression models, if appropriate. Post hoc comparisons for the identification of homogeneous subgroups were performed using Tukey’s HSD. Associations between title properties and the journals were analyzed using likelihood ratio tests. Effect estimates, i.e., odds ratios and corresponding 95% CI were reported for all analyses, and the journal BMJ was used as reference category. An odds ratio of x.x being greater than 1 indicates an x.x fold increased chance containing the specific item for an one-year difference adjusted for the variable journal.
Data and R code for all analyses are provided in S1 and S2 Files, respectively.
Results
A total of 500 randomly selected original research articles from 5 medical journals were analyzed regarding the selected title items (see Table 1). In Table 2, the descriptive statistics, i.e., absolute and relative frequencies for all title properties over the years are shown, respectively for each journal. Results of the meta-analyses are provided in detail in S3 File, sections 4 and 5.
Absolute and relative frequencies (parenthesis) are shown.
Items–Content
In terms of the title content topic methods, the NEJM deviated from the other journals regarding the methods mentioning. While methods were mentioned in at least 93% of the article titles in BMJ, Lancet and PLOS, about the half (47%) was in JAMA and 11% in NEJM article titles. Similar results were reported by Kerans et al. [15, 23] for BMJ, JAMA and Lancet, but proportions differed between Lancet titles (Table 3). The mentioning of methods increased over time (OR: 1.12 (95% CI: [1.01‒1.24]), p = 0.025, Fig 1 and S3 File, section 6.1.1), i.e., methods were mentioned more frequently in the article titles more recently.
Displayed are odds ratios (square) per increase by one year, corresponding 95% confidence intervals (whisker) and p-values (numbers).
Corresponding 95% confidence intervals (CI) are shown in brackets. Results of PLOS Medicine are missing because Kerans et al. did not examine article titles of this journal.
Lowest and highest numbers for the mentioning of the patient population were in the BMJ (62%) and the NEJM (78%), respectively. For the mentioning of the patient population, neither an increase over time (OR: 1.06 (95% CI: [0.99‒1.13]), p = 0.100, Fig 1) nor substantial differences between the journals (S3 File, section 6.1.2) could be observed.
About half of the PLOS titles (52%) contained any geographic information, but only 31% of the BMJ titles (see Table 2). Frequencies were only 16% and 17% for JAMA and Lancet, respectively, and 9% for NEJM titles. These findings are in line with Kerans et al. [15, 23], except for the BMJ, where Kerans et al. observed that 15.8% of the articles mentioned geographic information (Table 3). Mentioning of geographic information varied over time both within each journal (S3 File, section 4.1.3.1) and over the journals (S3 File, section 4.1.3.2). This is consistent with the results from the logistic regression analysis (OR: 1.07 (95% CI: [0.99‒1.16]), p = 0.072, Fig 1 and S3 File, section 6.1.3).
The clinical context was mentioned in 73% of BMJ titles, while it was mentioned at least 80% in the other four journals. This is in line with Kerans et al. [15, 23] (Table 3). Additionally, we observed an increase of clinical context mentioning over time (OR: 1.10 (95% CI: [1.01‒1.19]), p = 0.025, Fig 1 and S3 File, section 6.1.4).
Only 27% in PLOS and 30% in BMJ provided some treatment information in the title, while for the other three journals at least 50% of the article titles mentioned a treatment. Our results did not show any differences from those of Kerans et al. [15, 23] (Table 3). Over time the naming of treatments in the title increased (OR: 1.08 (95% CI: [1.02‒1.16]), p = 0.015, Fig 1 and S3 File, section 6.1.5).
There was no NEJM title containing a study name while Lancet had the highest usage of it (45%). The analysis over time showed a trend over time (OR: 1.13 (95% CI: [1.03‒1.24]), p = 0.008) and substantial differences between the journals (S3 File, section 6.1.6).
Regarding the title topic results, only 6 out of the total of 500 articles mentioned results in their titles. This is in line with the findings of Kerans et al. [15, 23], who reported that 1.9% of NEJM titles mentioned results. No article provided any quantitative information in its title, and only 4 of 500 articles provided semi-quantitative information in their title. Because of very low numbers, no further analyses were performed for these criteria.
A relation between variables was used least frequently in the NEJM (23%), followed by the Lancet (35%). The other three journals mentioned a relation in more than half of the articles (Table 2). These differences between journals were confirmed in regression analysis (S3 File, section 6.2.4). However, an increase over time could not be observed (p = 0.858, Fig 1).
Items–Form
In terms of the title form topic methods, abbreviations were less used in NEJM titles and most used in Lancet titles, 24% and 55 respectively (see Table 2). An increase use over time was observed (OR: 1.13 (95% CI: [1.05‒1.20]), p < 0.001, Fig 1) as well as differences between journals (S3 File, section 7.1.1).
Dashes were rarely used. Only three articles in BMJ and two articles in NEJM used a dash (Table 2). Further analyses were not performed because of these low frequencies.
A subtitle was used in at least 98% of the articles in BMJ, Lancet, and PLOS, while only 41% of JAMA titles and only 2% of NEJM titles used subtitles. These clear differences between the journals were confirmed with the regression analysis (S3 File, section 7.1.2). Moreover, the usage of subtitles increased over time (OR: 1.22 (95% CI: [1.07‒1.38]), p < 0.003, Fig 1).
Finally, regarding the title form topic discussion, not a single article had a declarative title in our analyses (Table 2). Phrasal titles were present in 3% of JAMA, 7% of NEJM, 11% of BMJ, 12% of Lancet, and 15% of PLOS titles. Significant differences between journals could not be observed (S3 File, section 7.2.2). A decrease of phrasal titles over time was observed in the regression analysis (OR: 0.90 (95% CI: [0.81‒1.00]), p < 0.044, Fig 1 and S3 File, section 7.2.2).
Only three of 500 article titles were written as a question (Table 2). Kerans et al. [15, 23] observed similar low frequencies; and they reported 3.9% for the BMJ and 1.3% for Lancet articles with a question symbol, and none for both JAMA and NEJM (Table 3).
Geographic information–Manual versus automated search with the maps package
The comparison of our hand search on the mentioning of geographic information revealed substantial differences to the automated search with the R package maps [9].
In detail, respectively, 31% vs. 13% for BMJ, 16% vs. 3% for JAMA, 17% vs. 9% for the Lancet, 9% vs. 3% for the NEJM and 52% vs. 29% for PLOS articles contained any geographical information in their titles for the hand and automatic search. The automated search thus led to fewer titles with any geographic information.
Discussion
Title content properties varied substantially between original research articles published in the general major medical journals. Furthermore, title content and form changed over time. Differences between journals were specifically observed in the use of subtitles. While almost all articles from the BMJ and PLOS had subtitles, only two of the NEJM articles had a subtitle. Previously, we and others showed that the colon was most used in titles to split a title into multiple parts rather than any other separator [9, 15, 23]. Here, we furthermore showed that the proportion of paper with subtitles increased over time.
Substantial differences between journals were also observed for the mentioning of methods, the patient population, the geography, the interventional treatment, and the use of an abbreviation in the title. In addition, there were substantial differences in the use of a study name in the title. For example, while no article published in the NEJM used a study name, almost half (45%) of the studies in the Lancet used one. Some content criteria were mainly not or rarely used in all considered journals, such as a dash, mentioning of results, using a declarative title, or a question mark. This was in contrast to Paiva, Lima [32] who showed for PLOS and BMC journals that approximately 40% of the articles mentioned the results, and such articles were more frequently cited than work mentioning methods. In our study, only 6 articles out of 500 mentioned results in the title, while 344 out of the 500 articles mentioned of methods. Our findings are in line with general guidelines that declamatory titles, i.e., titles that give study results should be avoided [33]; see, e.g., instructions to authors for the Lancet. Authors should thus avoid providing quantitative or semi-quantitative information in the title. In fact, since the title is a one-line summary, the conclusions could be spread out into the world without reading at least the abstract or the full text of the article. Aleixandre-Benavent and colleagues go a step further and provide recommendations what a title should contain, and how it should not be constructed [16].
Our work focused on the general major medical journals plus the online only journal PLOS. Between the printed journals, there were substantial differences regarding the content of article titles [9]. One of the reasons could be in the instructions for authors, which differ in the provided information on the construction of a title. Specifically, the NEJM title had the lowest number of frequencies for a couple of criteria, such as the subtitle, methods mentioning, geography, abbreviations, and relation. No NEJM title contained a study name. However, the clinical context and the patient population was most frequently described in NEJM article titles. Differences between printed and online journals were obvious using geographic information in the title or usage of a phrasal title occurring more often in the online journal PLOS.
Subtitles are now more frequently used than a decade ago. Furthermore, the mentioning of methods increased in the 10 years from 2011 to 2020. This change in the title may be caused by the increased use of reporting guidelines, such as the CONSORT statement [34], which states that a randomized controlled trial should be identifiable as randomized in the title. The instructions for authors of all considered journals state that subtitles should be used for reporting the study design and/or authors should follow the respective reporting guidelines of their study. In fact, authors should look out a copy of the target journal and identify its preferences [35].
Our results are in line with the recommendations from the journal-specific instructions for authors, except NEJM. The NEJM does not follow the CONSORT statement using subtitles for randomized controlled trials, see also [1]. For the other four journals, the mentioning of the study design or the type of analysis is almost always done using subtitles as recommended. Furthermore, our results for JAMA using no declarative titles, no results mentioning or using questions in the title match with its recommendations.
Research has so far concentrated on the form of article titles rather than its content. While some authors investigated title content in BMJ, JAMA, Lancet and NEJM for a specific time, generally a single year [15, 23, 36], the development of title content over time has rarely been studied [37]. A strength of our work thus is the availability of all original articles over a time span of 10 years [9]. From this database, we randomly selected a subset of articles for manual assessment. These articles were evaluated by two raters according to a pre-specified coding plan with examples and training. Title evaluations were then done blinded by year and journal.
We did not expect different journal-specific frequencies regarding the geographic information in the title compared to our recent work [9], in which we performed an automatic search for country and city mentioning in the title by the use of the R package maps [9]. However, frequencies differed substantially. The automated search led to fewer titles with any geographic information. For example, the maps package did not contain countries, such as ‘England’, continents, abbreviation, such as ‘U.S.’, or terms, such as ‘English’. The main reasons for the discrepancies were for the use of country-specific abbreviations and additional country-specific terms. However, other tools or packages might have been more appropriate for the geographical query than the maps package.
One limitation of our study is that we relied on the quality of the data provided by the PubMed database [38]. Another limitation of our work is that additional variables could have been considered, e.g., more complex title content [12, 16, 22].
A further limitation is the sample size of 500 articles, i.e., 10 articles per journal and year. With a sample size substantially larger than 1000 articles we would have been able to study the association of title characteristics with citation counts. For example, the total sample size of our previous study, which was based on an automated search was 8096 articles [9]. With 500 articles, 95% confidence intervals are approximately 4 times larger (√8096 / √500 = 4.02), and many results, such as the association between the number of citations would not have been significant. The sample size used in this study is approximately twice that of [15, 23], and this study with 500 articles was powered to reliably detect trends over time.
In future research, it would be of interest to analyze the effect of title content properties on citation frequencies. It would also be interesting to compare specific journals with general medical journals.
In conclusion, title content differed substantially between the five major medical journals BMJ, JAMA, Lancet, NEJM and PLOS. Furthermore, title content changed over time. We recommend that authors study titles of articles recently published in their target journal when formulating the manuscript title. Analyses of title content may generally require manual time-consuming inspections.
References
- 1. World Medical Association. Ethical principles for medical research involving human subjects Ferney-Voltaire: World Medical Association (WMA); 2018 [05.07.2022]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/.
- 2. Simera I. EQUATOR network collates resources for good research. BMJ. 2008;337:a2471. Epub 2008/11/13. pmid:19001485.
- 3. Day RA. The origins of the scientific paper: the IMRaD format. J Am Med Writers Assoc. 1989;4(2):16–8.
- 4. Sollaci LB, Pereira MG. The introduction, methods, results, and discussion (IMRAD) structure: a fifty-year survey. J Med Libr Assoc. 2004;92(3):364–7. pmid:15243643; PubMed Central PMCID: PMC442179.
- 5. Morgan PP. The importance of being cited. Can Med Assoc J. 1983;129(1):9. Epub 1983/07/01. pmid:20313928; PubMed Central PMCID: PMC1874924.
- 6. Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behavior. J Doc. 2008;64(1):45–80.
- 7. Tahamtan I, Safipour Afshar A, Ahamdzadeh K. Factors affecting number of citations: a comprehensive review of the literature. Scientometrics. 2016;107(3):1195–225.
- 8. Ale Ebrahim N, Salehi H, Amin Embi M, Habibi Tanha F, Gholizadeh H, Motahar SM, et al. Effective strategies for increasing citation frequency. Int Educ Stud. 2013;6(11):93–9.
- 9. Heßler N, Ziegler A. Evidence-based recommendations for increasing the citation frequency of original articles. Scientometrics. 2022;127(6):3367–81. Epub May 3, 2022.
- 10. Fox CW, Burns CS. The relationship between manuscript title structure and success: editorial decisions and citation performance for an ecological journal. Ecol Evolut. 2015;5(10):1970–80. pmid:26045949
- 11. Norman ER. Maximizing journal article citation online: readers, robots, and research visibility. Polit Policy. 2012;40(1):1–12.
- 12. Haggan M. Research paper titles in literature, linguistics and science: dimensions of attraction. J Pragmat. 2004;36(2):293–317.
- 13. Habibzadeh F, Yadollahie M. Are shorter article titles more attractive for citations? Cross-sectional study of 22 scientific journals. Croat Med J. 2010;51(2):165–70. pmid:20401960; PubMed Central PMCID: PMC2859422.
- 14. Guo F, Ma C, Shi Q, Zong Q. Succinct effect or informative effect: the relationship between title length and the number of citations. Scientometrics. 2018;116(3):1531–9.
- 15. Kerans ME, Anne M, Sabaté S. Content and phrasing in titles of original research and review articles in 2015: range of practice in four clinical journals. Publications. 2016;4(2):11.
- 16. Aleixandre-Benavent R, Montalt-Resurecció V, Valderrama-Zurián JC. A descriptive study of inaccuracy in article titles on bibliometrics published in biomedical journals. Scientometrics. 2014;101(1):781–91.
- 17.
Belcher WL. Writing Your Journal Article in 12 Weeks. A Guide to Academic Publishing Success. 2 ed. London: Sage; 2021.
- 18. Jamali HR, Nikzad M. Article title type and its relation with the number of downloads and citations. Scientometrics. 2011;88(2):653–61.
- 19. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. Epub 2015/10/30. pmid:26511519; PubMed Central PMCID: PMC4623764.
- 20. Diao J. A lexical and syntactic study of research article titles in Library Science and Scientometrics. Scientometrics. 2021;126(7):6041–58.
- 21. Ball R. Scholarly communication in transition: The use of question marks in the titles of scientific articles in medicine, life sciences and physics 1966–2005. Scientometrics. 2009;79(3):667–79.
- 22. Buter RK, van Raan AFJ. Non-alphanumeric characters in titles of scientific publications: An analysis of their occurrence and correlation with citation impact. J Inform. 2011;5(4):608–17.
- 23. Kerans ME, Marshall J, Murray A, Sabaté S. Research article title content and form in high-ranked international clinical medicine journals. English Specif Purp. 2020;60:127–39.
- 24. PLoS Medicine Editors. A medical journal for the world’s health priorities. PLOS Med. 2009;6(4):e1000072. Epub 2009/04/29. pmid:19399159; PubMed Central PMCID: PMC2667639.
- 25. Goodman NW. Survey of active verbs in the titles of clinical trial reports. BMJ. 2000;320(7239):914–5. pmid:10742001; PubMed Central PMCID: PMC27333.
- 26. Siegel PZ, Thacker SB, Goodman RA, Gillespie C. Titles of articles in peer-reviewed journals lack information on study design: a structured review of contributions to four leading medical journals, 1995 and 2001. Sci Ed. 2006;29(6):183–5.
- 27. Wilson VanVoorhis CR, Morgan BL. Understanding power and rules of thumb for determining sample sizes. Tut Quant Meth Psychol. 2007;3(2):43–50.
- 28. Green SB. How many subjects does it take to do a regression analysis? Multivariate Behav Res. 1991;26(3):499–510. pmid:26776715.
- 29.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed ed. New York: Lawrence Erlbaum; 1988.
- 30. DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemporary Clinical Trials. 2015;45:139–45. pmid:26343745.
- 31.
Lipsey MW, Wilson DB. Practical Meta-Analysis. Thousand Oaks: SAGE Publishing; 2000.
- 32. Paiva CE, Lima JPdSN, Paiva BSR. Articles with short titles describing the results are cited more often. Clinics. 2012;67:509–13. pmid:22666797
- 33. Sharp D. Kipling’s guide to writing a scientific paper. Croat Med J. 2002;43(3):262–7. Epub 2002/05/30. pmid:12035130.
- 34. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. Epub 2010/03/25. pmid:20332511; PubMed Central PMCID: PMC2844943.
- 35.
Albert T. Winning the Publications Game: The Smart Way to Write Your Paper and Get It Published. 4th ed. Boca Raton: CRC Press; 2016.
- 36. Wang Y, Bai Y. A corpus-based syntactic study of medical research article titles. System. 2007;35(3):388–99.
- 37. Heßler N, Rottmann M, Ziegler A. Empirical analysis of the text structure of original research articles in medical journals. PLOS ONE. 2020;15(10):e0240288. Epub 2020/10/09. pmid:33031425; PubMed Central PMCID: PMC7544105.
- 38. Cappell MS. Major bibliographic errors in PubMed: personal experience among 240 publications and proposed remediation process for errors. J Med Libr Assoc. 2016;104(1):83–4. Epub 2016/01/26. pmid:26807060; PubMed Central PMCID: PMC4722650.