Global Cardiovascular Research Output, Citations, and Collaborations: A Time-Trend, Bibliometric Analysis (1999–2008)

Introduction Health research is one mechanism to improve population-level health and should generally match the health needs of populations. However, there have been limited data to assess the trends in national-level cardiovascular research output, even as cardiovascular disease [CVD] has become the leading cause of morbidity and mortality worldwide. Materials and Methods We performed a time trends analysis of cardiovascular research publications (1999–2008) downloaded from Web of Knowledge using a iteratively-tested cardiovascular bibliometric filter with >90% precision and recall. We evaluated cardiovascular research publications, five-year running actual citation indices [ACIs], and degree of international collaboration measured through the ratio of the fractional count of addresses from one country against all addresses for each publication. Results and Discussion Global cardiovascular publication volume increased from 40 661 publications in 1999 to 55 284 publications in 2008, which represents a 36% increase. The proportion of cardiovascular publications from high-income, Organization for Economic Cooperation and Development [OECD] countries declined from 93% to 84% of the total share over the study period. High-income, OECD countries generally had higher fractional counts, which suggest less international collaboration, than lower income countries from 1999–2008. There was an inverse relationship between cardiovascular publications and age-standardized CVD morbidity and mortality rates, but a direct, curvilinear relationship between cardiovascular publications and Human Development Index from 1999–2008. Conclusions Cardiovascular health research output has increased substantially in the past decade, with a greater share of citations being published from low- and middle-income countries. However, low- and middle-income countries with the higher burdens of cardiovascular disease continue to have lower research output than high-income countries, and thus require targeted research investments to improve cardiovascular health.


Introduction
Economic development has been associated with transitions in health, notably shifts from communicable diseases to noncommunicable, chronic diseases [NCDs] including cardiovascular diseases [CVD] [1]. Health research is one mechanism to improve population-level health and should generally match the health needs of populations. Even though CVD is the leading cause of death and disability worldwide, there have been limited data to assess the trends in country-level cardiovascular research output [2,3]. Bibliometrics, a research evaluation methods used to quantitatively evaluate scientific literature, can be used to measure trends in research productivity over time. Prior cardiovascular bibliometric research has been largely confined to high-income country settings from more than 20 years ago [4].
Our objective was to evaluate trends in global cardiovascular research publications disaggregated by country from 1999 to 2008, the relative importance of these publications as measured by average number of citations per paper, and the degree to which countries collaborate. An additional aim was to compare cardiovascular research publications against measures of countrylevel development and CVD burden.

Bibliometric Filter Creation and Testing
We created, tested, and revised a cardiovascular bibliometric filter using a previously published approach to achieve .90% precision (specificity) and recall (sensitivity) to capture cardiovascular research articles, reviews, and conference proceedings from the Web of Knowledge (1999Knowledge ( -2008 [5,6]. Precision represents proportion of filter output considered to be cardiovascular research, and recall represents the proportion of known cardiovascular research publications retrieved [7]. Testing was undertaken by two cardiovascular research specialists (MDH, GSB). We amended a previously-used definition of cardiovascular research [5] to develop this filter: Study of the cardiovascular system (heart and blood vessels) and its functions in health and disease, including heart disease and stroke, invasive/interventional cardiology, cardiac surgery, and vascular surgery, which is the practice of diagnostic and therapeutic procedures that involve entry into the heart and major blood vessels.
With each filter iteration (developed by MDH), the tester (GSB) was given a random sample of 400 titles of publications drawn from the whole retrieval of the cardiovascular filter, mixed with 100 titles of publications from cardiovascular departments. The tester was asked to decide whether titles represented cardiovascular research publications. Iterative amendments were made to the filter. Keywords or headings were included or excluded on the basis of the findings, and tester was given new testing samples. Our initial filter produced precision and recall scores of 0.797 and 0.874, respectively, and after five iterations, our final filter produced precision and recall scores of 0.905 and 0.903.

Bibliometric Filter Application
We applied our final bibliometric filter to the Thomson Reuters Web of Knowledge Science Citation Index Expanded [SCI-EXPANDED] and Social Sciences Citation Index [SSCI] for publication years 1999-2008 [8]. We searched for all articles, reviews, and conference proceedings as citable items over the study period. After removing duplicates, we merged this output with the corresponding citation report from the Web of Knowledge for each publication to create actual citation indices [ACIs], the average of the running five-year citation count postpublication (total citations five years post-publication divided by five), including the year of publication, to provide comparable citation estimates over time through 2012 [9].

Bibliometric Filter Analysis
We performed geographical analysis through integer counts and fractional counts based on publication years, as we have previously published [10]. In brief, integer counts give a country a score of one for all authors from that country, which overestimates the number of publications by approximately 20%, depending on publication year and subject [7]. Fractional counts divide the number of publication addresses from an individual country (numerator) by the total number of publication addresses (denominator). For example, comparing a manuscript with one Chinese author and 9 US authors with a manuscript with 9 Chinese authors and one US author, the former example would be a 0.1 fractional count for China and 0.9 for the US, and the latter would be a 0.9 fractional count for China and 0.1 for the US. No weight is assigned by author order or reprint addresses [11]. We estimated international collaboration by the ratio of fractional to integer counts to derive a fractional contribution estimate.

Analysis by Income, Population, Development, and Burden of Disease
We estimated the relationship between time and cardiovascular publications through linear regression of log-transformed data within the entire dataset by World Bank Income group, including members of the Organization for Economic Cooperation and Development [OECD] [12]. A list of OECD member countries can be found at http://www.oecd.org/general/ listofoecdmembercountries-ratificationoftheconventionontheoecd. htm. World Bank income groups include low-income ($1025 or less), lower middle-income ($1026 to $4035), upper middle-income ($4036 to $12 475), and high-income ($12 476 or more) economies based on per capita gross national income. We created replicate models after removing data from the United States from the dataset as a sensitivity analysis (n = 199 vs. 198 countries). We also performed parametric and non-parametric pairwise comparisons on annual cardiovascular publications among World Bank Income groups.

Ethics
The Northwestern University institutional review board provided exemption for ethics review since this study did not meet criteria for Human Subject Research. Based on this exemption, no informed consent was obtained. Table 1 outlines trends in the total number of global cardiovascular research articles downloaded from Web of Knowledge each year, number of citations downloaded, and matched number of records. Cardiovascular integer counts increased from 40,661 publications in 1999 to 55 284 publications in 2008, which represents a 36% increase over that decade. Figure 1 shows these trends on a logarithmic scale stratified by World Bank income. High-income countries that are members of the OECD published more articles in aggregate per year than high-income, non-OECD countries, middle-, or low-income countries. However, the proportion of cardiovascular publications from high-income, OECD countries declined from 93% (1999) of the total share to 84% (2008) of the total share over the study period. The rate of publication increase was greatest in upper-middle-income countries over the study period (336%; driven primarily by China), followed by lower-middle-income countries (291%), high-income, non-OECD countries (246%), low-income countries (73%), and high-income, OECD countries (30%). The majority of this growth appears to be driven primarily by increases in the number of publication outlets (i.e., sources increased by 73% over the study period; data not shown). Table 2 shows the linear regression results demonstrating the association between time and cardiovascular research publications by income group. Substantial overlap is seen among the effect sizes (beta coefficients) and measures of imprecision (95% CI) by income group despite different baseline levels (intercepts), even after excluding the largest publisher of cardiovascular research as a sensitivity analysis (United States). Table 3 demonstrates trends in the median number of citations per publication by year, stratified by income group and shows a direct gradient by income country status. Median citations per publication increased across all country groups over the study period, and the relative difference between poorer and wealthier countries decreased over time. Median (interquartile range) citations per publication were lowest among publications from low-income countries (0 all years; 0 [0, 0] in 2001) and were highest among publications from high-income, OECD countries (18?4 [12?5, 19?7] in 2007). Lower citation counts in 2008 may reflect incomplete citations reported to Web of Knowledge during follow-up period (2012). Trends were statistically significant for all country groups except upper middle-income countries.

Results
The remaining figures evaluate multiple comparisons through a combination of x and y-axes, color, and bubble sizes to identify novel, country-level relationships. Interactive figures whereby users can manipulate combinations of variables are available at: http:// 115.115.223.246:8080/ccdcgmc-webapp/generate.ccdcgmc. For the remaining figures in this manuscript, the following format is used: each bubble represents an individual country, and the size of each bubble represents a country's population size. By way of orientation, the largest high-income, OECD country represents the United States, the largest upper-middle-income country represents China, and the largest lower-middle-income country represents India. For continuity, the same color scheme is used for stratification of income groups in all figures. Figures 2A and 2B compare country-level fractional contributions (ratio of fractional to integer counts; scale: 0-1) in 1999 (A) and 2008 (B) against logarithmic annual citation counts (countrylevel cardiovascular research output) by World Bank country income status to estimate the relative degree of international collaboration. Lower fractional contributions reflect a greater  proportion of publications with international collaboration. Highincome, OECD countries generally have higher fractional counts than lower income groups, which suggest less international collaboration. There is marked variability in fractional counts among countries with less than 100 annual citations (integer count).
In general, the majority of countries appear to publish a similar proportion of cardiovascular publications against total publications with a modest relative increase over the study period (mean over entire study period = 4?3% of total scientific and technical publications) (Figures S1A and S1B).

Summary of Results
In this study of global cardiovascular research output from 1999 to 2008, we demonstrate a 36% increase in cardiovascular research output over a ten-year period by applying a bibliometric filter to the Web of Knowledge. High-income, OECD countries have a 9% lower proportion of cardiovascular publications over the study period (93% to 84%). The relative rate of publication growth was highest among upper and lower middle-income countries, and China had the greatest relative increase in cardiovascular research output over the study period. The wider availability of the Internet over the study period for greater international collaboration, access to journals, and ease of   electronic submissions has likely contributed to these gains. The mean number of citations per publication has increased across all income groups, but the relative difference between high-income, OECD countries and low-income countries has decreased from a six-fold difference to a 3?5-fold difference between these groups. Richer countries tend to have more stable, higher fractional contribution estimates, which reflect a lower degree of international collaboration compared with poorer countries. On the other hand, some middle-income countries have had increases in fractional contributions, which may be associated with concomitant economic development. For example, China's fractional contribution has increased from 0?77 in 1999 to 0?85 in 2008, which reflects a higher proportion of Chinese authors of cardiovascular publications, perhaps due to increased cardiovascular research capacity in China. Cardiovascular research output is also directly associated with Human Development Index rather than cardiovascular disease burden, with which it has an inverse association.    [13]. The differences in these estimates may be due to differences in search strategies and lack of iterative filter testing by Prabhakaran et al. and Mendis et al.

Comparison with Prior Literature
While citations are increasing in the era of reference management software, increases in citations among cardiovascular publications may be a function of larger trends in the topic area under study. Kulkarni and colleagues found in a 2007 analysis of leading general medical journals that cardiovascular publications had, on average, 13 more citations than ''other'' research articles; however, general medical publications had, on average, nine more citations than ''other'' articles, suggesting that this reference group may not be particularly useful [14].
McKee and colleagues have suggested several potential reasons why overall research productivity is low among low-and middleincome countries: limited governmental or non-governmental funding, lack of health research strategy, lack of political will to engage globally or to prioritize health research, geographic isolation, and recent or ongoing conflict [15]. Some have argued that the increasing dominance of English language in scientific writing may be one barrier to cardiovascular research growth, particularly among low-and middle-income countries [16]. The recent increase in cardiovascular research output from China, however, appears to dispute this argument.

Implications
The ''10/90'' gap outlined by the 1990 Commission on Health Research for Development, which suggests that ,10% of the research resources are available for 90% of the world's health problems, appears to be waning modestly for cardiovascular research over the past decade [17]; however, a ''16/84 gap'' in terms of research productivity remains. The World Health Organization [WHO] has suggested that there is ''No Health Without Research'' in its planned World Health Report for 2012 in collaboration with PLoS Medicine, with an emphasis on health research system strengthening for better healthcare delivery and policymaking [18]. However, publication of this report was delayed for unclear reasons, and the WHO's World Health Report shifted its focus to ''the contributions of research to universal health coverage'' [19] rather than research capacity building in general. Further, the WHO's recent call to reduce premature (,70 years) mortality from NCDs, including CVD, by 25% by the year 2025 (''25625 goal'') has limited mention of NCD research capacity building as a means to improve health, although prior calls for enhanced research capacity have been made [20]. The inconsistent emphasis on research capacity building may stall sustained efforts required to develop a research workforce.
Mendis and colleagues highlighted the fact that poor research productivity is both ''a consequence and a contributory factor'' to the widening gap between cardiovascular morbidity and mortality in wealthier countries compared with poorer countries [13]. We argue that future development assistance for health and health research, such as the recent funding for the Global Alliance for Chronic Diseases (gacd.org) and the National Heart, Lung, and Blood Institute/UnitedHealth-supported Centers of Excellence network (www.nhlbi.nih.gov/about/globalhealth/centers/index. htm), should be expanded to fill this gap in research capacity to help poorer countries, which bear a disproportionate burden of CVD morbidity and mortality, as one tool to improve national cardiovascular health. While delays in discovery to application are common in both high-income and low-and middle-income research settings, the pursuit of research may not be solely evaluated by its potential effect on the burden of disease. In addition, the decision to support increases in health investments should be made in parallel with increases in research investments.

Strengths/Limitations
Our study has several strengths, including development and iterative testing of a cardiovascular bibliometric filter with high level of precision and recall; long study period with a large number of publications and citations downloaded to evaluate trends over time; and development of an interactive webpage for easy exploration of multivariable relationships capitalizing upon publicly available datasets.
However, our study also has limitations. First, small countries with few cardiovascular research publications had unstable measures of several data points, including fractional count as a measure of international collaboration due to their low sample sizes. These results, in particular, should be interpreted with caution. However, we know of no other method for assessing international research collaboration on a global scale. Second, citations are only one, limited measure of the relative importance of a research article to inform practice and improve care, though they are used widely for comparison across (and within) journals [14,21]. Third, the exploration of relationships between country income status, size, cardiovascular research publications and measures such as burden of disease and development are susceptible to reverse causality and confounding and should be considered hypothesis-generating. Fourth, we evaluated publications over the study period that were collected through the Web of Knowledge database, which may have excluded publications from other databases, including others that collect relatively more data from LMICs compared with the Web of Knowledge (Latin American and Caribbean Health Sciences Literature [LILACS], e.g.). However, the Web of Knowledge is the largest bibliometric database currently available and provides standardized author address information for analyses like ours.

Conclusions
Cardiovascular health research has increased substantially in the past decade, with the greatest relative increases in upper middle-income countries such as China. The United States and other high-income, OECD countries remain the dominant producers of cardiovascular research globally, although their share appears to be declining. However, low-and middle-income countries with the higher burdens of cardiovascular disease continue to have lower research output than high-income countries, and thus require targeted research investments to improve cardiovascular health. Figure S1 Country-level cardiovascular research publications against total scientific and technical publications in 1999 (A) and 2009 (B). Total scientific and technical publications as reported by the World Bank with color coding representing World Bank country income status. The size of each country's bubble represents the population size of that country. (TIFF)