https://orcid.org/0000-0003-2568-0930
Figures
Abstract
In science, sleeping beauties (SBs) denotes a special phenomenon of the diffusion of scientific knowledge based on citation trajectories, the awakening of which is also measured through changes in the citations index. However, the rapid advancement of social media has altered the mode of scientific communication and knowledge diffusion. This study aims to re-identify SBs and its Prince from the perspective of comprehensive indicators, which involves the analysis of Altmetrics indexes and Citation index, and investigate the awakening mechanism of A-SB to supplement the research on the awakening mechanism of SBs. By combining Ab index, we redefined the Prince, which makes A-SB receive high attention after a long Sleeping period and reflects the most prominent academic or social behavior that awakens and sustains the Awakening of A-SB. Then we conducted empirical research on the retrieved PLOS Biology collection and examined Prince after identifying the A-SB. The analysis and summary of the characteristics of the identified A-SB and Prince revealed the SBs’ awakening mechanism under the comprehensive trajectory based on Altmetrics from the three dimensions of the influence between the indicators, the overall evolution trajectory of A-SB, and literature bibliometric attributes. In the trajectory of Delayed Recognition stage of A-SB, we define the Dogsleep of SBs, which mirrors that the instability of the Sleeping of SBs will generate a specific negative impact on Prince of A-SB and Awakening intensity. Besides, the literature bibliometric attributes cannot reflect the tendency of users to read academic papers, which again proves that the traditional citation index cannot be neglected in the awakening mechanism of A-SB. Overall, this study demonstrates the addition of the Altmetrics indexes as a useful complement, illustrating the inheritance and connection between the SBs based on the comprehensive trajectory and the SBs based on the citation diffusion trajectory.
Citation: Hou J, Li H, Zhang Y (2020) Identifying the princes base on Altmetrics: An awakening mechanism of sleeping beauties from the perspective of social media. PLoS ONE 15(11): e0241772. https://doi.org/10.1371/journal.pone.0241772
Editor: Feng Xia, Federation University Australia, AUSTRALIA
Received: August 12, 2020; Accepted: October 21, 2020; Published: November 25, 2020
Copyright: © 2020 Hou et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available from Zenodo (doi: 10.5281/zenodo.4034611).
Funding: This research was supported by the National Social Science Foundation of China under Grant 17BGL031.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Van Raan referred to the phenomenon that scientific literature was not cited for a prolonged time after publication (“Sleeping”) and then suddenly received numerous citations (“Awakened by Prince”) as “Scientific Sleeping Beauty” (SBs) [1]. Then, Li & Ye proposed “All-Elements Scientific Sleeping Beauty” to supplement the concept discussed above [2]. Nevertheless, Prince literature is a necessity when it is to awaken SBs. Braun, Glanzel, and Schubert specifically explored the characteristics of Prince [3]. The phenomenon of the evolution of scientific literature has attracted the attention of researchers as early as the early 1960s, at which time the phenomenon of scientific literature evolution was also termed “Resisted Discovery” [4], “Premature Discovery” [5], and “Delayed Recognition” [6]. From the perspective of citation network, SBs is essentially a distinct phenomenon of the diffusion of scientific knowledge based on citation trajectories, which reveals a crucial mechanism for the diffusion of scientific information through citations [3], and is indispensable for scientific development [7,8]. We refer to the SBs under this perspective as Citation-based Sleeping Beauty (C-SB).
However, the rapid advancement of social media has revolutionized the mode of scientific communication and knowledge diffusion. A multitude of data, such as browsing, saving, and discussion of scientific literature on social media platforms, as well as the quantitative measurement index established by it, have offered a novel perspective and approach to explore the identification and evolution trajectory of SBs and Prince. The knowledge evolution trajectory of scientific literature after publication comprises not only the evolution trajectory based on the citations indicator but also the evolution trajectory based on the social media metrics. We refer to the SBs created by the combination of citations indicator and social media metrics as Altmetrics-based Sleeping Beauty (A-SB). Correspondingly, the all-elements scientific SB from the perspective of comprehensive influence is called Altmetrics-based all elements sleeping beauties (Aa-SB). From this standpoint, how should the Prince of SBs be defined? What type of Prince awakened A-SB? How has the awakening mechanism altered? Of note, reexamining the knowledge diffusion and evolution trajectory of scientific literature from the comprehensive perspective of citations and social media metrics serves a vital supplement and innovative development to the traditional citation-based SBs and Prince research. The contributions of this study are as follows:
- We define the Prince of SBs from the perspective of Altmetrics and perform an empirical study on the identification of Prince from the standpoint of Altmetrics.
- We analyze the correlation between A-SB’s Prince and the diffusion trajectory and bibliometric attributes of SBs, followed by exploring the awakening mechanism of A-SB, expanding the research on the awakening mechanism of traditional SBs.
Literature review
The evolution trajectory of SBs and prince based on citations
The current research on the evolution of SBs primarily focuses on the citation evolution of SBs, the reasons for the formation and influencing factors of SBs, the identification of Prince, and the awakening mechanism of SBs. Prior studies have defined the “Sleeping–Awakening” time of SBs (Table 1). The Sleeping and Awakening of SBs denote a continuous period, including (1) sleeping period—being cited two times a year on average within 5 years; (2) awakening period—obtain numerous citations (cited >20 times) in a specific period after the sleeping period (>4 years) [1,9–14].
In the citation evolution trajectory, the early long-tailed distribution of citation contributes to the Sleeping of SBs. Reportedly, the long tail can be quantified as the citation delay [15], and its distribution can be used to identify and measure SBs [16,17]. Simultaneously, it can also estimate the formation of SBs to some extent [2]. Particularly, academic development serves a critical factor in the formation of SBs [18]. However, early citation trajectories also merit careful consideration [19]. Although SBs could be highly cited during the later period of diffusion trajectory [13], it is essential to focus on self-citation in late Sleeping [20].
In the citation evolution trajectory, Citations is also the only indicator to study the awakening pattern of SBs. Citations awakens SBs and enables it to arouse considerable scientific attention [21]; these citations are called “Prince” [1]. Thus, the characteristics of Prince [2,3] and identification criteria [22] have become the primary method to explore the awakening mechanism of SBs. Regarding the relationship between Prince and SBs, some scholars believe that SBs embraces its necessity to be awakened [1]. However, some scholars consider that SBs is awakened at random [23]. At the same time, some scholars think that Prince does not necessarily cite SBs directly, but the two are cited together [24]. After this, some scholars believe that the identification of prince should not only analyze the citers of SBs, but also consider the case that the viewpoint of SBs is cited, but SBs is not listed as a reference [25]. The research object of this viewpoint focuses more on the “citation” of Prince than on Prince itself. Recently, van Raan (2015, 2017, 2018) found that SBs is more likely to be cited by patent literature, and the technology drive is the main awakening mechanism of SBs [26–28].
In previous research, the attention on SBs and its awakening mechanism primarily focused on the characteristics and standards of Prince, such as the investigation of the inherent characteristics of the subject content and author relationship of Prince. Owing to the existence of indirect citation and invisible citation, the research on “citation” is overlooked. Besides, the influence of citations during sleeping on the awakening mechanism of SBs and Prince is ignored. Moreover, SBs are cited in some documents during sleeping but have not been successfully awakened. “Heartbeat Spectrum” proposed by Li and Ye observed the frequency of citations during each year of Sleeping [29]; but they focused more on their impact on the formation of SBs. On the other hand, the studies mentioned above examined SBs based on the trajectory of citation evolution. However, with the rapid development of social media platforms, the influence of measuring the impact of scientific literature based on the trajectory of citation evolution has expanded markedly. Besides the citation trajectory of citations, the measurement of the dynamic evolution trajectory should also consider the evolution trajectory of Altmetrics indexes, including Viewed, Saved, Discussed, and Recommended.
The evolution trajectory of scientific literature based on social media metrics
With the rapid growth of social media, the evolution trajectory of scientific literature comprises not only perspectives based on the evolution trajectory of citations but also evolution trajectories based on social media [30–32]. Altmetrics are the measurement indexes used to measure the social influence diffusion trajectory of scientific literature [33]. Reportedly, the correlation between some Altmetrics and citations is weak [34,35] such as the weak correlation between the number of tweets and citations [36,37]. Several studies have revealed a strong correlation between Altmetrics indexes (Save, Discussion, Download, Read in Mendeley [save], Number of readers in Mendeley, Recommendation measures, The number of tweets, F1000, and Bookmarks) and citations index (Table 2) [38–52]. Nevertheless, Altmetrics indexes and citations index are not simply related, and the differences between different disciplines directly affect their correlation [53]. At present, Altmetrics cannot replace traditional Bibliometrics but has already been a crucial complement of Bibliometrics and Scientometrics [54,55].
Accordingly, researchers combined Citations and Altmetrics indexes to explain and define documents of different evolution types, revealing that the performance of highly cited documents on Altmetrics indexes differs from that of citation evolution [56,57]. By combining Bibliometric indexes and Alternative Metrological indexes, a comprehensive document scoring system was constructed [58–61] and was used to conduct empirical research [62–65]. These studies have, to a certain extent, verified the feasibility of comprehensive citations and Altmetrics to measure the influence of academic papers.
Existing studies have focused more on the relationship between Altmetrics indexes and citation, but have not extensively studied or focused on the comprehensive evolution trajectory of scientific literature under the combined action of citations and Altmetrics. Alternatively, the studies have focused on providing a comprehensive scoring system for scientific literature in terms of Altmetrics, in order to evaluate scientifically, but not revealing the specific evolution characteristics of scientific literature under the combined action of Altmetrics indexes and Citations. Recently, Hou & Yang proposed Social media-based Sleeping Beauty and reported the characteristics of SBs recognized on the basis of social media metrics. This study did not incorporate Citations, and the identified SBs embraced no citations [66]. Therefore, there exists certain differences between the evolutionary trajectory and characteristics of SBs under the action of comprehensive indicators combined with social media and citations and the evolutionary trajectory of SBs based on the function of social media indicators. This study preliminarily analyzed the causes of SBs's Awakening on social media platforms, but it did not involve the awakening mechanism in a deep level.
Thus, unlike SBs identified through citation diffusion trajectory of scientific literature [1], we termed the SBs identified through the comprehensive evolution of scientific literature (including Citations and social media indicators) as Altmetrics-based Sleeping Beauty (A-SB). Among them, for all elements SBs under the comprehensive influence trajectory, we termed it Altmetrics-based all elements sleeping beauties (Aa-SB), and Aa-SB is a special A-SB. This study combines Citations and Altmetrics indexes to reidentify SBs (A-SB) and its Prince from the perspective of comprehensive indicators, as well as explore the awakening mechanism of A-SB to complement the research on the awakening mechanism of SBs.
Data and methods
Data collection and processing
The literature data (3541 documents) were adopted from the PLOS Biology journal since its publication as a sample, and the data were obtained from the PLOS Biology Open Access Platform and the Core Collection database of Web of Science. Among them, the data of citations index of the literature were generated from the number of documents published each year by PLOS Biology, which is included in the Core Collection database of Web of Science, and the number of citations obtained each year for each article. The social media metrics of the literature include View, Save, Discussed, and Recommended indexes [67–69]. The data were derived from the open-access data of the PLOS Biology website; Table 3 shows their specific sources and definition. Among them, Viewed and Discussed are from Open Access Platforms and social networking sites, Saved are from the document management website, Recommended are from scientific paper online recommendation platform, and Citations from the scholarly search engine.
We summarized and cleaned up the relevant data of each article in PLOS Biology. Then, we removed the retraction, Correction, Letter, Biographical Item, and other documents, and used Excel and MATLAB 2018b to classify and calculate the selected target data.
We used SPSS Statistics 21 to perform correlation analysis of the indicator information at each stage to measure the correlation between social impact and citations and further determine whether Ab could reflect the comprehensive influence of a document to certify the feasibility of identifying ASB and Prince.
First, the normality test was performed on all indicator data in the sample literature (Table 4), which revealed that they did not conform to the normal distribution data. Thus, the Spearman correlation coefficient was used to analyze the correlation among various indicators. Table 5 shows the correlation coefficient between each indicator and the cited number and Ab.
As shown in Table, Citations have a significant correlation with Viewed, Saved, Discussed, Recommended, and Ab. The correlation coefficients between Citations, Viewed, Saved, and Ab were all >0.6. Besides, Citations and Discussed, and Citations and Recommended were 0.073 and 0.353, respectively, suggesting a weak correlation. Thus, it is feasible to construct a comprehensive impact evaluation system of Citations and Altmetrics indexes, as well as identify A-SB and its Prince. Moreover, when identifying Prince, we will also focus on Viewed, Saved, and Citations.
Metrics and methods
We used the Altmetrics-based beauty index (Ab Index) to measure the comprehensive impact generated monthly after the publication of a document, that is, the value of the function of the community of citations index (IA) and social media metrics (IS) as follows:
(1)
To explain the comprehensive evolution trajectory of a document since its publication, we used five types of indicators—Citations (C), Viewed (V), Saved (S), Discussed (D), and Recommended (R)—to elucidate changes of a document in the comprehensive evolution trajectory generated after its publication. These indicators illustrate the influence of a paper from different perspectives. However, what catches our attention is how to objectively assess the comprehensive impact of a document according to these indicators from different perspectives? Thus, we try to deal with these indicators from such a perspective and build a comprehensive trajectory model.
Thus, the dynamic change formula of the Ab Index for the i-th month since the publication of a document is as follows:
(2)
Among which, Wtv, Wts, Wtd, Wtr and Wtc are the corresponding weights of V, S, D, R and C respectively; i denotes time, which in this study is the i-th month after the publication of a document; and Abi denotes the comprehensive influence of the i-th month after the document’s publication. It should be noted that Web of Science does not provide the monthly citation of all literature, only the citation data of literature in each year can be obtained, so the Ci here is the average of the monthly citation quantity of literature in a certain year [70]. For example, a document is cited 12 times in the third year after publication, suggesting that each month in the third year is cited once.
For the determination of weightsWtvWtsWtdWtrWtc Wtv, Wts, Wtd, Wtr, Wtc, using the Analytic Hierarchy Process, we constructed a structural matrix, based on the effect of five types of indicators on the comprehensive evolution trajectory of a document and assigned different weight values to each indicator. Among these, we selected the value in the model matrix based on 9 scale, which can fulfill the needs of ranking the five indicators under a single rule. Furthermore, by using the MATLAB tool, we accurately evaluated the five indexes and obtained a comprehensive weight.
The basic steps of the analytic hierarchy process are as follows:
Compare the influence of n factors X1,X2,…,Xn on a factor (influence) at the previous level, you can choose Xi and Xj from X1,X2,…,Xn to compare their contribution to influence (Or importance). Assign values to Xi/Xj according to the following “1–9 scale.”
Building the model matrix:
By evaluating, the maximum of its eigenvalues is λmax = 5.2837, and the eigenvector at the maximum eigenvalue is as follows:
By performing consistency test on the results, , the consistency ratio was
. Thus, RI is random consistency test, when n = 5, RI = 1.12. Hence, the matrix passes the consistency test. By conducting the Normalization Processing on the eigenvector, the weights were obtained, that is, Wtv = 0.0321, Wts = 0.048, Wtd = 0.176, Wtr = 0.176, and Wtc = 0.5679. Hence, the Ab Index can be expressed as follows:
(3)
From a citation-based perspective, the time-statistical unit used by the C-SB Institute is based on years. However, since the publication cycle of several documents is monthly, it creates a large time gap between the documents published in January and December each year when the statistics are collected on an annual basis. If a document published in January awakens after 4 years of Sleeping, then its Sleeping time is ≥47 months; however, if a document published in December awakens after 4 years of Sleeping, its Sleeping time is ≥36 months.
Nonetheless, from the perspective of Altmetrics-based, the spread of literature on social media platforms is faster, and the diffusion trajectory of the literature in monthly units can more precisely denote the dynamic diffusion process of the literature. Thus, under the comprehensive evolution trajectory, we adopted Th = 36 months as the standard, that is, when a document awakens after Sleeping for ≥36 months, it was considered to be Altmetrics-based SB (Delayed Recognition).
Hence, the Combined Ab index, we redefined the Sleeping and Awakening of Altmetrics-based SB.
Awakening: In the C-SB identification research, it typically takes years as the unit of time. Scientific literature needs to be awakened for 4 consecutive years before it can be termed as true Awakening; however, it is not necessarily continuous in terms of monthly time. To ensure the research consistency, we selected four consecutive time intervals to determine the literature as Awakening. However, under the social media platform, we define SBs enter the Awakening stage the comprehensive influence value for 4 consecutive months (Abi) is higher than the average monthly comprehensive influence of all the literature of the journal We selected the indicator because in the study of the Awakening of SBs, it is unfair to compare publications from different scientific disciplines, as significant difference exists in speed and frequency of citation accumulation across different fields of science [70–72].
Sleeping: The average comprehensive influence of the literature for 4 consecutive months is (
Sleeping). Before entering the Awakening, we consider that SBs is in the Sleeping stage, which includes both Sleeping and Dogsleep.
Dogsleep: In a specific period, when the Awakening or Sleeping of a document does not change continuously (continuous Awakening or Sleeping does not exceed 4 months), the document is considered to have entered Dogsleep. Precisely, we defined that when , or when the comprehensive influence value (Abi) is not greater than the average value of the comprehensive influence of all literature (
) (
) in the journal for four consecutive months, the SBs is in Dogsleep.
Awakening time (Tw-k): The duration of Awakening, where k denotes the times Awakening, and when k = 1, Awakening time.
Dogsleep time (Tb-k): The duration of Dogsleep, where k denotes the times of continuous Dogsleep, and when k = 1, it denotes the first Dogsleep time.
Sleeping time (Ts-k): The Sleeping duration, where k denotes the times of continuous Sleeping, and when k = 1, it denotes the first Sleeping time.
Awakening intensity: To describe the literature characteristics at different stages, we used two variables, average () and standard deviation (σ). We assumed that the process of the Sleeping and Awakening is as follows: Sleeping/Dogsleep–Awakening–Sleeping/Dogsleep–Awakening–Sleeping/Dogsleep. When in Sleeping,
, the smaller the
and σ, the stronger the Sleeping intensity of the literature. Conversely, when in Awakening,
, the larger the
and σ, the stronger the Awakening intensity of the literature. Likewise, during Dogsleep, the larger the
and σ, the stronger the Dogsleep intensity of the literature.
In this study, we used the following definitions (Table 6) of the degree of continuous change in different states as the criteria for identifying A-SB and AA-SB.
Dogsleep: In the trajectory study of C-SB, scholars focused on the trajectory characteristics of Sleeping-Awakening. However, some scholars have noticed the low citation [29] and temporarily high citation of SBs during Sleeping [73]. Under this phenomenon, there is no continuous change in the Awakening or Sleeping of a document, which is known as entering Dogsleep.
Prince: The literature that awakens SBs under the citation track is called Prince. Under the social media platform, Citations is no longer the only indicator for awakening SBs. Citation a specific academic behavior. Under the social media platform, the Awakening of A-SB is achieved by the combined action of various indicators, but there must be a decisive indicator with the largest proportion, which reflects the most prominent academic or social behaviors that awaken and maintain A-SB's Awakening, and its appearance makes A-SB highly concerned after a long period of Sleeping. Specifically, we define that Prince needs to meet the following requirements: ①It will have the highest proportion of overall influence in the first month of the Awakening stage, thus awakening A-SB. ②It is expected to have a sustained high influence during the Awakening stage, to contribute most to maintaining the Awakening of A-SB.
“Prince” is not defined as the leading indicator in the first month of the Awakening stage because on social media platforms, the real Awakening of A-SB is a continuous process, not an instant; this is important which criteria distinguishes Awakening from Dogsleep. In addition, in the current academic communication environment, the definition of “Prince” only as the first citer is not conducive to considering the motivation of Awakening and accurately defining the Sleeping-Awakening stage [22].
Results
We used the methods and definitions mentioned above to conduct empirical research on the retrieved PLOS Biology collection. After identifying the A-SB, we examined Prince of A-SB, and explored the awakening mechanism of A-SB, supplementing the research of traditional SBs.
Identification of A-SB
We evaluated the monthly comprehensive influence of all documents in PLOS Biology. First, we calculated the average value () of the comprehensive influence. Thus, for the literature published in PLOS Biology, the Awakening is defined as (Abn⋯Abn−3)>5.38 and the Sleeping is Ab△n≤2.69. Then, we computed the monthly comprehensive influence value of 3541 documents since publication, as well as identified A-SB and Aa-SB in the sample (Table 7) based on the degree of continuous change in their impact value, discovering 4 A-SB (Figs 1–4) and 7 Aa-SB (Figs 5 and 6) [74–86]. We found that the trajectories of A-SB and Aa-SB are right skewed of long tail, and their Ab presented a high peak in the early stage, and then entered Dogsleep. After being awakened by Prince, they obtained a sustained high Ab. The difference was that the duration of the high Ab of A-SB did not exceed the four consecutive months defined by Awakening, and the duration of Aa-SB was no less than four months. Therefore, we considered that A-SB was in Dogsleep and was not awakened, while early Awakening occurred in Aa-SB. It should be noted that Aa-SB is a special form of A-SB. The time range we studied was the Delayed Recognition stage of A-SB, during which stage Aa-SB occurred early Awakening and second Awakening.
The identification of prince of A-SB
According to the definition of Prince in 3.2, we identified the Prince of A-SB.
Analysis of the proportion of various indicators in the awakening stage of A-SB.
We analyzed the proportion of each indicator of Ab in the Awakening stage of 11 A-SB. Given that 11 A-SB have not been affected by Recommended during Awakening, we will not take Recommended into consideration in the following analysis. As shown in Table 8, we found that the indicator of the largest proportion of Ab in 11 A-SB was Viewed. Among the average values of the proportions, the largest is Viewed and the smallest is Discussed.
In order to observe the changing trend of various indicators in different stages of A-SB trajectory, we summarized the proportion of the comprehensive influence of various indicators in different stages of A-SB in Delayed Recognition stage (Table 9). It can be found that the proportion of Viewed occupied the highest proportion in the overall influence of the Delayed Recognition stage, and that the proportion of it after A-SB is awakened is greater than that of A-SB in sleeping. Differently, the proportion of other indicators decreases after A-SB awakens. Among which, the decline of Citations is the most pronounced, and it works when A-SB is in Sleeping, but not strong enough to awaken A-SB. Only Viewed's influence soars before and after A-SB's awakening.
Therefore, according to the proportion of the comprehensive influence in the Awakening stage, Viewed is the most active indicator in awakening 11 A-SB, while Saved and Citations play an essential role in some A-SB. However, the proportion of indicators cannot explain the real function of them in awakening A-SB, because there may be uneven or relatively dispersed distribution of indicators. In order to determine whether these indicators really awaken A-SB, it needs to be analyzed in combination with their changing trend and specific trajectory before and after awakening.
Trajectory analysis of each indicator in the awakening stageof A-SB.
In order to explore the specific role of each indicator in Awakening of A-SB and find out the reason of A-SB's awakening, we presented the influence track of each indicator of A-SB in the form of stacked rectangle graph and analyzed them (Table 10). Among them, in order to compare the causes of two Awakening of AA-SB more clearly, the early Awakening of AA-SB is displayed on the same track as the second Awakening.
Among 4 A-SB, except Del Cul A(2007), A-SB can be awakened and maintain its awakening only by Viewed. Based on the track of Del Cul A(2007), we found that during the first, second, third, fifth, sixth and twelfth months of its Awakening, Viewed alone can not awaken A-SB, but requires cooperation with Citations and Saved. However, because Saved has less influence, it is far from being able to awaken the literature without Viewed. Citations has a certain influence, but due to the dispersion of its distribution, its monthly influence is not strong enough to dominate the Awakening of A-SB. Viewed's influence is close to A-SB's Awakening boundary and enjoys a continuity, so we believe that it is Viewed that dominates the awakening of Del Cul A(2007). This suggests that, from the perspective of comprehensive influence, academic researchers' citation of literature is still one of the reasons to awaken A-SB, but it is no longer the most significant factor. A-SB is also subject to mentions, discussions, sharing and other communication behaviors by users on social networking websites in the Awakening stage, but from the perspective of the influence of Discussed, it is not the reason for A-SB's awakening.
In 7 Aa-SB, we found that the indicator that contributes to their awakening was Viewed. In the second Awakening, Khalturin K (2008) and Powell K (2007) were awakened by Viewed alone, and Saved participated in the maintenance of Awakening; Alerstam T (2007) was also awakened by Viewed alone, but Citations participated in maintaining Awakening. However, because the influence of Viewed was near the Awakening boundary, the influence of Saved and Citations was minimal, and their role was more like assisting Viewed's awakening. The second Awakening of other Aa-SB was completely dominated by Viewed alone.
Although the leading inicators of A-SB and Aa-SB's early and second Awakening are Viewed, their trajectory and influence are different (Figs 7–9). We found that Viewed’s distribution curve of A-SB in the Awakening stage showed a slight vibration trend, and the whole was approaching the Awakening boundary line. Viewed’s distribution curve of Aa-SB showed a decreasing trend in the early Awakening, but Viewed showed no evident trend in the second Awakening, but all of them were near the Awakening boundary line. After separating the first and second Awakening of Aa-SB, we observed that the Viewed obtained during the early Awakening of Aa-SB was significantly greater than that of the second Awakening, and the Viewed of the second Awakening was generally greater than that of A-SB’s Awakening. In addition, the Discussed, Saved, and Citations of Aa-SB all focused on the second Awakening, which indicates that Aa-SB was highly viewed and downloaded by users shortly after publication, but then quickly lost the user's attention until being awakened by browsing and downloading again, it began to be widely discussed, saved and cited by website users and academic researchers.
The Boundary of awakening is formed by the influence of Viewed solely, in order to check whether the documents could enter awakening only by Viewed.
The Boundary of awakening is formed by the influence of Viewed solely, in order to check whether the documents could enter awakening only by Viewed.
The Boundary of awakening is formed by the influence of Viewed solely, in order to check whether the documents could enter awakening only by Viewed.
Therefore, considering the reason why 4 A-SBs and 7 Aa-SBs were awakened, we believe that Prince of 11 A-SB are Viewed, and their awakening is all dominated by the browsing and downloading of the users’ social behavior. Moreover, compared with normal A-SB, Aa-SB is more sensitive to Prince’s “kiss.” Thanks to Viewed, their overall influence in final Awakening is higher than that of A-SB. In addition, 11 documents have not been officially recognized and recommended by academic researchers during the Awakening stage. Citations have played a role in the awakening of some documents, but it is no longer the most important indicator of A-SB’s Awakening. However, because Saved and Citations have a certain proportion in the Awakening stage, although Viewed is Prince, some documents cannot maintain the Awakening of A-SB by Viewed solely, which makes their reason for awakening more complicated.
Research on the awakening mechanism of A-SB
Based on the results of the normality test in Section 3.1, we used the Spearman correlation coefficient to complete the following analysis.
Correlation analysis of the indicators of A-SB in awakening.
In the identification of Prince, we found that there were two awakening methods of A-SB: one is that Prince solely awakens A-SB; the other is that Prince awakens A-SB in corporation with other indicators. In order to further analyze the interaction mechanism of Prince and other indicators during the awakening process of A-SB, we conducted a Spearman correlation analysis on the identified Prince of 11 A-SB, Viewed, and other indicators in Awakening (Table 11). This indicates that in the Awakening of A-SB, the massive browsing of users, on one hand, can awaken A-SB solely, and on the other hand, can guide users’ saving behaviour, thereby assisting the awakening of Prince or improving the comprehensive influence of Awakening. In this interesting phenomenon, the relationship between Viewed and Saved resembles a prince and a follower, Saved accompanies Viewed and assists it to awaken A-SB.
In addition, there is no correlation between citations, Altmetrics indexes and Ab, because of the discontinuity of the distribution of citations in Awakening. This demonstrates that the role of citations in the Awakening of A-SB is not related to Altmetrics indexes. It reflects the academic influence of A-SB and is relatively independent in the process of awakening A-SB. However, citations have played an essential role in awakening some A-SB. From the perspective of comprehensive influence, we still need to pay attention to traditional Citations.
Therefore, based on the above findings, we believe that there is a deeper internal mechanism in the awakening mechanism in which Prince cooperates with other indicators to awaken A-SB. The indicators, including Prince, are not completely independent of each other when they play a role in Awakening, reflecting that the communication between different social behaviors under the social media platform makes the awakening mechanism of SBs more diverse.
Correlation analysis of prince and indicator information at each stage.
The trajectory of A-SB in Delayed Recognition on the social media platform is dynamic and continuous. In order to further study whether Prince is related to other stages of A-SB, and explore the awakening mechanism of A-SB from a holistic perspective, we conducted correlation analysis between Prince of A-SB, Awakening intensity, the intensity of other stages and Viewed (Table 12). Among them, we integrated the data of Aa-SB's two awakenings to ensure the consistency of A-SB and Aa-SB research.
As shown in the results of Spearman correlation analysis (Table 13), we found that there was a significant strong correlation between the Ab mean, standard deviation and Viewed of Dogsleep and the Ab standard deviation of the Sleeping stage, the Viewed of Dogsleep and the Ab standard deviation and Viewed of the Sleeping stage, the duration of Dogsleep and the mean of comprehensive influence of the Sleeping stage. What’s more, there was a significant moderate correlation between the Viewed of Dogsleep and its duration of Sleeping. These prove what the definition assumes about Dogsleep: the greater the Dogsleep intensity, the greater the Ab standard deviation in the Sleeping stage, and the more unstable the sleeping of A-SB. As for A-SB's awakening, we found that there was a significant moderate negative correlation between the Ab mean of Dogsleep and the Viewed of the Awakening stahe, the Ab standard deviation of the Sleeping stagge and the Ab mean of the Awakening stage, the viewed of Dogsleep and the Ab mean of the Awakening stage. This indicates that in the overall trajectory of A-SB, the distribution of Prince in Dogsleep and Dogsleep intensity have a negative effect on Awakening intensity and the scale of Prince, while the greater the Dogsleep intensity of A-SB, the less stable the sleeping. Therefore, we consider that the instability of A-SB’s sleeping will hinder the scale of Prince and the degree of attention after awakening to a certain extent. Additionally, the Viewed of Awakening presents a significant strong correlation with the duration and intensity of this stage, which once again demonstrates the leading role of Viewed in awakening A-SB.
Influence of A-SB’s bibliometric attributes on awakening.
Previous researches have studied in detail the relationship between the bibliometric attributes and diffusion trajectories of SBs. In order to explore whether the bibliometric attributes of A-SB are related to their awakening and whether they have some critical features of C-SB, we try to mine some important bibliometric attributes of A-SB (Table 14).
Through the correlation analysis of A-SB’s bibliometric attributes and the indicators in its Awakening stage (Table 15), we discovered that there was no correlation between the Prince of 11 A-SB—Viewed, the Ab of the Awakening stage and the bibliometric attributes, that there was a significant medium positive correlation between Discussed and the number of authors, and that Citations presented a strong positive correlation with A-SB’s title length, text length, number of references, and number of keywords. This reflects that, on the one hand, the longer the title and text of A-SB, the more documents cited; the more extensive the research topics involved, the more citations can be obtained in the Awakening stage, which is similar to C-SB, although Citations does not make a difference during the awakening process of 11 A-SB. On the other hand, it shows that the diffusion trajectory of scientific literature reflected by Ab is not consistent with the citation trajectory. Ab can measure the influence of literature from two aspects of citation diffusion and social media diffusion, and identify A-SB. Meanwhile, the greater number of authors is conductive to A-SB getting more communication and discussion in Awakening.
Viewed numerically, although Viewed and Ab have negative correlation coefficients with title length, number of authors and text length, they are not significantly correlated. It indicates that users, who comprise the public as main body, generally pay little attention to the length of title and text of the documents when browsing academic papers, which is inconsistent with the tendency of academic researchers when choosing documents.
Conclusions
This study extends SBs from a research perspective based on citation indicator diffusion trajectory to a comprehensive evolution trajectory research based on Altmetrics indexes. The research identified its Prince based on the recognition of A-SB, explored the specific reasons for the Awakening of A-SB, and excavated awakening mechanism of A-SB from the three dimensions of the influence between the indicators, the overall evolution trajectory of A-SB, and literature bibliometric attributes, making the SBs and identification of its Prince more accord with the actual diffusion and evolution of the scientific literature. During the research process, the following conclusions were mainly drawn:
- The study, which defines SBs princes under the Altmetrics perspective, points out the new characteristics of the Prince on social media platforms. Under the social media platform, the awakening of A-SB is realized under the action of a single indicator or the combination of multiple indicators. However, there is bound to be a decisive indicator with the largest proportion, which makes A-SB receive high attention after a long Sleeping period and reflects the most prominent academic or social behavior that awakens and sustains the Awakening of A-SB. Based on these features, we described the Awakening trajectory of SBs through quantitative Ab index, and recognized A-SB and its Prince from a new perspective.
- Based on Ab index, this study identifies four A-SB literature and seven Aa-SB literature among 3541 literature in the journal PLOS Biology, and conducts empirical research on Prince identification based on this literature. We found that there are two awakening methods in the 11 literature during Delayed Recognition stage, and users’ browsing and downloading of the information of online literature is considered as Prince, which can often awaken A-SB alone and make the greatest contribution in the way of awakening in cooperation with other indicators.
Different from the awakening mechanism of SBs under the view of traditional citation, Citation does not dominate the awakening of A-SB but, together with Saved, plays the role of assisting Prince to maintain the Awakening of A-SB. In addition, A-SB is rarely discussed or evaluated by researchers on social networking sites during its Awaking stage, which is closely related to the number of co-authors of A-SB. Nor has it been officially endorsed by researchers. These, on the one hand, reflect the importance of the dimension of social influence in A-SB’s awakening mechanism. Academic researchers are not the main body to awaken A-SB. On the other hand, these fully reflect that adding Altmetrics indexes into the study of SBs is a beneficial supplement, making the awakening mechanism of SBs more comprehensive and scientific. - This study reveals the internal mechanism within A-SB awakening mechanism from three different dimensions, and finds that during the awakening process of A-SB, there is a dynamic relationship between the indicators in the Awakening stage and the stages under the overall trajectory. We discovered that the spread between two different users’ social behaviors—users’ browsing and downloading of online literature information and users’ behavior of saving information in the literature manager such as adding bookmarks—has a significant effect on the Awakening of A-SB, an interesting phenomenon similar to that of Prince and his squire in the awakening mechanism of SBs under traditional citation trajectory. In the trajectory of Delayed Recognition stage of A-SB, we define the Dogsleep of SBs, which mirrors that the instability of the Sleeping of SBs will generate a certain negative impact on Prince of A-SB and Awakening intensity. It is also a supplement to the diffusion trajectory of the SBs. Besides, the title, length, the number of references and the amount of the topics involved are all conducive to the accumulation of Citations in the Awakening process of A-SB, however, from the point of Altmetrics, they cannot reflect the tendency of users to read academic papers, which again proves that traditional citation index cannot be neglected in the awakening mechanism of A-SB. These findings indicate that the awakening mechanism of A-SB is both related to and new to SBs in the traditional citation trajectory.
Discussions
In the traditional citation evolution trajectory, the Citations represents the academic influence of scientific literature. Contrarily, on social media platforms, Altmetrics indexes can not only reflect the social influence of scientific literature but also effectively evade the highly cited phenomenon caused by fake citations and the Matthew effect [87], thereby improving the accuracy of identifying literature with trajectory under the comprehensive evolution trajectory. However, some researchers believe that Altmetrics primarily reflects “attention” rather than influence or influence force [37,67]. When combing the dynamic distribution of the indicators of A-SB, we also observed that early surge existed in the Altmetrics indexes because, as an OA platform, PLOS access to literature is not subject to subscription, and researchers tend to view the latest published papers online, which can accumulate Viewed quickly in the early days of publication. Maybe this phenomenon should be explained from the perspective of “attention.” Therefore, from the point of Viewed, the long-tailed phenomenon of the time distribution of Viewed of A-SB denotes a rapid decline in user’s attention to them, that is, A-SB will enter into Sleeping stage because of being quickly forgotten. During Sleeping, the Ab of A-SB will reach a state that is neither fully sleeping nor truly awakening, causing it to fall into Dogsleep (similar to “heartbeat” [29]. Owing to the short duration, it occupies less proportion in the evolution trajectory of SBs, but it does not mean that its existence should be ignored, especially the prediction research of SBs. Besides, regarding the awakening of A-SB, we found the significance of Viewed as Prince. However, browsing behaviors on social media platforms include both web browsing and literature resource downloading, and the latter is obviously a more valuable use of literature. Thus, the problem that whether the effectiveness of Viewed on the Open Access Platform as a scoring indicator of the comprehensive evolution trajectory of scientific literature be recognized still remains. Besides, the interesting question is that A-SB is a supplement to the scientific Sleeping Beauty research, which, like the traditional sleeping beauty, would receive continuous attention after awakening. However, will A-SB receive continuous and more attention or reference after the Delay Recognition stage? This is related to the second stage of the diffusion track of literature, which is worth exploring. Moreover, despite interesting phenomena between the bibliometric attributes of the literature and the trajectory of the Awakening stage, some cannot be considered as effective conclusions due to the sample quantity of A-SB. The questions above need to be further discussed and studies.
This study has some limitations worth acknowledging. First, during the collection and evaluation of the citations index, considering that we currently cannot obtain monthly citation information for each literature in a longer interval in the Web of Science database, we adopted the average annually cited data of each literature on a monthly basis, which could have a certain impact on our research results. However, this impact will not cause major changes to the study results. In the future, with the gradual improvement of the development of Open Access, this issue will be solved. To date, various scholarly data have easily accessed and powerful data analysis technologies being developed, which will also help us to develop and improve new perspectives of scientific research [88]. Second, the disciplines covered by the sample journals are relatively single, and the data are generated from the published literature of PLOS Biology from 2003 to 2019. The disciplines are concentrated in the field of biology. In this case, whether the analysis of bibliometric attribute dimensions in the research has universal applicability remains unclear. However, we believe this issue could affect the conclusion of bibliometric attributes but not the trajectory of A-SB (the latter is our main concern), because our parameters are set according to the overall situation of the sample. Thus, with the expansion of the total number of samples, the parameters will be improved; this would help to improve our current conclusion, but it does not imply that our conclusion is completely invalid. Finally, in this study, the scope of our discussion is only the trajectory in Delayed Recognition stage of SBs, but we found during the research that some literatures have multi-stage Awakening with >3 stages. The question arises, does the Prince of that literature with multi-stage Awakening have a new feature? There is no denying that the breadth of the A-SB research still requires further expansion.
In the future, with the promotion and improvement of Open Access Platform, the information of journal literatures will be more abundant, and extending the time window of the literature will become feasible. Thus, we will focus on exploring the validity of the characteristic of the Prince of SBs under the perspective of Altmetrics in an interdisciplinary context after the expansion of data samples, and further improve the awakening mechanism of SBs.
References
- 1. van Raan AFJ. Sleeping Beauties in science. Scientometrics. 2004;59(3):467–72.
- 2. Li J. Citation curves of "all-elements-sleeping-beauties": "flash in the pan" first and then "delayed recognition". Scientometrics. 2014;100(2):595–601.
- 3. Braun T, Glanzel W, Schubert A. On Sleeping Beauties, Princes and other tales of citation distributions. Research Evaluation. 2010;19(3):195–202.
- 4. Barber B. Resistance by scientists to scientific discovery. Science (New York, NY). 1961;134(3479):596–602. pmid:13686762
- 5. Stent GS. Prematurity and uniqueness in scientific discovery. Scientific American. 1972;227(6):84–93. pmid:4564019
- 6. Garfield E. Premature discovery or delayed recognition—Why? Current Contents. 1980;4:488–93.
- 7. Hu Z, Wu Y. Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals. Journal of Informetrics. 2014;8(1):136–46.
- 8. Wang J, Ma F, Chen M, Rao Y. Why and how can "sleeping beauties" be awakened? Electronic Library. 2012;30(1):5–18.
- 9. Garfield E. Delayed recognition in scientific discovery: Citation frequency analysis aids the search for case histories. Current Contents. 1989;23:3–9.
- 10. Garfield E. More delayed recognition. Part 2. From inhibin to scanning electron microscopy. Current Contents. 1990;9:3–9.
- 11. Garfield E. More delayed recognition. Part 1. Examples from the genetics of color blindness, the entropy of short-term memory, phosphoinositides, and polymer Rheology. Current Contents. 1989;38:3–8.
- 12. Glanzel W, Garfield E. The myth of delayed recognition. Scientist. 2004;18(11):8–.
- 13. Glanzel W, Schlemmer B, Thijs B. Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon. Scientometrics. 2003;58(3):571–86.
- 14. Hou J, Yang X. Social media-based sleeping beauties: Defining, identifying and features. Journal of Informetrics. 2020;14(2):101012. https://doi.org/10.1016/j.joi.2020.101012.
- 15. Wang J, Thijs B, Glanzel W. Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity. Plos One. 2015;10(5).
- 16. Du J, Wu Y. A parameter-free index for identifying under-cited sleeping beauties in science. Scientometrics. 2018;116(2):959–71.
- 17. Sun J, Min C, Li J. A vector for measuring obsolescence of scientific articles. Scientometrics. 2016;107(2):745–57.
- 18. Egghe L, Guns R, Rousseau R. Thoughts on uncitedness: Nobel laureates and fields medalists as case studies (vol 62, pg 1637, 2011). Journal of the American Society for Information Science and Technology. 2012;63(2):429–.
- 19. Li J, Ye FY. Distinguishing sleeping beauties in science. Scientometrics. 2016;108(2):821–8.
- 20. El Aichouchi A, Gorry P. Delayed recognition of Judah Folkman's hypothesis on tumor angiogenesis: when a Prince awakens a Sleeping Beauty by self-citation. Scientometrics. 2018;116(1):385–99.
- 21. Huang T-C, Hsu C, Ciou Z-J. Systematic Methodology for Excavating Sleeping Beauty Publications and Their Princes from Medical and Biological Engineering Studies. Journal of Medical and Biological Engineering. 2015;35(6):749–58.
- 22. Du J, Wu Y. A Bibliometric Framework for Identifying “Princes” Who Wake up the “Sleeping Beauty” in Challenge-type Scientific Discoveries. Journal of Data and Information Science. 2016;1:50–68.
- 23. Burrell QL. Are "sleeping beauties" to be expected? Scientometrics. 2005;65(3):381–9.
- 24. Ohba N, Nakao K. Sleeping beauties in ophthalmology. Scientometrics. 2012;93(2):253–64.
- 25. Shen l, Yu G, Zhang X, Zhang WF. Identifying princes of Sleeping Beauty—Knowledge mapping in discovering princes. International Conference on Management Science and Engineering—Annual Conference Proceedings. 2014:912–8.
- 26. van Raan AFJ. Dormitory of Physical and Engineering Sciences: Sleeping Beauties May Be Sleeping Innovations. Plos One. 2015;10(10). pmid:26469987
- 27. van Raan AFJ. Sleeping beauties cited in patents: Is there also a dormitory of inventions? Scientometrics. 2017;110(3):1123–56. pmid:28255185
- 28. van Raan AFJ, Winnink JJ. Do younger Sleeping Beauties prefer a technological prince? Scientometrics. 2018;114(2):701–17. pmid:29449753
- 29. Li J, Shi D, Zhao S, Ye F. A study of the “heartbeat spectra” for “sleeping beauties”. Journal of Informetrics. 2014;8:493–502.
- 30. Bornmann L. What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology. 2013;64(2):217–33.
- 31. Bornmann L, Haunschild R, Adams J. Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF). Journal of Informetrics. 2019;13(1):325–40.
- 32. De Silva P, Vance C. Assessing the Societal Impact of Scientific Research. 2017. p. 117–32.
- 33. Prime J, Taraborelli D, Groth P et al. Altmetrics: A manifesto Retrieved. 2010. http://altmetrics.org/manifesto/
- 34. Costas R, Zahedi Z, Wouters P. Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology. 2015;66(10):2003–19.
- 35. Waltman L, Costas R. F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations. Journal of the Association for Information Science and Technology. 2014;65(3):433–45.
- 36. Haustein S, Peters I, Sugimoto CR, Thelwall M, Lariviere V. Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Science and Technology. 2014;65(4):656–69.
- 37. De Winter JCF. The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics. 2015;102(2):1773–9.
- 38. Sud P, Thelwall M. Evaluating altmetrics. Scientometrics. 2014;98(2):1131–43.
- 39. Eysenbach G. Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research. 2011;13(4). pmid:22173204
- 40. Xia F, Su X, Wang W, Zhang C, Ning Z, Lee I. Bibliographic Analysis of Nature Based on Twitter and Facebook Altmetrics Data. Plos One. 2016;11(12). WOS:000389482700013. pmid:27906981
- 41. Thelwall M, Haustein S, Lariviere V, Sugimoto CR. Do Altmetrics Work? Twitter and Ten Other Social Web Services. Plos One. 2013;8(5). pmid:23724101
- 42. Zahedi Z, Costas R. How well developed are Altmetrics? Cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics. 2014;101.
- 43. Mohammadi E, Thelwall M, Haustein S, Lariviere V. Who Reads Research Articles? An Altmetrics Analysis of Mendeley User Categories. Journal of the Association for Information Science and Technology. 2015;66(9):1832–46.
- 44. Ebrahimy S, Mehrad J, Setareh F, Hosseinchari M. Path analysis of the relationship between visibility and citation: the mediating roles of save, discussion, and recommendation metrics. Scientometrics. 2016;109(3):1497–510.
- 45. Bornmann L. Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics. 2015;103(3):1123–44.
- 46. Bornmann L. Usefulness of altmetrics for measuring the broader impact of research A case study using data from PLOS and F1000Prime. Aslib Journal of Information Management. 2015;67(3):305–19.
- 47. Bornmann L, Haunschild R. Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data. Plos One. 2018;13(5). pmid:29791468
- 48. Wardle D. Do 'Faculty of 1000' (F1000) ratings of ecological publications serve as reasonable predictors of their future impact? Ideas in Ecology and Evolution. 2010;3.
- 49. Shema H, Bar-Ilan J, Thelwall M. Do Blog Citations Correlate With a Higher Number of Future Citations? Research Blogs as a Potential Source for Alternative Metrics. Journal of the Association
- 50. Haustein S, Siebenlist T. Applying social bookmarking data to evaluate journal usage. Journal of Informetrics. 2011;5(3):446–57.
- 51. Bornmann L. Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics. 2014;8(4):935–50.
- 52. Shuai X, Pepe A, Bollen J. How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations. Plos One. 2012;7(11).
- 53. Fei Guo, Bin You, Jingyuan Xue. Analysis on Transmission Characteristics and Influence of Altmetrics Hot Papers. Library and Information Service. 2016;60(15):86–93. Chinese.
- 54. Adie E. TAKING THE ALTERNATIVE MAINSTREAM. Profesional De La Informacion. 2014;23(4):349–51.
- 55. Das A, Mishra S. Genesis of Altmetrics or Article-level Metrics for Measuring Efficacy of Scholarly Communications: Current Perspectives. Journal of Scientometric Research. 2014;3:82–92.
- 56. Zhang L, Wang J. Why highly cited articles are not highly tweeted? A biology case. Scientometrics. 2018;117(1):495–509.
- 57. Chapa J, Haq Z, Cifu AS. Comparative analysis of the factors associated with citation and media coverage of clinical research. Scientometrics. 2017;112(3):1271–83.
- 58. Wang M, Wang Z, Chen G. Which can better predict the future success of articles? Bibliometric indices or alternative metrics. Scientometrics. 2019;119(3):1575–95.
- 59. Wang X, Lv T, Hamerly D. How do altmetric sources evaluate scientific collaboration? An empirical investigation for Chinese collaboration publications. Library Hi Tech. 2019.
- 60. Zhao R, Bo Y. Analysis of the original influencing factors and secondary influencing factors of academic papers. Library Hi Tech. 2020;38(2):446–57.
- 61. Wang W, Ren J, Alrashoud M, Xia F, Mao M, Tolba A. Early-stage reciprocity in sustainable scientific collaboration. Journal of Informetrics. 2020;14:101041.
- 62. Zhao R, Bo Y. Analysis of the original influencing factors and secondary influencing factors of academic papers. Library Hi Tech. 2020;38(2):446–57.
- 63. Wang M, Wang Z, Chen G. Which can better predict the future success of articles? Bibliometric indices or alternative metrics. Scientometrics. 2019;119(3):1575–95.
- 64. Nocera AP, Boyd CJ, Boudreau H, Hakim O, Rais-Bahrami S. Examining the Correlation Between Altmetric Score and Citations in the Urology Literature. Urology. 2019;134:45–50. pmid:31560915
- 65. Mullins CH, Boyd CJ, Corey BL. Examining the Correlation Between Altmetric Score and Citations in the General Surgery Literature. Journal of Surgical Research. 2020;248:159–64. pmid:31901796
- 66. Hou J, Yang X. Social media-based sleeping beauties: Defining, identifying and features. Journal of Informetrics. 2020;14(2):101012. https://doi.org/10.1016/j.joi.2020.101012.
- 67. Moed H. Applied Evaluative Informetrics. 2017.
- 68. Sugimoto CR, Work S, Lariviere V, Haustein S. Scholarly Use of Social Media and Altmetrics: A Review of the Literature. Journal of the Association for Information Science and Technology. 2017;68(9):2037–62.
- 69. Haustein S. Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics. 2016;108(1):413–23.
- 70. Mcallister PR, Narin F, Corrigan JG. Programmatic evaluation and comparison based on standardized citation scores. IEEE Transactions on Engineering Management, 1983;30(4): 205±211.
- 71. Ravenscroft J, Liakata M, Clare A, Duma D. Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements. Plos One. 2017;12(3). pmid:28278243
- 72. Waltman L. A review of the literature on citation impact indicators. Journal of Informetrics, 2016;10(2): 365–391.
- 73. Ye FY, Bornmann L. “Smart girls” versus “sleeping beauties” in the sciences: The identification of instant and delayed recognition by using the citation angle. Journal of the Association for Information Science and Technology, 2018;69(3): 359–367.
- 74. Pan J, Keeney S. Molecular cartography: Mapping the landscape of meiotic recombination. Plos Biology. 2007;5(12):2774–7. pmid:18092891
- 75. Powell K. Going against the grain. Plos Biology. 2007;5(12):2748–53. pmid:18162047
- 76. Rodriguez-Girones MA, Santamaria L. Why are so many bird flowers red? Plos Biology. 2004;2(10):1515–9. pmid:15486585
- 77. Schmitz OJ. Restoration of Ailing Wetlands. Plos Biology. 2012;10(1).
- 78. Servedio MR. The what and why of research on reinforcement. Plos Biology. 2004;2(12):2032–5.
- 79. Tsuriel S, Geva R, Zamorano P, Dresbach T, Boeckers T, Gundelfinger ED, et al. Local sharing as a predominant determinant of synaptic matrix molecular dynamics. Plos Biology. 2006;4(9):1572–87.
- 80. MacDonald PE, Rorsman P. Oscillations, intercellular coupling, and insulin secretion in pancreatic beta cells. Plos Biology. 2006;4(2):167–71.
- 81. Del Cul A, Baillet S, Dehaene S. Brain dynamics underlying the nonlinear threshold for access to consciousness. Plos Biology. 2007;5(10):2408–23. pmid:17896866
- 82. DeRisi S, Kennison R, Twyman N. The what and whys of DOIs. Plos Biology. 2003;1(2):133–4.
- 83. Alerstam T, Rosen M, Backman J, Ericson PGP, Hellgren O. Flight speeds among bird species: Allometric and phylogenetic effects. Plos Biology. 2007;5(8):1656–62. pmid:17645390
- 84. Khalturin K, Anton-Erxleben F, Sassmann S, Wittlieb J, Hemmrich G, Bosch TCG. A Novel Gene Family Controls Species-Specific Morphological Traits in Hydra. Plos Biology. 2008;6(11):2436–49. WOS:000261187900014. pmid:19018660
- 85. Neugebauer KM. Keeping tabs on the women: Life scientists in Europe. Plos Biology. 2006;4(4):494–6. WOS:000237066500003. pmid:16602822
- 86. Montooth KL, Rand DM. The Spectrum of Mitochondrial Mutation Differs across Species. Plos Biology. 2008;6(8), e213. pmid:18752353
- 87. Rongying Zhao, fengjiao Guo, jie Tan. Evaluation of Academic Papers Impact Based on Altmetrics:A Case Study of Chinese Language and Literature. Journal of Library Science in China. 2015; 42(1): 96–108. Chinese.
- 88. Xia F, Wang W, Bekele TM, Liu H. Big Scholarly Data: A Survey. Ieee Transactions on Big Data. 2017;3(1):18–35. WOS:000415161400003.