Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Evaluation and Ranking of Researchers – Bh Index

Evaluation and Ranking of Researchers – Bh Index

  • D. Gnana Bharathi
PLOS
x

Abstract

Evaluation and ranking of every author is very crucial as it is widely used to evaluate the performance of the researcher. This article proposes a new method, called Bh-Index, to evaluate the researchers based on the publications and citations. The method is built on h-Index and only the h-core articles are taken into consideration. The method assigns value additions to those articles that receive significantly high citations in comparison to the h-Index of the researcher. It provides a wide range of values for a given h-Index and effective evaluation even for a short period. Use of Bh-Index along with the h-Index gives a powerful tool to evaluate the researchers.

Introduction

There are varied methods used for evaluation of journals and researchers. Journal Impact Factor (JIF) is the most popular and widely used method to evaluate the journals. However, over a period, JIF has been inadvertently used to evaluate the researchers for their performance. This practice is relatively offset by a new method called h-Index that was introduced by Jorge Hirsch [1]. Ranking being in a whole number, ease of calculations and a robust system has enabled the h-Index to become very popular in a short period of time [2].

h-Index is defined by Hirsch as “a scientist has index h if h of his or her Np papers have at least h citations each and the other (Np-h) papers have ≤ h citations”. For example, an author who has 5 or more articles with just 5 of them having at least 5 citations, then the author has an h-Index of 5.

However, on critical evaluation of the h-Index it is found that there are a number of issues with the method [3-6]. For example, an author who publishes few articles yet receives thousands of citations is poorly ranked in comparison to those authors of relatively more articles with limited number of citations in each of these individual articles.

Alternative methods are being proposed by a number of researchers such as g-Index [7], hg-Index [8], A-Index and AR-Index [9,10], R-Index [11] e-Index [12], p-Index [13], P-Index and C-Index [14], etc. These methods were put forth based on identification of specific problems with the h-Index. However, they do not fit for extreme or theoretical situations or they lack simplicity in calculation. For the g-Index, in case of the author with less publications but more citations, citations have to be assigned even to non-existing articles[15]. In the case of A-Index, better scientist is punished for having higher h-Index [16]. Indices such as m-quotient and AR-Index are time dependent.

The proposed new index can be used to evaluate researchers of different age groups or work experience. The index can increase on continuity of citations, even after suspension of publications and provides range of values for a given number of articles and citations.

The basis for the Bh-Index is ranking of the researchers in such a way that the value does not fall below the h-Index nor has a phenomenal increase in case of a few well cited articles. Further, the method is designed to allow researchers with few publications, and more citations, be relatively compared with the authors of large number of publications and limited citations.

Method

Bh-Index

A new methodology to evaluate the researcher, called Bh-Index, is based on threshold value as criteria for the selection of the articles or what is known as ‘h-core articles’. It gives value addition for the articles with very high citations in comparison with threshold value. As a result, author of low h-Index with high level of citations have more impact in comparison to authors of more h-Index with low level of citations.

In h-Index, each of the qualified articles gets exactly one value, irrespective of the citations. In Bh-Index, these qualified articles gets at least one value. Every article can increase its value by 2, 3, 4…. for doubling, quadrupling, eight times… of the citations over the threshold value – the h-Index.

The method is similar to the weighted voting system as followed in some voting systems of European Union and United States of America [17]. The difference between the sizes of the countries/states can be just over 10 times larger than the smallest one. But the peak citations of researchers can be more than 100 times of h-Index is common. Therefore, simple proportional values cannot be applied to the evaluation of researchers. The average value or direct proportional value will result in substantial changes in the ranking of the researchers, if such extreme values are equally considered with other articles. Bh-Index addresses these issues.

The excessive citations of the h-core articles have to be rationally valued so that there is no deficit or excessive gain in the ranking of the researchers who have more than h2 citations. Considering the extremity in ranking of the researchers, Bh-Index provides a judicial method for ranking the researchers. In the Bh-Index, the value addition is measured on the geometric sequence with ratio of two and the h-Index of the author as the first value.

The method can be expressed as citations of every qualified article factored on the geometric sequence of the h-Index with ratio of two, added together, forms the Bh-Index of the author.

Calculation of Bh-Index

As a first step, h-Index is calculated – by ordering citations received by the articles in the descending order. Only those articles in the h-Index, also known as h-core articles, are considered for the calculation of Bh-Index. Making h-Index as the first term, a geometric sequence with ratio of two is derived. The number of citations between the geometric sequences is grouped into mutually exclusive classes.

A class constitutes tn to 2tn-1.

Geometric sequence, tn=t1* r(n-1)

Where n is the factor, t1 is the first term of factor, tn is the nth term of factor and r is common ratio.

Therefore, nth term of the geometric sequence of Bh-Index= (h-Index)*2(n-1)

Citations of articles falling in the nth class of the nth term are assigned n value. Frequency of the articles in every class is calculated. The factor, n is multiplied with the frequency, f of the class. The total is the Bh-Index of the author.

If h-Index is 8, the geometric sequence will be 8, 16, 32, 64…for the classes constituting citations in the range of 8-15, 16-31, 32-63, 64-127….for the factors 1, 2, 3, 4…. Similarly, if h-Index is 15 the geometric sequence will be 15, 30, 60, 120.for the classes constituting citations in the range of 15-29, 30-59, 60-119, 120-239……for the factors 1, 2, 3, 4….

h-Index remains as base of the Bh-Index. Researchers with fixed h-Index can increase their Bh-Index, if the articles continue to get more citations to reach each of the benchmarks as 2h, 4h, 8h…. Bh-Index necessitates increase in citations of every article for the increase of each h-Index. For every increase in the h-Index, the benchmarks change to 2(h+1), 4(h+1), 8(h+1)….

The citations of every article that reach the benchmarks increase the Bh-Index by one. This enables researchers with given h-Index to show the impact of citations of their articles with reference to that h-Index. If none of the article reaches even the first benchmark, then Bh-Index is equivalent to h-Index. If all the articles meet the first benchmark, the Bh-Index will be 2h, if all the articles meet the second benchmark then Bh-Index will be 3h-Index and so on.

Analysis

Evaluation of researchers

An example to calculate the Bh-Index is given in Table 1. As each of the six articles has six or more citations, h-Index is 6. As a result, the geometric sequence for this case will be 6, 12, 24, 48, 96..… Citation classes will be 6-11, 12-23, 24-47, 48-95…. and factor for the classes will be 1, 2, 3, 4… respectively. The frequency of citations in each of the class is multiplied by the factor of each of the citation class. The values added together give the Bh-Index of 16.

ArticlesCitationsGeometric sequenceClassFactor, nFrequency, fn*f
1218192192-383616
2349696-191500
3274848-95400
4152424-47326
581212-23212
6666-11122
74Bh-Index16
82
91
100

Table 1. Calculation of Bh-Index for the h-Index of 6.

CSV
Download CSV

Figure 1 exhibits how a Bh-Index compares with h-Index. There are three graphs each with h-Index of 6 and 12. As citations in Figure 1A and 1D has one as a factor, Bh-Index is same as h-Index. Figure 1B exhibits that the citations are spread over three different classes of factor 1, 2 and 4. As a result, for the h-Index of 6, Bh-Index is 19. In Figure 1E also citations are in the same factors. However, most of the citations are in the lower class. Hence, h-Index of 12 gets Bh-Index of 19.

In Figure 1C, citations of the each article follow the geometric sequence. As a result, author with 6 h-Index gets Bh-Index 21. In Figure 1F, though the factors remain same, number of articles and citations has increased. Therefore, the h-Index of 12 gets Bh-Index of 42. It is also clear that the increase in h-Index necessitates more citations for every factor. Calculation of Bh-Index for the Nobel laureates of 2012 is given in Appendix S1.

Figure 2 gives Bh-Index for the articles with h-Index of 2, 10, 25, 50 and 100. The lines represent the idealistic conditions from most homogeneous citations to most skewed citations. Each line has sharp bend. The bottom part of the line represents the most skewed citations pattern – one in which only one article get high level of citation and others get less than 2h citations. The top part of the line represents the most homogenous citations where all articles equally reach a given benchmark. The other possibilities of citations fall between the lines.

thumbnail
Figure 2. Bh-Index coverage for the h-Index of 2, 10, 25, 50, 75 and 100.

https://doi.org/10.1371/journal.pone.0082050.g002

Bh-Index gives more values for those articles whose citations are equal or cross every benchmark. As increase in the h-Index increases the benchmark, those articles whose citations equaled or crossed the earlier benchmark may need to get more citations to remain in the increased benchmark. Failing to do so will lead to reduction in Bh-Index. This reduction can continue until it equals the h-Index. However, it is unlikely due to the fact that increase in h-Index also necessitates increase in citations of existing articles to certain extent, failing which h-Index remains stagnant.

Evaluation of researchers over a period of time

Citations trends of Nobel laureates in chemistry for 15 years starting from 1995 to 2009 are studied. The citations trends, up to the h-Index of the individual Nobel laureates, are obtained between September 2010 and August 2013 from Web of Science database. The changes in citations trends over the years, for two individuals with identical increase in h-Index are given in Figure 3 and Table 2.

Name20102013Difference
h-core citationsh-indexBh-indexh-core citationsh-IndexBh-Indexh-core citationsh-IndexBh-Index
Boyer PD310329483484305038112
Kohn W321520534050215883515
Hershko A104103997122734299186332
Shimomura O96219301246223728437
Chalfie M7301357887024085140157
Curl RF418335595027406884459
Molina MJ5399448066955283129683
Ertl G139327911117198871283266817

Table 2. Change in h-Index and Bh-Index in 2010 and 2013.

Data collected from WoS in September 2010 and August 2013.
CSV
Download CSV

In Figure 3A and 3B increase in citations of the researchers resulted in increase of h-Index by one. In Figure 3B, though almost all the articles increased their citations, only five articles have increased their citations to the next level of their respective geometric sequence of the current h-Index. Therefore, Bh-Index increased by five. In Figure 3A, only two articles have increased to the next level of their geometric sequence. Hence, Bh-Index is increased by two.

In Figure 3C, there is an increase of 1863 citations, which is substantially high, in comparison to Figure 3D where citations is increased by 284. Though h-Index increased by three for both the researchers, Bh-Index is increased by two for the former and by seven for the later. This is due to the fact that the geometric sequence of the former starts at 42 while for the later it is 22. As it is clear from the graph, newly added citations in the Figure 3C is concentrated in highly cited articles. There is no clear increase in citations for the poorly cited articles. As the citations are highly skewed, Bh-Index increases by just two values. In Figure 3D every article increased its citations of which citations of seven articles reached the next level. Therefore, Bh-Index increased by seven.

h-Index, A-Index, e-Index, R-Index, g-Index and gh-Index along with the Bh-Index are given in Table 3. g-Index is not dependent on h-Index and h-core citations and therefore calculated by inclusion of additional citations from the non h-core articles. Other indices are calculated based on h-Index and/or citations. gh-Index is the square root of g and h indices.

Nameh-core citationsg-Index citationsh-IndexA-Indexe-IndexR-Indexg-Indexgh-IndexBh- Index
Knowles WS208210452.0013.8614.42147.4815
Rose I5185941339.8518.6822.762417.6622
Tanaka K6457471349.6221.8225.402718.7328
Skou JC8168551081.6026.7628.572917.0326
Shimomura O124614282256.6427.6035.303728.5337
Chauvin Y195122732481.2937.0844.174733.5943
Shirakawa H309337983296.6645.4955.616144.1858
Boyer PD3484390930116.1350.8359.036243.1350
Rowland FS354343204088.5844.0859.526550.9961
Kohn W4050417921192.8660.0763.646436.6658
Yonath AE4067482834119.6253.9563.776948.4460
Curl RF5027591140125.6858.5470.907655.1468
Molina MJ6695823752128.7563.1781.829068.4183
Fenn JB7783785319409.6386.1588.228840.8962
Chalfie M8702912440217.5584.2793.289561.6485
Ramakrishnan V102361142153193.1386.18101.1710674.95101
Hershko A122731263442292.21102.51110.7811268.5999
Kroto HW130031567366197.0292.99114.0312590.83110
Zewail A130391616481160.9880.49114.19127101.42113
Walker JE133981601966203.0095.09115.7512691.19110
Schrock RR134461660581166.0082.98115.96126101.02110
Kornberg RD159271907680199.0997.61126.20138105.07125
Macdiarmid AG169762033076223.37105.83130.29142103.88136
Ertl G171982154587197.6898.13131.14146112.70128
Crutzen PJ173662027778222.64106.22131.78142105.24127
Agre P174412087581215.32104.31132.06144108.00131
Ciechanover A185332092959314.12122.69136.1414492.17123
Noyori R237682706384282.95129.27154.17164117.37161
Steitz TA244652777383294.76132.57156.41166117.38159
Mackinnon R251322652470359.03142.24158.53162106.49161
Wuthrich K304793689298311.01144.48174.58192137.17161
Sharpless KB318643617183383.90158.03178.50190125.58159
Pople JA332373512963527.57171.08182.31187108.54160
Grubbs RH3466341724101343.20156.40186.18204143.54180
Tsien RY4372149052108404.82179.04209.10221154.49216
Heeger AJ5048561147118427.84191.21224.69247170.72213
Smalley RE5497860824119462.00202.03234.47246171.10233

Table 3. Indices of Nobel laureates of chemistry from 1995 to 2009.

Data collected in August 2013.
CSV
Download CSV

A-Index gives high values whenever the h-Index is small. As e-Index is square root of excess citations of the h-core articles, its values can widely vary from zero to infinity. Researchers with more h-core citations have high R-Index. g-Index also follows the same method, based on inclusion of increased number of citations. The Bh-Index on the other hand, measures by evaluating the individual h-core articles, based on its citations. Thus, it gives more appropriate values.

Discussion

Bh-Index has a range of values for the publication of articles and their citations. Range of Bh-Index for given h-Indexes for various citation ranges is given in the Figure 4. For easy understanding h-Index of 1, 2, 3, 4, and 5 are considered. Author with only one article can have more Bh-index, if the article continues to receive citations. As Bh-Index follows geometric sequence, double citations are necessary for every additional increase in Bh-Index. Author with h-Index of 2 can get Bh-Index of 16, if each of the articles gets citations in the range of 256 to 511. However, if one article is cited between 2 to 3 and other is cited in between 256 to 511, Bh-Index will be 9.

thumbnail
Figure 4. Bh-Index for extreme level of citations for h-Index of 1, 2, 3, 4 and 5.

https://doi.org/10.1371/journal.pone.0082050.g004

If one author with large number of citations from a few articles has Bh-Index for a given h-Index, other author can make the same Bh-Index with relatively lower level of citations, if all articles are equally cited. For example, author of h-Index of 4 can get Bh-Index of 8, if only one article gets citations between 64 and 127 and others get citations in between 4 to 7. Other researchers can have same Bh-Index for the same h-Index if all four articles get citations in between 8 and 15. If the first author continues to receive citations only from one article, the Bh-Index will be 12, if the citations of the article are between 1024 and 2048. On the other hand, the second author can get Bh-Index of 12 if all four articles get citations between 16 and 31.

Therefore, the author with high h-Index can score more Bh-Index when more articles get high citations. If citations are confined to fewer of articles, even if the total number of citations is equal, the Bh-Index will be low. Author with low h-Index has to have more citations from most of the articles to have high Bh-Index. If no article gets double or more citations than the h-Index, Bh-Index equals the h-Index. This characteristic enables young researcher with few articles having high citations to be compared with the experienced author with more articles.

For every increase in h-Index, the author needs to get additional citations from the non h-core articles. The authors may also have to get increase in citations from the lowest cited article(s) that are already included in the h-Index. In case of Bh-Index, for every increase in the h-Index, there has to be increase in citations of almost all the articles. This is essential to retain the Bh-Index, if the h-Index increases, even by one value.

As a result, citations of an article that received n factor become n-1 factor, if the number of citations of the article falls below the factor. Therefore, if the author who fails to increase the citations of all the articles, especially those close to the lower limit of the class, Bh-Index will decrease, marginally, for every increase in h-Index. This may prevent self-citations to boost the h-Index, to certain extent, if both the indices are used to evaluate the researchers.

For example, a researcher has h-Index of 3 with citations as 4, 6 and 12. As per the calculations of geometric sequence 3, 6, 12.…Bh-Index will be 6. Let us assume the researcher's one more article received 4 citations. Hence, the citations are 4, 4, 6 and 12. Accordingly, the h-Index is 4. However, as per the redefined geometric sequence of 4, 8, 16…. Bh-Index will be 5.

Conclusion

Bh-Index provides a comprehensive measurement to evaluate the researcher. It gives a range of values for the citations for the h-core articles. While h-Index evaluates the overall performance by fixing the benchmark for the author, Bh-Index evaluates the individual articles of the h-Index, as well as the author, effectively.

For every increase of h-Index, citations of marginal articles of the existing h-core articles may be necessary. In such cases, if the marginal articles fail to increase their citations, h-Index remains stagnant. For every increase of h-Index, Bh-Index demands increase in citations of all the articles. For every article that fails to reach the new benchmark from the existing one, Bh-Index is reduced by one. On the other hand, any article that reaches every additional benchmark increases the Bh-Index by one, irrespective of the increase in h-Index.

More citations from every h-core article increases the Bh-Index for reaching every benchmark of the geometric sequence. Bh-Index of author of more publications and h-Index with skewed citations can be equaled by an author with less publications and h-Index, if most of the articles receive high citations.

Further, Bh-Index takes into consideration the citations of every qualified article and ranks accordingly. Similar to h-Index, use of the Bh-Index alone will not show the true picture of the citation behavior of the articles of the researcher. However, if associated with the h-Index it will prove the true characteristics of citations and real impact of the researchers.

Supporting Information

Appendix S1.

Examples of calculation of Bh-Index for seven Nobel Laureates of 2012. The data were collected in the month of April 2013 from Web of Science database.

https://doi.org/10.1371/journal.pone.0082050.s001

(DOCX)

Acknowledgments

The author acknowledges the Director, Central Leather Research Institute, Chennai for the permission to publish the article. Author thanks Ms. Janani Natarajan for editing the text.

Author Contributions

Conceived and designed the experiments: DGB. Performed the experiments: DGB. Analyzed the data: DGB. Contributed reagents/materials/analysis tools: DGB. Wrote the manuscript: DGB. Contribution type: DGB.

References

  1. 1. Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 102: 16569-16572. doi:10.1073/pnas.0507655102. PubMed: 16275915.
  2. 2. Vanclay JK (2007) On the robustness of the h-index. Journal of the American Society for Information Science and Technology 58: 1547-1550. doi:10.1002/asi.20616.
  3. 3. Bornmann L, Mutz R, Daniel HD (2008) Are there better indices for evaluation purposes than the h index? a comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology 59: 830-837. doi:10.1002/asi.20806.
  4. 4. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2009) h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics 3: 273-289. doi:10.1016/j.joi.2009.04.001.
  5. 5. Costas R, Bordons M (2007) The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. Journal of Informetrics 1: 193-203. doi:10.1016/j.joi.2007.02.001.
  6. 6. Gibb BC (2012) Lies, damned lies and h-indices. Nat Chem 4: 513-514. doi:10.1038/nchem.1388. PubMed: 22717424.
  7. 7. Egghe L (2006) An Improvement of the h-Index: G-Index. ISSI Newsletter 2: 2.
  8. 8. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2010) hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 82: 391-400. doi:10.1007/s11192-009-0047-5.
  9. 9. H JB (2006) h-index: an evaluation indicator proposed by scientist. Science Focus 1: 2.
  10. 10. Bornmann L, Daniel HD (2009) The state of h index research Is the h index the ideal way to measure research performance? EMBO Rep 10: 2-6. doi:10.1038/embor.2008.233. PubMed: 19079129.
  11. 11. Schreiber M (2008) An empirical investigation of the g-index for 26 physicists in comparison with the h-Index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology 59: 1513-1522. doi:10.1002/asi.20856.
  12. 12. Zhang CT (2009) The e-Index, Complementing the h-Index for Excess Citations. PLOS ONE 4: e5429. PubMed: 19415119.
  13. 13. Prathap G (2010) The 100 most prolific economists using the p-index. Scientometrics 84: 167-172. doi:10.1007/s11192-009-0068-0.
  14. 14. Stallings J, Vance E, Yang J, Vannier MW, Liang J et al. (2013) Determining scientific impact using a collaboration index. Proceedings of the National Academy of Sciences. PubMed: 23720314.
  15. 15. Egghe L (2006) Theory and practise of the g-index. Scientometrics 69: 131-152. doi:10.1007/s11192-006-0144-7.
  16. 16. Jin BH, Liang LM, Rousseau R, Egghe L (2007) The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin 52: 855-863. doi:10.1007/s11434-007-0145-9.
  17. 17. Gelman A, Katz JN, Tuerlinckx F (2002) The mathematics and statistics of voting power. Statistical Science 17: 420-435. doi:10.1214/ss/1049993201.