ASSAM U

IVE N

RSITY

Assam University Journal of Science & Technology : Physical Sciences and Technology Vol. 5 Number II 50-52, 2010

The Impact of Impact Factors in Measuring Research Quality

Sudip Choudhury1 and Uday Shankar Chakraborty2
1 2

Department of Chemistry, Gurucharan College, Silchar-788004, Assam, India Department of Mathematics, Assam University, Silchar-788011, Assam, India Correspondence; 2e-mail: uday_aus06@yahoo.co.in

Abstract
The Impact Factor (IF) is an important bibliometric indicator which is currently being used in most of the countries to evaluate the quality of a journal. Despite of its global acceptance, it has got some serious criticism also from scientists and researchers working in various disciplines round the globe. In this present paper, an effort has been made to highlight some of the advantages and limitations of IF to weigh up the quality of journals. Apart from IF, some other bibliometric indices are also briefly discussed. Kew words: Impact Factor, Journals, Journal Citation Report, Bibliometric Indicator

Introduction Research papers from all over the world are published in thousands of journals every year. The quality of these papers essentially has to be weighed up, not only to determine their accuracy and contribution to fields of research, but also in making decisions about rewarding researchers with funding and appointments to research positions (Simons, 2008). The pursuit of such an appropriate measure to evaluate the quality of research papers continued until Eugene Garfield, a linguist and the remarkable founder of the Institute for Scientific Information (ISI) (now part of Thomson Reuters ) came up with a fascinating concept, that has become indispensable to the scientific community. The concept is that citations in a scientific paper can be used to determine the importance of not only scientific research but also researchers and research journals (Garfield, 1979). Based on the idea of Garfield, the concept of impact factor of a journal has been developed (Janick, 2008). Impact factor is an index based on the frequency with which a journals articles are cited in scientific publications, a marker of journal quality (Somnath et al., 2003). Impact factor for a journal in a year is considered as the ratio of the number of times the papers published in that journal, are cited in indexed journals during last two years to the total number of times papers published in the last two years. In the modern world, IF has got the acceptance as the qualitative measure of journal quality (wikipedia website). There have been many innovative applications of journal impact factors. The most common one involves market research for publishers and others (Tandon et al., 2007). Impact factor is used by librarians in selecting journals for library collections, and, in some countries, it is used to evaluate individual scientists and institutions for the purposes of academic promotion and funding allocation (Seglen, 1997; Lowy, 1997). Though, it is useful in many respect, there have been numerous criticisms over the years of its use as a measure of the quality of individual research papers. In this paper, the merits and limitations of this index have been carefully highlighted. Merits of Impact Factor IF provides us a guide to what should be read. It provides quantitative tools for ranking, evaluating, categorizing and comparing journals. It eliminates some of the bias of such counts which favour

LC SI

HAR

- 50 -

The Impact of Impact Factors...........

large journals over small ones or frequently issued journals over less frequently issued ones and of older journals over newer ones. Particularly in the latter case such journals have a larger citable body of literature than smaller or younger journals. The impact factor can be used to provide a gross approximation of the prestige of journals in which individuals have been published (Tandon et al., 2007). Limitations of Impact Factor The use of impact factor as an index of journal quality, relies on the theory that citation frequency accurately measures a journals importance to its end users. This theory is credible for journals whose audiences are primarily researchers, most of whom write manuscripts for publication. By citing articles from a given journal in their own manuscripts, researchers are in essence casting votes for that journal. Impact factor serves as a tally of those votes. By merely counting the frequency of citations per article and disregarding the prestige of the citing journals, the impact factor becomes merely a metric of popularity, not of prestige (Tandon et al., 2007). One major limitation of IF is that it varies significantly among journals of various disciplines, which opposes it to become a globally acceptable indicator for the quality of journals. For instance, journals of biological sciences enjoy much higher IF in comparison to the journals published on mathematical sciences. Thus a journal of biological sciences can not be compared with a journal of mathematical sciences in terms of IF. Again a newly published journal needs at least three years to have an impact factor. Thus, before the completion of three years a journal publishing papers of very good quality also can not get IF. Thus, the quality of this type of journals can not be judged in terms of impact factor. The timing of publication can also affect the IF. Considering the sample period of 2 years, a good paper published in January has 11 months longer to be cited than one published in December of the same year (Epstein, 2007)! Generally, journals which publish mainly review papers possess higher IF as compared to other journals. Thus, a journal can easily bump up its IF by publishing more review papers. Again since Impact factors are calculated

yearly for those journals only which are indexed in Thomson Reuters Journal Citation Reports, therefore, a good quality journal also wont be credited with IF unless it is indexed in Thomson Reuters Journal Citation Reports. In that respect also, impact factor fails to asses the quality of research. Other Journal Ranking Measures H-index: Unlike Impact factor Hi-index provides a measurement of both the scientific productivity and the apparent scientific impact of a scientist (wikipedia website). The index is based on the set of the scientists most cited papers and the number of citations that they have received in other peoples publications. The index can also be applied to the productivity and impact of a group of scientists, such as a department or university or a country. The index was suggested by Jorge E. Hirsch, a physicist at UCSD, as a tool for determining theoretical physicists relative quality (Hirsch, 2005) and is sometimes called the Hirsch index or Hirsch number. Immediacy index: An Immediacy index is a measure of how topical and urgent work published in a scientific journal is. Along with the better known Impact factor measure, it is a calculated each year by the Institute for Scientific Information for those journals which it indexes; both impact factors and immediacy indices are published annually in the Journal Citation Reports. Eigenfactor score: It is a measure of the overall value provided by all of the articles published in a given journal in a year. Eigenfactor score of a journal is calculated as the ratio of the total number of citations (except self citation) to the total number of articles published. Eigenfactor scores are scaled so that the sum of the Eigenfactor scores of all journals listed in Thomsons Journal Citation Reports (JCR) is 100. Article influence score: Article Influence measures the average influence, per article, of the papers in a journal. As such, it is comparable to the Impact Factor. It is calculated as the ratio of the journals Eigenfactor score to the fraction of articles published by the journal. Article Influence scores are normalized so that the mean

- 51 -

The Impact of Impact Factors...........

article in the entire Thomson Journal Citation Reports (JCR) database has an article influence of 1.00. A score greater than 1.00 indicates that each article in the journal has above-average influence. A score less than 1.00 indicates that each article in the journal has below-average influence. Cited Half-life: Cited Half-life measures the number of years, going back from the current year, that account for half the total citations received by the cited journal in the current year. The Table 1 gives a comparison among Impact Factor, Eigenfactor Score and Article Influence Score of some important journals for the year 2007.

Conclusion Of the many conflicting opinions about Impact factors, it can be said that though Impact factor is not a perfect tool to measure the quality of articles but there is nothing better till date and it has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation (Hoeffel, 1998). Thus to evaluate the quality of articles and journals publishing them, one has to depend on IF until a better and more adequate device is developed to replace it.

Table 1: Comparison of 2007 Impact Factors, Eigenfactor and Article Influence Scores (adapted from www.library.illinois.edu/export/biotech/docs/ImpactFactors.ppt)
Journal JCRs Impact Factor 26.372 29.887 28.751 9.598 3.493 5.039 Eigenfactor Score 1.69272 0.67067 1.83870 1.74485 0.04648 0.15579 Article Influence Score 16.539 18.188 16.996 4.929 1.608 2.406

Science Cell Nature PNAS BMC Bioinformatics Bioinformatics

References
Epstein, D. (2007). Impact factor manipulation. The Write Stuff. 16: 133134 Garfield, E. (1979). Citation Indexing. Its Theory and Application in Science. Technology and Humanities. Wiley. New York. Hirsch, J. E. (2005). An index to quantify an individuals scientific research output. Pnas 102 (46): 1656916572 Janick, J. (2008). The Tyranny of the Impact Factor. Chronica Horticulturae. 48(2): 3 Lowy, C. (1997). Impact factor limits funding. Lancet. 350 :1035 Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal. 314: 497-502 Simons, K. (2008). The Misused Impact Factor (editorial). Science. 322 (5899): 165 Somnath, S.; Sanjay, S.; Dimitri, A.C. (2003). Impact factor: a valid measure of journal quality? J Med Libr Assoc. 91(1): 4246 Tandon, V.R.; Mahajan, A.; Sharma, S.; Gupta, S.K. (2007). Impact Factor: Indian Contest. Journal Knowledge. 9(3): 107-108 Available online: http://en.wikipedia.org/wiki/H-index last accessed on 1-11-2009 Available online: http://en.wikipedia.org/wiki/ Impact_factor last accessed on 1-11-2009 Available online: www.library.illinois.edu/export/ biotech/docs/ImpactFactors.ppt last accessed on 111-2009 Available online: http://eigenfactor.org/ last accessed on 1-11-2009

- 52 -