![[Board accepted]](http://www.istl.org/graphics/b2.gif){kind=small}
Comparison of Journal Citation Reports and Scopus Impact Factors for Ecology and Environmental Sciences Journals
Public Services Librarian
teddy.gray@duke.edu
Sarah Z. Hodkinson
Library Assistant
sarah.hodkinson@duke.edu
Biological and Environmental Sciences Library
Duke University
Durham, North Carolina
Abstract
Impact factors for journals listed under the subject categories "ecology" and "environmental sciences" in the Journal Citation Reports database were calculated using citation data from the Scopus database. The journals were then ranked by their Scopus impact factor and compared to the ranked lists of the same journals derived from Journal Citations Reports. Although several titles varied significantly in impact factor and rank, the Journal Citation Reports and Scopus lists had a high degree of statistical similarity.
Introduction
The Thomson Scientific impact factor is a popular tool that is used to measure the influence of scholarly journals. Nearly fifty years ago, Eugene Garfield developed this method as a way to use the Thomson Scientific (formerly Institute for Scientific Information, ISI) citation indexes to analyze the frequency with which the average journal article is cited (Garfield & Sher 1963; Garfield 1972). The impact factor of a journal is calculated by dividing the references cited in one year by the number of citable articles published in the same journal over the previous two years. This ratio is published annually in Thomson Scientific's Journal Citation Reports (JCR), along with a number of other quantitative tools for comparing and evaluating scholarly journals. For years, the impact factor has been used to "clarify the significance of absolute citation frequencies" (Thomson Scientific 1994), and as a result its applications have gained great significance in the world of academia (Monastersky 2005). Academic librarians frequently use impact factors to help them decide which journals are important enough to subscribe to and which subscriptions may be canceled (Barschall 1988; Coleman 2007).While impact factor data were originally used extensively in journal marketing to rank and compare scholarly journals for prospective subscribers, it has become a way of ranking the scientists who publish articles in these journals. Since the importance of scientific research is notoriously difficult to evaluate quantitatively, many universities now make funding and tenure decisions based on the average impact factor value, or prestige, of the journals that a scientist has published in. This may seem to be a reliable tool to assess the importance of a particular scientist's research, but there have been numerous reports that show the dangers of overestimating the value of the impact factor (Seglen 1997; Opthof 1997; Coleman 1999). The impact factor of a journal may be artificially inflated when it frequently includes review articles and letters, and there is also a bias against articles that are not published in English. Journals that publish articles on botanical or zoological taxonomy may also have lower impact factors because they often rely on older references, and references to taxonomic names are not included in the references list at the ends of these papers (Werner 2006).
Since the widely used JCR impact factors are all based on data from journals indexed in Web of Science, the researchers of the present study were interested in comparing the impact factors calculated from citation data from another database. Elsevier's Scopus, which became available in 2004, has been considered a competitor to Web of Science because of its citation tracking capabilities (Bakkalbasi et al. 2006). Scopus does not rank journals according to "impact factor," using instead a measure called the h-index that was developed in 2005 by Hirsch to evaluate the impact of journals. However, the data required to calculate an impact factor using Garfield's ratio are all available in the Scopus database (Pislyakov 2007). If impact factor is the most widely used tool for evaluating the importance of a journal and thus the worth of its writers, it would be wise to determine whether or not calculated impact factors vary significantly between JCR and Scopus.
Methods
Impact factors were obtained for the 116 ecology titles and the 160 environmental sciences titles listed in the 2007 edition of JCR. In this latest edition of JCR, cites to recent articles (the numerator of the impact factor equation) were from 2007, while citable articles (the denominator) were from 2005 and 2006. JCR does not state what constitutes a citable article. Generally, the number of citable articles in JCR corresponds to the number of research and review articles indexed in Web of Science for any particular journal, however, the two numbers don't always correspond.Impact factors were generated in Scopus by individually searching in the source title field each journal from the JCR ecology and environmental sciences lists. Dates were limited to 2005 through 2006. If Scopus didn't index a journal from the JCR list during this time period, the title was eliminated from further consideration. Once results were obtained for each journal, all the citations were selected and the citation tracker feature was used to determine the number of times these articles were cited in 2007. These numbers were placed in an Excel spreadsheet and were divided against the number of citable articles from JCR to come up with Scopus impact factors.
Using the rank formula in Excel, the journals were ranked by both their JCR Scopus impact factors. For each journal, the Scopus rank was subtracted from the JCR rank to determine a change in rank. Finally, the rankings were entered into an online statistical application to determine their Spearman rank-order correlation coefficients, which determine statistical similarity between two lists (Lowry 2008).
Results and Discussion
Tables 1 and 2 show the JCR and Scopus impact factors, the JCR and Scopus ranks, and the change in rank from JCR to Scopus for the top 50 ecology and environmental sciences titles, respectively. Although several of the titles changed greatly in rank (e.g., Wildlife Monographs dropped 38 places from 17 in JCR to 55 in Scopus for the ecology list), the JCR and Scopus ranks had a high degree of statistical similarity. The Spearman rank-order correlation coefficient was .9818 for the ecology titles and .9823 for the environmental science titles.Several journals warrant closer examination because of their changes in impact factor and rank. Although Bulletin of the American Museum of Natural History only dropped one spot from number one in JCR ecology rank to number two in Scopus ecology rank, its Scopus impact factor (16.385) was nearly 4.7 points less than its JCR impact factor (11.692). Several factors could account for this steep drop in impact factor. JCR listed 13 citable articles that determined its impact factor. Scopus, which did not match Web of Science in its indexing of this title, only returned 11 articles for this journal. Because we relied on the number of citable articles from JCR to create our Scopus impact factor, any title that isn't completely indexed by Scopus will have an underinflated impact factor. Had we used the 11 articles indexed in Scopus as our number of citable articles, the impact factor would have increased by over two points to 13.818.
The high JCR impact factor for Bulletin of the American Museum of Natural History might also result from the data in JCR not matching the data in Web of Science. When we performed a publication search in Web of Science for Bulletin of the American Museum of Natural History for 2005 and 2006 and selected the "create citation report" feature, the database showed that the journal received only 153 cites in 2007; JCR (which is supposedly using Web of Science data) returned 213 cites. Using 153 cites instead of 213 cites creates a JCR impact factor of 11.769, which is more in line with the Scopus number. In case JCR is using incorrect citations in Web of Science to come up with a cite count, we also performed a search for the journal in the cited reference search. Articles from 2005 and 2006 were cited 203 times in 2007. If we use 203 cites, we get a JCR impact factor of 15.615. Because JCR doesn't indicate what cites are used in their data, we can't be certain if the discrepancy between the JCR and Scopus impacts is the result of incomplete indexing by Scopus, poor data in JCR, our methodology of relying on JCR for the number of citable articles, or some combination of all three.
Wildlife Monographs was another journal that shows a major change between the two different sets of impact factors as that title dropped from 17th in the JCR rank to 55th in the Scopus rank for ecology titles. Both Web of Science and Scopus indexed 10 articles for the journal from 2005 to 2006, so incomplete indexing by Scopus can't be the cause of the difference. Using the "create citation report" feature in Web of Science, we got only 10 cites for Wildlife Monographs compared to the 39 listed in JCR. Using the cited reference search, we got 33 cites for the journal. If we rely on the Web of Science numbers instead of the JCR numbers, the impact factor for Wildlife Monographs is either 1.000 or 3.300, which places the Scopus impact factor of 1.700 nearly in the middle.
Table 1: Impact Factors, Ranks, and Change in Ranks for the Top 50 Ecology Journals
Journal Title |
JCR Impact Factor |
Scopus Impact Factor |
JCR Rank |
Scopus Rank |
Change in Rank |
Bulletin of the American Museum of Natural History |
16.385 |
11.692 |
1 |
2 |
-1 |
Trends in Ecology & Evolution |
14.797 |
14.993 |
2 |
1 |
1 |
Annual Review of Ecology Evolution and Systematics |
10.340 |
10.440 |
3 |
3 |
0 |
Ecology Letters |
8.204 |
8.292 |
4 |
4 |
0 |
Ecological Monographs |
8.117 |
8.117 |
5 |
5 |
0 |
Molecular Ecology |
5.169 |
5.151 |
6 |
6 |
0 |
Ecology |
4.822 |
4.775 |
7 |
8 |
-1 |
Global Change Biology |
4.786 |
5.000 |
8 |
7 |
1 |
American Naturalist |
4.543 |
4.457 |
9 |
9 |
0 |
Evolution |
4.502 |
4.441 |
10 |
10 |
0 |
Global Ecology and Biogeography |
4.435 |
4.259 |
11 |
12 |
-1 |
Journal of Ecology |
4.422 |
4.422 |
12 |
11 |
1 |
Frontiers in Ecology and the Environment |
4.269 |
4.241 |
13 |
13 |
0 |
Journal of Applied Ecology |
4.220 |
4.174 |
14 |
14 |
0 |
Conservation Biology |
3.934 |
3.926 |
15 |
15 |
0 |
Journal of Evolutionary Biology |
3.920 |
3.911 |
16 |
16 |
0 |
Wildlife Monographs |
3.900 |
1.700 |
17 |
55 |
-38 |
Journal of Animal Ecology |
3.747 |
3.720 |
18 |
17 |
1 |
Ecological Applications |
3.571 |
3.562 |
19 |
18 |
1 |
Journal of Biogeography |
3.539 |
3.503 |
20 |
19 |
1 |
Biological Conservation |
3.296 |
3.354 |
21 |
20 |
1 |
Paleobiology |
3.225 |
3.270 |
22 |
21 |
1 |
Functional Ecology |
3.157 |
3.112 |
23 |
24 |
-1 |
Oikos |
3.136 |
3.094 |
24 |
25 |
-1 |
Ecography |
3.066 |
3.132 |
25 |
23 |
2 |
Perspectives in Plant Ecology Evolution and Systematics |
3.043 |
3.261 |
26 |
22 |
4 |
Behavioral Ecology |
3.018 |
3.075 |
27 |
26 |
1 |
Oecologia |
2.973 |
3.022 |
28 |
28 |
0 |
Diversity and Distributions |
2.965 |
3.035 |
29 |
27 |
2 |
Evolutionary Ecology |
2.905 |
2.068 |
30 |
45 |
-15 |
Biogeosciences |
2.813 |
2.520 |
31 |
34 |
-3 |
Behavioral Ecology and Sociobiology |
2.754 |
2.699 |
32 |
29 |
3 |
Ecosystems |
2.684 |
2.637 |
33 |
30 |
3 |
Microbial Ecology |
2.558 |
2.522 |
34 |
33 |
1 |
Marine Ecology-Progress Series |
2.546 |
2.533 |
35 |
32 |
3 |
Animal Conservation |
2.495 |
2.392 |
36 |
36 |
0 |
Ecotoxicology |
2.405 |
1.954 |
37 |
49 |
-12 |
Aquatic Microbial Ecology |
2.385 |
2.366 |
38 |
37 |
1 |
Agriculture Ecosystems & Environment |
2.308 |
2.505 |
39 |
35 |
4 |
Journal of Vegetation Science |
2.251 |
2.152 |
40 |
43 |
-3 |
Basic and Applied Ecology |
2.247 |
2.351 |
41 |
38 |
3 |
Journal of the North American Benthological Society |
2.217 |
2.224 |
42 |
40 |
2 |
Ecology and Society |
2.215 |
2.049 |
43 |
46 |
-3 |
Ecological Engineering |
2.175 |
2.330 |
44 |
39 |
5 |
Biological Invasions |
2.125 |
2.165 |
45 |
42 |
3 |
Ecological Modelling |
2.077 |
2.196 |
46 |
41 |
5 |
Landscape Ecology |
2.061 |
2.624 |
47 |
31 |
16 |
Theoretical Population Biology |
1.950 |
1.917 |
48 |
50 |
-2 |
Journal of Chemical Ecology |
1.941 |
1.959 |
49 |
48 |
1 |
Restoration Ecology |
1.928 |
1.993 |
50 |
47 |
3 |
Table 2: Impact Factors, Ranks, and Change in Ranks for the Top 50 Environmental Sciences Journals
Journal Title |
JCR Impact Factor |
Scopus Impact Factor |
JCR Rank |
Scopus Rank |
Change in Rank |
Environmental Health Perspectives |
5.636 |
5.804 |
1 |
1 |
0 |
Global Change Biology |
4.786 |
5.000 |
2 |
2 |
0 |
Critical Reviews in Environmental Science and Technology |
4.615 |
4.615 |
3 |
3 |
0 |
Environmental Science & Technology |
4.363 |
4.285 |
4 |
6 |
-2 |
Global Biogeochemical Cycles |
4.335 |
3.947 |
5 |
8 |
-3 |
Frontiers in Ecology and the Environment |
4.269 |
4.241 |
6 |
7 |
-1 |
Annual Review of Environment and Resources |
4.036 |
4.357 |
7 |
5 |
2 |
Conservation Biology |
3.934 |
3.926 |
8 |
9 |
-1 |
Global Environmental Change-Human and Policy Dimensions |
3.915 |
4.559 |
9 |
4 |
5 |
Environmental Science and Pollution Research |
3.894 |
3.173 |
10 |
17 |
-7 |
Ecological Applications |
3.571 |
3.562 |
11 |
10 |
1 |
Water Research |
3.427 |
3.223 |
12 |
15 |
-3 |
Biological Conservation |
3.296 |
3.354 |
13 |
12 |
1 |
Applied Catalysis A-General |
3.166 |
3.290 |
14 |
13 |
1 |
Environmental Pollution |
3.135 |
3.231 |
15 |
14 |
1 |
Geobiology |
3.114 |
2.864 |
16 |
21 |
-5 |
Remote Sensing of Environment |
3.013 |
3.207 |
17 |
16 |
1 |
Journal of Environmental Science and Health Part C- Environmental Carcinogenesis & Ecotoxicology Reviews |
3.000 |
3.429 |
18 |
11 |
7 |
Environmental Research |
2.962 |
2.996 |
19 |
18 |
1 |
Climatic Change |
2.890 |
2.826 |
20 |
22 |
-2 |
Environmental Chemistry |
2.809 |
2.351 |
21 |
30 |
-9 |
Environment International |
2.797 |
2.977 |
22 |
19 |
3 |
Chemosphere |
2.739 |
2.820 |
23 |
23 |
0 |
International Journal of Hydrogen Energy |
2.725 |
2.893 |
24 |
20 |
4 |
Atmospheric Environment |
2.549 |
2.729 |
25 |
24 |
1 |
Biogeochemistry |
2.534 |
2.592 |
26 |
26 |
0 |
Environmental Reviews |
2.467 |
2.600 |
27 |
25 |
2 |
Ecotoxicology |
2.405 |
1.954 |
28 |
46 |
-18 |
Environmental and Molecular Mutagenesis |
2.361 |
2.340 |
29 |
31 |
-2 |
Aerosol Science and Technology |
2.350 |
2.070 |
30 |
42 |
-12 |
Journal of Hazardous Materials |
2.337 |
2.442 |
31 |
28 |
3 |
Marine Pollution Bulletin |
2.334 |
2.368 |
32 |
29 |
3 |
Journal of Environmental Quality |
2.331 |
2.218 |
33 |
36 |
-3 |
Environmental Toxicology and Chemistry |
2.309 |
2.218 |
34 |
37 |
-3 |
Agriculture Ecosystems & Environment |
2.308 |
2.505 |
35 |
27 |
8 |
Journal of Paleolimnology |
2.287 |
2.182 |
36 |
39 |
-3 |
Science of the Total Environment |
2.182 |
2.298 |
37 |
34 |
3 |
Ecological Engineering |
2.175 |
2.330 |
38 |
32 |
6 |
Water Resources Research |
2.154 |
1.914 |
39 |
48 |
-9 |
Estuaries |
2.133 |
1.867 |
40 |
51 |
-11 |
Environmental Modelling & Software |
2.099 |
2.293 |
41 |
35 |
6 |
Ecotoxicology and Environmental Safety |
2.014 |
2.118 |
42 |
41 |
1 |
Journal of Industrial Ecology |
1.962 |
2.308 |
43 |
33 |
10 |
Boreal Environment Research |
1.951 |
2.122 |
44 |
40 |
4 |
Marine Environmental Research |
1.930 |
1.930 |
45 |
47 |
-2 |
Journal of Aerosol Science |
1.902 |
2.019 |
46 |
43 |
3 |
Energy Policy |
1.901 |
2.185 |
47 |
38 |
9 |
Journal of Contaminant Hydrology |
1.852 |
1.955 |
48 |
45 |
3 |
Journal of Environmental Monitoring |
1.833 |
1.792 |
49 |
54 |
-5 |
Reviews of Environmental Contamination and Toxicology |
1.818 |
1.818 |
50 |
52 |
-2 |
Conclusion
Impact factors are useful for ranking and comparing scholarly journals, and are one of the most widely used indicators of journals' significance to the scholarly community. With all the weight that is given to the impact factor, it is important that these values do not significantly vary from one database's calculation to another. This study's comparison between JCR and Scopus impact factors in the fields of ecology and environmental sciences showed no statistically significant difference, which indicates that one or the other may be used for evaluative purposes.While most of the impact factors and journal ranks for Web of Science and Scopus were fairly similar, there were a few journals that had significantly different impact factors. In some instances, these differences could be accounted for by incomplete indexing in Scopus. Because we relied upon the number of citable articles in JCR to calculate our Scopus impact factors, our methodology may have underinflated the Scopus impact factor and rank for some titles. Our quick examination of the data in Web of Science also shows that number of cites used in JCR might be incorrect for some titles. Finally, the collection of titles indexed by the two citation databases can affect the differences in rank and impact factors seen in JCR and Scopus.
Since JCR impact factors are the standard used by most journal advertisers and university administrators, we may now be confident that these values are close enough to Scopus impact factors to justify the use of a single database's impact factor. This is valuable information because it proves that it is not necessary to calculate Scopus impact factors in addition to the JCR data. It is, however, still important to use the JCR impact factor carefully. There are many variables that can affect the impact factor of a journal that must be considered when using Garfield's ratio to evaluate the importance of a journal.
References
Bakkalbasi, N., Bauer, K., Glover, J. & Wang, L. 2006. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3(1):7.Barschall, H.H. 1988. The cost-effectiveness of physics journals. Physics Today. 41(7):56-59.
Coleman, A. 2007. Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology 58(8):1148-1161.
Coleman, R. 1999. Impact factors: Use and abuse in biomedical research. The Anatomical Record 257:54-57.
Garfield, W. 1972. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 178(4060):471-479.
Garfield, E. & Sher, I.H. 1963. New factors in the evaluation of scientific literature through citation indexing. American Documentation 14(3):195-201.
Hirsch, J.E. 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America. 102(46):16569-16572.
Lowry, R. 2008. VassarStats: Web Site for Statistical Computation. [Online]. Available: http://faculty.vassar.edu/lowry/VassarStats.html [July 8, 2008].
Monastersky, R. 2005. The number that's devouring science: The impact factor, once a simple way to rank scientific journals, has become an unyielding yardstick for hiring, tenure, and grants. The Chronicle of Higher Education 52(8):A12.
Opthof, T. 1997. Sense and nonsense about the impact factor. Cardiovascular Research 33:1-7.
Pislyakov, V. 2007. Comparing two "thermometers": Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus. E-LIS. [Online]. Available: http://eprints.rclis.org/archive/00011865 [July 2, 2008].
Seglen, P.O. 1997. Why the impact factor should not be used for evaluating research. BMJ 314:498-502.
Thomson Scientific. 1994. The Thomson Scientific impact factor. [Online]. Available: http://scientific.thomsonreuters.com/free/essays/journalcitationreports/impactfactor [June 27, 2008].
Werner, Y.L. 2006. The case of impact factor versus taxonomy: a proposal. Journal of Natural History 40(21-22):1285-1286.
| Previous | Contents | Next |
