Extramural Research

Bibliometrics

Grantee Research Project Results

2005 Combined Bibliometric Analysis for Papers on Topics Relating to Water Quality Research (Papers Published 1995-2005)
November 2006

This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of the U.S. Environmental Protection Agency (EPA) on topics related to water quality research that were published from 1995 to 2005. For this analysis, 800 papers were reviewed. These 800 papers were cited 10,234 times in the journals covered by Thomsons Web of Science.¹ Of these 800 papers, 668 (84%) have been cited at least once in a journal.

The analysis was completed using Thomsons Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomsons databases. For influence and impact measures, ESI employs both total citation counts and cites per paper scores. The former reveals gross influence while the latter shows weighted influence, also called impact. JCR presents quantifiable statistical data that provide a systematic, objective way to evaluate the worlds leading journals and their impact and influence in the global research community.

Summary of Analysis

Nearly one-seventh of the water quality publications are highly cited papers. A review of the citations indicates that 114 (14.2%) of the water quality papers qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. Twelve (1.5%) of the water quality papers qualify as highly cited when using the criteria for the top 1%. Three (0.38%) of the papers meet the 0.1% thresholds for very highly cited papers, and two (0.25%) of the papers qualify as very highly cited when using the criteria for the top 0.01%.

The water quality papers are more highly cited than the average paper. Using the ESI average citation rates for papers published by field as the benchmark, in 10 of the 12 fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the water quality papers are more highly cited than the average papers in those fields.

Approximately one-eighth of the water quality papers are published in very high impact journals. One-hundred six (106) of the 800 papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 13.2% of EPAs water quality papers. Ninety-one (91) of the 800 papers appear in the top 10% of journals ranked by JCR Immediacy Index, representing 11.4% of EPAs water quality papers.

Nine of the water quality publications qualified as hot papers. ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorterpapers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Using the current hot paper thresholds established by ESI as a benchmark, nine of the water quality papers, representing 1.1% of the water quality publications, were identified as hot papers in the analysis.

The authors of the water quality papers cite themselves less than the average self-citation rate. Four hundred seventeen (417) of the 10,234 cites are author self-cites. This 4.1% author self-citation rate is below the accepted range of 10-30% author self-citation rate.

Highly Cited Water Quality Publications

The 800 water quality papers reviewed for this analysis covered 12 of the 22 ESI fields of research. The distribution of the papers among these 12 fields and the number of citations by field are presented in Table 1.

Table 1. Water Quality Papers by ESI Fields

No. of Citations	ESI Field	No. of EPA Water Quality Papers	Average Cites/Paper
8,092	Environment/Ecology	581	13.93
500	Pharmacology & Toxicology	40	12.50
432	Engineering	47	9.19
320	Biology & Biochemistry	32	10.00
310	Microbiology	23	13.48
215	Chemistry	33	6.52
170	Plant & Animal Science	29	5.86
99	Multidisciplinary	4	24.75
47	Geosciences	5	9.40
36	Agricultural Sciences	3	12.00
8	Clinical Medicine	2	4.00
5	Physics	1	5.00
Total = 10,234		Total = 800	12.79

There were 114 (14.2% of the papers analyzed) highly cited EPA water quality papers in 10 of the 12 fieldsEnvironment/Ecology, Engineering, Pharmacology & Toxicology, Biology & Biochemistry, Microbiology, Chemistry, Multidisciplinary, Plant & Animal Science, Geosciences, and Agricultural Scienceswhen using the ESI criteria for the top 10% of papers. Table 2 shows the number of EPA papers in those 10 fields that met the top 10% threshold in ESI.

Twelve (1.5%) of the papers analyzed qualified as highly cited when using the ESI criteria for the top 1% of papers (see Table 3). These papers were categorized in the fields of Environment/ Ecology and Engineering. The citations for these papers are presented in Tables 4 and 5. Three of the 800 papers (0.38%) met the 0.1% threshold, which is nearly four times the number expected, and two of the 800 papers (0.25%) met the 0.01% threshold for very highly cited papers, which is nearly 25 times the number expected. The citations for the papers that met the 0.1% and the 0.01% thresholds for very highly cited papers are provided in Tables 6 and 7, respectively.

Table 2. Number of Highly Cited Water Quality Papers by Field (top 10%)

No. of Citations	ESI Field	No. of Papers	Average Cites/Paper	% of EPA Papers in Field
4,647	Environment/Ecology	69	67.35	11.88%
349	Engineering	18	19.39	38.30%
293	Pharmacology & Toxicology	7	41.86	17.50%
155	Biology & Biochemistry	1	155.00	3.13%
149	Microbiology	3	49.67	13.04%
88	Chemistry	6	14.67	18.18%
85	Multidisciplinary	3	28.33	75.00%
57	Plant & Animal Science	5	11.40	17.24%
30	Geosciences	1	30.00	20.00%
10	Agricultural Sciences	1	10.00	33.33%
Total = 5,863		Total = 114	51.43

Table 3. Number of Highly Cited Water Quality Papers by Field (top 1%)

No. of Citations	ESI Field	No. of Papers	Average Cites/Paper	% of EPA Papers in Field
2,191	Environment/Ecology	9	243.44	1.55%
148	Engineering	3	49.33	6.38%
Total = 2,339		Total = 12	194.92

Table 4. Citations of Highly Cited Water Quality Papers in the Field of Environment/Ecology (top 1%)

No. of Cites	First Author	Paper
792	Van den Berg M	Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792.
323	Daughton CG	Pharmaceuticals and personal care products in the environment: agents of subtle change? Environmental Health Perspectives 1999;107(Suppl 6):907-938.
304	Howarth RW	Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139.
230	Folmar LC	Vitellogenin induction and reduced serum testosterone concentrations in feral male carp (Cyprinus carpio) captured near a major metropolitan sewage treatment plant. Environmental Health Perspectives 1996;104(10):1096-1101.
170	Poff NL	Functional-organization of stream fish assemblages in relation to hydrological variability. Ecology 1995;76(2):606-627.
130	Boynton WR	Inputs, transformations, and transport of nitrogen and phosphorus in Chesapeake Bay and selected tributaries. Estuaries 1995;18(1B):285-314.
128	Ankley GT	Technical basis and proposal for deriving sediment quality criteria for metals. Environmental Toxicology and Chemistry 1996;15(12):2056-2066.
111	Erickson RJ	The effects of water chemistry on the toxicity of copper to fathead minnows. Environmental Toxicology and Chemistry 1996;15(2):181-193.
3	Lackey RT	Economic growth and salmon recovery: an irreconcilable conflict? Fisheries 2005;30(3):30-32.

Table 5. Citations of Highly Cited Water Quality Papers in the Field of Engineering (top 1%)

No. of Cites	First Author	Paper
57	Jaworski NA	Atmospheric deposition of nitrogen oxides onto the landscape contributes to coastal eutrophication in the northeast United States. Environmental Science & Technology 1997;31(7):1995-2004.
50	Ankley GT	Effects of light-intensity on the phototoxicity of fluoranthene to a benthic macroinvertebrate. Environmental Science & Technology 1995;29(11):2828-2833.
41	Montgomery DR	Process domains and the river continuum. Journal of the American Water Resources Association 1999;35(2):397-410.

Table 6. Citations of Very Highly Cited Water Quality Papers (top 0.1%)

Field	No. of Cites	First Author	Paper
Environment/Ecology	792	Van den Berg M	Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792.
	323	Daughton CG	Pharmaceuticals and personal care products in the environment: agents of subtle change? Environmental Health Perspectives 1999;107(Suppl 6):907-938.
	304	Howarth RW	Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139.

Table 7. Citations of Very Highly Cited Water Quality Papers (top 0.01%)

Field	No. of Cites	First Author	Paper
Environment/Ecology	792	Van den Berg M	Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792.
	323	Daughton CG	Pharmaceuticals and personal care products in the environment: agents of subtle change? Environmental Health Perspectives 1999;107(Suppl 6):907-938.

Ratio of Actual Cites to Expected Citation Rates

The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present. Using the ESI average citation rates for papers published by field as the benchmark, in 10 of the 12 fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the EPA papers are more highly cited than the average papers in those fields (see Table 8).

Table 8. Ratio of Actual Cites to Expected Cites for Water Quality Papers by Field

ESI Field	Total Cites	Expected Cite Rate	Ratio
Multidisciplinary	99	16.67	5.94
Engineering	432	121.31	3.56
Agricultural Sciences	36	12.51	2.88
Environment/Ecology	8,092	4,511.15	1.79
Geosciences	47	26.78	1.76
Physics	5	3.32	1.51
Plant & Animal Science	170	118.87	1.43
Pharmacology & Toxicology	500	359.62	1.39
Chemistry	215	187.34	1.15
Microbiology	310	284.98	1.09
Clinical Medicine	8	8.86	0.90
Biology & Biochemistry	320	392.86	0.81

JCR Benchmarks

The Impact Factor is a well known metric in citation analysis. It is a measure of the frequency with which the average article in a journal has been cited in a particular year. The Impact Factor helps evaluate a journals relative importance, especially when compared to others in the same field. The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years.

Table 9 indicates the number of water quality papers published in the top 10% of journals, based on the JCR Impact Factor. One-hundred six (106) of the 800 papers were published in the top 10% of journals, representing 13.2% of EPAs water quality papers. This exceeds the expected number of 80 papers (10%) published in the top 10% of high impact journals.

Table 9. Water Quality Papers in Top 10% of Journals by JCR Impact Factor

EPA Water Quality Papers in that Journal	Journal	Impact Factor (IF)	JCR IF Rank
20	Environmental Science & Technology	3.557	540
14	Environmental Health Perspectives	3.929	439
14	Limnology and Oceanography	3.024	737
10	Applied and Environmental Microbiology	3.810	470
6	Ecological Applications	3.287	623
5	Analytical Chemistry	5.450	243
5	Ecology	4.104	394
5	Toxicological Sciences	3.391	591
5	Journal of Chromatography A	3.359	602
3	TRAC-Trends in Analytical Chemistry	3.888	452
2	Nature	32.182	9
2	Proceedings of the National Academy of Sciences of the United States of America	10.452	88
2	Electrophoresis	3.743	482
2	Ecosystems	3.283	624
2	Remote Sensing of Environment	3.185	666
2	Bioscience	3.041	730
1	Lancet	21.713	20
1	Progress in Nuclear Magnetic Resonance Spectroscopy	6.885	175
1	Molecular Ecology	4.375	351
1	Drug Metabolism and Disposition	3.836	461
1	Journal of the American Society for Mass Spectrometry	3.760	479
1	Proceedings of the Royal Society of London Series B-Biological Sciences	3.653	509
1	Frontiers in Ecology and the Environment	3.362	600
Total = 106

Immediacy Index

The journal Immediacy Index is a measure of how quickly the average article in a journal is cited. It indicates how often articles published in a journal are cited within the year they are published. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Table 10 indicates the number of EPA water quality papers published in the top 10% of journals, based on the JCR Immediacy Index. Ninety-one (91) of the 800 papers appear in the top 10% of journals, representing 11.4% of EPAs water quality papers. This exceeds the expected number of 80 papers (10%) published in the top 10% of high impact journals.

Table 10. Water Quality Papers in Top 10% of Journals by JCR Immediacy Index

EPA Water Quality Papers in that Journal	Journal	Immediacy Index (II)	JCR II Rank
20	Environmental Science & Technology	0.623	617
14	Environmental Health Perspectives	1.202	202
6	Ecological Applications	0.747	466
6	Freshwater Biology	0.664	558
5	Analytical Chemistry	0.885	346
5	Hydrobiologia	0.681	532
5	Ecology	0.590	676
4	Ecotoxicology	1.450	151
3	Journal of Geophysical Research	0.617	630
3	TRAC-Trends in Analytical Chemistry	0.583	681
2	Nature	6.089	5
2	Ecosystems	2.048	76
2	Proceedings of the National Academy of Sciences of the United States of America	1.923	89
2	Bioscience	0.863	356
2	Electrophoresis	0.575	697
1	Lancet	5.017	12
1	Journal of Paleolimnology	1.581	132
1	Ambio	1.435	156
1	Climatic Change	1.235	195
1	Journal of Oceanography	0.692	519
1	Molecular Ecology	0.674	545
1	Progress in Nuclear Magnetic Resonance Spectroscopy	0.667	551
1	Drug Metabolism and Disposition	0.590	676
1	X-Ray Spectrometry	0.580	685
1	Journal of the American Society for Mass Spectrometry	0.575	697
Total = 91

Hot Papers

ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorterpapers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers. There were no hot papers identified for the most recently completed 2-month period (i.e., September-October 2005), but there were nine hot papers identified from previous periods.

Using the current hot paper thresholds established by ESI as a benchmark, nine of the water quality papers, representing 1.1% of the water quality publications, were identified in the fields of Environment/Ecology and Multidisciplinary. The hot papers are listed in Table 11.

Table 11. Hot Papers Identified Using Current ESI Thresholds

Field	ESI Hot Papers Threshold	No. of Cites in 2-Month Period	Paper
Environment/ Ecology	7	9 cites in March-April 2003	Angradi TR, et al. Vegetation type and the intertidal macroinvertebrate fauna of a brackish marsh: Phragmites vs. Spartina. Wetlands 2001;21(1):75-92.
	7	20 cites in November-December 2000	Van den Berg M, et al. Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792.
	8	13 cites in December 1996	Berry WJ, et al. Predicting the toxicity of metal-spiked laboratory sediments using acid-volatile sulfide and interstitial water normalizations. Environmental Toxicology and Chemistry 1996;15(12):2067-2079.
	8	11 cites in December 1996	Hansen DJ, et al. Chronic effect of cadmium in sediments on colonization by benthic marine organisms: an evaluation of the role of interstitial cadmium and acid-volatile sulfide in biological availability. Environmental Toxicology and Chemistry 1996;15(12):2126-2137.
	8	10 cites in December 1996	Hansen DJ, et al. Predicting the toxicity of metal-contaminated field sediments using interstitial concentration of metals and acid-volatile sulfide normalizations. Environmental Toxicology and Chemistry 1996;15(12):2080-2094.
	7	10 cites in November-December 1996	Pesch CE, et al. The role of acid volatile sulfide and interstitial water metal concentrations in determining bioavailability of cadmium and nickel from contaminated sediments to the marine polychaete Neanthes arenaceodentata. Environmental Toxicology and Chemistry 1995;14(1):129-141.
	8	9 cites in December 1996	Liber K, et al. Effects of acid-volatile sulfide on zinc bioavailability and toxicity to benthic macroinvertebrates: a spiked-sediment field experiment. Environmental Toxicology and Chemistry 1996;15(12):2113-2125.
Environment/ Ecology	6	6 cites in July-August 1997	Howarth RW. Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139.
Multidisciplinary	6	7 cites in October-November 2001	Steidinger KA, Rublee PA. Heteroduplex mobility assay-guided sequence discovery: elucidation of the small subunit (18S) rDNA sequences of Pfiesteria piscicida and related dinoflagellates from complex algal culture and environmental sample DNA pools. Proceedings of the National Academy of Sciences of the United States of America 2000; 97(8):4303-4308.

Author Self-Citation

Self-citations are journal article references to articles from that same author (i.e., the first author). Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the water quality papers. Of the 10,234 total cites, 417 are author self-citesa 4.1% author self-citation rate. Garfield and Sher² found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time. MacRoberts and MacRoberts³ claim that approximately 10% to 30% of all the citations listed fall into the category of author self-citation. Therefore, the 4.1% self-cite rate for the water quality papers is below the expected range for author self-citation.

¹ Thomson's Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,500 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching.

² Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation 1963;18(July):195-201.

³ MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. Journal of the American Society of Information Science 1989;40(5):342-349.

You will need Adobe Reader to view some of the files on this page. See EPA's PDF page to learn more.

Top of page