Extramural Research
Bibliometrics
Grantee Research Project Results
Bibliometric Analysis for Papers on Topics Related to Water Quality Research (Papers Published 1995 to 1999)
January 5, 2006
This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of the U.S. Environmental Protection Agency (EPA) on topics related to water quality research that were published from 1995 to 1999. For this analysis, 294 papers were reviewed.1 These 294 papers were cited 7,687 times in the journals covered by Thomson’s Web of Science.2 Of these 294 papers, 289 (98.3%) have been cited at least once in a journal.
The analysis was completed using Thomson’s Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomson’s databases. The chief indicators of output, or productivity, are journal article publication counts. For influence and impact measures, ESI employs both total citation counts and cites per paper scores. The former reveals gross influence while the latter shows weighted influence, also called impact. JCR presents quantifiable statistical data that provide a systematic, objective way to evaluate the world’s leading journals and their impact and influence in the global research community.
Summary of Analysis
Nearly one-quarter of the water quality publications are highly cited papers. A review of the citations indicates that 73 (24.8%) of the water quality papers qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. Eleven (3.7%) of the water quality papers qualify as highly cited when using the criteria for the top 1%. Three (1.0%) of the water quality papers qualify as very highly cited (in the top 0.1%), and two (0.7%) actually meet the top 0.01% threshold.
The water quality papers are more highly cited than the average paper. Using the ESI average citation rates for papers published by field as the benchmark, in 7 of the 7 fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the water quality papers are more highly cited than the average papers in those fields.
More than one-eighth of the water quality papers are published in very high impact journals. Thirty-nine(39)of 294 papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 13.3% of EPA’s water quality papers. Approximately one-eighth of the water quality papers are published in the top 10% of journals ranked by JCR Immediacy Factor. Twenty-nine (29) of the 294 papers appear in the top 10% of journals, representing 9.9% of EPA’s water quality papers.
Seven of the water quality publications qualified as hot papers. ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Using the current hot paper thresholds established by ESI as a benchmark, seven of the water quality papers, representing 2.4% of the water quality publications, were identified as hot papers in the analysis.
The authors of the water quality papers cite themselves less than the average self-citation rate. Two hundred-nineteen(219) of the 7,687 cites are author self-cites. This 2.8% author self-citation rate is well below the accepted range of 10-30% author self-citation rate.
Highly Cited Water Quality Publications
The 294 water quality papers reviewed for this analysis covered 7 of the 22 ESI fields of research. The distribution of the papers among these 7 fields and the number of citations by field are presented in Table 1.
Table 1. Water Quality Papers by ESI Fields
|
No. of Citations |
ESI Field |
No. of EPA Water Quality Papers |
Average Cites/Paper |
|
6,512 |
Environment/Ecology |
238 |
27.36 |
|
324 |
Engineering |
17 |
19.06 |
|
298 |
Pharmacology & Toxicology |
14 |
21.28 |
|
234 |
Biology & Biochemistry |
7 |
33.43 |
|
183 |
Microbiology |
8 |
22.88 |
|
72 |
Plant & Animal Science |
5 |
14.40 |
|
64 |
Chemistry |
5 |
12.80 |
Total = 7,687 |
Total = 294 |
26.15 |
There were 73 (24.8% of the papers analyzed) highly cited EPA water quality papers in 7 of the 7 fields—Environment/Ecology, Engineering, Pharmacology & Toxicology, Biology & Biochemistry, Microbiology, Plant & Animal Science, and Chemistry—when using the ESI criteria for the top 10% of papers. Table 2 shows the number of EPA papers in those seven fields that met the top 10% threshold in ESI.
Eleven (3.7%) of the papers analyzed qualified as highly cited when using the ESI criteria for the top 1% of papers. This number is nearly four times higher than would be expected. These papers were categorized in the fields of Environment/Ecology and Engineering. Three (1.0%) of the papers qualified as very highly cited when using the ESI criteria for the top 0.1% of papers, and two of these papers (Van den Berg and Daughton) actually met ESI’s highest threshold for very highly cited papers(top 0.01%). These numbers are 10 times higher (for the top0.1%) and 68 times higher (for the top 0.01%) than would be expected. Tables 3, 4 and 5 report the number of papers that met the top 1% threshold in ESI, the top 0.1% threshold in ESI, and the top 0.01% threshold in ESI, respectively. The citations for the highly cited papers in the top 1% are presented in Tables 6 and 7, and the citations for the very highly cited papers in the top 0.1% are listed in Table 8.
Table 2. Number of Highly Cited Water Quality Papers by Field (top 10%)
No. of Citations |
ESI Field |
No. of Papers |
Average Cites/Paper |
% of EPA Papers in Field |
4,580 |
Environment/Ecology |
55 |
83.27 |
23.11% |
270 |
Engineering |
9 |
30.00 |
52.94% |
222 |
Pharmacology & Toxicology |
5 |
44.40 |
35.71% |
155 |
Biology & Biochemistry |
1 |
155.00 |
14.29% |
54 |
Microbiology |
1 |
54.00 |
12.50% |
37 |
Chemistry |
1 |
37.00 |
20.00% |
26 |
Plant & Animal Science |
1 |
26.00 |
20.00% |
Total = 5,344 |
Total = 73 |
73.20 |
Table 3. Number of Highly Cited Water Quality Papers by Field (top 1%)
No. of Citations |
ESI Field |
No. of Papers |
Average Cites/Paper |
% of EPA Papers in Field |
2,188 |
Environment/Ecology |
8 |
273.50 |
3.36% |
148 |
Engineering |
3 |
49.33 |
17.65% |
Total = 2,336 |
Total = 11 |
212.36 |
|
Table 4. Number of Highly Cited Water Quality Papers by Field (top 0.1%)
No. of Citations |
ESI Field |
No. of Papers |
Average Cites/Paper |
% of EPA Papers in Field |
1,419 |
Environment/Ecology |
3 |
473.00 |
1.26% |
Total = 1,419 |
Total = 3 |
473.00 |
|
Table 5. Number of Highly Cited Water Quality Papers by Field (top 0.01%)
No. of Citations |
ESI Field |
No. of Papers |
Average Cites/Paper |
% of EPA Papers in Field |
1,115 |
Environment/Ecology |
2 |
557.50 |
0.84% |
Total = 1,115 |
Total = 2 |
557.50 |
|
Table 6. Citations of Highly Cited Water Quality Papers in the
Field of Environment/Ecology (top 1%)
No. of Cites |
First Author |
Paper |
792 |
Van den Berg M |
Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792. |
323 |
Daughton CG |
Pharmaceuticals and personal care products in the environment: agents of subtle change? Environmental Health Perspectives 1999;107(Suppl 6):907-938. |
304 |
Howarth RW |
Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139. |
230 |
Folmar LC |
Vitellogenin induction and reduced serum testosterone concentrations in feral male carp (Cyprinus carpio) captured near a major metropolitan sewage treatment plant. Environmental Health Perspectives 1996;104(10):1096-1101. |
170 |
Poff NL |
Functional-organization of stream fish assemblages in relation to hydrological variability. Ecology 1995;76(2):606-627. |
130 |
Boynton WR |
Inputs, transformations, and transport of nitrogen and phosphorus in Chesapeake Bay and selected tributaries. Estuaries 1995;18(1B):285-314. |
128 |
Ankley GT |
Technical basis and proposal for deriving sediment quality criteria for metals. Environmental Toxicology and Chemistry 1996;15(12):2056-2066. |
111 |
Erickson RJ |
The effects of water chemistry on the toxicity of copper to fathead minnows. Environmental Toxicology and Chemistry 1996;15(2):181-193. |
Table 7. Citations of Highly Cited Water Quality
Papers in the
Field of Engineering (top 1%)
No. of Cites |
First Author |
Paper |
57 |
Jaworski NA |
Atmospheric deposition of nitrogen oxides onto the landscape contributes to coastal eutrophication in the northeast United States. Environmental Science & Technology 1997;31(7):1995-2004. |
50 |
Ankley GT |
Effects of light-intensity on the phototoxicity of fluoranthene to a benthic macroinvertebrate. Environmental Science & Technology 1995;29(11):2828-2833. |
41 |
Montgomery DR |
Process domains and the river continuum. Journal of the American Water Resources Association 1999;35(2):397-410. |
Table 8. Citations of Very Highly Cited Water Quality Papers (top 0.1%)
Field |
No. of Cites |
First Author |
Paper |
Environment/Ecology |
792 |
Van den Berg M |
Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792. |
|
323 |
Daughton CG |
Pharmaceuticals and personal care products in the environment: agents of subtle change? Environmental Health Perspectives 1999;107(Suppl 6):907-938. |
|
304 |
Howarth RW |
Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139. |
Ratio of Actual Cites to Expected Citation Rates
The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present. Using the ESI average citation rates for papers published by field as the benchmark, in seven of the seven fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the EPA papers are more highly cited than the average papers in those fields (see Table 9).
Table 9. Ratio of Average Cites to Expected
Cites for
Water Quality Papers by Field
ESI Field |
Total Cites |
Expected Cite Rate |
Ratio |
Environment/Ecology |
6,512 |
2,989.26 |
2.18 |
Engineering |
324 |
82.84 |
3.91 |
Pharmacology & Toxicology |
298 |
189.02 |
1.58 |
Biology & Biochemistry |
234 |
160.67 |
1.46 |
Microbiology |
183 |
167.25 |
1.09 |
Plant & Animal Science |
72 |
44.03 |
1.64 |
Chemistry |
64 |
59.53 |
1.08 |
JCR Benchmarks
The Impact Factor is a well known metric in citation analysis. It is a measure of the frequency with which the average article in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal’s relative importance, especially when compared to others in the same field. The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years.
Table 10 indicates the number of water quality papers published in the top 10% of journals, based on the JCR Impact Factor. Thirty-nine (39) of 294 papers were published in the top 10% of journals, representing 13.3% of EPA’s water quality papers. This exceeds the expected number of 29 papers (10%) published in the top 10% of high impact journals.
Table 10. Water Quality Papers in Top 10% of Journals by JCR Impact Factor
EPA Water Quality Papers in that Journal |
Journal |
Impact Factor (IF) |
JCR IF Rank |
11 |
Environmental Science & Technology |
3.557 |
540 |
10 |
Limnology and Oceanography |
3.024 |
737 |
7 |
Environmental Health Perspectives |
3.929 |
439 |
6 |
Applied and Environmental Microbiology |
3.810 |
470 |
2 |
Ecological Applications |
3.287 |
623 |
1 |
Analytical Chemistry |
5.450 |
243 |
1 |
Ecology |
4.104 |
394 |
1 |
Toxicological Sciences |
3.391 |
591 |
Total = 39 |
|||
Immediacy Index
The journal Immediacy Index is a measure of how quickly the average article in a journal is cited. It indicates how often articles published in a journal are cited within the year they are published. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.
Table 11 indicates the number of EPA water quality papers published in the top 10% of journals, based on the JCR Immediacy Index. Twenty-nine (29) of the 294 papers appear in the top 10% of journals, representing 9.9% of EPA’s water quality papers. This is equivalent to the expected number of 29 papers (10%) published in the top 10% of high impact journals.
Table 11. Water Quality Papers in Top 10% of Journals by JCR Immediacy Index
EPA Water Quality Papers in that Journal |
Journal |
Immediacy Index (II) |
JCR II Rank |
11 |
Environmental Science & Technology |
0.623 |
617 |
7 |
Environmental Health Perspectives |
1.202 |
202 |
2 |
Ecological Applications |
0.747 |
466 |
2 |
Hydrobiologia |
0.681 |
532 |
1 |
Freshwater Biology |
0.664 |
558 |
1 |
Analytical Chemistry |
0.885 |
346 |
1 |
Ecology |
0.590 |
676 |
1 |
Ecotoxicology |
1.450 |
151 |
1 |
Climatic Change |
1.235 |
195 |
1 |
Ambio |
1.435 |
156 |
1 |
X-Ray Spectrometry |
0.580 |
685 |
Total = 29 |
|||
Hot Papers
ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers. There were no hot papers identified for the most recently completed 2-month period (i.e., September-October 2005), but there were two hot papers identified from previous periods.
Using the current hot paper thresholds established by ESI as a benchmark, seven of the water quality papers, representing 2.4% of the water quality publications, were identified in the field of Environment/Ecology. The hot papers are listed in Table 12.
Table 8. Hot Papers Identified Using Current ESI Thresholds
|
Field |
ESI Hot Papers Threshold |
No. of Cites in 2-Month Period |
Paper |
Environment/ Ecology |
7 |
20 cites in November-December 2000 |
Van den Berg M, et al. Toxic equivalency factors (TEFs) for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 1998;106(12):775-792. |
8 |
13 cites in December 1996 |
Berry WJ, et al. Predicting the toxicity of metal-spiked laboratory sediments using acid-volatile sulfide and interstitial water normalizations. Environmental Toxicology and Chemistry 1996;15(12):2067-2079. |
|
Environment/ Ecology |
8 |
11 cites in December 1996 |
Hansen DJ, et al. Chronic effect of cadmium in sediments on colonization by benthic marine organisms: an evaluation of the role of interstitial cadmium and acid-volatile sulfide in biological availability. Environmental Toxicology and Chemistry 1996;15(12):2126-2137. |
8 |
10 cites in December 1996 |
Hansen DJ, et al. Predicting the toxicity of metal-contaminated field sediments using interstitial concentration of metals and acid-volatile sulfide normalizations. Environmental Toxicology and Chemistry 1996;15(12):2080-2094. |
|
7 |
10 cites in November-December 1996 |
Pesch CE, et al. The role of acid volatile sulfide and interstitial water metal concentrations in determining bioavailability of cadmium and nickel from contaminated sediments to the marine polychaete Neanthes arenaceodentata. Environmental Toxicology and Chemistry 1995;14(1):129-141. |
|
8 |
9 cites in December 1996 |
Liber K, et al. Effects of acid-volatile sulfide on zinc bioavailability and toxicity to benthic macroinvertebrates: a spiked-sediment field experiment. Environmental Toxicology and Chemistry 1996;15(12):2113-2125. |
|
6 |
6 cites in July-August 1997 |
Howarth RW. Regional nitrogen budgets and riverine N&P fluxes for the drainages to the North Atlantic Ocean: natural and human influences. Biogeochemistry 1996;35(1):75-139. |
Author Self-Citation
Self-citations are journal article references to articles from that same author (i.e., the first author). Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the water quality papers. Of the 7,687 total cites, 219 are author self-cites—a 2.8% author self-citation rate. Garfield and Sher3 found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time. MacRoberts and MacRoberts4 claim that approximately 10% to 30% of all the citations listed fall into the category of author self-citation. Therefore, the 2.8% self-cite rate for the water quality papers is well below the range for author self-citation.
1 The number of publications analyzed does not include all of the papers published by EPA’s Water Quality Research Program from 1995 to 1999. It includes only those identified by the National Health and Environmental Effects Research Laboratory and the National Center for Environmental Research.
2 Thomson's Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,500 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching.
3 Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation 1963;18(July):195-201.
4 MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. Journal of the American Society of Information Science 1989;40(5):342-349.