Bibliometrics

Bibliometric Analysis for the U.S. Environmental Protection Agency/Office of Research and Developments Drinking Water Research Program
May 2007

This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of the U.S. Environmental Protection Agency’s (EPA) Drinking Water Research Program. For this analysis, 910 papers were reviewed, and they were published from 1996 to 2006. These publications were cited 13,632 times in the journals covered by Thomson’s Web of Science¹ and Scopus². Of these 910 publications, 783 (86.04%) have been cited at least once in a journal.

Searches of Thomson Scientific’s Web of Science and Elsevier’s Scopus were conducted to obtain times cited data for the drinking water journal publications. The analysis was completed using Thomson’s Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomson’s databases. For this analysis, the ESI highly cited papers thresholds as well as the hot papers thresholds were used to assess the influence and impact of the drinking water papers. JCR is a recognized authority for evaluating journals. It presents quantifiable statistical data that provide a systematic, objective way to evaluate the world’s leading journals and their impact and influence in the global research community. The two key measures used in this analysis to assess the journals in which the EPA drinking water papers are published are the Impact Factor and Immediacy Index. The Impact Factor is a measure of the frequency with which the “average article” in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal’s relative importance, especially when compared to other journals in the same field. The Immediacy Index is a measure of how quickly the “average article” in a journal is cited. This index indicates how often articles published in a journal are cited within the same year and it is useful in comparing how quickly journals are cited.

The report includes a summary of the results of the bibliometric analysis, an analysis of the 910 drinking water research papers analyzed by ESI field (e.g., chemistry, microbiology, pharmacology & toxicology), an analysis of the journals in which the drinking water papers were published, a table of the highly cited researchers in the Drinking Water Research Program, and a list of patents and patent applications that have resulted from the program.

Summary of Results

1. More than one-fifth of the drinking water publications are highly cited papers. 193 (21.21%) of the drinking water papers qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. This is 2.12 times the 10% of papers expected to be highly cited. 26 (2.86%) of the drinking water papers qualify as highly cited when using the ESI criteria for the top 1%, which is 2.86 times the number expected. 1 (0.11%) of these papers qualify as very highly cited when using the criteria for the top 0.1%, which is exactly the number anticipated. None of the papers actually meet the 0.01% threshold for the most highly cited papers, which is not surprising given that the number of papers expected to meet this threshold for this program is 0.09.
2. The drinking water papers are more highly cited than the average paper. Using the ESI average citation rates for papers published by field as the benchmark, in 14 of the 19 fields in which the 910 EPA drinking water papers were published, the ratio of actual to expected cites is greater than 1, indicating that the drinking water papers are more highly cited than the average papers in those fields. For all 19 fields combined, the ratio of total number of cites to the total number of expected cites (13,632 to 8,944.60) is 1.52, indicating that the drinking water papers are more highly cited than the average paper.
3. More than one-third of the drinking water papers are published in high impact journals. 411 of the 910 papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 45.16% of EPA’s drinking water papers. This number is 4.52 times higher than the expected 91 papers. 278 of the 910 papers appear in the top 10% of journals ranked by JCR Immediacy Index, representing 30.55% of EPA’s drinking water papers. This number is 3.06 times higher than the expected 91 papers.
4. Fourteen of the drinking water papers qualify as hot papers. Using the hot paper thresholds established by ESI as a benchmark, 14 hot papers, representing 1.54% of the drinking water papers, were identified in the analysis. Hot papers are papers that were highly cited shortly after they were published. The number of drinking water hot papers identified is 15.4 times higher than the expected 0.91 hot papers.
5. The authors of the drinking water papers cite themselves much less than the average author. 661 of the 13,632 cites are author self-cites. This 4.85% author self-citation rate is well below the accepted range of 10-30% author self-citation rate.
6. Twenty of the authors of the drinking water papers are included in ISIHighlyCited.com,which is a database of the world’s most influential researchers who have made key contributions to science and technology during the period from 1981 to 1999.
7. There was 1 patent issued to an investigator from 1996 to 2006 for research that was conducted under EPA’s Drinking Water Research Program. This patent was cited by another patent.

Highly Cited Drinking water Publications

All of the journals covered by ESI are assigned a field, and to compensate for varying citation rates across scientific fields, different thresholds are applied to each field. Thresholds are set to select highly cited papers to be listed in ESI. Different thresholds are set for both field and year of publication. Setting different thresholds for each year allows comparable representation for older and younger papers for each field.

The 910 drinking water research papers reviewed for this analysis were published in journals that were assigned to 19 of the 22 ESI fields. The distribution of the papers among these 19 fields and the number of citations by field are presented in Table 1.

Table 1. Drinking Water Papers by ESI Fields

There are 193 (21.21% of the papers analyzed) highly cited EPA drinking water papers in 15 of the 19 fields—Agricultural Sciences, Biology & Biochemistry, Chemistry, Clinical Medicine, Economics & Business, Engineering, Environment/Ecology, Geosciences, Immunology, Microbiology, Multidisciplinary, Neuroscience & Behavior, Pharmacology & Toxicology, Physics, and Social Sciences—when using the ESI criteria for the top 10% of papers. Table 2 shows the number of drinking water papers in those 15 fields that meet the top 10% threshold in ESI. Twenty-six (2.86%) of the papers analyzed qualify as highly cited when using the ESI criteria for the top 1% of papers. These papers cover 7 fields—Chemistry, Clinical Medicine, Engineering, Environment/Ecology, Microbiology, Multidisciplinary, and Pharmacology & Toxicology. Table 3 shows the 26 papers by field that meet the top 1% threshold in ESI. The citations for these 26 papers are provided in Tables 4 through 10. There was 1 (0.11%) very highly cited drinking water paper in the field of Multidisciplinary. The paper, which met the top 0.1% threshold in ESI, is listed in Table 11. None of the drinking water papers met the top 0.01% threshold in ESI, which is not surprising given that the expected number of papers that should meet this threshold for this analysis is 0.09.

Table 2. Number of Highly Cited Drinking Water Papers by Field (top 10%)

Table 3. Number of Highly Cited Drinking Water Papers by Field (top 1%)

Table 4. Highly Cited Drinking Water Papers in the Field of Chemistry (top 1%)

Table 5. Highly Cited Drinking Water Paper in the Field of Clinical Medicine (top 1%)

Table 6. Highly Cited Drinking Water Papers in the Field of Engineering (top 1%)

Table 7. Highly Cited Drinking Water Papers in the Field of Environment/Ecology (top 1%)

Table 8. Highly Cited Drinking Water Papers in the Field of Microbiology (top 1%)

Table 9. Highly Cited Drinking Water Papers in the Field of Multidisciplinary (top 1%)

Table 10. Highly Cited Drinking Water Papers in the Field of
Pharmacology & Toxicology (top 1%)

Table 11. Very Highly Cited Drinking Water Paper (top 0.1%)

Ratio of Actual Cites to Expected Citation Rates

The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present. Using the ESI average citation rates for papers published by field as the benchmark, in 14 of the 19 fields in which the EPA drinking water papers were published, the ratio of actual to expected cites is greater than 1, indicating that the drinking water papers are more highly cited than the average papers in those fields (see Table 12). For all 19 fields combined, the ratio of total number of cites to the total number of expected cites (13,632 to 8,944.60) is 1.52, indicating that the drinking water papers are more highly cited than the average paper.

Table 12. Ratio of Actual Cites to Expected Cites for Drinking Water Papers by Field

JCR Benchmarks

Impact Factor. The JCR Impact Factor is a well known metric in citation analysis. It is a measure of the frequency with which the “average article” in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal’s relative importance, especially when compared to others in the same field. The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years.

Table 13 indicates the number of drinking water papers published in the top 10% of journals, based on the JCR Impact Factor. Four hundred eleven (411) of 910 papers were published in the top 10% of journals, representing 45.16% of EPA’s drinking water papers. This indicates that nearly one-half of the drinking water papers are published in the highest quality journals as determined by the JCR Impact Factor, which is 4.52 times higher than the expected percentage.

Table 13. Drinking Water Papers in Top 10% of Journals by JCR Impact Factor

Immediacy Index. The JCR Immediacy Index is a measure of how quickly the average article in a journal is cited. It indicates how often articles published in a journal are cited within the year they are published. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Table 14 indicates the number of drinking water papers published in the top 10% of journals, based on the JCR Immediacy Index. Two hundred seventy-eight (278) of the 910 papers appear in the top 10% of journals, representing 30.55% of the drinking water papers. This indicates that nearly one-third of the drinking water papers are published in the highest quality journals as determined by the JCR Immediacy Index, which is 3.06 times higher than the expected percentage.

Table 14. Drinking Water in Top 10% of Journals by JCR Immediacy Index

Hot Papers

ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers. There were no hot papers identified for the current 2-month period (i.e., March-April 2007), but there were a number of hot papers identified from previous periods.

Using the hot paper thresholds established by ESI as a benchmark, 14 hot papers, representing 1.54% of the drinking water papers, were identified in four fields—Economics & Business, Environment/Ecology, Multidisciplinary, and Pharmacology & Toxicology. The number of drinking water hot papers is 15.4 times higher than expected. The hot papers are listed in Table 15.

Table 15. Hot Papers Identified Using ESI Thresholds

Field	ESI Hot Papers Threshold	No. of Cites in 2-Month Period	Paper
Economics & Business	3	3 cites in November 2002	Irwin EG, Bockstael NE. The problem of identifying land use spillovers: measuring the effects of open space on residential property values. American Journal of Agricultural Economics 2001;83(3):698-704.
	4	4 cites in November 2002	Irwin EG, Bockstael NE. Interacting agents, spatial externalities and the evolution of residential land use patterns. Journal of Economic Geography 2002;2(1):31-54.
Environment/ Ecology	7	10 cites in July-August 2004	Styblo M, et al. The role of biomethylation in toxicity and carcinogenicity of arsenic: a research update. Environmental Health Perspectives 2002;110(Suppl 5):767-771.
Environment/ Ecology	6	12 cites in December 2006-January 2007	Glassmeyer ST, et al. Transport of chemical and microbial compounds from known wastewater discharges: potential for use as indicators of human fecal contamination. Environmental Science & Technology 2005;39(14):5157-5169.
	3	3 cites in September 2006	Chiu WA. Issues in the pharmacokinetics of trichloroethylene and its metabolites. Environmental Health Perspectives 2006;114(9):1450-1456.
	3	4 cites in June 2006	Craun GE, Calderon RL. Observational epidemiologic studies of endemic waterborne risks: cohort, case-control, time-series, and ecologic studies. Journal of Water and Health 2006;4(Suppl 2):101-120.
	3	5 cites in June 2006	Colford Jr. JM, et al. A review of household drinking water intervention trials and an approach to the estimation of endemic waterborne gastroenteritis in the United States. Journal of Water and Health 2006;4(Suppl 2):71-88.
	3	5 cites in June 2006	Craun GF, et al. Assessing waterborne risks: an introduction. Journal of Water and Health 2006;4(Suppl 2):3-18.
	3	5 cites in June 2006	Calderon Rl, Craun GF. Estimates of endemic waterborne risks from community-intervention studies. Journal of Water and Health 2006;4(Suppl 2):89-100.
	3	6 cites in June 2006	Craun, et al. Waterborne outbreaks reported in the United States. Journal of Water and Health 2006;4(Suppl 2):19-30.
	3	3 cites in May-June 2006	Groffman P, et al. Ecological thresholds: the key to successful environmental management or an important concept with no practical application? Ecosystems 2006;9(1):1-13.
Multidisciplinary	9	9 cites in September-October 2004	Abrahamsen MS, et al. Complete genome sequence of the apicomplexan, Cryptosporidium parvum. Science 2004;304(5669):441-445.
Pharmacology & Toxicology	6	7 cites in November-December 2002	Thomas DJ, et al. The cellular metabolism and systemic toxicity of arsenic. Toxicology and Applied Pharmacology 2001;176(2):127-144.
	9	11 cites in August-September 2004	Nesnow S, et al. DNA damage induced by methylated trivalent arsenicals is mediated by reactive oxygen species. Chemical Research in Toxicology 2002;15(12):1627-1634.

Author Self-Citation

Self-citations are journal article references to articles from that same author (i.e., the first author). Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the drinking water papers. Of the 13,632 total cites, 661 are author self-cites—a 4.85% author self-citation rate. Garfield and Sher³ found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time. MacRoberts and MacRoberts⁴ claim that approximately 10% to 30% of all the citations listed fall into the category of author self-citation. Kovacic and Misak⁵ recently reported a 20% author self-citation rate for medical literature. Therefore, the 4.85% self-cite rate for the drinking water papers is well below the range for author self-citation.

Highly Cited Researchers

A search of Thomson’s ISIHighlyCited.com revealed that 20 (0.93%) of the 2,142 authors of the drinking water papers are highly cited researchers. ISIHighlyCited.com is a database of the world’s most influential researchers who have made key contributions to science and technology during the period from 1981 to 1999. The highly cited researchers identified during this analysis of the drinking water publications are presented in Table 16.

Table 16. Highly Cited Researchers Authoring Drinking Water Publications

Patents

There was 1 patent issued to an investigator during the period 1996 to 2006 for research that was conducted under EPA’s Drinking Water Research Program. This patent has been cited by one other patent. The patent and the patent that cites it are listed in Table 17.

Table 17. Patent from the Drinking Water Research Program (1996-2006)

Patent No.	Inventor(s)	Title	Patent Date	Patents that Referenced This Patent
6,365,048	Masten SJ Yavich AA	Method for treatment of organic matter contaminated drinking water	4/2/02	Referenced by 1 patent: (1) 6,893,559 System and method for removing organic compounds from waste water by oxidation

Top of page