![[logo] US EPA](../gif/logo_epaseal.gif){kind=logo}
Comments from Aquatic Peer Input Panel
Barry, T.A.
Review of the Draft Report of the Ecological Committee on FIFRA Risk Assessment Methods (ECOFRAM)
Terrell Barry, Ph.D.
California Department of Pesticide Regulation
Environmental Monitoring and Pest Management Branch
June 4, 1999
On this Page
- Is the draft report scientifically sound?
- Did the ECOFRAM Workgroup address the "Charge to the Terrestrial and Aquatic Workgroups" identified in the background document, "evaluating Ecological Risk: Developing FIFRA Probabilistic Tools and Processes" (Attachment #3)?
- What are the limitations for predicting risk using the approach described in the draft report?
- Taking into account your answers to the three questions above, what areas of the report need to be strengthened?
- At what point in the risk assessment process is the certainty level high enough to support the consideration of risk mitigation?
What is the minimum level of technical information and scientific understanding that is necessary to evaluate whether risk mitigation would be necessary and/or effective? - Literature Cited
This is a review of the draft report of the Ecological Committee on FIFRA Risk Assessment Methods (ECOFRAM). The committee is to be commended for the comprehensive and thorough review of the literature related to the topic of ecological risk assessment. All Panel Members were asked to respond to the following five questions:
Is the draft report scientifically sound?
This report presents a comprehensive review and description of recent advances in the area of ecological risk assessment in general and more specifically as it pertains to pesticides and FIFRA guidelines. The basic concepts presented in this report are scientifically sound. The tier system for assessing pesticide risk is a logical and efficient method of prioritizing efforts. This document does a good job of laying out each tier for both exposure and effects characterization. Various appropriate quantitative tools are included and each of the quantitative tools included is well described. Difficult issues such as pulsed exposures and time to event analysis are addressed.
Chapter 2
Chapter 2 gives a clear overview of the tier system and prepares the reader for the more detailed presentation of methods for each tier in Chapters 3 and 4. This chapter also gives a clear description of the decision process that a registrant or Agency may go through during the registration process for a particular product. This description is useful to orient readers who may be familiar with the basic ecological risk assessment process but not with the registration process for a pesticide.
All the recommendations to improve risk characterization listed in Section 2.6.2. are important and should be supported and/or implemented within OPP. Recommendation 5), that EPA OPP EFED develop and maintain a web page carrying up to date details of recommended models, databases and approaches for handling exposure or toxicology data is crucial to the success of this new initiative.
Chapter 3
Section 3.5, The Conceptual Models of Key Components of the Agricultural Landscape is a good introduction to the key factors that potentially influence the transport and fate of pesticides to and in surface waters. The various summary tables in this section are excellent complements to the text of this section. However, the figures were difficult to evaluate because of poor copy quality.
Section 3.6.4, Recommendations for Reporting Environmental Fate Study Findings and Exposure Modeling Input Data is probably one of the most important recommendations in this report. Standardized reporting of results will facilitate development and maintenance of a data base of model input parameters such as the ARS Pesticide Properties database. The development and maintenance of such a database should be advocated by OPP. The example summary given in Appendix 3-8 appears to be a reasonable first step to standardizing the reporting format.
The Risk Assessment tool to evaluate Duration and Recovery (RADAR) is an extremely important tool. The recommendation of the ECOFRAM aquatic group that this tool be made available on an EPA sponsored web site should be implemented. Although it is certainly possible to perform similar analyses with other software, this tool will enable much easier in depth analysis of exposure data within a standardized format. As the use of this tool becomes more widespread, the types of analysis available and the output format will become more familiar. The standardized output formats will greatly facilitate communication of results. It should be noted that RADAR is an analysis tool and not a model, therefore there are no actual assumptions other than statistical assumptions that accompany the various analysis techniques.
The landscape level analysis is an extremely important component of the process because this type of analysis puts into proper perspective the likelihood that significant off-site movement of a pesticide into surface water will occur following an application of a pesticide under a certain application scenario. This likelihood is not only a function of the physico-chemical properties of the pesticide but also the landscape characteristics discussed in the example given in the report. Proper accounting of land use patterns in various watersheds and the quantification of the distribution of distances between treated fields and surface water bodies will allow a significant refinement to risk assessments. It is likely that over a period of time GIS databases of land use, cropping patterns and locations of waterways and sensitive sites will be developed, thus simplify a landscape level analysis for many new pesticides.
Chapter 4
Chapter 4 identifies some of the major contributors to uncertainty in quantifying ecological effects of exposure of natural systems to toxics or other stressors. In general, Chapter 4 is well organized, being a logical progression from introductory concepts to the series of white papers (Sections 4.3 through 4.9) that present techniques and tools having the highest potential to yield analysis results useful in addressing these contributors to uncertainty. However, this chapter mixes exposures characterization and risk characterization concepts. The clarity of the report would be improved if the risk characterization tools were presented in a chapter devoted to those tools only. For example, the Joint Probability Curve (JPC) is a powerful tool for integrating the exposure and effect data but the JPC is a risk characterization tool. Presentation of risk characterization results in this format should become a standard part of the risk characterization in any pesticide ecological risk assessment.
Section 4.3, Time-to-Event Analysis, is the first step away from the use of the standard 48-hour or 96-hr LC50 in the characterization of the effects of pesticides on aquatic communities. Through the more wide spread use and availability of this type of data, a better match between relevant exposure duration and the response can be achieved. The suggestion that Time-to-Event analysis be used for interpretation of standard toxicity test data would aid in this transition. In addtion, the suggestion sin Section 4.3.5 for modifying current test protocols should be considered.
Section 4.4, The Use of Population Models in Aquatic Effects Assessment, presents a review of various population models with the potential for use in pesticide ecological risk assessments. Many of the population models reviewed were developed for terrestrial vertebrate populations. It is not clear how applicable to aquatic systems, aquatic invertebrates in particular, theses population models would be. It is likely that, at least initially, the only population models that are applicable for many aquatic ecological risk assessments are the life table and logistic approaches. However, use of these types of models would allow refinement of the risk assessment.
The effects characterization concepts and tools presented in Section 4.5 are probably the easiest to implement at this time. Standard single species toxicity test results are used in this method to produce an easily understandable ordering and analysis of the full set of toxicity data available for the pesticide under evaluation. An attractive characteristic of this method is that it requires a minimal amount of data. However, if the minimum number of data points are used it may be difficult to obtain a reasonably well characterized percentile function. The issues of data selection brought up in Section 4.5.1.2 should also be carefully addressed. A standard set of rules should be developed so that each exposure profile is developed in a uniform manner. Addressing responses at concentrations above the solubility limit for the substance are especially problematic and should be given careful attention. The possible groupings for toxicity/sensitivity data sets shown in Figure 4-18 is logical. In addtion, existing exposure profiles employing this type of grouping scheme for various pesticides (atrazine, diazinon, and chlorpyrifos) are well characterized. Lognormal model is reasonable and fits many of the existing examples. However, other models could be employed in the characterization of the exposure profile if the lognormal model clearly did not provide an adequate fit.
Section 4.6 Effects of Time-Varying or Repeated Exposures. This section is extremely detailed and complex. It should be shortened to include only those tools that are presently well developed enough to be likely to be employed in a risk assessment. In addtion, concepts and tools that are more appropriately risk characterization should be moved in a chapter on risk characterization. Pulsed scenarios are the norm for many of the waterways of concern in California. Therefore, the recommendations listed in Section 4.6.10 concerning the characterization of time-varying exposure testing and modeling are important for addressing California exposure scenarios.
The ECOFRAM proposal in Section 4.8.3.2 for the use of regression techniques for the determination of an ECx value should be considered. As noted in the ECOFRAM report, there are a considerable number of advantages to regression analysis over the standard hypothesis testing and NOEC determination. Guidance should be developed to direct the most efficient experimental designs for chronic testing.
Did the ECOFRAM Workgroup address the "Charge to the Terrestrial and Aquatic Workgroups" identified in the background document, "evaluating Ecological Risk: Developing FIFRA Probabilistic Tools and Processes" (Attachment #3)?
According to the "Charge to the Terrestrial and Aquatic Workgroups" (Attachment #4)
"Workgroups are charged with developing a process and tools for predicting the magnitude and probabilities of adverse effects to non-target aquatic and terrestrial species resulting from the introduction of pesticides into their environment."
The charge had three requirements for candidate methods listed:
The methods should be standardized procedures integrating estimates of pesticide exposure with knowledge about the potential adverse effects
The majority of the methods can be considered standardized procedures. However, some of the methods described are clearly still in the research and development phase. Those methods still under development are for the most part noted as such in the report.
The methods should account for sources of uncertainty
Section 4.1. of the report provides a listing of uncertainties attributed to Suter and Barnthouse. Those sources of uncertainty are:
- Natural stochasticity
- Parameter error
- Model error
The methods presented and discussed in this report address characterization of natural stochasticity and reduction of parameter error. However, potential model error is not addressed to a large extent.
The methods should be developed within the context of the FIFRA regulatory perspectives and following the outline provided by the Framework for Ecological Risk Assessment (U.S.EPA, 1992).
The methods appear to fit within the context of the FIFRA regulatory perspective. However, there is not enough integration with the outline provided by the Framework for Ecological Risk Assessment. This report appears to assume a through familiarity and understanding of the organization and philosophy behind the Framework for Ecological Risk Assessment. In some areas the report is poorly organized because it is difficult to relate the report sections to the Framework.
What are the limitations for predicting risk using the approach described in the draft report?
The ECOFRAM Report states under Section 2.2.2 Rational for Probabilistic Risk Assessment, Topic Area #4: Guideline Field Testing and Research that there should be a reinstatement of field tests and that field studies need to be carried out in order to validate the assessment models that are used. In light of that those recommendations, the most significant limitation for reliably predicting risk using the approach described in this report is the heavy reliance on simulation modeling to characterize exposure. Although there are brief references to need for model validation, there is not adequate discussion about the reliability and uncertainty in the refined modeling results. Refined modeling, by design, is not as conservative as the Tier 1 calculations because theoretically this is more "realistic" modeling scenario based upon more refined knowledge of the input parameters. However, the value of those input parameters can vary by an order of magnitude or more (e.g. field dissipation). Choosing the appropriate value of highly variable input parameters is problematic. Therefore, Recommendations 1) and 2) in Section 3.1.2.1.3 should be implemented:
"EPA OPP EFED should work together with a group of academic and industry experts, to develop and agree a flexible guidance document based on consensus to specify an approach for calculating rate constants from FIFRA environmental study data in order to standardize comparative risk assessments and simplify Agency review of studies",
"EPA OPP EFED should provide clear guidance on how to select appropriate values for any model input parameters and also how best to express the variation around the single value initially selected."
It is usually assumed that even refined modeling results will be conservative relative to actual environmental concentrations (give predicted concentrations higher than those observed in the field) and there are indications that this is often the case. However, no specific evidence is presented that the models (both exposure and effects models) listed in this report are always conservative or that they reliably perform under all expected scenarios. It is not clear from the discussion in this report whether the recommended models have been adequately validated individually. In addition, there is considerably potential for propagation of errors in the simulation modeling results since the majority of this report discusses the linkage of several simulation models, both exposure and effects models, to obtain characterization of risk as a product at the end of the analysis. Two models validated independently can not be guaranteed to yield reliable results when they are linked.
These reservations about the reliance on simulation models are further strengthened by the OPP/EFED report entitled: "Proposed Methods for determining watershed-derived percent crop areas and considerations for applying crop area adjustments to surface water screening models. May 27, 1999" (Effland et al., 1999). The main concern listed in the overview of the documents is "...PRZM/EXAMS may not be realistically capturing basin-scale processes of all pesticides or for all uses." In California, the main focus of ecological risk assessments is on basin level processes. Furthermore, in California, most water bodies of concern are creeks, rivers, bays, and estuaries. The farm pond scenarios is not common in California. It is not apparent that the PRZM/EXAMS modeling tool will reliably estimate environmental concentrations at the basinscale. The OPP/EFED document lists the steps OPP believes need to be taken to evaluate the effectiveness PRZM/EXAMS as a basin-scale screening model:
conduct sensitivity analysis of PRZM, EXAMS and the linked models (italics added) to determine what input factors most influence the model results and identify potential conditions (site, chemical, weather) under which the models may not work as expected.
conduct a more thorough survey of modeling and monitoring comparisons for all pesticides in which such data is available.
Through a systematic comparison of modeling and monitoring data and the ancillary data, attempt to identify specific chemical or scenario characteristics that could lead to inconsistencies in the modeling results. This evaluation could then be used to determine whether, for certain pesticides or uses, corrections are needed or whether another form of screening is necessary.
In addtion, the Final Report from the December 10, 1997 SAP report states on page 8 "The need to validate EXAMS has so far been ignored by the FIFRA Exposure Model Validation Task Force, but this is probably the model most often used for risk assessment purposes. It is recommended that this effort should be rectified as soon as possible...." It is not apparent from the ECOFRAM document or from the web site for the FIFRA Exposure Model Validation Task Force (FEMVTF) project whether this validation is close to completion. The ECOFRAM recommendations related to the current Tier 2 process indicate that the validation of EXAMS code in terms of pesticide endpoints has not been completed (Section 3.1.2.2.4). The ECOFRAM recommendation that the MUSCRAT tool be refined and issued formally via an EPA sponsored web site should be predicated on the proper review and validation before this occurs. The important issue is whether these models are ready to use now for products not currently registered. If not, how long will it be before they are ready to use and what process will be used to perform the exposure assessments above Tier 1 in the interim while these model are validated?
For currently registered products undergoing re-evaluation it is likely that monitoring data exist. There is not enough emphasis on the use of surface water monitoring data as a complement to modeling results. The two pesticide related ecological risk assessments published in peer reviewed publications, for atrazine (Solomon et al., 1996) and chlorpyrifos (Giesy et al., 1999), used monitoring data for the final characterization of risk. Both of these risk assessments would be considered Tier 4 assessments and were conducted in response to re-evaluation needs. The atrazine ecological risk assessment found significant discrepancy between modeling and monitoring results that initiated improvements to the PRZM2 model, leading to PRZM3. The chlorpyrifos risk assessment included edge-of-field runoff simulation modeling for the Tier 2 and Tier 3 exposure assessment. However, the Tier 4 exposure assessment conducted at a watershed scale relied on an existing database of monitoring results because the text states "...there are no simulation models that can reliably estimate pesticide EECs in surface waters on a watershed scale." Various monitoring programs can be expected to be on-going efforts simply because, as discussed in Table 3-17, there tends to be a greater acceptance of measured data than modeled data. Therefore, it is well worth the time and effort to explore methods for extracting the maximum amount of information and benefit from those monitoring results rather than relegating them to a secondary role. It is also important to develop these methods in order to anticipate and respond pro-actively to newly emerging pesticide related surface water quality problems. It is true that for short-term monitoring programs, monitoring data may represent a small "window" across a few seasons as discussed in Chapter 2. However, that small "window" can be placed in historical context both through modeling environmental fate of the pesticide of interest but also by analyzing the weather data during that window relative to historical weather data.
Taking into account your answers to the three questions above, what areas of the report need to be strengthened?
Integration with the U.S. EPA Ecological Risk Assessment Guidelines
There should be greater integration with the U.S. EPA Ecological Risk Assessment Guidelines. (1998). The report seems too technical for that part of the intended audience without extensive background in U.S.EPA process and procedure related to ecological risk assessment. If the assumption is that any reader will have a working knowledge of the U.S. EPA Ecological Risk Assessment Guidelines, then that should be clearly stated in the introductory paragraphs of the report. The clarity and organization of the report will be greatly improved with the addition of the Executive Overview and Chapter 1. The intended audience can be clearly stated in either of those portions of the report.
Addition of a separate chapter on risk characterization
This report currently has three main sections:
- introduction (chapter 2),
- exposure characterization (chapter 3),
- and effects characterization (chapter 4).
Chapters 2 and 3 are actually the two subsections of the analysis phase of an ecological risk assessment. The report should have a separate chapter on techniques for the risk characterization phase of an ecological risk. This separate chapter on risk characterization should integrate the concepts and techniques included in chapters 2 and 3 and illustrate general methods to obtain a probabilistic risk characterization. This suggestion is related to the comment that this report should be better integrated with the U.S. EPA Guidelines on Ecological Risk Assessment (1998). This ECOFRAM report should have a chapter that corresponds to each major section in an ecological risk assessment according to the Guidelines (problem formulation, analysis and risk characterization) and then discuss the concepts and techniques at each phase that are most relevant to a pesticides oriented risk assessment. In that case, the exposure and effects characterizations would be two subsections of the analysis phase chapter. A separate, chapter discussing risk characterization concepts and techniques for integrating the methods and results from chapters 2 and 3 would complete and strengthen the report. This organization format would give reader a basic, standard outline that fits into the familiar U.S. EPA guidelines format. Risk characterization issues are presently mentioned in each chapter and some of these techniques are presented in detail. However, by including these techniques throughout the report the concepts get confused. Chapter 4 tends to interchange effects characterization and risk characterization throughout the chapter. For example, the Joint Probability Curve concept more appropriately belongs in an integration chapter on risk characterization rather than Chapter 4. And, Section 4.1 is an excellent presentation of the uncertainty issues in ecological risk assessments in general. The general uncertainty issues and concepts discussed also apply to the exposure assessment and ultimately affect the risk characterization. Therefore, this discussion should probably be included in a chapter on risk characterization.
Expansion of the discussion on coordination with other programs and regulatory mandates
This report discusses in several sections coordination with other OPP projects (e.g. FEMVTF, FIFRA EMWG, FQPA) to avoid duplication of efforts with regard to the exposure assessments. Although brief discussion us devoted to the need to coordinate with other regulatory programs such as Office of Water (Clean Water Act mandates), this discussion should be expanded. This coordination is especially important when considering the issue mentioned in Chapter 3 in Section 3.5.1 where the text states, "A major issue that the ECOFRAM process has not been able to address is how regulators, the regulated community and society at large can better understand which water bodies need to be protected and to what degree." Coordination of overlapping regulatory mandates can be anticipated to become increasingly important.
At what point in the risk assessment process is the certainty level high enough to support the consideration of risk mitigation?
What is the minimum level of technical information and scientific understanding that is necessary to evaluate whether risk mitigation would be necessary and/or effective?Initial mitigation measures could be proposed at Tier 2 using the Joint Probability Curves developed during that tier. In some cases, proposing mitigation measures at the conclusion of the Tier 2 analysis may be more cost effective that progressing to a Tier 3 or Tier 4 analysis. This view is consistent with the ECOFRAM report Section 2.3.6.5. The minimum level of technical information and scientific understanding is a well characterized response profile coupled with a conservative exposure scenario.
Literature Cited
Effland, W.R., N.C. Thurman, and I. Kennedy. 1999. Proposed methods for determining watershed derived percent crop areas and considerations for applying crop area adjustments to surface water screening models. U.S. EPA Office of Pesticide Programs, Environmental Fate and Effects Division. Presentation to FIFRA Science Advisory Panel, May 27, 1999.
Giesy, J.P. K.R. Solomon, J.R. Coates, K.R. Dixon, J.M. Giddings, and E.E. Kenaga. 1999. Chlorpyrifos: Ecological risk assessment in North American aquatic Environments. Rev. Environ. Contam. Toxicol. 160:1-129.
Solomon, K.R., D.B. Baker, P. Richards, K.R. Dixon, S.J. Klaine, T.W. LaPoint, R.J. Kendall, J.M. Giddings, J.P. Giesy, L.W. Hall Jr., C.P. Weisskopf, M. Williams. 1996. Ecological risk assessment of atrazine in North American surface waters. E.Tox.Chem. 15:31-76.
U.S. EPA (Environmental Protection Agency). 1992. Framework for ecological risk assessment. EPA/600/R-92/001. Risk Assessment Forum, Office of Research and Development, U.S. Environmental Protection Agency, Washington, D.C.
U.S. EPA (Environmental Protection Agency). 1998. Guidelines for ecological risk assessment. EPA/630/OR-95/002F. Risk Assessment Forum, U.S. Environmental Protection Agency, Washington, D.C.