Jump to main content.

Ecological Committee on FIFRA Risk Assessment Methods (ECOFRAM)
Terrestrial Work Group Report: I. Introduction and Problem Formulation

On this Page

Introduction

The Ecological Committee on FIFRA Risk Assessment Methods (ECOFRAM) was initiated in June 1997 in response to recommendations from the USEPA Science Advisory Panel (SAP). The ultimate goals are to develop and validate probabilistic risk assessment tools and address increasing levels of biological organization.

Top of page

Why Probabilistic Risk Assessment?

The SAP considered that the methodologies and endpoints used for risk assessment by the Office of Pesticide Programs (OPP) have several limitations. While they can serve as a screen to identify possible environmental damage, more information on the degree of uncertainty and the likelihood of damage would be desirable in balancing risks and benefits as required under FIFRA. Consequently, the SAP recommended that OPP develop the necessary databases and methodologies to conduct probabilistic assessments of risk.

Probabilistic Risk Assessments can be used to:

Dealing with uncertainty

Sources of uncertainty in risk assessment include:

By iteratively refining models to reflect new data and understanding of ecological relationships, we may achieve greater certainty that our predictions are a reasonable reflection of field responses. By using probabilistic methods to quantify natural variation, our predictions will reflect the tremendous variability in risks to individuals that exists in terrestrial systems.

Top of page

What is Probabilitic Risk Assessment?

Definition

ECOFRAM was charged with developing methods to predict both the magnitude and probability of effects. ECOFRAM uses the term "probabilistic risk assessment" to describe such methods, but they have also been described as risk assessment quantitative policy analysis, quantitative risk analysis, stochastic modeling, probabilistic analysis and Monte Carlo Analysis.

Basic Approach

The basis for probabilistic risk assessment is relatively simple:

  1. Identify the uncertain variables that most influence the effect of concern

  2. Define or estimate their distribution parameters (mean, variance) and any correlations with other uncertain variables

  3. Use the laws of mathematical statistics or Monte Carlo analysis to combine the uncertain variables and estimate the distribution of effects.

The Challenge

Commercial software packages have been developed that can perform probabilistic analyses with relative ease. However, applying these methods to ecological risk presents substantial difficulties:

A large proportion of the Terrestrial ECOFRAM report addresses limitations in the available data and suggests ways to estimate or collect additional data to reduce uncertainty.

There is a practical limit to how much uncertainty can be reduced by obtaining additional data. However, several techniques have been developed to incorporate or account for the absence of knowledge in assessing risk, e.g. the use of conservative assumption and/or subjective (expert) judgement. ECOFRAM proposes to employ these approaches in initial assessments and concentrate additional studies on quantifying uncertainty on the most important variables in more refined assessments (see below).

Top of page

Guiding Principles

The Terrestrial ECOFRAM approach takes into account the following 16 guiding principles for probabilistic type assessments, developed by the U.S. Environmental Protection Agency.

Selecting Input Data and Distributions

1. Use sensitivity analysis to identify the most important exposure pathways, model inputs and parameters.

2. Restrict the use of probabilistic assessment to significant pathways and parameters.

3. Use data to inform the choice of input distributions.

4. Use surrogate data for distributions when justified.

5. Use appropriate sampling methods to obtain empirical data and pay particular attention to the quality of information at the tails of the distribution.

6. Be very explicit about the use of expert judgment.

Evaluating Variability and Uncertainty

7. Variability and uncertainty should be distinguished. The decision about how to track them must be made separately for each variable.

8. There are methodological differences regarding how variability and uncertainty are addressed in a Monte Carlo analysis.

9. Investigate the numerical stability of the moments and tails of the distributions.

10. Identify areas of uncertainty and include them either quantitatively or qualitatively.

Presenting the Results of a Monte Carlo Analysis

11. Describe the model, its equations, limitations and results.

12. Provide detailed information on the input distributions.

13. Provide detailed information and graphs for each output distribution.

14. Discuss the influence of dependencies and correlations.

15. Calculate and present point estimates.

16. Tailor the presentation to the audience.

Top of page

Levels of Refinement - A Key Concept for Terrestrial Ecofram

The Terrestrial ECOFRAM identified a large number of variables which needed to be considered, and each of these could be treated at various levels of refinement. We decided to formalize the concept of Levels of Refinement (LoRs) as a means of organizing the enormous variety of tools available for probabilistic risk assessment.

The following diagram illustrates the continuum between the lowest and highest Levels of Refinement. Four levels are sufficient to describe the range of tools for all the parameters we have considered.

Levels of Refinement
Level 1Level 2 Level 3Level 4
  • Deterministic inputs
  • Deterministic outputs
  • Simple models
  • Conservative assumptions
  • Ignore minor pathways and effects
  • Use only standard studies
  • Use only existing field data
 double arrow
  • Probabilistic inputs
  • Probabilistic outputs
  • Complex models
  • Assumptions replaced by data
  • Include all significant pathways and effects
  • Include special studies where needed
  • Include focussed field studies where needed

The concept of Levels of Refinement is used throughout this presentation.

By including deterministic tools at the lowest level, we enable the assessor to restrict probabilistic methods to key pathways and parameters (USEPA Guiding Principle no. 2, above). At the other end of the continuum, Level 4 includes field studies for most parameters. Unlike field studies of the past, these are focussed on improving specific input parameters for the risk assessment.

The fact that tools for probabilistic assessment are organized into four Levels of Refinement does not imply that these levels have to be used as rigid "Tiers" for risk assessment, in the conventional sense. It may be more efficient to use tools from different levels for different parameters, according to need.

Top of page

Problem Formulation

Scope of Terrestrial ECOFRAM

As specified in the charge, the majority of the Terrestrial ECOFRAM Workgroup agreed to first address direct acute and chronic effects of pesticides to birds and mammals. The focus on direct toxicity does not imply this is more important than indirect ecological effects, since this was not substantially debated. However, assessments of direct toxicity drive the current pesticide registration process and are more tractable than addressing indirect effects. While several members felt that indirect effects are a significant issue, it was generally felt that the issue is too complex to adequately address within the timeframe of ECOFRAM. The focus on birds and mammals does not imply they are the most important taxonomic groups, but the larger databases of toxicity and life history information on these species makes them amenable for developing a new process for risk assessment. Several members felt that a complete probabilistic assessment would also consider other groups of non-target vertebrates and invertebrates in an assessment.

Assessment endpoints

After a great deal of discussion the Terrestrial ECOFRAM Workgroup identified the assessment endpoints listed below as important to address in evaluating the effects of pesticides to non-target species. And of these the Workgroup, due to time considerations, narrowed the endpoints to be considered as indicated.

*Primary endpoints to be considered by the current ECOFRAM initiative.

Definition of agco-ecosystem, spatial scale and focal species for assessment

A pesticide may be intended for use on several crops over a large geographical range, in a variety of conditions (e.g. different climate, soils, agricultural practices, wildlife communities etc.). It is impractical to assess risk separately for every combination of conditions. In practice the range of conditions in which the pesticide is used will fall into a number of broad types, for which separate assessments are appropriate. We refer to these as "agro-ecological scenarios" or "agro-ecosystems".

We envisage many aspects of the agro-ecosystem as being defined in similar terms throughout the assessment, i.e. at all Levels of Refinement. However, we propose that two aspects should differ between levels: the spatial scale of the assessment, and the definition of the species at risk.

Levels of Refinement
Spatial Scale and Species at Risk
Level 1Level 2 Level 3Level 4
AGRO-ECOSYSTEM

  • Agro-ecosystem broadly characterized in terms of crops, landscape type, soils, agricultural practices, wildlife communities etc.

  • Ideally need to develop a comprehensive set of standard scenarios for use by registrants and regulators alike, preferably coordinated with those used in aquatic risk assessment

SPATIAL SCALE

  • Assume animals live entirely within treated area - "the world is a field"

  • Represents a literal worst-case: pesticides which raise no concern need not proceed to higher levels

  • World conceptually divided into two habitat types: treated and untreated

  • Animals divide their time between treated and untreated areas

  • World conceptually divided into three habitat types: treated area, area receiving spray drift, and untreated area

  • Animals divide their time between treated area, untreated area and drift zone

 More complex and realistic spatial models. For example:

  • Abstract models based on matrices of square fields

  • GIS models of real or hypothetical landscapes

Development of these approaches likely to be led by registrants with specific needs

FOCAL SPECIES

  • Generic species, e.g. small granivorous bird, large grazing bird etc.

  • Need complete set of standard, generic species for use by registrants and regulators alike

  • Base assessment on appropriate "focal species". Define groups of species with similar ecology, and from these select the species with highest potential for exposure.

  • Conduct risk assessment for using specific data for focal species. If the assessment protects them, other species in the group should also be protected.

  • In selecting focal species, exploit the wealth of existing information on ecology, diet, foraging patterns.

Top of page

Publications | Glossary | A-Z Index | Jobs

Local Navigation

Jump to main content.