Comments from Terrestrial Input Panel
Dwayne Moore

This section provides one of many typologies of sources of uncertainty in ecological risk assessment. This typology (and most others) is somewhat limited in that it does not acknowledge that probability can be thought of in several ways, and uncertainties are not always probabilistic in nature (uncertainty.probability). Classical probability is based on the notion that past frequencies make good predictors. Relative frequencies are a way of characterizing uncertainties. It is not the only way, however, and ultimately may prove to be a rather impractical approach to characterizing uncertainties in ecological risk assessments where data are lacking on past performance of environmental parameters.

Another approach for characterizing uncertainties is to reason back from observed effects to cause, or more precisely, from evidence to hypothesis. Consider the example of an urn filled with black and white balls. A classical or frequentist approach would ask what is the chance that the next ball will be black, and base the answer on the number of black and white balls drawn previously from the urn. Bayesian or subjective probability would ask what is the ratio of black and white balls in the urn. Baye's theorum is essentially a formula that allows probabilities to be updated as new evidence becomes available. In the complete absence of information, Laplace's principle of indifference applies - all alternatives are equally likely. That is, prior to extracting any balls from the urn, we start with the belief that the ratio of black and white balls in the urn is 1:1. This non-informative prior is then updated as balls are extracted. Eventually, the updated probability (the posterior) converges on the true ratio of black to white balls in the urn. Bayesian probabilities do not depend on a track record of past performance. As such, they are subjective and can be thought of as degrees of belief, rather than relative frequencies. Intuitively, this would seem to be a more useful paradigm for ecological risk assessments in which serious data gaps exist, but for which we may have much expert (i.e., subjective) knowledge to draw upon.

Most Bayesians would argue that subjective probability is the only way to handle uncertainties in ecological risk assessments involving multiple stressors. There are, however, several types of uncertainties for which probability may be ill suited to describe. Consider the ambiguities inherent in our use of words to classify objects or describe systems. Suppose, for example, that one of the goals of the U.S. EPA is to protect sustainability of wildlife populations. The task for the risk assessors and managers would be to determine the probabilities of reductions in sustainability occurring given projections about pesticide usage and consequent effects in the region. Probability, however, does not capture the uncertainty that exists because the term sustainability lacks a precise definition. It is hard to estimate the probability of an event when the event covers a range of possibilities, some closer to the implied definition than others. Klir and Folger (1988. Fuzzy Sets, Uncertainty, and Information. Prentice-Hall, Englewood Cliffs, New Jersey) refer to this non-probabilistic type of uncertainty as fuzziness or vagueness. Another non-probabilistic source of uncertainty, nonspecificity or ambiguity, arises when the evidence is nonspecific and therefore uninformative as to the cause of the observed adverse effects. If, for example, we had evidence that songbird populations were declining in an area, we would not be able to determine whether this decline was due to pesticide use, habitat fragmentation, climate change or some other factor or combination of factors. In Klir and Folger's typology, the other non-probabilistic source of uncertainty (i.e., confusion) arises when the very meaning of the evidence is unclear. In the end, probability only deals with dissonance (i.e., pure conflict). A pesticide is or is not a carcinogen, or a pesticide may or may not kill a particular individual. Typically, some evidence supports one hypothesis, different evidence the other, and we are uncertain between the two. When the evidence is combined, probabilities may be assigned to each hypothesis. Thus, a bioassay may indicate that at a particular pesticide concentration, there is a 30% probability that a particular individual will die and a 70% probability that it will not die. With dissonance, a hypothesis cannot be both true and false at the same time, an outcome of Aristotle's law of the excluded middle.

The bottom line for this chapter is that I think it would be useful to educate readers as to the different schools of thought on probability, and recognize that non-probabilistic sources of uncertainty exist. Monte Carlo analysis (or Baye's theorum) cannot deal with these latter sources of uncertainty, but other methods exist that can (e.g., fuzzy arithmetic).

In the absence of information, the text recommends that conservative models or assumptions be used to "represent reasonable worst case scenarios". This approach is fine for early tier assessments that are designed to screen out negligible risk scenarios (the conservative quotient approach). I do not feel it is an appropriate recommendation in probabilistic risk assessment, however, because the resulting risk estimates will be biased high. Worse, we will have no idea how biased the risk estimates are. This is often the criticism leveled at conservative quotients (e.g., Cullen, A.C. 1994. Risk Anal. 14: 389-393; Bogen, K.T. 1994. Risk Anal. 379-381). Extending an analogy by Reckhow (1994. Ecol. Model. 72: 1-20), a forecast of "it will very likely rain" when rain is highly unlikely is not helpful; rather, we would like to know the true odds, and act according to our attitude toward risk. Thus, if the potential for severe effects exists, but uncertainty is high, risk managers may appropriately invoke conservatism by choosing to hedge their decisions away from the potentially serious effects by, for example, banning a pesticide use. By introducing conservatism into an assessment, assessors are in effect making policy decisions, decisions that should be made by risk managers and interested parties after being presented with an unbiased assessment. Use of expert judgment to estimate uncertainties about model structure or characterize input distributions is one way of dealing with some of the subjective uncertainties mentioned in this section. Meyer and Booker (1991. Eliciting and Analyzing Expert Judgment: A Practical Guide. Academic Press, NY) provides guidance on how to elicit information from experts.

The Lee and Wright reference should be 1994, not 1964.

The discussion on PDFs and CDFs (section 1.9.1) will lose most readers. Given that the target audience for this document includes many assessors and managers not familiar with uncertainty analysis, I suggest that a more userfriendly description of normal and lognormal distributions be used here. Readers do not need to know the equations, just the concepts and assumptions behind the distributions. Figures could be used to show distribution shapes for both PDFs and CDFs.

Section 1.9.1 presents a lot of detail on normal and lognormal distributions, and no information on other distributions likely to be encountered in ecological risk assessments (e.g., beta, logistic, Weibull, chi square, Poisson, binomial). The concept of probability mass functions should also be introduced here.

The section on computerized exposure models is generally well written, except that there is little mention of whether the models (PRZM3, EXAMS, etc) have undergone verification and validation studies. It would be very useful to know the predictive capabilities of these models. Also, in this section it would be helpful to describe the different kinds of statistics that can be used to measure model performance. Potential techniques include lumped measures of average model goodness-of-fit, correlation measures, parametric and non-parametric statistical tests, spatial analysis of goodness-of-fit (e.g., kriging), and Bayesian measures of estimation error. Errors due to model structure can be included in uncertainty analyses (a topic not discussed in this chapter).

In several places in this section, there is a call to develop a "comprehensive terrestrial exposure model". Presumably, such a model would be massive and have perhaps 100s of input variables. Before embarking on such a task, it would be useful to read Reckhow's (1994. Ecol Model. 72: 1-20) article cautioning against the use of large, mechanistic fate and transport models (e.g., WASP4 and EXAMS) in regulatory decision making because of their poor track record in producing reasonably accurate predictions. In an excellent review paper, Beck (1987. Water Resour. Res. 23: 1393-1442) concluded that "most of the evidence suggests that the current models of water quality, in particular, the larger models, are easily capable of generating predictions to which little confidence would be attached". I doubt the situation is any better with terrestrial models.

Seiler and Alvarez (1995) were highly critical of the use of triangular and uniform distributions in ecological risk assessment because the distributions simply do not describe environmental phenomena. The explanation on this page that their criticism was "due in part to discontinuities in those distributions" is obtuse and misses the point for most readers.

The use of expert judgment to derive input distributions when data are limited should be discussed in this section.

Section 4.3.1 notes the importance of considering indirect effects and the current limitations of the pesticide registration process for addressing such effects, but stops short of making any recommendations for improving the situation. What should be done in the future to give assessors the capability to address indirect effects?

I have no idea what the first sentence in section 4.3.2 means.

Mineau et al. (1996) found that the mean allometric scaling factor for chemical toxicity to birds was 1.15 and ranged as high as 1.55. Sample and Opresko (1996. Toxicological benchmarks for wildlife: 1996 revision. ES/ER/TM-86/R3. Oak Ridge National Laboratory, Oak Ridge, TN, USA), however, state that an allometric scaling factor of one is still appropriate for birds. This is because, for the majority of chemicals in the Mineau et al., study factors were not significantly different from one. As a result,

ED_p^w = ED_p^t [(bw^t) / (bw^w)]⁰      (1)

where ED is the effects dose, p is proportion affected, w is avian wildlife species, t is avian test species, and bw is body weight. Therefore, they argue that toxicity test results normalized to body weight (i.e., µg/kg body weight/day) can be used to estimate effects doses for other avian wildlife species without use of an allometric scaling factor.