Note: This information is provided for reference purposes only. Although the information provided here was accurate and current when first created, it is now outdated.

TRAC 5/28-29/98

Staff Paper # 3.1

TOPIC: MONTE-CARLO TECHNIQUES AND THE 99.9TH PERCENTILE

BACKGROUND:

Introduction

Until recently, the Office of Pesticide Programs (OPP) has been limited to conducting its acute risk assessments with its Dietary Risk Evaluation System (DRES) software. Assessments conducted with DRES assumed that all crops with registered uses of a pesticide were treated with that pesticide and that all such treated crop items contained pesticide residues at the tolerance level. The resulting DRES acute risk estimates could be considered "high-end" or in some cases "bounding" estimates.⁽¹⁾ However, it was not known where our "high-end" pesticide exposure estimates from the DRES software fit in the overall distribution of exposures. Thus risk management decisions were being made by the Office not only without a full picture of the distribution of risk among the population, but also without full knowledge of where in the distribution of risk the DRES risk number lay.

A Monte-Carlo assessment, on the other hand, provides a probabilistic or statistsical assessment of dietary risk, using more refined information than is used in the DRES calculation. A Monte-Carlo analysis uses the actual distribution of pesticide residue level results from the experimental field trials performed by the registrant and incorporates the percentage of the crop which is treated. With the advent of Monte-Carlo and probabilistic techniques, it becomes possible to estimate more accurately the complete distribution of exposures to the entire population.

Using Monte-Carlo, OPP is no longer required to assume that all registered crops are treated with the pesticide of interest or that all residues are present in crops at tolerance levels. Rather than the crude "high end,"single- point estimates provided by DRES, Monte-Carlo provides better, more accurate information on the range and probability of possible exposure and their associated risk values.

For example, a given food item (e.g., apples) can have several dozen or more individual residue values generated from experimental field trials for a certain pesticide.

• In an acute DRES analysis, only the highest residue value (or tolerance level) would be used and all registered crops would be assumed to be treated and contain these high residue values.

• In a Monte-Carlo run, the entire set of actual residue data points generated by the registrant in the crop field trials and the percent of the crop which was treated would be considered.

The differences between the risk numbers generated by these two techniques can be substantial, with the Monte-Carlo generated values (at the 99.9^th percentile) frequently many times lower than DRES-generated values (at the 95^th percentile). The advantage of this probabilistic technique is that it can enhance the risk estimates by more fully taking advantage of all available information and minimizing reliance on values chosen more for their regulatory and administrative convenience than their scientific merit.

The DRES model does have some advantages over Monte-Carlo. First, because the mathematics of running DRES are simpler than those of a Monte-Carlo analysis, DRES is faster and cheaper to run. A second advantage is that a DRES run is the "sum of its parts". That is, with a DRES run all the individual risks from each individual crop for which a given pesticide is registered can be added up to obtain the total risk for that pesticide. This is not the case with a Monte-Carlo analysis. A Monte-Carlo run requires that the computer run be redone to represent an addition or deletion of a crop: unlike DRES with which one can simply "subtract out" any individual risk component attributable to a specific crop, there is no single component of a Monte-Carlo analysis that can be ascribed to a particular crop or commodity. Thus, a change in input parameters will require an entirely new (sometimes lengthy) computer simulation. EPA does not currently require a "subtraction" analysis for Monte-Carlo submissions. To do so would improve our ability to look at individual food item contributions to exposure but would increase the cost greatly.

More on Monte-Carlo

The output of a Monte-Carlo analysis can be thought of as a series of numbers representing the classic bell curve distribution where the given risk (or exposure) is on the horizontal axis and the associated probability is on the horizontal axis. The total area under the curve represents the entire population of the United States. As shown above, as one moves from the left-hand side of the distribution to the right-hand side of the distribution along the horizontal axis to the tails (extremes) of the distribution, an ever higher percentage of people fall under the curve.

Where Do We Regulate?

One of the central issues now before the Agency is where on the curve to regulate. The consequences of a lower regulatory threshold (e.g., regulating at the 99th percentile or 95th percentile as opposed to the 99.9th percentile) is that a greater proportion of the population will be exposed to pesticides at levels which exceed the regulatory threshold (e.g., a Margin of Exposure of 100). The number of such excursions in any year can be calculated if one assumes that they are independent events and that they can occur on any of the 365 days of each calendar year. The number of exceedances in any given year varies dramatically depending upon the regulatory percentile established.

The information in this table translates into a probability of 50% that any individual will not experience any exceedance of the regulatory threshold over a period of:

• about 2 years for the 99.9^th percentile;
• about 2 months for the 99^th percentile; and
• about 2 weeks for the 95^th percentile.

And, conversely, a probability of 50% that an individual will experience at least one exceedance over those time periods.

It is important to note that an exceedance of the regulatory threshold does not translate into a potential health effect -- the regulatory threshold is generally at least two orders of magnitude (100 times) below a dose at which no effect occurred in animal studies. Thus, the number of individuals at or above a certain percentile is that number at or above a specified exposure (which itself is well below any levels known to cause harm), not the number of individuals who experience a health effect.

Office's Interim 99.9th Percentile Proposal

The Office has proposed on an interim basis that we regulate at the 99.9th percentile of exposure when probabilistic techniques are used, citing consistency with Agency Guidelines for Exposure Assessment. While these guidelines recommend caution in extrapolating beyond the 99th percentile of exposure, they indicate that for populations greater than 100, more stringent regulation may be warranted.

The SAP considered this issue of where it might be appropriate to regulate in March 1998 and expressed divergent views on whether setting a criteria at the 99.9th percentile is an (adequately) conservative approach. They noted that if the 99.9th percentile is utilized, a percentage of the population (e.g., 23,000 children per day) would still be exposed to levels that exceed the regulatory threshold. The following additional remarks were made by the panel:

• To judge whether any given percentile criterion is conservative for acute affects or not, it would be necessary to consider the margin of safety which is already incorporated into the toxicological portion of the risk evaluation.

• They believed that to identify the level of risk, variability not only in exposure levels but also in human thresholds for the toxic effects under consideration would be needed. That is, a probabilistic "toxicity" component of a risk assessment should be incorporated into the analysis as well.

• The panel pointed out that by recognizing and separately modeling subpopulations, it may be possible to choose a lower, less statistically tenuous percentile at which to make regulatory decisions. This lower percentile may also be warranted, they indicate, if the risk assessment contains a number of "conservative" assumptions that might result in overestimates of risk even at the 99.9th percentile.

Concerns About Regulating at the 99.9th Percentile

A number of concerns have been raised by some about EPA's proposal to regulate at the 99.9th percentile. These include concerns about moving from the DRES 95^th percentile to a Monte-Carlo 99.9th percentile. Additional questions have been raised about the quality of the food consumption data at these extreme tails of the distribution. These two issues are addressed, in turn, below.

DRES 95^th percentile vs. Monte-Carlo 99.9th percentile

The Agency has in the past regulated at the 95^th percentile of an acute DRES analysis and concerns have been raised about what is seen by some as a significant "raising of the bar" by now choosing to regulate at the 99.9th percentile from a Monte-Carlo analysis. While it may appear at first that the Agency has increased its threshold of regulation, this is not so. Exposure at the 99.9th percentile (as calculated by Monte-Carlo) is lower than exposure calculated by DRES at the 95^th percentile for most cases reviewed by OPP to date. This is shown on the following page for one widely used agrichemical which has recently been evaluated:

Comparison of 95th Percentile DRES Exposure and Margin of Safety Estimates with Monte-Carlo 99.9th Percentile Exposure and Margin of Safety Estimates
Population Subgroup	Exposure (mg/kg bw/day)		Margin of Safetya
	DRES 95th Percentile Estimate	Monte-Carlo 99.9th Percentile Estimate	DRES 95th Percentile Estimate	Monte-Carlo 99.9th Percentile Estimate
U.S. Population	0.005	0.000542	10	92
Infants	0.008	0.000804	6.2	61
Children 1-6	0.008	0.000905	6.2	55
Females 13+	0.0036	0.000468	14	107
Males 13+	0.0038	--b	13	--b
aThe Margin of Safety is equivalent to the Margin of Exposure (MOE) and is calculated by dividing the NOEL (no observed effect level) generated from animal studies by the estimated exposure. It represents the degree to which (or factor by which) the estimated exposure is lower than the dose at which no effects were seen in animal studies. b Not calculated

As can be seen, exposures are significantly lower (and corresponding MOEs or safety margins are significantly greater) at the 99.9th percentile Monte-Carlo analysis than they are at the 95^th percentile DRES analysis. This is almost invariably the case. In fact, at all comparable percentiles the exposure estimates derived from Monte Carlo are lower than the corresponding DRES estimates. There are several reasons for this. DRES assumes:

• tolerance level residue levels in all crops that have registered uses and

• that all the crop is treated at the maximum (label) application rate and is harvested at the minimum (label) pre-harvest interval (i.e., the time period between last application and harvest).

In general, Monte-Carlo techniques will provide lower (but more accurate) estimates of exposure than previous DRES techniques when

1) a lower percentage of the crop is treated (e.g., 10% rather than 100%);

2) when a greater number of crops are registered (e.g., 10 crops instead of 2 crops); and

3) when the bulk of residue values from crop field trials are present at low levels and there are only a few high values.

In short, Monte-Carlo analysis tends to provide a lower (but more reliable ) estimate of actual exposure in exactly those situations where DRES is least realistic. We propose to continue to regulate at the DRES 95^th percentile when actual tolerance levels are used during risk assessment, but recognize that this can severely overestimate actual exposure levels. In those cases where DRES exposure estimates are greater than the regulatory threshold, we propose to use Monte-Carlo techniques to assess exposure at the 99.9th percentile using more refined data. In practice, risk assessments done at the 99.9th percentile using more refined data almost invariably result in lower estimated exposures (and risk) than risk assessment performed at the 95^th percentile using DRES and less refined data.

Food Survey Data Quality

A concern has arisen about regulating at the 99.9th percentile of exposure given perceived inadequacies in the quality of the food survey data⁽²⁾. The food survey data used in our risk assessments are collected by the U.S. Department of Agriculture. The USDA has been conducting such food surveys since the 1930's by means of personal interviews in which survey interviewers ask individuals to recall everything they ate and drank over the previous 24 hours. Intake amounts are reported as well as energy intakes. Usually more than one day's intake is obtained on non-consecutive days. When such data are collected for large numbers of survey participants who have been scientifically selected so that results could be projected from the sample to the U.S. population, this is identified as a nationally representative sample. Very high food or energy intakes may not be common in these nationally representative samples, but they do occur.

USDA survey interviewers are trained to probe for additional information when unusual intakes of various kinds are reported, and to ask questions clarifying large reported amounts, and also if the day's intake was typical or not. If not typical, queries are made about what was atypical, such as the occurrence of a holiday, a social occasion or the like. Some high intakes by young males were noted to be nearly entirely alcoholic beverage intake over a holiday period; other high intakes reflect the ravenous appetites of some adolescent males. On preliminary review of survey data, USDA staff identify high intakes in various areas (i.e., high consumption of certain foods or high energy intake) and evaluate the reported intake for feasibility, including use of notations made by the survey interviewer which may relate to the perceived validity of the reported consumption. In sum, a variety of data checks and validations occur in the data collection and analysis procedures to ensure that the reported intakes are as accurate as possible. All reported high intake values (or "outliers") have been checked by USDA staff and resolution or adjudication of values outside specified ranges has been accomplished. Given that very high energy intakes do occur in the American population (even though they are not common) and that USDA has chosen to retain those individual consumption figures in its database, we in the Agency believe that they should, by no means, be disregarded or automatically be declared in error.

NEXT STEPS

OPP is interested in developing more extensive, comprehensive, and consistent criteria and guidelines for risk managers to use in evaluating quantitative estimates of risk and in making the consequent regulatory decisions which adequately and appropriately consider the full distribution of risk. Paramount to this endeavor is a decision as how much risk is acceptable. We note that Monte-Carlo techniques are, in and of themselves, neither more conservative nor less conservative than the DRES system they supplement: The "conservatism" is determined by the risk manager when he or she establishes the required MOE (e.g., 100) and its associated required percentile (e.g., 99.9th percentile). Monte-Carlo and probabilistic techniques are simply a tool that allows the risk manager to see a more accurate distribution of risks among the population. With this new capability, it now becomes possible to decide what level of protection we wish to provide at various exposure levels⁽³⁾. In short, the advent of Monte-Carlo permits us to decide in a quantitative sense "how safe is safe enough?". The question, and decision, is a policy issue, not a science issue, and thus can only be aided by science, not answered by it.

QUESTIONS THAT THE TRAC MIGHT ADDRESS:

1. Regarding the Monte-Carlo analysis, at what percentile is it appropriate for the Agency to regulate? 99.9%, 99%, 95%, 99.9999%?

2. Instead of regulation at one percentile for all toxicological effects (regardless of severity), should the Office regulate at a variety of percentiles, depending upon the toxic effect observed? For example, a lower threshold of regulation (perhaps 98th percentile) might be warranted for fully-reversible effects (such as mild anemia)while a more stringent threshold (perhaps 99.9th percentile or higher) might be justified for severe, non-reversible effects (e.g., birth defects).

3. Given that a Monte-Carlo analysis is not a 'sum of the parts' and therefore risks posed by each crop cannot be broken out, should Monte-Carlo runs routinely be done for different permutations of crops.

1. A "high end" estimate is one that is, conceptually, above the 90th percentile of the actual distribution but not greater than exposure to the person in the population who has the highest exposure. It is a plausible estimate of the individual risk for those persons at the upper end of the risk distribution. A "bounding estimate", on the other hand, purposely overestimates the exposure or dose in an actual population for the purpose of developing a statement that the risk is "not greater than..." (U.S. EPA, 1992. "Guidance on Risk Characterization for Risk Managers and Risk Assessors").

2. It is important to note that we are regulating at the 99.9th percentile of exposure not the 99.9th percentile of food consumption. The difference is significant: the 99.9th percentile of exposure represents the joining of the entire consumption data set from USDA and the entire residue data set from EPA. It represents neither the 99.9th percentile of consumption nor the 99.9th percentile of exposure, but rather some lower percentile of each which, when multiplied together represents, the 99.9th percentile of exposure (the joint distribution).

3. For example, we could decide that an MOE of 100 (the level of protection) is required at the 99th percentile exposure level.

Next Level Up | OPP Home | EPA Home | Comments
Site Map | Search OPP | Search EPA

www.epa.gov/oppfead1/trac/trac3.htm
updated May 22, 1998