![[logo] US EPA](../gif/logo_epaseal.gif){kind=logo}
Limitations in the 1996 National-Scale Air Toxics Assessment: Uncertainty
Note: EPA no longer updates this information, but it may be useful as a reference or resource.What are the Components of Uncertainty
The uncertainty analysis performed for the national-scale assessment helps in understanding the confidence with which specific statements (such as the lifetime excess risk of cancer due to inhalation of benzene in North Carolina) can be made based on the information available for the assessment. No scientific statement (in risk assessment or other areas of science) can be made with complete confidence. Uncertainties always exist in these statements due to issues that will be discussed below. Transparency and openness, therefore, require an assessment of these uncertainties and the ways in which they raise or lower confidence. The national-scale assessment produced statements about variability in ambient air concentrations, exposures and risks across geographic regions for typical individuals, as described in the section on Limitations, and so the uncertainty analysis was designed to characterize the confidence with which these statements may be made. It is important to note that uncertainty does not prevent a statement of risk from being made; nor does it prevent reasonable actions from being taken. It does, however, require that the magnitude of the uncertainty, and the implications for decisions, be understood so the degree of support for the statement is not misinterpreted.
Uncertainty can arise from a variety of sources. Some of these sources can be described quantitatively, while others are best described qualitatively. A decision must be made as to how to combine these quantitative and qualitative aspects of uncertainty into an overall measure of the confidence that may be placed in a specific statement (such as the statement that the lifetime risk of cancer from air toxics compounds in a specific census tract in North Carolina is 1 in a million). The goal of an uncertainty analysis is to characterize uncertainty and confidence as rigorously as possible, trying to avoid purely subjective judgments, while recognizing the inherently qualitative nature of some aspects of uncertainty and the need for human judgment at times.
What are the components of uncertainty? To understand this issue, consider the process by which a study such as the national-scale assessment is performed:
- Problem Formulation. First, the problem to be addressed must be defined precisely. Is the
problem one of human health effects being produced by industrial facilities? Is
it one of ecosystem effects being produced by mobile sources? Is it some
combination of these or other issues? It may not be clear what the study is
intended to address, or how the results will be used. This step in the analysis
introduces problem formulation uncertainty. The issue is dealt with in
the section on Limitations on this web site, where the
question addressed in the assessment is defined as precisely as possible (i.e. that
the study is limited to effects in human populations resulting from a variety
of sources). The issue of problem formulation uncertainty is not considered
further in this section on Uncertainty.
- Defining the factors to be considered. Second, a conceptual diagram must be drawn
for the problem. This diagram describes the parts of the world that influence
the answer to the problem. In the national-scale assessment, this clearly includes emissions
from a variety of sources (mobile, area, stationary, etc), atmospheric
dispersion, activity patterns for different receptor populations, Unit Risk
Estimates, Reference Concentrations, and so on. Where the science is poorly developed,
it may be unclear what processes must be included in the conceptual diagram.
There may also be limitations in resources, making it infeasible to include all
factors in the study. This step in the analysis introduces conceptual
uncertainty. Again, the issue is dealt with in the section on Limitations on this web site, where the aspects of the
problem that are (and are not) included in the study are addressed (i.e. that
the study addressed inhalation only). The issue of conceptual uncertainty is
not considered further in this section on Uncertainty.
- Selecting models. The national-scale assessment is based on a series of mathematical models. There is
a model that produces the emissions inventory, a model that calculates ambient
air concentration (ASPEN), a model that calculates exposure (HAPEM4), and a
model that calculates risk (for cancer and non-cancer effects). All scientific
models involve uncertainties, since the model must reduce a very complex set of
chemical, biological, physical and social processes down to manageable
equations from which calculations may be performed. These simplifications
introduce inaccuracies. There usually are competing models that can produce
different results, and there is uncertainty as to which model result should be
used. As a simple example, the national-scale assessment uses a linear model relating
exposure and cancer risk; i.e. the cancer risk equals the exposure (air
concentration) times a Unit Risk Estimate. Uncertainty analysis involves asking
a series of questions: Are we certain this linear relationship is correct?
Could there be a quadratic relationship (risk equals exposure times the square
of the dose)? Could there be a threshold relationship (there is no risk until
the exposure gets sufficiently large)? What are the implications for estimates
of risk if these different models are used? What are the implications for
decisions if we cannot choose decisively from amongst these models? And so
on.
This step in the analysis introduces model uncertainty. In judging model uncertainty, there are both quantitative and qualitative issues. Qualitative issues involve the scientific plausibility of the model. Does the model include all important processes? Does the model explain the phenomenon (such as atmospheric dispersion) well? Is the model well accepted in the scientific community, having passed critical tests and being subject to rigorous peer review? And so on. The national-scale assessment did not consider the effect of alternative models, and so this component of uncertainty was not addressed directly (although it will be discussed later on this web site).
Quantitative issues involve comparisons of the model against sets of data (although this also involves issues of parameter uncertainty discussed in the next bullet). Does the model generally predict these data accurately? Are the predictions accurate to within a factor of 2; a factor of 4? What is the effect of any approximation methods used in the model? And so on. In the end, both the qualitative and quantitative aspects of model uncertainty are important.
- Selecting parameter values. The models used in the national-scale assessment require
parameter values such as emissions rates, stack heights, fractions of time
spent indoors, and Unit Risk Estimates. While models describe general
relationships between properties of the world (e.g. the linear relationship
between exposure, Unit Risk Estimate and cancer risk), parameters quantify
these properties in specific cases (e.g. the numerical value of the Unit Risk
Estimate for benzene). They provide the numbers needed in the models. There
always are competing bodies of data from which these parameters may be
estimated, and the methods used to collect the data introduce uncertainties.
This introduces parameter uncertainty.
While there are both quantitative and qualitative aspects of parameter uncertainty, it is common to characterize this source of uncertainty quantitatively, with some qualitative caveats. For example, the parameter uncertainty might be characterized by a confidence interval, saying that the true value of the parameter (such as the stack height for a facility) probably lies somewhere between 40 and 60 meters, or that the stack height is “known to within” a factor of 1.2, or that the stack height is “accurate to within” 20%. Attached to this quantitative characterization of uncertainty will be a qualitative caveat such as: the estimate of this uncertainty is based on measurements made in 1990 at facilities similar to the one considered in this study, but there may have been a change in the design of stacks since 1990. This qualitative statement gives some idea of the confidence with which the quantitative assessment of uncertainty can be applied.
This section on Uncertainty considers both model uncertainty and parameter uncertainty,
and explains how these were developed and combined for the national-scale assessment.
More Details About the "Overall Confidence" Rankings
Which components of uncertainty did the national-scale assessment include?
How was the uncertainty analysis conducted?
Return to the main Uncertainty Page