Water: Regulatory Guidance
Quality Objectives for the RIA Process
Regulatory Impact Analysis
One of the major avenues of change instituted by the Safe Drinking Water Act Amendments of 1996 is in the area of Regulatory Impact Analysis (RIA). For the first time, as part of that analysis, the Environmental Protection Agency (EPA) is required to show that a proposed regulation maximizes the benefits due to health risk reduction. The law also requires that the information developed to support a RIA be scientifically defensible (peer-reviewed) and available to the public. In addition, estimates of costs and benefits of a regulation need to be quantified, to the degree possible, within boundaries of uncertainty. (1)
A set of Data Quality Objectives (DQOs) will be an integral component of the RIA. These objectives will serve dual but related purposes. First, they will define attainable levels of precision of compliance cost estimates, given the uncertainty and variability of the various input parameters and/or assumptions to the cost model. Second, they will be used to drive new data collection efforts in areas where limited or poor information has had a significant effect on the quality of the estimate. An example of a DQO in the first case would be a 95% confidence interval for a cost estimate. In the second case, a DQO would be a determination of how many sample points are needed to achieve a certain level of precision in the cost estimate. Estimates of uncertainty, in the form of confidence intervals, are important to decide among possible regulatory options (e.g., maximum contaminant levels) that result in significant differences in costs.
The "SafeWater Suite of Drinking Water Standards Development Models" are software tools, developed under contract to the U.S. EPA, in part to estimate national compliance costs of proposed regulations. An estimation of cost relies on many model assumptions including parameter calibrations and probability distributional assumptions. The SafeWater model is flexible enough to determine changes in costs for a broad range assumptions. Because of the large number of variables and assumptions in the model, however, there are many opportunities for cost estimation error. The various input parameters to the cost model, their relationships to each other and to the outputs of the model are depicted in the attached flow chart. All of the parameters and assumptions are subject to uncertainty and/or variability which necessarily impact the precision of the cost estimate.
In order for the "SafeWater Suite" to be a reliable tool for future Regulatory Impact Analyses, EPA has initiated a study of the model's logic and sensitivity to changes in input parameters and assumptions. Results of the study will include: (1) a determination of the relative effects of the uncertainty of the input factors and assumptions in the model on the cost estimate's precision (e.g., how does the quality of the input parameters affect the output?) and, (2) a determination of whether errors can be controlled by, for example, new sampling to meet DQOs and improve the cost estimate's precision. A separate but related study will investigate the effects on costs of variability caused by inherent heterogeneities of the input parameters, such as real differences in contaminant occurrence.
Data Quality Objectives will directly affect and are affected by EPA's separate efforts to better define unit treatment costs (through development of a design manual) and model systems. As new and more rigorous information is obtained through these efforts, the cost model will be updated and refined and more precise cost estimates are expected. Conversely, the results of the data quality analysis will motivate specific work in other areas of regulation development- especially in those areas that have been shown to significantly impact cost estimates.
Summary of Activities:
Determination of how the number and range of water system size categories and corresponding flow estimates influence estimates of compliance cost.
Determination of the degree to which estimates of contaminant occurrence drive variability in compliance costs by quantifying the sensitivity of the cost estimate to changes in occurrence distribution assumptions.
Determination of the degree to which contaminant occurrence and unit treatment costs together affect variability in compliance costs by looking at the cases of arsenic, uranium and radon.
Determination of what are achievable data quality objectives for cost estimates, given there is variability in input data due to real differences across facilities, and other sources of uncertainty or error.
Future work will be dedicated to standardizing methods for quantifying the relative uncertainties in the various input parameters, defining variability (real differences among utilities) and determining cumulative impacts on cost estimates. Another aspect of the work will be to develop a clear presentation of results so that the robustness and completeness of supporting data to the RIA can be evaluated and various regulatory options weighed.
Strawman Position Paper Data Quality Objectives; will include results of ongoing uncertainty/sensitivity study of the Safewater Suite and discussion of how conclusions of the study will be used to design a standard program of uncertainty/sensitivity analysis for each proposed regulation.
Final Review Manual on Data Quality Objectives. Manual will be in the form of a simple and user-friendly "how-to" manual on quantifying and portraying variability and uncertainty of the costs of regulation.
Duplication of Uncertainty/Sensitivity analysis but this time in support of Groundwater Disinfection Rule.
Based on results of ongoing Uncertainty/Sensitivity analysis, blue-print for new data collection efforts to improve the cost estimates.
Report on how co-occurrence issues affect cost model relationships; analysis of the interaction effects between different regulations and treatments.
Develop methods for defining uncertainty for benefits/risk analyses.
|Strawman Position Paper Data Quality Objectives
|Final Review Manual on Data Quality Objectives
|Duplication of Uncertainty/Sensitivity analysis
|Blue-print for new data collection efforts
|Report on co-occurrence issues
|Due date unspecified
|Develop methods for defining uncertainty for benefits/risk analyses
|Due date unspecified
Questions for Stakeholders:
Does EPA's approach to defining uncertainty in cost estimates address the statutory requirement?
How should EPA use the results of the sensitivity and uncertainty analysis in regulation development? What are the links between the DQO work and other areas of the RIA process?
EPA's early approach is to use the Safewater model as a framework for the DQO process. Do stakeholders feel that this approach is flawed and will cause important input parameters and assumptions to be overlooked?
What are the priority areas in the uncertainty/sensitivity analysis?
How do stakeholders want to participate in the DQO process?
How can stakeholders help to strike a balance between reducing monitoring/reporting burdens and filling gaps to reduce uncertainty?
What are other areas for future work?
- SEC. 1412(b)(3)(c)(I) of the Act states: ...When proposing any national primary drinking water regulation that includes a maximum contaminant level, the Administrator shall, with respect to a maximum contaminant level that is being considered in accordance with paragraph (4) [goals and standards] and each alternative maximum contaminant level that is being considered pursuant to paragraph (5) [additional health risk reduction and cost consideration] or (6)(A) [additional health risk reduction and cost considerations-in general] publish, seek public comment on, and use for the purposes of paragraphs (4), (5) and (6) an analysis of .. the quality and extent of, the information, the uncertainties in the analysis supporting subclauses (I) through (VI) [quantifiable and nonquantifiable health risk reduction, quantifiable and nonquantifiable costs, incremental costs and benefits associated with each alternative maximum contaminant level considered, effects of the contaminant on general population and sensitive subpopulation, increased health risk as the result of compliance, including risks associated with co-occurring contaminants] and factors with respect to the degree and nature of the risk.