Minnesota Pollution Control Agency (MPCA)

Quality Assurance Plan

Quality Assurance Plan
for
Socioeconomic Projects Related to Pollution Prevention

Advising, Monitoring, and Evaluating a Minnesota Pollution Control Agency Pilot Project for Flexible, Multi-media Permitting

Professor Alfred A. Marcus and Dr. Donald A. Geffen
Strategic Management Research Center (SMRC)
Carlson School of Management, University of Minnesota

Professor Ken Sexton
School of Public Health, University of Minnesota

Dr. Brett A. Smith
Environmental Consultant and Conservation Chairperson, Minnesota chapter of the Sierra Club and member of the National Sierra Club's Conservation Governance Committee

1. Project Description

The Minnesota Pollution Control Agency (MPCA) is planning to conduct up to 5 permitting pilots, under Project XL, as an experiment to test the benefits of flexible, multi-media permits. We have been asked by the Agency to participate in this program to advise in the shaping and implementation of the pilots, to monitor their progress, and to measure and analyze their outcomes. Our roles in this experiment will be to encourage the MPCA and pilot companies to maximize the incentives for innovation in the permits negotiated, observe how each facility goes about satisfying the requirements of its permit and measure the economic and environmental advantages of the new approach. We will determine the extent to which pollution prevention (P2) solutions are adopted at the test facilities and what changes if any should be made before expanding the program to other companies and industries. We will verify that compliance requirements have been met and determine the nature and extent of individual chemical emissions to all media. We are particularly interested in examining the management and organizational responses to the flexible multi-media permits at both the corporate and plant levels. We will try to measure the cost savings enjoyed by the company that resulted from this regulatory approach and the potential cost savings to the MPCA. Finally, we will work in constant consultation with responsible officials from the air, water, and hazardous waste divisions of the MPCA and monitor the inter-media problems that develop during the pilot project.

During this entire process, we will be working closely with the Pilot Project Committee (PPC), a multi-stakeholder working group consisting of representatives from the business, environmental advocacy, government, legal and engineering consultant, and academic communities. The PPC, as advisors to the MPCA, will be examining the project during all its phases to provide advice, monitor the implementation and evaluate the results.

2. Project Objectives

1. To help make this experiment in flexible, multi-media environmental regulation as productive as possible by maximizing the incentives for pollution prevention and by making the pilots as broadly applicable to other industries as possible.
2. To determine the benefits resulting from the pilot, including the extent to which pollution prevention solutions have been implemented and the amount of cost reduction that results.
3. To work towards making the pilot project, if successful, a permanent option for as many companies and industries as possible through dissemination of our reports and analysis and through working with the PPC to develop stakeholder support.
4. Finally, as a consequence of our research, we hope to develop an understanding and analysis of how management responds to the challenge of a changed regulatory structure and to the opportunities presented by it.

The MPCA and the business community have a large stake in the success of the project so that we expect a high level of cooperation from all concerned. We have a high level of confidence, therefore, that most of the objectives listed above will be attained. We would like to observe the long term effects of the experiment but this may require additional funding beyond the two year period covered by the grant. We have, as might be expected, only a moderate degree of confidence that we will be able to garner public support among all interested parties.

3. Project Design

Our work plan naturally divides into three phases: the shaping phase of the project during which time the facilities participating are chosen and the permits are designed through negotiations between the MPCA and the participating companies with the principals and the PPC playing an advisory role; the monitoring phase when the pilot project companies carry out the required modifications at the pilot facilities and the principals and volunteers from the PPC monitor and study the changes and outcomes; and the analysis and dissemination phase when the principals analyze and report on their findings.

The critical aspect of the shaping phase is being able to establish strong enough communication links with the MPCA so as to interact with them in a timely fashion. The critical aspects of the monitoring phase include, receiving all the needed data and in a timely manner, and being able to verify the accuracy of the data provided. The critical aspects of the analysis and dissemination phase are, understanding the technical aspects of the data in a wide variety of industries, and being able to separate the true cost differences between conventional permits and the experimental one from the special costs arising from the new and experimental nature of the program. The critical aspect of the dissemination phase will be enlisting sufficient support of the multi-stakeholder groups to help disseminate the conclusions as broadly as possible to the public.

Uncertainties in the analysis will be discussed explicitly at appropriate sections of our report. Quantitative errors related to data collection, such as toxic chemicals emission data, will be determined to the full extent possible and presented along with the data. Cost data uncertainties will be estimated roughly and the methodology discussed but will present some difficulties. Lastly, qualitative uncertainties such as those related to observing changes in management approaches and culture arising from the new permit will be also discussed but, as might be expected, with very little precision. The analysis of well crafted surveys will help in this endeavor.

4. Schedule

October -- December, 1995	· Develop and negotiate relationships and roles of the principals and PPC with the MPCA and participating companies. · Working with the PPC, develop criteria for data needed to establish baselines and monitor and evaluate pilot programs at participating facilities, advise the MPCA on this issue.
November, 1995 -- January, 1996	· Develop criteria and procedures to monitor and evaluate changes at the MPCA brought about by the experimental permits. · Consult with the PPC advisory group.
October 1995 -- June, 1996	· Advise the MPCA on selection of facilities participating in Project XL.
January, 1996 -- April, 1996	· Monitor permit negotiations between the MPCA and 3M as the first pilot is initiated. Work with the MPCA and 3M to establish baseline data. Consult with the PPC for feedback. · Initiate interviews of 3M and MPCA staff.
May, 1996 -- September , 1997	· Monitor and evaluate the implementation of the 3M permit at its Hutchinson facility. Special emphasis will be placed on P2 efforts that are fostered by the permit. · Analyze our observations and prepare interim and final reports summarizing our findings for this facility.
?, 1996 -- September, 1997	· Repeat the process developed with the 3M pilot for the remaining 2 to 4 pilot facilities as they are phased in. Suggest modifications to the MPCA as we learn from mistakes made in each previous pilot program.
April, 1997 -- September, 1997	· principals prepare analysis and reports of the pilot project's outcomes with assessment of benefits, disadvantages, and needed modifications, working with members of the PPC; final report incorporates these inputs from PPC
April, 1997 -- September, 1997	· dissemination phase: principals prepare papers for publication, seeking advice of the PPC. Dialogue members make presentations to their respective stakeholder groups, etc. · The PPC, facilitated by the principals, develop their final report with conclusions and recommendations for presentations to constituencies and the public.

5. Project Organization and Responsibilities

Alfred Marcus, Professor, Strategic Management Research Center, Carlson School of Management, University of Minnesota, Minneapolis, MN 55455.
Telephone: 612-624-2812; Fax: 612-625-2873; E-Mail: amarcus@csom.umn.edu
Professor Marcus is the Project Manager and will be responsible for the overall coordination of the project. Professor Marcus' primary focus in the project will be the management, organizational, and regulatory issues involved.
Donald Geffen, Research Associate, Strategic Management Research Center
Telephone & Fax: 612-377-5704; E-Mail: dgeffen@csom.umn.edu
Dr. Geffen will work closely with Professor Marcus and with the other participants involved in the more technical aspects of the projects. Dr. Geffen will conduct and analyze interviews and will facilitate many of the meetings with the Pilot Project Committee (PPC). Geffen, applying his scientific and technical background, will take responsibility for operating data collection and analysis.
Ken Sexton, Professor, School of Public Health, University of Minnesota, Minneapolis, MN 55455.
Telephone: 612-626-4244; Fax: 612-626-0650; E-Mail: ksexton@mail.eoh.umn.edu
Professor Sexton will help with the analysis of emissions data both to establish the facilities' base lines and to determine the health, safety, and environmental benefits of the new permit. He will assist in all other aspects of the project.
Brett Smith, Consultant, 5300 Irving Ave. S., Minneapolis, MN 55419.
Telephone: 612-920-9569; Fax: 612-929-0903; E-Mail: bsmith4442@aol.om
Dr. Smith will focus primarily on the permit negotiations and implementation with particular emphasis on the incentives for pollution prevention that can be built into the permits. He will help track the extent of P2 solutions at each facility and will provide analysis and reports of these efforts and will be of assistance in examining the economic and environmental benefits of the pilot.
Sharon Hansen is the Administrator for the Strategic Management Research Center and will be administrating the budget. She will assist in our work with the PPC by arranging for meetings and maintaining contacts with members. Ms. Hansen will assist in managing the data that will be collected for this project.

6. Assessment of Quality Assurance

Quantitative Pollution Release Data

A lot of the data regarding items such as quantities of waste (all media) produced and disposed of, emissions monitoring and mass balance analyses, will be data submitted to the MPCA by the participating facilities and obtained by us from that agency. Other additional data that we deem to be necessary for our program will be obtained directly from the pilot companies under confidentiality agreements. The principals will explore and test the accuracy of the data in a variety of ways; by cross-checking MPCA monitored data with the companies' own figures, by working with plant managers and engineers to verify the methodology they employ, and by looking for internal inconsistencies that are signals for data errors. We will attempt to verify each facility's mass balance analysis by carefully working through the equations used and spot checking a few instances of their application.

Cost Data

We will be relying on cost data collected and provided to us by the participants. Since properly accounting for the costs of environmental management to companies remains challenging, we will take special care to make the data we request as complete as possible. We will also check carefully for internal inconsistencies. This kind of data will, nevertheless be the most uncertain and we will provide an analysis of that expected uncertainty.

Qualitative Data

Qualitative data will include interviews and surveys of project participants at both the MPCA and the pilot project companies. We will employ, where appropriate, the methodology developed in the policy implementation analysis literature and have at our disposal well crafted surveys that have already been used for such studies. These surveys, which will also form the basis of the questions asked in our interviews, provide tests of the accuracy and consistency of individual responses. The uncertainties of our qualitative data due to sampling limitations will be discussed fully.

7. Performance Assessment

Our success will be measured by our ability to adequately determine and evaluate the outcomes of Project XL. These outcomes are best framed in terms of the five core hypotheses Project XL will attempt to test. They are that beyond compliance multi-media, flexible permits will result in:

1. lower direct and indirect costs to both the companies and the MPCA;
2. superior environmental, health, and safety performance by the permitted facilities;
3. more innovative environmental management approaches such as pollution prevention;
4. increased involvement of stakeholders with the environmental, health, and safety issues at the plant facilities; and
5. beneficial organizational and "cultural" changes at both the permitted companies and at the MPCA.

This research project will be a success to the extent that we are able to reliably determine whether or not of these hypotheses are true.

Minnesota Pollution Control Agency (MPCA)

Local Navigation