Jump to main content.

Multi-criteria Integrated Resource Assessment (MIRA)

You will need the free Adobe Reader to view some of the files on this page. See EPA's PDF page to learn more.

On this page

What is the Multi-criteria Integrated Resource Assessment (MIRA)?

MIRA is an approach that facilitates stakeholder engagement for collaborative multi-objective decision making. MIRA is designed to facilitate and support an inclusive, explicit, transparent, iterative learning-based decision process.

Multi-criteria Integrated Resource Assessment (MIRA): When Clumsy Solutions are Superior (PDF) (2pp, 182KB)

The process is used to:

What kinds of analytical problems can MIRA handle?
  1. Policy/decision problems with many and diverse stakeholder interests
  2. Mix of quantitative and qualitative data/information, including chemical and non-chemical stressors and human health and ecosystem/ecosystem services
  3. Data/model uncertainties as well as decision uncertainty
  4. Policy/decision problems that utilize highly technical, expert information as well as stakeholder values
  5. Rank ordering and comparing things
MIRA is designed on these decision analytic principles:
  1. Consideration of Stakeholders
  2. Context
  3. Criteria and their metrics
  4. Significance of the data metrics
  5. Value judgments
Why MIRA?

Tame vs. Wicked Problems

Tame problems:

Wicked problems are:

Wicked problems (PDF) (15pp., 1.3MB) Exit EPA Disclaimer - Rittel and Webber (1973)

Simple way to think about tame vs. wicked:

Environmental problems are complex and deciding what to do about them is complicated by diverse stakeholder perspectives, a combination of science, economic and socially relevant data/information, and sometimes, regulatory constraints. Defining the problem and finding the necessary common ground in which to make policy decisions are difficult. This is a wicked problem.

What’s wicked got to do with it?

All decision making problems are wicked problems. Environmental decision making problems typically also contain tame problem components (i.e., we want to use scientific data in decision making). Knowing which components are tame vs. wicked in the environmental decision making problem flags those components that are best resolved with mathematical and scientific data and models (elegant solutions) vs. those components that are best resolved with facilitated stakeholder discussion and the application of values (clumsy solutions).

Examples of Wicked Environmental Problems: Global climate change, Sustainability, Environmental risk management

Tame Problems are those that can be clearly defined and agreed to by all stakeholders and have a clear right answer. A math problem is an example of a tame problem. Tame problems can be very complex. Going to Mars is an example of a tame but complex problem.

Environmental Solutions: MIRA facilitates "clumsy" solutions

Clumsy solutions are those that exhibit the following features:

Elegant solutions address tame problems. Examples of elegant solutions are the answer to an algebra problem, the experimental results from a toxicological study, or the engineering design of the Mars landing vehicle.

Learn more about how clumsy solutions work from Vermeij et al. 2006. Clumsy Solutions for a Complex World: The Case of Climate Change. Public Administration 84 (4): 817-843 and about how MIRA facilitates clumsy solutions in Applications.

Top of page

Tools in the MIRA Toolbox

Diagram of MIRA toolbox tools. Shown are the relationships between the Data Collection Manager, Geostatistical Indicators Module, Model Outputs and Programmatic and Budget Decision Analysis Module. Examples of Model Outputs shown are Mechanistic Models, Empirical Models and Integrated Modeling.

How the MIRA modules connect with each other and with outside information:
Data Collection Manager
Geostatistical Indicators Module
Programmatic and Budget Decision Analysis Module
Model Outputs
Mechanistic Models:
Explicitly include the mechanisms or processes between the state variables; unlike empirical models. The parameters in mechanistic models should be supported by data and have real-world interpretations (EPA, 2009b). Examples are Fate and Transport models like Air Quality Models (AERMOD, CMAQ) and Water Quality Models (HSPF, SWAT)
 
Empirical Models:
Rely upon the observed relationships among experimental data typically because information about the underlying mechanism is not understood or available. These can be thought of as ‘best-fit’ models whose parameters may or may not have real-world interpretation. Examples are probabilistic models like regression models and exposure models.
 
Integrated Modeling:
A systems analysis-based approach to environmental assessment. It includes a set of interdependent science based components (models, data, and assessment methods) that together form the basis for constructing an appropriate modeling system. The constructed modeling system is capable of simulating the environmental stressor-response relationships relevant to a well specified problem statement. (Integrated Modeling for Integrate Environmental Decision Making, EPA100/R-08/010) An example is integrated environmental modeling Technologies (iemTechnologies)developed by EPA’s National Exposure Research Laboratory (NERL). It is a suite of tools for integrated environmental modeling, including Data for Environmental Modeling (D4EM), Framework for Risk Analysis for Multimedia Environmental Systems (FRAMES), and Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE).
 
Data for Environmental Modeling (D4EM):
D4EM interfaces National, Regional, State, Local and user defined databases with modeling systems, such as FRAMES.
Framework for Risk Analysis for Multimedia Environmental Systems (FRAMES):
FRAMES allows models to communicate with each other, facilitating the passage of data, resulting in the simulation of complex environmental processes.
Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE):
SuperMUSE combines a software and hardware to allows analysts to select one or more components of the MIRA-FRAMES architecture depending on their analytical needs and also give them the capability to perform model uncertainty and/or sensitivity analysis.

Top of page

Six Steps of the MIRA Process

  1. Initialize/Define the Context
  2. Initialize/Define the Criteria/Narratives/Data and Organization of Indicators applicable to the Context
  3. Initialize/Apply significance to the data
  4. Initialize/Apply values
  5. Learning via iteration
  6. Understanding the consequences of each choice (i.e., decision uncertainty)
MIRA process flow diagram showing the six steps.

MIRA Process Flow

Top of page

Applications

Select the number to the right of each application or keyword to jump to its corresponding reference.

Application Reference Number
Creating an Environmental Index 7, 8
Ranking Environmental Elements
  • Environmental Condition
  • Pollution Program Management
  • Uncertainty
 1, 5, 10
2, 4, 5, 9, 10
2, 3, 11
Find Applications by Keywords
Keyword Reference Number
Agriculture 1
Air Toxics 4, 5, 9, 10
Budget prioritization 5, 10
Chesapeake Bay 1
Clumsy solutions 9, 23
Decision Analysis/Decision Making 15, 16, 17, 18, 19, 20, 22, 23
Economics 1, 2, 3, 23
Ecosystem 5, 10, 20
Energy 1, 5, 10
Environmental Condition 2, 5, 10, 12, 21, 22
Indicators 1, 2, 5, 6, 10, 12, 14, 20, 21
Multi-media 5, 10
Multi-pollutant 4, 5, 10
Nitrogen Deposition 1
Ozone 2, 4, 5, 6, 9, 10, 12, 14, 23
PM2.5 4, 5, 9, 10
Public Health 4, 5, 6, 9, 10, 12, 14, 22
Risk Management 4, 5, 10, 12, 20, 22, 23
Stakeholders 1, 3, 4, 5, 6, 10, 12, 14, 15, 16, 17, 18, 19, 20, 23
Sustainability 5, 10, 20
Uncertainty 2, 3, 11, 21, 22, 23
Watershed Condition 5, 10
  1. Using Environmental, Energy, Climate, Economic and Social Indicators to Evaluate Selected Crop Fertilizer Practices in the Chesapeake Bay Watershed (PDF) (1 pg, 533K)
  2. Air Quality Data - A Methodology to Assess Optimal Ozone Monitoring Network Design (PDF) (21 pp, 766K)
  3. Environmental Policy Analysis: How Much Uncertainty is Too Much and How Do We Know? (PDF) (29 pp, 1.3MB)
  4. Philadelphia Air Toxics Study: Evaluation of Risk Management Options Using MIRA (PDF) (1 pg, 83K)
  5. Trans-Disciplinary Learning: A Case Study Linking Science to Budgets (PDF) (18 pp, 1.3K)
  6. Evaluating Ozone Nonattainment: Technical Support Document for the 8-hour Ozone Designations 11-Factor Analysis
  7. PM2.5 Public Health Community of Practice - Baltimore, MD (PDF) (19pp, 287K)
  8. Indicators and Implicit Weighting Scheme of the Hazard Ranking System (HRS) (PDF) (17pp, 78K)

Top of page

Learn More

Please contact your local library or EPA's Philadelphia library to obtain the unlinked documents listed below.

  1. Stahl, C.H. and A.J. Cimorelli. 2013. A Demonstration of the Necessity and Feasibility of Using a Clumsy Decision Analytic Approach on Wicked Environmental Problems. Integrated Environmental Assessment and Management 9(1): 17-30.
  2. Stahl, C.H., A.J. Cimorelli, C. Mazzarella and B. Jenkins. 2011. Toward Sustainability: A Case Study Demonstrating Transdiciplinary Learning Through the Selection and Use of Indicators in a Decision Making Process. Integrated Environmental Assessment and Management 7(3): 483-498.
  3. Sanderson, H., C. H. Stahl, R. Irwin and M. D. Rogers. 2005. Reflections on uncertainty in risk assessment and risk management by the Society for Environmental Toxicology and Chemistry's (SETAC) precautionary principle workgroup. Water Science and Technology 52: 73-79.
  4. Cimorelli, Alan J. and Cynthia H. Stahl. 2005. Tackling the Dilemma of the Science-Policy Interface in Environmental Policy Analysis Exit EPA Disclaimer. Bulletin of Science, Technology, and Society 25: 46-52.
  5. Stahl, Cynthia H. and Alan J. Cimorelli. 2005. How Much Uncertainty is Too Much and How Do We Know? A Case Example of the Assessment of Ozone Monitoring Network Options. Risk Analysis 25:  1109-1120.
  6. Stahl, Cynthia H., Cristina Fernandez and Alan J. Cimorelli. April 15, 2004. Technical Support Document for the Region III 8-hour Ozone Designations 11-Factor Analysis. Philadelphia, PA: U.S. Environmental Protection Agency, Region III.
  7. Stahl, Cynthia H. 2003. Multi-criteria Integrated Resource Assessment (MIRA): A New Decision Analytic Approach to Inform Environmental Policy Analysis. University of Delaware. For the degree of Doctor of Philosophy.
  8. Stahl, Cynthia H., Alan J. Cimorelli and Alice H. Chow. 2002. "A New Approach to Environmental Decision Analysis: Multi-criteria Integrated Resource Assessment (MIRA)" (PDF) (17 pp, 146K) Exit EPA Disclaimer, Bulletin of Science, Technology, and Society 22:  443-459.
  9. EPA Office of the Inspector General (August 15, 2002), "Consistency and Transparency in Determination of EPA's Anticipated Ozone Designations" (PDF) (36 pp, 115K), Report No. 2002-S-00016
  10. Cimorelli, Alan J., Cynthia H. Stahl, Alice H. Chow and Cristina Fernandez. June 1999. Decision Consequence Model (DCM):  Integrating Environmental Data and Analysis Into Real-Time Decision Making paper presented to the Air and Waste Management Association Conference in St. Louis, Missouri.
  11. Stahl, Cynthia H., Hong-Jin Kim, Alan J. Cimorelli and Alice H. Chow. 1999. Decision Consequence Model (DCM): Integrating Environmental, Social, Political and Economic Impact Assessment into Real-Time Decision Making.  Proceedings of the 19th Annual Meeting of the International Association for Impact Assessment, Glasgow, Scotland
  12. Stahl, Cynthia H. and Todd S. Bridges. 2013. "Fully baked" sustainability using decision analytic principles and ecosystem services Exit EPA Disclaimer. Integrated Environmental Assessment and Management 9(4): 551-553.
  13. Cimorelli, Alan J. and Cynthia H. Stahl. 2014. Avoiding "Proofiness": Addressing uncertainty in environmental characterization Exit EPA Disclaimer. Integrated Environmental Assessment and Management 10(1): 141-142.
  14. U.S. EPA, Office of the Science Advisor, Risk Assessment Forum. July 2014. Risk Assessment Forum White Paper: Probabilistic Risk Assessment Methods and Case Studies (PDF) (98pp., 5.2MB), EPA 100/R-14/004.
  15. Cynthia H. Stahl. 2014. Out of the Land of Oz: the importance of tackling wicked environmental problems without taming themExit EPA Disclaimer. Environmental Systems and Decisions 34 (4):473-477.

Top of page

Contacts

Cynthia Stahl (stahl.cynthia@epa.gov)
Decision Analysis Module
Office of Environmental Information & Analysis (3EA10)
Environmental Assessment & Innovation Division
US EPA Region 3
1650 Arch St.
Philadelphia, PA 19103-2029
215-814-2180

Alice Chow (chow.alice@epa.gov)
Data Collection Manager
Office of Air Monitoring and Analysis (3AP40)
Air Protection Division
US EPA Region 3
1650 Arch St.
Philadelphia, PA 19103-2029
215-814-2144

Janet Kremer (kremer.janet@epa.gov)
Model Outputs / Integrated Modeling
Office of Environmental Information and Analysis (3EA10)
Environmental Assessment & Innovation Division
US EPA Region 3
1650 Arch St.
Philadelphia, PA 19103-2029
215-814-2147

Top of page

Mid-Atlantic Region || Mid-Atlantic Env'l Assessment & Innovation || Mid-Atlantic Data

Local Navigation

Jump to main content.