Draft Summary of the Federal Advisory Committee Act (FACA)

Science and Technical Support Work Group Face-to-Face Meeting

Omni Hotel

June 9, 1997

Durham, North Carolina

The Science and Technical Support Work Group (STSWG) of the FACA Subcommittee held a Face-to-Face meeting on Monday, June 9, 1997 in Durham, North Carolina. A list of the participants and a summary of the meeting follows.

Work Group Participants

Name	Affiliation

Tom Helms, Co-Chair	U.S. EPA

Harvey Jeffries, Co-Chair	University of North Carolina - Chapel Hill

Diana Andrews	Kentucky

John Cabaniss	AIAM

Glen Cass	CalTech

Kirit Chaudhari	Virginia DEQ

David Chock	Ford Research Laboratory

Ellis Cowling	NCSU

Bruce Hill	Appalachian Mountain Club

David McNeill	Utah DEQ

Leon Outlaw	Santee-Cooper

S.T. Rao	New York DEC

Mark Scruggs	DOI-NPS

Jeffrey West	NARSTO-NE

Steve Ziman	Chevron Corporation

Work Group Members Not Participating

John Core	WESTAR

Cyril Durrenberger	TNRCC

David Kelly	Navaho EPA

Russell Koch	Technical Coatings Laboratory

George Lauer	ARCO

Tom Moore	Arizona

Rich Poirot	Vermont

Jay Rosenthal	U.S. Navy

Jay Turner	Washington University/DOT

Manop Vanichchagorn	Louisiana DEQ

Dan Weiss	Cinergy Corporation

Non-Work Group Participants

Bob Wayland, SAIC; John Dege, DuPoint; Ken Gallupi, UNC; Bob Palzer, Sierra Club; Ken Schere, U.S. EPA/NERL; Ellen Baldridge, U.S. EPA/AQMG; David Sanders, John Silvasi and Bob Stallings, U.S. EPA/OPSG.

Summary of the Meeting

Harvey Jeffries opened the meeting stating that the group really had two issues to deal with today. He continued saying that these two issues had been discussed off and on by the Science and Technical Support Work Group (STSWG) for the past two months: (1) Reasonable Further Progress (RFP) issues (meteorological variability, air quality metrics); (2) Ted Russell's modeling and inventory paper.

Mr. Jeffries said that STSWG had been asked to answer three questions by the National and Regional Strategies Work Group (NRSWG) on RFP issues: (1) air quality metrics; (2) time intervals; and, (3) meteorological variability. He stated that the group had discussed these on the last phone call on Tuesday, and that the group agreed that the responses were in good enough shape to discuss with the NRSWG in our joint meeting later this morning. Mr. Jeffries indicated that Cyril Durrenberger and Steve Ziman participated on the RFP metrics issues, while David Chock, S.T. Rao, Jay Rosenthal, Steve Ziman and Cyril Durrenberger participated in the meteorological variability discussions. In addition, David Sanders, John Silvasi and Bob Stallings provided EPA support on both of these issues.

John Silvasi began the presentation on the June 5, 1997 version of the RFP Metrics Paper. Mr. Silvasi said that the contractor had pulled together some initial draft outlines, which were then edited by Cyril Durrenberger and Steve Ziman. He concluded saying that Bruce Hill (STSWG member) and CARB had also provided written comments, which would be attached to this document as we moved to the next room to join the NRSWG.

Harvey Jeffries reminded the group that a good deal of the comments were linked heavily towards policy. He encouraged the STSWG membership to remain focused on only the science issues in today's forum. He reminded the group that we were only concerned as to how viable and scientifically sound the metrics were. Mr. Jeffries stated that this process was interesting to him because one state helped write this document (e.g., Texas), and now another state agency (e.g., CARB) was questioning the approach. Thus, he encouraged the membership to take a careful look at each of these issues.

Mr. Jeffries suggested that we deal with Question 3 first, and then come back and look at Questions 1 and 2. He reminded everyone that Question 3 dealt with the removal of meteorological variability from time series data, and essentially involved in detecting a signal versus detecting noise. Mr. Jeffries said that the STSWG did not like the way the question was originally phrased, so the group re-stated the question. He continued saying that the STSWG response was broken down into two component pieces:

(1)	In the establishment of a long-term trend in ambient air quality and meteorological data, a minimum of 5-6 years of data would be required.

Glen Cass asked if anyone had actually run the numbers to determine whether 5-6 years of data was adequate? He questioned if we were simply pulling this number out of the hat?

S.T. Rao responded that the number had been determined quantitatively. However, Mr. Rao continued that the group had come to consensus that 1-year of data was definitely insufficient for the determination of a trend. He confessed that to determine this number precisely, the group would need to do some detailed analyses. Mr. Rao stated that the 5-6 year time period was not a bad starting point; but, to categorically say that 5-6 years of data was sufficient, the group would need to do some further analyses to determine this period quantitatively.

Harvey Jeffries replied that we would rephrase this response to indicate that this was a starting point, and that it defined a minimum period of record. He stated, however, that further analysis would be required to determine this interval quantitatively.

Mr. Jeffries stated that the second issue dealt with not throwing away data points because they were statistical outliers. He said that this approach would require that all data sets be adequately reviewed and verified. Mr. Jeffries indicated that the third point dealt with the existence of analytical tools for determining the meteorological variability in time series data; however, the subgroup was not recommending any specific tool. The consensus of the group was simply that there did currently exist tools for detecting signals in time series data, and adequately removing noise from these data sets.

David Chock suggested that the response to Point 3 could be improved by stating that analytical tools exist for separating ambient air quality data into appropriate time scales.

John Dege questioned what the original question was - he indicated that he thought this question was addressing long-term trends?

John Silvasi referred Mr. Dege to the original language in Question 3..

John Dege responded that it was the cool summer issue which he was interesting in clarifying.

Harvey Jeffries said the response did not say that you can only do an assessment every 5- 6 years. Mr. Jeffries indicated that this answer states that once you have enough data to do an appropriate analysis, then you can do it every year. However, the group is stating that a minimum of 5-6 years of ambient data is required from which to determine your baseline or starting point.

John Dege stated that this concept did not come-out in this paper.

Harvey Jeffries stated that for ozone, there was currently enough data from which to establish the baseline and begin to perform the trend analyses; however, for fine particulates it would probably take another 5-6 years to gather the appropriate data and perform the necessary analyses.

Bob Palzer asked that since ozone formation was linked heavily to precursors, could you make corrections for years when the temperature was abnormally high or low?

Harvey Jeffries responded that it was possible to make such calculations.

S.T. Rao interjected that the group tried to capture the concept that the analysis approach was a continuous process. He emphasized that the key here was to determine a trend versus environmental change.

John Dege suggested that part of the analysis should be to go back and see if you were seeing the ambient reductions that you thought you were getting.

Jeffrey West stated that he thought the definition was too narrow, because it specifically referred to ozone.

Harvey Jeffries agreed, and said that he would strike the reference to ozone in the title.

David Chock suggested that a fourth point be considered. Mr. Chock stated that when observational data was used for determining a trend, then we should consider that a certain significance test was used to determine the significance of the trend.

S.T. Rao disagreed, and argued that this was implicit in the statement that we had already made.

Harvey Jeffries agreed with Mr. Rao, and said that this would be too technical to bring up the ladder this afternoon.

Ellen Baldridge acknowledged that this information was great; however, she stated that EPA would be better served if STSWG would generate some examples of these analyses and procedures.

John Silvasi urged the group to remember what the process was here. He stated that the NRSWG was developing recommendations on the RFP issues, and in doing so, they reached an impasse and said that they could not go further without some answers from STSWG. Mr. Silvasi indicated that the next step would be for NRSWG to take STSWG's answers, and then move forward. Thus, if at that point they say that they recommend a certain approach to EPA, then at that point, it may be appropriate that we identify these sorts of tools and how they might be applied. However, Mr. Silvasi concluded that it would be inappropriate at this point in the process for STSWG to make such recommendations.

Harvey Jeffries replied that if the Source Apportionment Workshop was successful, then you could expect a second workshop immediately thereafter to deal with these statistical tools.

S.T. Rao questioned whether these changes would be made prior to joint the NRSWG later this morning?

Harvey Jeffries responded that this would not be possible; however, he would highlight the changes which resulted from the current discussions.

Jeffrey West asked when the August FACA meetings were?

John Silvasi replied that the meetings were scheduled for August 12-14, 1997 in Portland, Oregon.

Harvey Jeffries suggested in the interest of time, that the group move on to Questions 1 and 2 and the RFP Metrics Paper. Mr. Jeffries stated that the idea here was to establish a hierarchy of metrics, which one could choose from. Initially, it would simply involve air quality data, then you would move-up to monitoring precursor emissions, and finally you would be measuring the secondary responses within the atmosphere. This latter concept has three metrics: (1) peak concentrations; (2) areal metrics; and, (3) population-weighted metrics.

Mr. Jeffries said that part of the requirements here dealt with hammering out whether you had the appropriate density of monitors and that they were in fact in the appropriate locations. He also reminded that group that you could have differential success rates on these metrics. The guiding response here was not that you did not make reductions, rather the issue was what kind of reductions you made. He continued that there were two kinds of reductions: dumb reductions and smart reductions. The hierarchy was supposed to guide us towards making smart reductions. The metrics are simply measures of usefulness.

S.T. Rao questioned how this paradigm would work in reality? He said that he had real concerns about this approach. Mr. Rao indicated that he viewed these metrics as part of some type of retrospective (e.g., tracking) analysis.

Tom Helms stated that he did not believe this group could address this question from a policy perspective.

John Dege stated that there could be more than one metric chosen, and in fact, different metrics for different parts of the country.

Harvey Jeffries reminded the group that this was only the groundwork, or the infrastructure. He concluded that we had a ways to go yet on getting to the finish line.

John Dege suggested that we would need more guidance on these metrics issues.

Harvey Jeffries offered that this was also a way to institutionalize some of the approaches and techniques that were first included in the Demerjian and Roth paper, which was done for Basil Dimitriades. He reminded the group that how this actually gets converted into requirements, would be well beyond the scope of this paper. Mr. Jeffries questioned if anyone had any major concerns or comments about this paper?

Jeffrey West commented that he did not see specific guidance on the pros and cons of the individual metrics.

John Silvasi stated that this material was originally in the document, but had been lifted on one of the subsequent revisions.

Harvey Jeffries stated that he would relay to the NRSWG personnel that this type information existed if they were interested in reviewing it. Mr. Jeffries questioned where the figures came from in this document?

John Silvasi replied that this information came from Cyril Durrenberger, and was based on analyses of data from Houston and Galveston, Texas. Mr. Silvasi reminded the group that these results were based on modeling and not ambient data.

Ken Schere responded that he had some reservations on the population and areal metrics, basically because of the high degree of uncertainty with respect to scaling issues.

Harvey Jeffries acknowledged that Steve Ziman had promised to provide some additional information on the monitoring requirements for implementing these metrics.

Glen Cass said that he was interested in pursuing David Chock's earlier point. When air quality was moving in the same direction then there was generally not a problem. However, when air quality was getting better for some people and worse for others - e.g., the Long Beach versus Riverside area - then there real issues to be resolved. In Long Beach, the ozone levels are well-below the world background level; at Riverside these concentrations are through the roof. Thus, if you increase the NOx, then you generate a huge disbenefit for this region. Thus, you must be careful about these situations - you don't want to solve your ozone problem by emitting huge tons of NOx.

David Chock stated that it made sense to look into this dichotomy issue. He concluded that we needed a qualification on the proper use of population weighting.

Joint Meeting - NRSWG and STSWG

Emissions Inventories and Air Quality Modeling: Availability, Evaluation, Use and Limitations (by Ted Russell)

Introduction

-	Paper focuses on directions that the work groups and FACA subcommittee would like EPA to go

-	Refer to Conceptual Model for supporting material

Air Quality Models

-	Central to air quality management

-	Questions about current approach

-	Model flexibility

-	Uncertainties

-	Attainment demonstration

-	Issues on future model use

-	Availability

-	Confidence

Issues

-	Model limitations

-	Air quality Model availability

-	Resource needs

-	Attainment demonstration

-	Dealing with uncertainty

-	Process flexibility

-	Pre-AOI identification

Model Limitations

-	Current models have limitations

-	Future models will have limitations

-	Major limitations

-	MHAPS - We need better emissions and air quality data to apply the models with confidence (e.g., limitations are in the model inputs!!)

-	What is EPA's role in collecting, facilitating and managing the necessary data?

(Motherhood and Apple Pie Statement - MHAPS)

John Heuss interjected that he would also raise the question of meteorological inputs, which were also a limitation.

Ted Russell responded that the major limitations were in the inputs; however, he agreed that there were some problems with model formulation. He concluded, however, that the major limitations were in the model inputs.

Glen Cass offered that in general, photochemical airshed modeling works best where the emissions data were the best.

Ted Russell agreed with Mr. Cass, saying that when you had the appropriate emissions data, then the model results start to look very good.

Glen Cass said that if you had good emissions, your model would (in general) work well. He continued saying that the model was nothing more than a huge mass balance - e.g., the conservation of mass. Mr. Cass indicated that it was very straightforward to verify that mass conservation was in fact working. Thus, Mr. Cass concluded that it was very easy to identify that the limitations were in the input data and not in the model formulation.

Harvey Jeffries asked the group to remember that we were dealing with models that were several years down the road. He disagreed with Mr. Cass, and stated that there were still scientific questions in these models that needed to addressed, and these questions would continue to evolve because the science would move forward. Mr. Jeffries said the key question here was whether the process was capable of accepting this new science as the framework moves forward in the out years?

Model Availability

-	What modeling capability is available, and what do we do if the necessary model is not available when desired?

-	Integrated ozone, pm and haze

-	Models are available

-	Confidence built up by successful applications

-	We need to develop appropriate modeling data to increase confidence

(Confidence will be built through actual model application - if we don't apply them and test them, then little confidence will be gained.)

John Heuss stated that he would agree that the models existed. He encouraged the STSWG membership to take a hard look at the gap in funding and knowledge between academia and the individual state governments which would have to implement these concepts.

Glen Cass reiterated his point that if you were running mass conservation calculations for precursors, then your ozone calculations should be right.

John Heuss agreed, but stated that you first had to be making the necessary precursor observations, which were in fact very costly.

David Chock agreed that model performance was the key issue here.

Glen Cass said that if you were running multi-species model comparisons - e.g., NO2, nitric acid vapor and PAN- and, you know the chemical split is okay then you could make these calculations for organics and individual aromatics. He concluded that if all of these numbers looked good, then you were probably getting the right ozone answer for the right reason.

David Chock replied that we currently did not have adequate VOC data to make these types of calculations.

David Mobley called time-out on this discussion and asked that Mr. Russell move forward.

Ellen Baldridge stated that public access was also very important as was the development of codes which were modular. She said the Agency needed to be able to easily update the science as well as educate the public.

Resource Issues

-	Developing good emissions inventories take time/money/trained personnel

-	Getting good air quality and meteorological data takes time/money/trained personnel

-	Modeling - ditto

-	Computational limitations diminishing

-	Developing confidence will take time

-	Resources at odds with regulatory constraints

-	Time, money, personnel

-	Must deal with uncertainty

Resource Questions

-	What is EPA's role in providing the resources?

-	How can the limited resources be best utilized?

Use of Models Attainment Demonstration

-	Questions as to the use of air quality models in the attainment demonstration process:

-	Models can not give an accurate estimate of needed controls.

-	Use of historical episodes for future air quality

-	Options as to how to proceed

Attainment Demonstration Options

-	Do not use models

-	Continue as now

-	With or without successful evaluation

-	only with successful evaluation

-	Models are central, but not alone tool

-	Weight of evidence approach

Dealing with Uncertainty

-	Environmental modeling and emissions inventory development is fraught with uncertainty

-	Always will be present

-	How should we deal with this uncertainty?

-	Recognize that it is there, but not treat it in detail

-	Identify major areas of uncertainty

-	Quantify uncertainties at each stage, communicate uncertainties

-	Use uncertainty in developing uncertainty in developing strategies

-	Design

-	Goal

Flexibility in Emissions and Air Quality Modeling

-	At present, parts of the air quality modeling process are very prescriptive

-	others are not (meteorological fields development, treatment of biogenics)

-	Should the process continue to be prescriptive, or should it be more flexible?

-	Similar to present

-	Increased flexibility

-	Mid-course corrections

Pre-AOI Identification

-	How should pre-AOIs be identified?

-	Single approach

-	Detailed approach

-	How rigidly should the approach be specified?

-	Complete flexibility

-	Limited flexibility

-	Who should conduct the analysis?

-	RAMPs

-	EPA

RAMP Modeling and EI Roles and Responsibilities

-	Need to identify the roles in the RAMP paradigm

-	Process will require continual involvement of the scientific and regulatory communities

Informational

-	NARSTO critical reviews underway

-	State-of-the-Science assessment

-	Updated look at "Rethinking the Urban Ozone Problem"

-	Provide information to interested parties

-	Not available in time for this effort

-	Source Attribution Methods Workshop

-	Overview of current and coming capabilities

Comments on the Issue Paper

-	Issues

-	Are these the appropriate Issues?

-	Are the options all there? No, absolutely not.

David Mobley stated that the NRSWG had anticipated presenting this paper to the FACA Subcommittee in April, May and June. However, he continued, it was now looking like August would be the "lucky" meeting. Mr. Mobley stated that the current e-mail process was simply not working. He suggested that the group generate an action plan, and bring this paper to closure. Mr. Mobley proposed that 4-5 people get together with Mr. Russell and finish this effort.

John Dege asked what was wrong with this paper? Mr. Dege indicated that he had read it and critiqued it, and really wondered why people were having difficulty with the document?

Ted Russell responded as the author, there was a lot wrong with it!

John Dege replied that you would never get consensus within the scientific community. He indicated that each person had different ideas and proposed different solutions.

Ted Russell stated this paper reflected his perspective. He reminded the group that the paper did not contain the industry and regulatory perspectives, and that this information definitely needed to get in this paper.

David Mobley stated that EPA could provide the nucleus that was required to bring this together.

John Heuss offered that he though Mr. Russell had done a good job of stating the policy issues. However, Mr. Heuss continued, that he did not here were the nuts and bolts issues. If you measure a lot of different things, you can then do good modeling. If you do not, then your modeling is somewhat suspect. PAMS data can only be taken so far. If we proceed with these standards, we are going to have problems dealing with pollutants on a whole wealth of spatial and temporal scales. We simply need more research by EPA and other agencies on these issues. The issue of resources and pulling this stuff together is something that FACA has totally dropped the ball on. Mr. Heuss concluded that Mr. Russell's paper assumes that all of this would come together, and he did not believe that to be the case.

David Mobley replied that he thought Mr. Russell's paper brushed those issues. The reality of the limitations of the resources needs to be addressed up front. There were tremendous costs which need to be addressed with respect to what Mr. Russell's paper was discussing. We simply have not addressed these issues.

Steve Ziman said that the real need from the California perspective was that more data was needed to do adequate performance evaluations of model results. He also cautioned the group that the community needed to be very straightforward about the use of models, and what they were designed to be used for. He reminded the group that models were not absolute determiners. They allow us to look at different issues and ask the "what if" questions. Were the alternative strategies working? Can we look at alternative base cases? How could we define strategies that were optimized for the maximum environmental benefit? He concluded that models were not the panacea; however, if we make the assumption that they were, then we would be bound to failure.

David Mobley proposed that Ted Russell, Harvey Jeffries and the EPA core continue to work on this paper, and move it toward consensus.

Steve Ziman volunteered to join this group, along with S.T. Rao, David Chock, Sarah Peirce-Sandner, Ken Gallupi, Alex Johnson and Jeff West.

David Mobley requested in the interest of time, that we move forward the other agenda items at this time.

Reasonable Further Progress (RFP) Paper

David Chock - Meteorological Variability

Steve Ziman - RFP Metrics

John Silvasi reminded the group that all responses today should be considered drafts, and a final response would be coming from STSWG shortly.

David Chock showed some time series plots of meteorological. From these examples you could determine daily and weekly patterns. Mr. Chock highlighted that the fluctuations were tremendous. He continued saying that we could clearly choose the specific time scales for the determination of trends. There are indeed tools available for determining these trends. The more fluctuation present, the more difficult to determine the trend.

Mr. Chock then answered the 3 basic questions before the STSWG:

(1)	What is the appropriate time interval for detecting a trend?

Because of large variability, a minimum of 5-6 years of ambient monitoring data should be used.

(2)	Do we have tools that will enable us to determine changes in air quality due to control policies in the presence of meteorological variability?

Tools exist for separating air quality data into different time scales and analyzing the air quality trends of different time scales under different meteorological regimes.

(3)	Is there a valid method for identifying meteorological events that result on ozone or pm- fine concentrations that should be considered statistical outliers?

No real data observations should be discarded as long as the data can be validated.

Sarah Peirce-Sandner stated that she was confused since the question had been rephrased. She asked when considering ozone, do we already have that data, or do we need an additional 5- 6 years after the control strategy has been implemented to determine if progress was being made?

David Chock responded that for ozone, sufficient data already existed.

Sarah Peirce-Sandner questioned if for ozone, since we have historical data, if we implemented a control strategy now, could we determine whether meteorology was affecting the control strategy?

Harvey Jeffries responded that since you already have 5-6 years of data, at the end of every year you would reevaluate. He indicated that it would be some fashion of a rolling analysis. However, Mr. Jeffries concluded, that it may take you 5-6 years after the implementation of the control strategy to discern the change.

S.T. Rao stated that there were two different issues here. Mr. Rao said that Mr. Chock was talking about long-term trends, and if you implemented a control strategy today, you could detect the change in a much shorter period than 5-6 years; however, it would not be detectable tomorrow, but it should be a period shorter than 5-6 years. Mr. Rao concluded saying that the long-term trend that Mr. Chock was referring to, would in fact take 5-6 years of data from which to make a quantifiable estimation.

Harvey Jeffries reminded the group that STSWG had not quantitatively determined that 5-6 years of data was the magic period. He said that the group needed to conduct some further analyses on this issue.

Ted Russell stated that there had to be a huge delta (e.g., difference) in order to detect anything. He indicated that he tried a similar analysis with Carbon Monoxide (CO) time series last year, and a 20% change was undetectable. He cautioned the group that when dealing with a complex pollutant like ozone, it could be even more difficult.

David Chock confirmed that Mr. Russell was absolutely correct.

Alex Johnson asked if what Mr. Chock was really saying was that 5-6 years of good monitoring data was required for establishing a trend: If this was the case, then certain individuals would argue that 5-6 years of "good" ozone did not exist.

David Chock acknowledged that the 5-6 year guideline would require good monitoring data; however, he agreed that the question was whether this data actually was in existence at this time.

Harvey Jeffries stated that the STSWG would agree that this data did exist for ozone. However, Mr. Jeffries continued, obviously for particulate matter (e.g., PM-fines) we really do not have the data available for meaningful data analysis.

Alex Johnson replied that based on some work by S.T. Rao, you could conclude that sufficient ozone data does not exist.

David Chock responded that he believed in certain domains that the data did exist.

Joe Belanger stated that we were talking about 10-12 year periods when dealing with RFP, and we were talking about reviewing this in 2-3 year periods. Thus, Mr. Belanger was concerned that this approach did not lend itself to telling people whether they were on track or not. Mr. Belanger concluded saying that he believed this was the point Ms. Peirce-Sandner was trying to make earlier.

Sarah Peirce-Sandner responded that the way the RFP paper was currently structured, it did not suggest substituting air quality metrics for emission reduction metrics. She stated that it asks how the two will work in tandem together.

John Silvasi recognized Steve Ziman, who began the RFP Metrics presentation via telephone link.

STSWG SUBGROUP RESPONSE TO NRSWG QUESTIONS # 1 AND #2

REASONABLE FURTHER PROGRESS (RFP) METRICS

JUNE 9, 1997

STSWG RFP Metrics Subgroup

•	Cyril Durrenberger (TNRCC) and Steve Ziman (Chevron)

•	EPA Support Staff: David Sanders, John Silvasi, and Bob Stallings

Excerpt of Original Questions from the RFP Group

Question #1:	What guidance can the Science and Technical Support Work Group provide with regard to the appropriate length of time needed to determine if a true air quality trend is taking place or whether fluctuations in air quality are random noise? What guidance can STSWG provide with respect to an appropriate interval for assessing changes in air quality for ozone? For fine particulates?

Question #2:	What guidance can STSWG provide regarding which metrics are reasonable from a science perspective, recognizing that the ultimate goal of this customized RFP process is to produce by the attainment date? Are there alternate metrics that should be used in lieu of the ones listed here? Does the STSWG have a preference - based on scientific principles - for any of the metrics listed here and any other metrics the STSWG proposes? For the emission reduction metric, does the STSWG agree that emission inventories are the preferred metric? If not, what emission reduction metrics should be used and in what priority?

STSWG should account for the following:

Technical Considerations

•	Uncertainty

•	Data Requirements

Uses of the Metric

•	Prospective versus Retrospective

•	Trading

•	SIP Revisions

Tiered Metrics

•	Primary Metrics

•	Secondary Metrics

OVERVIEW OF STSWG RESPONSE

Response based on ozone data; however, RFP Metrics Subgroup believes transferrable to fine particulates.

RFP Metrics response reflects hierarchal approach -- fosters the development of an iterative, "customized" approach.

Focus is on a set of metrics, not an individual metric.

Certain metrics better-suited prospectively (planning); other metrics may better retrospectively (tracking). Metrics cannot be used in a stand alone fashion; both prospective and retrospective methods should be used. Prospective metrics can look at ways to optimize.

OVERVIEW OF STSWG RESPONSE (cont'd)

Selection of one (or more) metrics relies on the strength of the underlying ambient monitoring network and the credibility of the numerical models chosen for the analyses. Limitations:

(1)	Existing ambient monitoring networks will not meet data requirements of the air quality metrics for some regions (perhaps most) of the United States.

(2)	Need for better model performance evaluation and finer model resolution.

(3)	For some areas that want to use these metrics, upgrades to monitoring and modeling will be needed.

TYPES OF METRICS--INTRODUCTION

Air quality measurements can be subject to variation in meteorology and must be statistically sound (e.g., remove random noise). This concept implies the following:

(1)	Establishment of a Trend: The Subgroup recommends that a minimum of 5-6 years of ambient air quality data be used to establish such trends. However, once the baseline trend is established, analyses can be performed on an annual basis.

(2)	Statistical Descriptions of Air Quality: The Subgroup recommends the use of statistical techniques in the processing of air quality data to be used in the air quality metrics (e.g., t-test, etc.).

TYPES OF METRICS--SUMMARY

Air Quality Metrics

•	Population-Weighted Exposure

•	Areal Exposure

•	Peak Value Concentrations

Emissions Metrics

•	Direct Emissions Metrics

•	Administrative Emissions Metrics

•	Air Quality Emissions Metrics

AIR QUALITY METRICS

Population-Weighted Exposure:

•	The preferred approach, if sufficient ambient monitoring information available.

•	Allows tracking design value and also exposure above the standard.

•	Allows for resolution of both the temporal and spatial characteristics of pollutant concentrations coupled with the extent and magnitude of population exposure.

Planning:	Metric values determined for each grid cell for each day modeled. Population in each cell multiplied by hourly pollutant concentration greater than the standard, then summed for each hour of the day (e.g., on your way to attainment).

AIR QUALITY METRICS (cont'd)

Population-Weighted Exposure (cont'd):

Tracking:

•	Metric values determined for each monitor for each day being analyzed.

•	Population within each monitor's area of influence multiplied by the hourly pollutant concentration above the standard, then summed for each hour of the day.

•	Area of influence for each monitor must be carefully determined and non- overlapping.

•	Limiting factor -- sufficiently dense monitoring network -- most of the population lives within a reasonable definition of the monitor area of influence.

•	Approach has been used for planning in California and Texas and has been used for tracking in California. Metric valuable to develop control program to minimize exposure.

AIR QUALITY METRICS (cont'd)

Areal Exposure:

•	Use if population weighted exposure metric is not used, or in addition to the population-weighted metric..

•	Includes both spatial and temporal variability in ambient air quality data.

•	Best used when target level established and variation above target level is determined.

Planning:	Metric values determined for each grid cell for each day modeled. For each grid cell and hour, target concentration (e.g., standard) is subtracted from any pollutant concentration above the target, then differences summed for each hour of the day.

Tracking:

•	Metric values determined for each monitor for each day being analyzed.

•	For each monitor and hour, target concentration subtracted from each monitored concentration above the target, then these differences summed for each hour of the day.

•	Limiting factor -- sufficiently dense monitoring network to establish reasonable measure of areal exposure.

•	Used in planning analyses in Texas and currently being evaluated as a prospective tool in California. Modification of this metric has been utilized extensively in the Ozone Transport Assessment Group (OTAG) process.

AIR QUALITY METRICS (cont'd)

Peak Value Concentrations:

•	Metric useful, but only addresses changes to maximum concentration.

•	Cannot measure reductions obtained over a large area.

•	Possible to have significant decrease in frequency and area of exceedances, but not have change in the peak value.

•	Relate to design value.

•	This metric should not be used alone, but in conjunction with areal exposure and/or population-weighted exposure metrics.

Planning:

•	Maximum concentration modeled over the domain used.

•	Possible that this value is significantly higher than the highest concentration measured, because a monitor may not be at peak value location.

•	In many cases location of maximum concentration shifts when controls are modeled.

•	Not now possible to model a large number of episodes, so maximum concentration over modeled days is only practical value for this metric. Results can be placed in perspective with weight of evidence analyses.

AIR QUALITY METRICS (cont'd)

Peak Value Concentrations (cont'd):

Tracking:

•	Maximum concentration, second high, or some other value from monitors in the domain used.

•	May not indicate that significant progress has been made since location of this value can change.

•	Poor metric to use by itself for planning. Some control programs show extensive improvement in pollutant exposure over a large area, but may not reduce the maximum substantially. Conversely some control programs reduce the maximum in the domain, but increase the exposure of the population to the pollutant over a large area. Such an approach may not be improve public health.

EMISSIONS METRICS

•	Difficult in many cases to obtain direct measurements to indicate performance of control.

•	First, and most important, source reduction metric used should determine if precursor emissions have been reduced based on air quality monitoring.

•	Ultimate goal is to optimize strategy that best meets the air quality metrics that have been chosen, as a locale moves toward attainment.

The following types of metrics apply:

•	Direct Emissions Metrics

•	Administrative Emissions Metrics

•	Air Quality Emissions Metrics

EMISSIONS METRICS (cont'd)

Direct Emission Metrics: Applies to specific point sources whose emissions can be directly determined. The best method -- continuous emissions monitors (CEMs), or parametric monitoring.

Planning:	The amount of reduction expected from a control program can be estimated during the planning stages.

Tracking:	When applying a source control to a specific source the actual reductions may vary depending on source configurations and parameters. These variations will have to be accounted for during the use of this metric.

EMISSIONS METRICS (continued)

Administrative Emissions Metrics:

•	Many sources -- not practical or possible to do CEM or parametric monitoring (e.g., area sources and mobile sources).

•	May be possible to determine administratively if reductions can be expected. This metric is very "soft" and should be used with care. Best use is to determine if state has made "good faith" efforts to implement the control.

•	Guiding principals --

(a).	Rules to control the sources adopted?

(b).	Rules adopted in a timely fashion?

(c).	Subject sources inspected for compliance with the rules?

(d).	Impact of population growth and/or shifts?

Planning:	Development of control rules so that appropriate record and tracking mechanisms are established.

Tracking:	Development of appropriate parameters to track source performance.

EMISSIONS METRICS (continued)

Air Quality Emission Metrics:

•	Determine from air quality data if emission reductions have been made.

•	Advantage -- based on actual measurements and measurements in the atmosphere where the reactions occur.

•	Disadvantages--

--	atmospheric chemical reactions can occur rapidly and not provide accurate information on emissions trends.

--	air quality measurements may be subject to permutations caused by meteorology and random error.

•	Exact pollutants to track and the value to track are SIP specific.

•	Use with care and in conjunction with direct emissions metric and administrative emissions metric.

EMISSIONS METRICS (continued)

Air Quality Emission Metrics (cont'd):

•	Considerations:

(1)	Expected trend (e.g., a target % reduction of precursor in atmosphere that results from controls) established by modeling to determine the response to the metric.

(2)	Trends based on a minimum of 5-6 consecutive years of data.

(3)	Trends established by first removing effects of variable meteorological conditions. Seasonal data may be required. E.g., some fuel programs are implemented only during summer months.

(4)	Actual trend established from air quality monitoring to determine if the expected trend has been met.

RFP METRICS - EXAMPLE 1

Example 1:	The ozone control strategy is to only reduce NOx. The SIP would identify the sources of NOx to be reduced and the amount of NOx that would be reduced. Modeling would identify the level of ozone to be reached and indicate the level of NOx that would be expected at each monitoring station. For the time period to when attainment was required, primary tracking would be performed for ozone based on monitoring data using the Population-Weighted Metrics, Areal Exposure, and Peak Value Air Quality. If the expected reductions were not attained, then NOx emissions reductions be would be tracked with the Direct Emissions, Administrative Emissions, and Air Quality Emissions Metrics where applicable. If it appeared that the expected NOx reductions had been made and these reductions could not be identified in the atmosphere, then further study would be warranted to determine a mid-course correction.

RFP METRICS - EXAMPLE 2

Example 2:	The ozone control strategy includes reduction of VOC. The SIP would identify the sources of VOC to be reduced and this could be used to identify reductions of specific compounds that can be measured in the ambient air. As in Example 1 above, modeling would be performed to identify the amount of reduction of ozone required and the amount of specific compounds that would be expected to be reduced. Progress would be tracked with the applicable Air Quality and Emission Metrics. For example, acetylene could be used to track mobile source exhaust emissions. The tracking would be performed as above, except that the data would come from the Photochemical Assessment Monitoring Station (PAMS) network. If the Air Quality metrics did not show the expected improvements, then the Emission Metrics could be used. If the Administrative Emission Metric indicated that the program should show benefits, but the Air Quality Emissions Metric did not show improvement, then further study would be necessary. It could be that the emissions calculation methods were inaccurate or that there were problems with the design of the program.

RFP METRICS - EXAMPLE 3

Example 3:	The PM fine control strategy is to reduce sulfates by reducing only SO2. Modeling would be performed to indicate the expected levels of PM-fine, sulfates, and SO2. The levels of PM-fine, sulfates, and SO2 would be tracked with Air Quality and Emissions Metrics as indicated in Example 1. If it was found that the levels of sulfates and SO2 matched the expected levels, but the PM-fine level did not meet the expected level, then a mid-course correction would be needed to determine the additional controls needed to attain the standard.

*** End of Ziman's Presentation Slides ***

Mr. Ziman concluded saying that the basic premise of this approach was to deal with customized RFP from a scientific perspective. He said that the establishment of the hierarchal approach was intended to provide flexibility for the process. Mr. Ziman cautioned the group that using only an emissions metric was dangerous, because it did not allow one to optimize their strategy on the road to attainment.

John Silvasi noted that the STSWG did not reach consensus on these issues; however, it was relatively close to consensus. He acknowledged that Bruce Hill and CARB did not totally subscribe to the metrics as just presented. Mr. Silvasi indicated that Mr. Hill's comments were attached to the back to the paper; however, CARB's concerns were submitted too late to be included in this document. He stated that copies of the CARB comments would be left on the back table.

Bruce Hill said that it appeared to him that the population-weighted exposure metric failed to actually track the design value. Mr. Hill questioned how you would implement this metric in rural areas? He also asked if this technique depended on modeling in areas where monitoring was insufficient?

Steve Ziman replied that he and Mr. Durrenberger had proposed a hierarchy of metrics. If a state needed to improve their networks to implement the population-weighting metric, then they would need to do so. However, Mr. Ziman continued that this approach would also improve model performance evaluations. He concluded that if a state or region chose not to install a sufficient monitoring network, then they would be required to go back to the lower echelon metrics, like the peak value concentration.

John Silvasi stated that there were a number of background pieces provided by Cyril Durrenberger and Steve Ziman, and that there were a limited number of copies of this material in the back of the room.

Harvey Jeffries reminded the membership that STSWG was going to add the pros and cons of each metric (that were in a previous version of this paper) back into the document. He also stated that it would be possible that two different metrics could perform dichotomously in any given region.

Steve Ziman agreed with Mr. Jeffries, and stated that this situation developed in California. He said that it was quite a common occurrence when using the peak value concentration approach. Mr. Ziman indicated that this situation could occur in the hydrocarbon versus NOx preferred strategy approach. He also state that another concern was that there were going to be areas that would have a very difficult time attaining the standard. For those areas, which may not be able to identify enough emissions reductions to meet the standard, then we would prefer that they chose strategies that would reduce exposure.

Alan Johnson reiterated that he and Bruce Hill had real concerns about the population- weighted approach. Mr. Johnson reminded the group that when you calculate population

exposure, it is exposure for the episodes modeled, and that this does not recognize the overall long-term exposure.

Steve Ziman responded that he did recognize this limitation. He concluded that this approach would require the inclusion of more episodes in the SIP process for evaluation, and this was why you needed to couple both prospective and retrospective analyses. He emphasized that you would need to order your controls such that you got the most bang for your buck in the initial years. Currently, we were choosing a strategy that only reduces the peak, and you in fact may not be reducing exposure.

Alan Johnson replied that he did not think Mr. Ziman could substantiate that conclusion.

Steve Ziman stated that this would have to be addressed at a later date.

Glen Cass acknowledged that the paper included both the air quality metrics and emissions metrics. However, Mr. Cass stated that the emissions metrics section included only one discussion of examples. He said that he would expand this section to include mobile emissions (e.g., tunnel studies, and remote sensing). He urged Mr. Ziman not to limit the discussion to tall stack emissions. Mr. Cass concluded saying that the area dealing with solvents, and other solvent-type products would be another area for inclusion in the examples.

Steve Ziman responded that he had actually crossed-out "point sources" in his draft version.

Sarah Peirce-Sandner thanked all the members of STSWG who had participated in this effort. However, she indicated that the task now would be to discuss how we could fold an air quality metric into the RFP paper, if this was considered a feasible approach.

Steve Ziman reminded Ms. Peirce-Sandner that this framework was conceptual. Until we have completed making measurements, there is nothing that states that the annual standard will be the driving mechanism. In the San Joaquin Valley area, we are looking at late Fall and Winter for exceedances. In Los Angeles, this period is a little bit longer. The annual standard will not be uniform across the country.

Glen Cass stated that if you wanted to do a good job of monitoring for PM-fines, you wanted to be monitoring the major chemical subgroups of PM-fines. For example, if you want to reduce sulfates, you need to be monitoring the subgroups that contribute to sulfates.

Steve Ziman agreed with Mr. Cass. However, Mr. Ziman stated that it was simply going to take a lot longer to implement this approach for PM-fines. He concluded that we could move forward relatively quickly on ozone.

Dan McNerney replied that the group was overloading on modeling here. He stated that if you did not have good precursor performance, then your PM and ozone estimates would not be very good. He concluded that for RFP, this was simply too much reliance on modeling.

Rich Bradley inserted that meteorological variability was the real problem in defining air quality metrics. There were some benefits for looking at air quality metrics, but using it for RFP metrics was questionable. As you get closer to the standard, you tend to have greater uncertainty, especially with respect to population-weighted exposure. Using population- exposure was really a reduction in the protection of health-effects, because it represented a broad spatial average.

Steve Ziman acknowledged the last two comments, and stated that he had discussed these with CARB many times, and that the they simply did not agree. Mr. Ziman agreed to maintain a dialog, and to try and work these out in the future.

David Mobley asked Ms. Peirce-Sandner where the group should go from this point?

Sarah Peirce-Sandner responded that the STSWG had done an excellent job; but, now it was time for the policy group to digest this information. She stated that it was possible that we would need another conference call with STSWG to work through some of these issues. Ms. Peirce-Sandner said that one more iteration with STSWG would most likely be required to reach consensus, and she suggested that this interaction take place next week.

David Mobley reminded everyone that Ted Russell had suggested that the Models and Inventories Group meet next Tuesday at 1:00 p.m. to discuss that paper.

Harvey Jeffries reminded Mr. Mobley that this was a conflict with the regular STSWG call.

David Mobley replied that Robin Dunkins would resolve this conflict and notify the pertinent individuals of the appropriate schedule(s).

Harvey Jeffries informed the joint session that STSWG would be discussing Models-3 in the afternoon, and welcomed anyone that was interested to attend. He said the meeting would start promptly at 1:00 p.m.

*** LUNCH ***

Models-3 Discussion

Community Perspective for Environmental Modeling - Models-3

Joan H. Novak

National Exposure Research Laboratory

Program Goals

Credible modeling and decision support tools for direct use by Federal, State, and industry groups

-	User friendly graphical user interface

-	PC to scalable parallel computing

Community framework for continued advancement of environmental assessment tools

-	Easily incorporate new science advances

-	Rapidly adapt to changes in computing technology

Community Multi-Scale Air Quality Model

-	"One atmosphere" approach

-	The framework is generic; what you put into it is your own choice.

Community Multi-Scale Air Quality (CMAQ) Model

-	Multi-pollutant ("One Atmosphere" Concept)

-	Ozone, acidic deposition, nutrients

-	Secondary fine particles (sulfates, nitrates, organics)

-	Visibility

Science Components

-	MM5

-	Meteorology-Chemistry Interface Processor

-	Atmospheric Chemistry - Transport (CMAQ)

-	RADM-2, CB-IV, SAPRC-90 and generalized gas chemistry

-	Wet and dry deposition, aqueous chemistry

-	Advection and diffusion

-	Aerosol and cloud dynamics

-	Plume dynamics and reactive plume submodels

-	Generalized coordinates and generic grid system

Models-3 Adaptability

-	Meteorological Models, Emissions Models, Analysis Packages

-	Chemistry Transport Model (1) Chemistry - RADM-2, CB-IV, SAPRC-90; (2) Advection - Bott Scheme, Smolarkiewicz; (3) Diffusion - K-theory, TKE, 2nd Order Closure; (4) Clouds, etc.

Science Components (continued)

-	Emission/Cost Estimation for Future Year Scenarios

-	Multiple Projections System (MPS)

-	Economic Growth Analysis System (EGAS)

-	Emission Reduction and Cost Analysis Model (ERCAM)

-	Emissions Processing for CMAQ Input

-	Modified GEMAP/EMS-95

-	Mobile 5a

-	Biogenic Emissions Inventory System (BEIS II)

-	Future Enhancements

-	Sparse Matrix Operator Kernel Emissions (SMOKE)

-	Control Strategy Optimization

Models-3 Framework - An Air Quality Management System

Different Components of the System:

Data Management, Study Planner, Strategy Manager, Program Manager, Science Manager, Model Builder, Tool Manager, Source Code Manager, Framework Administrator

Models-3 Framework Capabilities

Data Manager

-	Data access from any networked system

-	No need to convert data between computers

-	Uses Federal Geospatial Metadata Standard

Science Manager

-	Define global shared information

-	Model domain and resolution

-	Coordinate systems and map projections

-	Chemical reactions and species

Model Builder

-	Select components to create a model specific to the problem

-	"Plug and Play" science modules via global data structures

-	Customize model components (i.e., redefine chemical reactions).

Study Planner

-	Automated multiple-platform execution of a series of interlinked programs/models/independent modules

Strategy Manager

-	Evaluate alternative emissions control options and associated costs

Tool Manager

-	Prepare emission inputs compatible with selected model

-	Invoke visualization and analysis tools

-	VIS5D - 3D analysis and animation

-	PAVE - 2D analysis and remote viewing

-	IBM DX - specialized analysis tools

-	SAS

Client-Server Architecture

Code will run at any site, and on any supporting machine (e.g., any UNIX box, UNIX Server, NT PC-system, etc.). Models-3 will run a single UNIX server; however, we recommend that you link several machines to increase model performance and execution times.

Object-Oriented Data Base Structure is utilized. This is the key to this system - it knows the locations of all required data bases. This is a commercial application, and EPA has negotiated with the owner to distribute this with Models-3 at no charge. However, this does not include SAS.

System provides seamless computing and data management from PC to Scalable parallel computer.

Bruce Hill asked what components of the system were available through public access?

Joan Novak responded that you would register your local Metadata through the INTERNET.

Bruce Hill again questioned what would be public access?

Joan Novak explained that if you were sitting in your local office, running your own system, could you have access to the information of others? The answer is yes, as long as you have the authorization to be on the different systems, which contain the information that you are trying to access.

Harvey Jeffries stated that you may also want to do your own analysis on an existing run on your own machine.

Glen Cass asked if there was such wide and easy access across the INTERNET, how could there be any security on these files?

Joan Novak responded that the system was designed on the underlying basic UNIX security features. The data resides on a specific system; the master database tells you where to go and find the data, through the use of Metadata. It is not like you have to call someone or write a letter - if the data is publicly available, then the system will take you to these data files, and allow you access too them.

Visualization and Analysis Tools

-	Package for Analysis and Visualization (PAVE) 2D Graphics

-	2 and 3D scatter plots, time series and color tile and contour	(http://www.iceis.mcnc.org/EDSS/pave_doc/Pave.html

-	VIS-5D: 3D graphics and animation

-	SAS: Standard Statistical Analysis

-	IBM Data Explorer (DX)

-	ARC/Info for emissions gridding and QA

Harvey Jeffries asked what the limitations with SAS were?

Joan Novak indicated that GEMAP and EMS-95 were linked to SAS - the rest of the system is not dependent on it. However, the emissions system was based on SAS. SMOKE was a tool that was developed by MCNC that would get away from this dependency on SAS; however, it was still missing the geographic component, which was needed for the spatial emissions allocation.

Schedule

Beta Release - end of June 1997

Evaluation - simultaneous with Beta Testing

Tech Transfer - following beta testing and tech transfer

Public Release - Summer of 1998

-	EPA Air Pollution Training Institute - Satellite Broadcasts

Models-3 Training

-	Computational Atmospheric Science

-	http://shodor.org/cas

-	Air Quality Meteorology

-	http://www.shodor.org/metweb/

Advantages of Models-3 Framework

-	Unifying foundation for continual community evolvement of environmental modeling and assessment tools

-	Encourages rapid infusion of new science

-	Facilitates inter-discipline coordination and data sharing

-	Promotes wide-spread use of advanced models

-	Adaptable to changing hardware and software

-	Cross-platform data management and computing

-	Effective use of computing resources

-	Integration of visualization

Future Direction

-	Vision

-	Community Environmental Problem Solving Framework(s) for rapid advance of practical tools, science and technology

-	Strategy

-	Community-developed consensus approach involving federal agencies, academia and industry

-	Useful targeted pilot projects with multi discipline teams

-	Define guidance and methods for interoperability and data sharing

-	Multi-agency grant programs promote adherence to guidance

-	Total Risk Assessment Vision (e.g., complete inter-system linkage of all modeling tools from the atmospheric transport to system ecological effects.)

Steve Ziman asked Ms. Novak if the documentation would address the data needs for Models-3 - both in terms of running the models and evaluating the model?

Joan Novak responded that this would be addressed in the documentation. The actual data requirements for the newer system would be much different than the current model data needs.

George Lauer stated that he would be highly skeptical, until proven otherwise.

Joan Novak replied that she would leave the "convincing" up to Dr. Jason Ching, who was the project scientist for Models-3.

Ted Russell commented that Models-3 would get us one step closer to the Microsoft User Friendly environment. However, he questioned how was there any insurance that this framework was applied correctly? He asked what were some of the thoughts on how all of this would be handled?

Joan Novak responded that this had to be handled very carefully, and that a great deal of guidance needed to be issued on this subject. The goal was to make the system user friendly and easier to run; however, we would have to deal with some of these other questions by issuing the appropriate guidance. Any runs that are made would be fully documented, and this information was written into every data file, such that you could review exactly what was done on any specific run.

Ted Russell asked about the high degree of model flexibility contained in the Models-3 framework. Mr. Russell continued saying that there would be a regulatory version of Models-3; however, he questioned who would make the determination of the regulatory version?

Joan Novak responded that EPA most likely make that determination. However, she admitted that this was more of an OAQPS decision, based on the science recommendation from NERL. The statement was meant to identify that there would be a "fixed" version of the model which could be used comparatively between different groups.

Ted Russell questioned in RAMS could be used instead of MM5 in the Models-3 framework?

Joan Novak responded that NERL would like to incorporate RAMS into this system, such that you could look at one meteorological model versus another meteorological model.

Robin Dennis replied that the generalized coordinate system would take care of this - you would be able to use MM5, RAMS and ETA-type models interchangeably.

Ted Russell inquired how Ms. Novak would ensure evolution of the system? Mr. Russell stated that he was not sure that Ms. Novak had the scientific community lining-up to use the Models-3 system as opposed to their own models.

Joan Novak responded that the process had to involve everybody. It could not just be one group coming-up with a model or a process. Everyone must take their tools, and move them into the scientific and regulatory community. There has to be some group that will take on this responsibility and continue to foster the development of good modeling science. The paradigm is to rapidly advance science and improve environmental management.

Dave McNeill stated that NERL's schedule seemed to be extremely ambitious. Mr. McNeill said this schedule looked similar to Mobile. He said that he just did not see Models-3 happening under the schedule that was presented. He continued saying that according to this schedule, I can go to my legislature in December and start telling them what I need over the next two years. Mr. McNeill stated that he just did not think that NERL should expect Models-3 to be evaluated properly within two months. He reminded Ms. Novak that it took two years and millions of dollars to evaluate UAM, and it was not anywhere near as complicated Models-3. Thus, Mr. McNeill questioned whether two quarters was enough time to do an adequate evaluation of this tool.

Joan Novak responded that even if the framework was not right, that NERL had accomplished something. She said that the system had been beta-tested for the past three or four years (e.g., UAM-Guides, EDSS, etc.). We have been through 2 rounds of beta testing over the past 4 years. We feel comfortable that we will have this system ready for release on schedule. However, Ms. Novak acknowledged that Mr. McNeill could be right; but, she concluded that she was going to be optimistic that NERL could meet the schedule.

Jason Ching Presentation

Models-3 CMAQ Modeling System (Community Multi-Scale Air Quality)

Models-3 is a framework, which deals with several key problems. My comments reflect this paradigm of air quality management. This is your system, and it will be as open as you the scientists want it to be. Then the policy people can take your results and plan for future air quality management. Science only does hypothesis testing, and the policy can evolve from there.

"The environmental management problem"

-	We are doing retrospective air quality predictions

-	NAAQS must be dealt with

-	AQRVs must also be dealt with

-	Ecosystem Protection

1970's

-	Dispersion models

1980's

-	Air quality issues - regional and urban scales

-	Deposition Models - acid deposition

1990's

-	Multi-scale, multi-pollutant and multi-media models

"Today's science will almost certainly be inadequate for tomorrow's needs"

-	Changing form, types of requirements, standards

-	Better more complete data bases are required

-	Improved spatial resolution

Models-3 Current Capabilities

-	Multi-pollutant ("One Atmosphere" Concept)

-	Ozone, acidic deposition, nutrients

-	Primary and secondary fine particles

-	Visibility

-	Current configuration - 36/12/4 - grid resolution; with anywhere from 15-30 vertical layers (current recommendations)

Meteorological Fields

-	Meteorology Processor

-	Dynamic prediction system, hydrostatic/non-hydrostatic

-	2-way coarse; 1-way nested

-	4-D Data Assimilation

-	Multi-Scale, 108-36-12-4 km resolution

-	Cloud, precipitation fields

-	Linked soil/vegetation/canopy Boundary Layer

-	Cloud fields/parameterization

-	Generalized coordinate system

Emissions Modeling

"Models-3 Emissions Processing and Projection System (MEPPS)"

-	Biogenics Emissions - BEIS II

-	Scale dependent

-	Speciation

-	Anthropogenic Emissions

-	Gridded

-	Speciation

David Chock voiced some concern about the generalized coordinate system and about using different coordinate systems with different map projections.

Robin Dennis replied that you would always standardize to the map projection of your meteorological fields. Mr. Dennis stated the group would not be completing map projection interfaces.

Harvey Jeffries asked how difficult it would be to move from MM5 to RAMS?

Jason Ching responded that he could not answer this question definitively at this time. However, Mr. Ching continued, there was a group in Japan currently doing this type of a switch.

Robin Dennis stated that the biggest stumbling block right now was with clouds. He reminded the group that dealing with a dry model was much less complex.

Jason Ching said that his group had added an within MM5 to link the soil and vegetation models into MM5. He indicated that this was necessary in order to resolve clouds at the current targeted resolution of 4 km. He stated that they had worked for four years to get soil and canopy moisture into the MM5 models to better predict air quality.

Robin Dennis reminded the group that MM5 used climatological moisture. However, in order to do accurate pollution forecasting, then you need day-specific moisture, and this was what Mr. Ching has added into MM5.

Harvey Jeffries questioned if this effort was complete?

Jason Ching replied that there was still more evaluation and testing that needed to be done.

Chemistry Dynamics

-	Gas Phase Chemistry: CB-IV, RADM-2, SAPRC-90)

-	Solvers: SMV Gear, QSSA

-	Aqueous Phase: RADM

-	AeRosol Equilibrium System: ARES

-	Aerosol Dynamics: Modal

-	Photolysis Rates: JPROC

Dry Deposition

-	Species dependent

-	size distributed

-	vegetative classes

Cloud Processes

-	Scavenging, Aqueous Chemistry

-	Vertical transport, venting

-	Affects photolysis/Actinic Fluxes

Transport

-	Advection

-	Smolarkewiecz

-	Bott

-	Vertical Diffusion

-	Eddy Diffusivity, ACM, TKE, TM

-	Plume-In-Grid

-	Chemical maturation

-	Reactive, dispersive

Plume-In-Grid Approach

-	Photochemistry, aerosol chemistry

-	Generalized chemistry mechanism

-	Plume dispersion, plume rise

-	Dry deposition

-	Decision logic

-	Multiple nest procedures

-	Methodology evaluation

Steve Ziman suggested that alternative approaches to addressing and evaluating the plume-in-grid procedure be implemented.

Glen Cass asked how the Carbon Bond IV (CB-IV) mechanism could be used to handle aerosols?

Harvey Jeffries replied that it was called a "fudge".

Jason Ching acknowledged that this was correct; however, he concluded that he did not want this limitation to be a "show stopper".

Models-3/CMAQ Evaluation - Robin Dennis

NARSTO Model Evaluation

Model Evaluation - Schedule/Approach

-	Acceptance Testing (12/97 - 1/98)

Preliminary Evaluation (5/97 - 1/98)

(EMEFS data period -- summer 1988; winter-spring 1990)

-	Move RADM at 80/20 km/MM4 and 21-layer to 80/20 km/MM5 and 21-Layer

-	Move RADM at 80/20 km/MM5 and 21-layer to 36 km/MM5 and 21-layer

(2/98 - 7/98)

-	RADM/RPM and CMAQ models-to-data at 36/12 km

-	RADM/RPM-to-CMAQ at 36 km

Public Release (6/98)

Phase I Evaluation (6/98 - 7/99)

Phase II Evaluation (8/99 - ??)

Model Evaluation Issues

-	NOx cycling; aging of NOy

-	Photochemical processing; response surface testing (ridge line determination; NOx- limited regime)

-	Regional Ozone (temporal and spatial)

-	Regional isoprene

-	Regional sulfate, nitrate and organic particulates; partitioning of nitric acid

-	Wet deposition (water, sulfate, nitrate)

-	Dry deposition (SO2, O3, H2O2, HNO3, NO2, PAN,Partiuclates)

-	Urban (Nashville) photochemical processing

-	Plume chemistry

-	Chemical mechanisms (RADM2, RADM2Ci, CB-IV)

-	Urban-Regional Interactions (production by isoprene versus anthropogenic VOCs)

-	Meteorology - PBL and soil moisture issues (MM5 and MM5-PX)

-	Meteorology - transport issues (clouds, intertial oscillations)

-	Meteorology - radiative issues, actinic fluxes

-	Meteorology - others

Most of the meteorology issues will use the Nashville data to make progress.

Model Evaluation - Comparison Periods

Preliminary Evaluation

EMEFS-I:	July 19 to August 25 to September 8, 1988

EMEFS-II:	March 20 to April 19 to May 30, 1990

Phase I Evaluation

Nashville 1995 Data

Meteorology June 1 to September 15, 1995

Chemistry June 25 to July 27, 1995

NARSTO-NE 1995 Data

NARE 1993 Data (August, 1993)

Steve Ziman asked if Mr. Dennis was going to at the SAQM data base at all?

Robin Dennis replied that this was not on the schedule at this time.

Model Application/Evaluation - Domains and Resolution

-	Anticipate tested capability to handle approximately four levels of nesting (one way)

-	Continental applications (national acid rain, etc.)

David Chock asked if we knew enough about the vertical diffusivity to justify the number of vertical layers that Mr. Dennis was proposing?

Robin Dennis replied that we probably did not. He indicated that some model runs were going to use as many as 50 vertical layers; however, we must make some approximations.

Evaluation Methodology

We need to extend the methodology - the old approaches are not very effective.

We will approach this as follows:

(1)	Better diagnostic evaluation tests. Scientific objectivity of judgements made under conditions of uncertainty required in an evaluation.

(2)	Design better model guidance for conducting and interpreting model evaluation testing.

(3)	Expand or extend the use of scientific sensitivity analyses in an integrative manner.

Possible descriptors are as follows:

-	Likely to be acceptable for this application;

-	Not likely to be acceptable for this application; and,

-	Acceptability is indeterminate at this time.

Steve Ziman asked if Mr. Dennis would have anything that told you how far you could take model outputs in the determination of the validity of those numbers?

Robin Dennis responded that if this was an issue, then the intent was that this methodology would be capable of walking you through this process, in order to better inform your judgement. However, Mr. Dennis acknowledged that this was not high on his list at this time. He agreed that he would like to work as many things as possible prior to the public release.

Jeff West responded that he had heard very little about 4 km domains.

Robin Dennis replied that they were running a 4 km grid over the northeast and one over Nashville.

NARSTO-NE Presentation - Jim Vickery

How do we put together a Research Plan to answer the key issues on a national ozone and PM plan? Mr. Vickery stated that he wanted to take advantage of this group's experience and enlist support, and also ask for some volunteers, and of course, resources.

The Executive Steering Committee members of the NARSTO Program are currently working on this issue for tropospheric ozone. They have also recognized that a good deal of the questions will be similar for PM as well. They are currently looking at the risk assessment aspects as well as the risk management piece of this puzzle. The question is, should the NARSTO group expand into the arena of effects? There still needs to be a joint effort; however, we are not sure who should take the lead on this effort.

Goals of Combining PM and NARSTO Research

-	Reflect the emerging importance of pm, with the need for coordinated national program on the scale of NARSTO. However, we probably do not need a program that is as large as NARSTO.

-	We need your data, and your resources to bring this thing to fruition. We are looking at holding a workshop in the September 1997 time frame to get down to specifics. We want to have a joint research program in place to begin in 1999.

Glen Cass stated that Mr. Vickery was inviting people to bring resources to the table. However, Mr. Cass asked how much was already on the table in terms of PM research?

Jim Vickery responded that EPA has almost $25M on the table for PM research; however, it could conceivably double. We have a similar investment for tropospheric ozone, and this is currently matched through public and private partnerships. Thus, approximately $60M is currently available on an annual basis for tropospheric ozone research. I would hope that a similar level of support could be found for PM.

David Chock - T-Test Presentation

The need for a Statistical Test to Determine the Attainment Status and the Trend of a Metric of RFP

-	Meteorological variability exists

-	Upper extreme values inherently more variable

-	Actual stringency versus perceived stringency (present standard more stringent than generally assumed)

-	Statistical test must be used when uncontrollable fluctuations are present

-	Multi-year mean has reduced standard error but does not obviate statistical tests (because the mean still fluctuates within its own range).

-	n-year mean is less responsive to trend change as n increases (e.g., the closer we are to the mean, the less variability we will have)

The t-Test

-	Compare the mean of a sample against a given mean (the level of the standard); or compare the means of two samples (current mean against previous mean for trend determination).

-	Standard deviation unknown a priori and must be estimated.

-	Most useful for small sample sizes; distribution need not be symmetrical.

-	Basic idea:	If the variability is large (less precise), the ability to distinguish two means is reduced.

One-Sided Test versus a Two-Sided Test

-	Trend test for RFP metrics; need two-sided test because the trend can be positive or negative relative to a previous mean.

-	Attainment test; two-sided test is meaningless when (mean - std dev) is negative because design value already below the level of the standard. One-sided test is meaningful for (mean - std dev) > 0. Stringency can be maintained if t-test is used only for areas with attainment designation and Too Close To Call (TCTC) designation is allowed for no more than, say, three years.

T-Test Easy Steps

(1)	Determine the 3-year mean and the standard deviation.

(2)	The TCTC interval is then equal to SD X 2 degrees of freedom X 0.356

(3)	Compare the 3-year mean minus the level of the standard against the TCTC interval. The sign is of course important. But for the comparison below, we ignore the sign of the difference.

If (mean - std dev) > TCTC interval, the area is considered outside the TCTC interval.

If (mean - std dev) <= TCTC interval, the area may belong to the TCTC category.

This only reduces the number of flip-flops; however, it does not remove them altogether.

Glen Cass stated that he understood what Mr. Chock was presenting; however, he questioned how this could be used under a slightly different agenda? He asked how do we plan in advance such that an area will not go into nonattainment?

David Chock responded that part of the violation comes from the variability in meteorology, and thus he could answer Mr. Cass' question.

Bruce Hill asked if you were already in nonattainment, was it possible to estimate the probability of encountering nonattainment again at some later date?

S.T. Rao asked what it was that was really being proposed here - to use this test in designating attainment/nonattainment?

David Chock stated that he had not looked at emissions at all here.

S.T. Rao responded that it seemed that this was an explanation for what had happened - after the fact. Logically, we should proceed with the designation process in the way we have been doing it. Then in the post-mortem, we could use this test to evaluate the effects of meteorological variability on attaining the standard.

Harvey Jeffries stated that what Mr. Chock was saying was that the amount fluctuating out of attainment is insufficient to designate an area as nonattainment.

S.T. Rao said that he could understand this. However, it did not agree that this test defined the air quality that prevailed during a certain time period. Designations are designed to assess the air quality that the people are breathing. Using this in the designation process could be problematic.

David Chock agreed that this was a true policy issue.

S.T. Rao stated that you had to deal with the physics here - if you separate out the processes involved, you see daily variations superimposed on the long-term component. The overall goal of this group was that this long-term component be directed downward. What is the distribution of ozone - it will be a function of the baseline - if you change the baseline, you will change the distribution of ozone. Mr. Chock is simply doing statistical manipulations.

David Chock disagreed that he had done a statistical manipulation; rather, he concluded that Mr. Rao had done likewise in his analyses.

S.T. Rao responded that he was only using ambient data, and that he had done no manipulations. You might question how well I have separated the constituents of the time series, but there is no manipulation in my analyses. I am looking at the role of emissions, meteorology, etc. in the overall signal process of air quality.

David Chock replied that it was ultimately up to EPA to decide if we could or should use a long-term analysis for determining the long-term trend. What was our purpose here? It was to make sure that the air quality trend was going down.

Ellis Cowling reminded the group that they were the Science and Technical Support Work Group for the implementation of the ozone and PM standards, and regional haze. Mr. Rao has brought-out that meteorology is important, Glen Cass has suggested some more robust statistical analyses, and you have done a pretty good job in your effort. I would suggest that we merge this information, and hand this up the chain of command, because I think you are on to something that could be very useful in implementing the ozone policy for the United States. I would suggest that this Work Group endorse your concept and conduct further analyses. The rate of change of emissions from year to year is considerably smaller than the inherent variability and the subsequent meteorological variability. I think that you are on to something.

David Chock responded that the Subcommittee needed to deal with the implementation issues.

S.T. Rao said that the STSWG was not really dictating how the designation was going to be made.

Harvey Jeffries concurred, and said that it was simply an example of applying statistical analysis to ambient data and evaluating the results. It was simply another tool that we have on the table. Mr. Jeffries encouraged the membership to read the paper by S.T. Rao that was distributed to you today: "Discerning Changes in Anthropogenic Ozone in the Presence of Meteorological Fluctuations". A full-version will be appearing in the peer reviewed literature in the next year.

David Sanders - Source Apportionment Workshop

Source Attribution Workshop - July 16-18, 1997

- Purpose of Workshop

- Principle Client - EPA and STSWG

- Workshop Format - 10 Key Presentations

-	1st Day for Ozone

-	2nd Day for Particulate Matter

-	3rd Day for Summary and Discussion

- 6 Member Peer Review Panel

- Ken Demerjian, SUNY-Albany, Chairman; Julius Chang, SUNY-Albany; Neil Donahue, Harvard; George Hidy, University of Alabama; Phil Roth, ENVAIR; Michael Trainer, NOAA Aeronomy Laboratory.

S.T. Rao suggested that a recognized meteorologist be added to the panel. He said that we were missing a key component of the process here.

David Sanders stated that he would review this with Tom Helms at a later date.