Draft Summary of the Federal Advisory Committee Act (FACA)

Science and Technical Support Work Group Meeting

Mutual Building, Tom Helms' Office

May 20, 1997

Durham, North Carolina

The Science and Technical Support Work Group (STSWG) of the FACA Subcommittee met on Tuesday, May 20, 1997 in Durham, North Carolina. A list of participants and a summary of the meeting follows.

Work Group Participants

Name	Affiliation

Tom Helms, Co-Chair	U.S. EPA

Harvey Jeffries, Co-Chair	University of North Carolina - Chapel Hill

John Cabaniss	AIAM

Kirit Chaudhari	Virginia DEQ

David Chock	Ford Motor Company

Bruce Hill	Appalachian Mountain Club

Dave McNeill	Utah DEQ

Tom Moore	Arizona

Jay Rosenthal	U.S. Navy

Jay Turner	Washington University/Dept. Of Transportation

Dan Weiss	Cinergy Corporation

Work Group Members Not Participating

Diana Andrews	Kentucky

Glen Cass	Cal Tech

John Core	WESTAR

Ellis Cowling	SOS

Cyril Durrenberger	TNRCC

Jay Hudson	Santee Cooper

David Kelly	Navajo EPA

George Lauer	ARCO

Rich Poirot	Vermont

S.T. Rao	N.Y. Department of Environmental Conservation

Mark Scruggs	DOI

Manop Vanichchagorn	Louisiana DEQ

Jeffrey West	GPU Services Corporation

Steve Ziman	Chevron Corporation

Non-Work Group Participants

Bob Wayland, SAIC; Pradeep Saxena, EPRI; Deborah Popejoy, CARB; Bob Judge, U.S. EPA/Region I; Ellen Baldridge, U.S. EPA/AQMG; Bill Kuykendal, U.S. EPA/EFIG; Charlene Spells, U.S. EPA/IPSG; David Sanders, John Silvasi and Bob Stallings, U.S. EPA/OPSG.

Summary of the Call

David Sanders opened the call by providing a recap of the joint Base Program Analysis and Policies (BPAP) Work Group and National and Regional Strategies Work Group (NRSWG) meeting, which was held in Durham, North Carolina on May 15/16, 1997. Mr. Sanders explained that the Reasonable Further Progress (RFP) issue paper was discussed, and that the STSWG inputs were stated (e.g., answers to the three questions). He indicated that John Silvasi would expand on this latter topic next.

John Silvasi replied that the three questions really did not come up at the meetings last week, because the RFP subgroup was focusing on other issues: who is responsible for when there is a failure of meeting RFP; and, measures that are not quantifiable and enforceable. Mr. Silvasi concluded that these two issues were the recipient of the majority of the air time last week. He continued, reminding the participants that STSWG has two subgroups, which are currently meeting: (1) Meteorological Variability (David Chock, S.T. Rao, Jay Rosenthal, Bruce Hill, Steve Ziman and Cyril Durrenberger); and, RFP Metrics (Cyril Durrenberger and Steve Ziman). Mr. Silvasi indicated that it had become apparent during discussions last week, that these two groups would both like to review the RFP metrics issue, so the two groups were jointly reviewing the issues. Mr. Silvasi concluded that it was unfortunate that neither Cyril Durrenberger or Steve Ziman would be able to join the call today.

Harvey Jeffries questioned what the titles of these two papers were, and where one could find them on the TTN/BBS?

John Silvasi responded that David sanders had sent these two documents out via either e- mail or facsimile to all STSWG members.

David Chock questioned if Mr. Silvasi had received his comments?

John Silvasi responded, that the comments were received, and that the contractor (e.g., SAIC) had incorporated those statements into the Meteorological Variability response.

David Chock stated that he was not referring to Meteorological Variability, rather he was referring to the comments he submitted to Mr. Silvasi pertaining to RFP metrics.

John Silvasi responded that he had been out of the office some last week, and this information had not yet been distributed to either Cyril Durrenberger or Steve Ziman. Mr. Silvasi stated that he would do this, and get those comments incorporated into a final version of the RFP metrics paper.

Bruce Hill requested that Mr. Silvasi provide the participants with a brief summary of the two papers currently being developed.

John Silvasi responded that he would give this a try, based on the information he had available to him at this time. Mr. Silvasi began with the RFP metrics paper, and made the following statements:

•	RFP metrics should reflect a hierarchal approach, which fosters an iterative methodology.

•	There are some metrics more suitable for prospective approaches, and others for retrospective approaches.

•	Primary Metrics - (1) Establish a trend, using 5-6 years of ambient monitoring data; one year of data is insufficient for establishing a trend.

Harvey Jeffries interrupted and stated that this could not be currently done for PM-fines, since the database has little or no information in it at this time.

John Silvasi continued with the examples of primary metrics.

•	(2) Statistical descriptions of air quality (e.g., the t-test of David Chock); (3) Design Value considerations; (4) Areal Exposure; and, (5) Population-Weighted Exposure.

Tom Moore interjected that these latter two approaches probably would not work well for PM-related issues.

Bruce Hill questioned if David Chock could expand on the Design Value Considerations metric?

John Silvasi stated that it is essentially the fourth highest concentration at an area's worst monitoring location, over a three year period.

Bruce Hill stated he was still confused, and was probably missing something in this discussion.

John Silvasi stated that the design value was based on the standard. Mr. Silvasi continued saying that you were basically tracking the progress of your "hotspot monitor". He concluded that the subgroup was recommending a combination of these metrics, and not inferring that any one area might implement all of the metrics. The scheme would definitely be flexible and hierarchal in nature.

David McNeill stated that he did not like the use of the Design Value nomenclature. He suggested that we simply refer to this as "the fourth highest value at your worst monitoring location" approach.

John Silvasi agreed, and indicated he would work with the subgroup to incorporate this comment.

Tom Moore questioned whether we needed to be absolutely clear that this metric was prescribed for a specific site. Mr. Moore cautioned the group to be conscious of the variable temporal and spatial scales, which had been discussed previously by the STSWG.

Bob Judge questioned is this paper was the appropriate place to discuss emissions metrics?

John Silvasi responded that he had not finished the paper yet, and that there was currently a short discussion of emissions metrics included.

Bruce Hill argued that emissions reductions should be the primary metric and that the air quality indicators or trends should be the secondary metrics.

Harvey Jeffries responded that there would be others on this Work Group who argue the exact opposite approach.

David Chock stated that if certain emissions reductions resulted in specific air quality improvements, then a linear approach would be great. However, he continued, in a non-linear system, where emissions reductions may actually provide a disbenefit , emissions metrics are not practical or feasible.

Harvey Jeffries said that one approach might be to develop an air quality indicator that was more closely linked to measuring emissions reductions. For example, he stated that if you claim you are having a NOx reduction, then the NOy values should be going down. If this is in fact not true, then you have a problem. Mr. Jeffries concluded that it is an "either-or" proposition, that both air quality and emissions metrics will be needed.

John Silvasi asked if could please try and finish the rest of the draft paper, and reminded the group that neither Cyril Durrenberger or Steve Ziman had seen this draft document. Mr. Silvasi stated that it was compiled late last week by the contractor, and had not been reviewed or discussed at this time with the subgroup members. He indicated that there would be Secondary Metrics, which should determine if precursor emissions had actually been reduced.

Harvey Jeffries interrupted saying that this was the goal; to make sure that you were getting the emissions reductions that you thought you were getting, and then getting the appropriate air quality improvements.

John Silvasi briefly outlined the Ancillary Analyses, and stated that these were from a document that Cyril Durrenberger had faxed to David Sanders. Mr. Silvasi replied that one thing the RFP group did specifically ask about during last week's meeting was a question related to measuring precursor emissions in the ambient air. He indicated that the group was interested in determining if this concept was feasible or not. He said that currently, there is the Photochemical Assessment Monitoring Station (PAMS) network, which was only in 21 or 22 areas. Thus, Mr. Silvasi questioned if states could really implement this type of approach for many geographic locations? Mr. Silvasi asked the Work Group to review this question, and to please forward any ideas/concepts to his attention.

Bob Judge questioned how this discussion/paper fit into the attainment demonstration process? He questioned what was going to happen when we crossed the 5-6 year mark?

Harvey Jeffries reminded Mr. Judge that the choice of metrics would be key in this determination. He stated that different metrics could in fact result in different results, and ultimately in different strategies being developed. For example, peak ozone metrics could perform poorly, while population-weighted metrics could do quite well. Mr. Jeffries stated that you could not assume that we have enough knowledge about what is going on in the system to make these types of metric decisions up front. However, he continued saying that he did not think you would wait until the 5-year mark to assess whether you were making progress or not; he suggested that you would do this each year. Mr. Jeffries concluded that he had not heard anyone say that if you were ahead of the game you would back-off the controls. He said that the form of the new standard could make the situation so difficult, that it would be very difficult to determine exactly what was going on. Thus, the implementation of flexible metrics are extremely important to allow a region to assess progress.

Bruce Hill questioned if you could normalize the meteorology and metrics such that you could turn one program on and turn another off on an annual basis?

Harvey Jeffries replied that this was not feasible. However, Mr. Jeffries said that you did need to do the work every year, and then decide whether your program were working or not. If you find that you were not making suitable progress, then you would need to try something different. He stated that the 5-6 year period did not mean that you reviewed progress only every 5-6 years; you have to be doing this every year.

Tom Moore echoed his agreement with Mr. Jeffries. He stated that there would be a lot of year-to-year variations, in addition to the spatial variation. Once a track record was established, then the year-to-year variability could be better understood. Mr. Moore indicated that the 5-6 year period was crucial.

Harvey Jeffries surmised that based on the discussions today, that this group had a long ways to go on the metrics paper.

John Silvasi cautioned Mr. Jeffries that this was only a first draft, and that neither Cyril Durrenberger or Steve Ziman had the opportunity to review this material. Mr. Silvasi then moved to a discussion of the Meteorological Variability Paper.

Bruce Hill asked if this paper was based on the concepts of S.T. Rao?

John Silvasi responded that S.T. Rao's approach could be used here; but, the purpose of this paper was not to select a specific tool, simply to provide a framework or methodology.

Harvey Jeffries agreed, saying that the Chock and Rao tools did not exhaust the universe of all possible tools.

Bruce Hill asked if someone could review the statistical outliers issue, which was part of the Meteorological Variability paper?

John Silvasi re-read the statistical outliers statement from the paper: "The STSWG Meteorological Variability Subgroup does not recommend treating any observed data to be statistical outliers so long as they are properly validated."

Tom Moore said that he thought this was related to the time interval required to establish a trend.

David Chock stated that this was a tricky issue. He reminded the group that a "smoothed" result was normally used to establish a long-term trend, and thus the trend was generally small.

John Silvasi indicated that the form of the standard was supposed to be reflective of the level of health protection, keyed back into health effects information. This was the intent when the form of the standard was set. It was not set with the ease of detecting a trend in mind.

Tom Moore responded that this was obvious by the repeated use of long-term averages.

Harvey Jeffries said that he liked this approach.

Tom Moore offered that there were trade-offs in implementation, and this provided direction for the policy which the scientists would be using. Mr. Moore stated that he liked what was here.

Harvey Jeffries concluded that the issue of Meteorological Variability was moving along nicely, and should be ready for presentation at the June Meeting.

John Silvasi concurred.

Harvey Jeffries suggested that before this went to the Subcommittee, we would need to generate some figures, graphics and examples. Mr. Jeffries stated that simple words would not explain what we are trying to get across here. He concluded that without figures and diagrams, we would not be able to convey our thoughts to the FACA Subcommittee.

Bruce Hill agreed, and suggested that S.T. Rao be contacted for some examples, specifically his work in the Chicago region.

David Chock reiterated that one trend does not mean that all trends should be supporting it (e.g., one upward trend does not mean that all trends are upward).

Harvey Jeffries replied that was exactly what everyone has been saying this afternoon. For example, one metric may be trending upwards, while another one was trending down.

John Silvasi asked that the group close the discussion on RFP and move on.

Modeling Requirements, Emissions Inventories and Statements Paper

David Sanders reminded the Work Group that Ted Russell had agreed to take the lead on this paper and that Mr. Russell had given a presentation at the joint meeting last week. The presentation was divided into two pieces, with a presentation on modeling on June 15, 1997, and a second presentation on inventories the following day. Mr. Sanders indicated that in summarizing, Mr. Russell cautioned the group about Models-3. He said that Mr. Russell stated that Models-3 was not the panacea - it would not become the end-all tool. In fact, Mr. Russell argued that Models-3 could be outdated by the time it reached the table later this year. He also stated that Models-3 was not as scientifically advanced as we would like in terms of integrated implementation.

Harvey Jeffries interrupted and said that he would not be as pessimistic as Mr. Russell about the future of Models-3. Mr. Jeffries indicated that EPA's Office of Research and Development (ORD) was facing a major struggle in the middle of reduced funds and political egg shells. He assured the Work Group that the science components were present and operational; however, what was missing were the front-end user-friendly interfaces. Mr. Jeffries stated that PM code had been reviewed extensively by himself and his students, and it was currently considered to be state-of-the-art. Additionally, he said that the post-processors for visibility were in place and operational. In conclusion, Mr. Jeffries argued that as a framework and an integrated tool, Models-3 was the best thing that was available, and that it would be this way for some time to come. However, Mr. Jeffries did question whether the resources and manpower would be available within ORD to adequately test Models-3 and train enough personnel prior to 1998. He stated that ORD was currently behind schedule on both training and documentation development. Mr. Jeffries stated that Models-3 could continue to grow, and could stay on the cutting edge, as long as the resources were available. Mr. Jeffries offered that he did not mean that Models-3 was the only tool available; there were other pieces that could do part of the issue, but there were not other comprehensive tools that could replace Models-3 at this time. He strongly urged the STSWG not to do anything that would undermine the credibility of Models-3.

Tom Moore questioned if it was fair to state that there could be fatal errors in the implementation of Models-3 due to lack of user inputs?

Harvey Jeffries replied "absolutely"!

Pradeep Saxena replied that the model should be tested and judged based on the scientific merits of the algorithms. He stated that there were new concepts for organics which needed to be incorporated into the model.

Harvey Jeffries agreed, and indicated that the model needed another component to adequately deal with PM.

Pradeep Saxena replied that the scientific community had done the best possible job with particulate matter; however, he stated that the state-of-science was about 20-years behind on testing for PM.

John Silvasi reminded the group that there was a good deal of discussion last Friday with Ted Russell about the lack of input data for running these types of models. Mr. Silvasi stated that both Ellen Baldridge and Dave McNeill were present for these discussions. He indicated that Ted Russell is supposed to be reworking these aspects of the paper, and then getting back to STSWG for a formal discussion.

David Sanders continued his discussion of Mr. Russell's presentation, indicating that Mr. Russell talked about the models, and questioned how models could serve as a foundation for demonstrating attainment for certain areas. He said that Mr. Russell posed two options: (1) models cannot be used to show what is in attainment; (2) models can have limited use for determining attainment status. Mr. Sanders stated that Mr. Russell incorporated several aspects of what STSWG has discussed for RFP (e.g., iterative process; weight of evidence, etc). Mr. Sanders said that Mr. Russell questioned whether we should be prescriptive in terms of the types of control strategies we chose; or, should be more flexible? Mr. Russell also questioned how we provide incentives for the emissions inventory process? He also questioned if we had the tools to categorize emissions? Mr. Russell then responded that these tools did exist, and that the Source Apportionment Workshop would identify these tools. Mr. Russell also discussed ways that we could strengthen emission inventory inputs for models. Mr. Russell indicated that the community was making progress in this arena, and pointed to the OTAG process as a starting point. Finally, Mr. Russell questioned how the emissions inventory process could be streamlined - how do we get better emissions inventory inputs? He also stated that we needed to address the culpability issue. Mr. Sanders stated that Mr. Russell ended his discussion by asking for immediate input from the STSWG membership, and any other interested parties.

Harvey Jeffries replied that he did not agree with approach. He stated that the Source Attribution Workshop was focused on analysis techniques that would work within a specific modeling framework; however, this does not say anything about the overall modeling framework. Mr. Jeffries interjected that an analysis technique that was applied to a lousy modeling result would not give you worthwhile answers. This approach would not give you an understanding of how well the model was performing. He reminded the group that there was a difference between model performance and model source apportionment. Mr. Jeffries concluded that he did not think the workshop would identify any specific set of models for use in any given situation. He concluded saying that the Source Attribution Workshop would not define a universe of models, from which one could pick and choose a suitable tool.

David Sanders interjected that this was exactly the kind of feedback that Ted Russell was looking for from STSWG. In fact, Mr. Sanders stated that Mr. Russell was interested in getting "a second shot" at this paper in August, if all of the facts/details were not completed by the June FACA meeting.

Dave McNeill replied that he told Ted Russell that this paper should have titled strictly a modeling paper, because it simply did not deal with emissions and inventories. Mr. McNeill stated that his you were a modeler, then you would understand the enormous amount of emissions work that still needed to go into this effort. He concluded saying, that there is simply insufficient trained personnel currently available to run these models.

Harvey Jeffries concurred with Mr. McNeill. Mr. Jeffries replied that a model is not simply limited to the computer code that drives the system. Even if the money was available, you would be looking at 3-5 years before you could develop and educate a staff in the setup, implementation and analysis of these types of modeling protocols.

Tom Moore offered that is was too premature to make generalized statements about regional pollutant control. He stated that perhaps you could make this case for regional haze, but for PM this was simply not clear.

Harvey Jeffries stated that Mr. Russell's paper had a long ways to go, prior to being presented to the Subcommittee.

John Silvasi reminded the Work Group that this paper was due to go to the Subcommittee in June, 1997; however, he recognized that it was obviously not ready to go before the Subcommittee in June.

Dave McNeill agreed, and stated that he had conveyed this concept to Mr. Russell.

John Silvasi replied that perhaps this could get pushed back to the August or October FACA agenda?

Harvey Jeffries voiced an opinion that the paper could be presented in draft outline format in June, similar to what Sarah Peirce-Sandner did for RFP.

Tom Helms agreed, that there was no problem presenting a basic draft outline in June.

John Silvasi agreed, stating that Mr. Russell did have a basic understanding of what the critical path items were. He indicated that most of these came directly from Ned Meyer's straw man document.

Harvey Jeffries questioned why the June schedule was so critical? He asked if this was the only time this paper could come before the FACA Subcommittee?

Tom Helms replied that the thought a strong case could be made to change this schedule. Mr. Helms indicated that the reasons brought forward by the STSWG membership today would make a good case for delaying this issue.

Harvey Jeffries questioned if this was not what Ted Russell meant by getting "a second shot"?

Ellen Baldridge stated that she believed the STSWG was headed in the right direction. Ms. Baldridge recommended a basic outline approach in June, and then come back with a more detailed paper at a later date. However, she suggested that either Tom Helms or Harvey Jeffries go to the Coordination Committee and ask for this delay, as opposed to asking Ted Russell to do this.

John Silvasi replied that he would take care of this.

Tom Moore reminded the Work Group that a good deal of what was contained in Mr. Russell's document was very speculative, because of the lack of information. He suggested that the paper state this fact up front. He offered that it should be approached as here is a general outline, and this is what we want to do; however, we cannot do these things until the standards are available.

John Silvasi asked to push back and review Mr. Moore's comments. He reminded the group that the whole concept of the FACA process was trying to avoid making regulations in a vacuum. Mr. Silvasi stated that the Agency did not want the STSWG to design the entire system; rather, EPA wanted STSWG to comment to the Agency on what you would like to see done. He concluded saying that telling EPA that you have no input on the PM rules because you don't have a PM standard is not really helping us get to where we need to go.

Harvey Jeffries questioned Mr. Moore's statement about ORD having "previously set the course"?

Tom Moore responded that Harvey Jeffries had just indicated to the Work Group that ORD has a model (e.g., Models-3), and that they are moving forward.

Harvey Jeffries retorted that the issue here was really model application. He stated that ORD had done an excellent job in formulating the scientific aspects of the model; however, it is the application of the model that has fallen way short.

Tom Moore replied that it appeared to him that we were missing a lot more information here on this issue that we were collectively willing to admit.

John Silvasi asked if it was not possible to state certain assumptions about how the preliminary designations could be made under the RAMP/AOI framework? He questioned if this paper could be the framework where this information could be released?

Tom Moore responded that this was plausible; however, he seriously questioned whether there was enough time to do this.

Harvey Jeffries replied that it was the job of the STSWG to do this, and to do it clearly and concisely. Mr. Jeffries stated that STSWG must outline the computer hardware and software requirements. He continued saying that he had heard estimates anywhere from $8M to $25M for these type of modeling requirements. Mr. Jeffries concluded that he thought this was what the inventory portion of this paper was all about. He finished saying that STSWG needed to make sure that there was a realistic understanding of what was involved in this entire process.

Dave McNeill reminded the group that inventories were not the only issue here. He said that meteorological data, especially in the western United States, was extremely deficient for these types of model applications.

Bill Kuykendal offered that David Mobley had challenged Ted Russell and the other authors (e.g., both STSWG and NRSWG members) to make this paper more focused. Mr. Mobley indicated that the overall direction and focus of this paper was too general. Mr. Kuykendal stated that if STSWG really wanted to help EPA, then it needed to be telling the Agency what it should be doing to adequately resolve these issues. Mr. Kuykendal concluded that this was the level of information this paper should contain, and that this was the type of information the Agency was trying to get.

Harvey Jeffries stated that he could not agree more with Mr. Kuykendal's statement. Mr. Jeffries replied that he should have sent something to Ted Russell earlier, outlining these concerns. Mr. Jeffries agreed that the paper was not very focused, and stated that he would attempt to work with ORD and focus the specifics of this paper, especially for Models-3.

David Sanders indicated the time was up for today's call, and reminded the Work Group that the next call would be on Tuesday, 5/27/97 at 1:00 pm. The next STSWG face-to-face meeting is scheduled for Monday 6/9/97, and the FACA Subcommittee is slated for 6/10-11/97 at the Durham Omni Hilton.