To STSWG

Re: Strawman Draft # 1 (5 March) Modeling tools

I have read the draft, or at least from mid section 3.2 on. My first, quick comments reflect a California bias, and having been involved in too many monitoring/modeling studies-to the point that I have become really critical of what we can/should expect from models and modeling.

I will comment on the individual topics as they are in the paper:

However, before making specific comments, it is very important to acknowledge up front the need for an adequate aerometric data base that has been collected for two purposes. The purposes are to all a stressful evaluation of the model and its modules-a mechanistic evaluation that may uncover formulation defects and underlying compensating errors; and use in evaluating planning options. Most areas of the country still do not have a data base that even approaches the desirable level. Without such a data base, there are serious risks of getting wrong answers. This comment is made with the historical perspective for ozone modeling, but at this point, there has been no guidance on real data base needs for evaluation of the aerosol module that would be part of the overall modeling frame work for fine (2.5 micron) particles. Equally as important is the improvement of the emissions inventory that is needed to carry out simulations and evaluation. In almost every study I have been in, we have identified significant major problems with the emissions inventory. This has been true for VOC and for NOx, in terms of not only emission factors and activity levels, but in source characterization for placement into the appropriate vertical layers. It is even more of an issue for fine PM.

The bottom line is that we can not continue to do modeling with inadequate data bases-which is what has transpired for the last twenty five years.

Trajectory Models: There probably needs to be some exploration of trajectories and back trajectories as compared with other methods which utilize a eulerian system, and for which the model has undergone rigorous evaluation in order to understand whether the results may be an upper limit, or should not be used for areas with less aerometric data.

Observational Models: Receptor models: With respect to this use, it is critical to understand the radius of influence with regard to application of CMB/rollback techniques. This is important in that assumptions on the relationships of proportionate reduction for sources at various distances from the receptor may not hold at all. Thus, any planning using this approach may overestimate or underestimate benefits.

4.0 Steps in Performing Modeling/Analysis

I would suggest that the present EPA Guidance for both regulatory application and evaluation are not adequate to allow us to test models. I would suggest that consideration should be given to the report "A Process for the Stressful Evaluation of Photochemical Model Performance", Reynold, Roth and Tesche, Western States Petroleum Association, 1994. This report, which has been shared with OAQPS, sets up a mechanism that provides for a much more rigorous evaluation so as to minimize compensating errors. In addition to this, the use of alternative base case analysis can provide more information with respect to bounding resultant modeling outputs. This is described in "The Importance of Using Alternative Base Cases for Photochemical Modeling, Reynolds et. al., American Petroleum Institute publication 4616, 1994. Again, this report has been shared with OAQPS.

Beyond this, there is the issue of the use of a model for predicting absolute values-yet this is still a major part of SIP modeling, even with EPA's move to other mechanisms for SIP modeling (EPA 1996 guidance document).	OAQPS should seriously consider mechanisms by which to eliminate the demonstration of attainment altogether. One of the first issues that the STSWG discussed and reached consensus on was the inappropriate use of the model for demonstrating attainment. On a practical level, such practice has led to gaming the results, which is not beneficial to anyone.

4.1 Developing a Protocol:

It is not clear that a generic protocol will be realistic. With many different areas, each with some significantly different meteorological or topographical issues, there will be need to have issues relevant to that specific area discussed in the protocol. Further, a protocol also needs to discuss how the various parties to the modeling will interact, the acceptable technical uses of modeling results (for example, what is a realistic lower threshold for determining when a model output is significant, or how should one use an episode which is not at the design value to estimate reductions needed). In reality, the protocol needs to be well thought out and needs to address all potential issues prior to any simulations. This way, all parties can try to objectively define what will be involved with the modeling exercise.

Most of our experience has been with development of protocols for ozone modeling. Because we have (essentially) no experience with episodic PM fine modeling, it may not be possible to yet devise a well thought out PM modeling protocol.

For both protocols, there must be recognition of the time needed to accomplish the technical needs versus the regulatory time frame. My experience has suggested that we have always sacrificed model evaluation and ability to make enough simulations to select an optimal strategy. Instead, we have pushed aside these issues and made the attainment demonstration simulations-without knowing how good those simulations are. And worse, we never seem to find the time to go back to the modeling. This will be an important issue if the approach being discussed by the Reasonable Further Progress sub work group is pursued. It will require making simulations in order to select a strategy that moves towards attainment, but is optimized to meet the selected RFP metric(s).

4. 2 Episode Selection:

There are three major issues here. First is the recognition that modeling with an inadequate data base will lower confidence in the modeling results, thus the need for a non-routine data base, as described above. Given that, it is not likely that the episodes with good data will also be the design value episode (this is even more so with the idea of a design value which is synthesized from the three year mean of the third highest concentration) or a three year mean of the 98% of the 24 hour PM fine. Second, the proposed ozone standard is no more robust than the present one hour standard-it is still an extreme value. Given that, there is a need to reject episodes, regardless of data availability, which are in the extreme tail of the distribution. Ideally one would want to model episodes which are representative of those which have high recurrence rates. The 1996 EPA recognized this issue, but instead took a tact of modeling toward an absolute value which was higher than 125 ppb as a means of modeling episodes which of themselves were not ones of high recurrence, but rather ones that were scaled. This approach gets back into using the model in an absolute predictive model, and using the model for episodes which not have the same meteorology. This is a very troubling issue and needs some innovative thinking.

With regard to PM fine, it is not clear until analysis of monitoring data throughout the country indicates the linkage or lack of linkage to ozone. Further, it may be that there are specific episodic conditions associated with different seasons for PM fine and it is these representative episodes which need to be modeled rather than attempting any form of annualized modeling. The point here is that there is not enough adequate aerometric data, not analysis of that data for PM fine to really develop guidance at this point in time.

4.3 Preparation of Emissions Data:

There are significant problems with the present inventories today for both ozone and PM fine. Assumptions made on EPA estimates, especially for PM are quite questionable. What is truly needed is source testing for development of adequate factors, activity levels and profiles. Based on the experience of the San Joaquin Valley Air Quality Study and now the California Regional PM Air Quality Study, (as well as understanding some problems in LMOS) too little effort and money has been devoted to identifying major trouble spots within the inventory. Both bottom up and top down approaches are needed.

Because of these concerns, there is all the more need for rigorous model/module evaluation in order to identify problems, and to use the alternative base case approach to try to understand the implications of the problems. And again, data analysis of ambient measurements (from a field program to get data for modeling) can also identify major issues.

If a combined approach is not taken, with less reliance on EPA factors and more on area specific concerns, then the inventory will still be questionable. The crux of this is that we have long underspent and not put in the work on this critical area.

4.4 and 4.5 Model Resolution:

Based on a number of monitoring/modeling studies, it would appear that one should not presuppose the grid cell resolution. Rather, this determination should be made by a set of diagnostic simulations that are tailored to address resolution for the specific area of concern. One also needs to remember that decisions should not only be made on physical (land sea breeze, mountains etc) constraints. Rather the inability to have resolution of emissions at larger grid cells will affect performance. We have found in the SARMAP (the SJVAQS modeling effort) exercise that there is significantly better results with respect to evaluation with a 4 Km. Obviously one can use nesting, but even here, one needs to carefully determine from diagnostic simulations where the larger grid should be. Note, we have never run any simulations at 36 km. With respect to the resolution of the met model and mapping of that into the air quality model, the same issues apply. It is particularly important to ensure that there are enough vertical layers especially near the surface, to treat different sources, as well as getting night time ozone right. For SAQM (the model), there are three layers in the first 60 meters, as that is what is needed to get the appropriate vertical distribution. Also, at least from the SARMAP experience, 4 DDA is a necessity for development of a reasonable prognostic wind field

If anything, PM resolution may be even more critical, rather than less. Until there is significant understanding of PM fine relative to meteorology, it is inappropriate to make any statements about resolution. In fact, it is not clear that the present met models are appropriate for treatment of what is essentially a stagnant condition with no organized flow (preliminary analysis of CRPAQS Integrated Monitoring Study 1995 is trying to assess this issues).

For both of these pollutants, the size of the domains must be based, to the extent possible on analysis of minimizing boundary conditions. It is necessary to try to include all sources within the model domain, and still have a model domain that will also capture these emissions when they can be mistaken for boundary (this is an issue in the West Coast relative to land/sea breeze recirculation ). Relative to Section 4. 3, this means that there will be a need to upgrade the emissions for the present attainment areas. They are nowhere near ready for use in modeling.

4. 6 and 4.7 Model Performance:

See my comments at the beginning of this memo. But to briefly restate. Modeling systems (models and the data) must be put through a rigorous evaluation before they can be used. It is inappropriate to use a flawed modeling system because "it is the best we have, and we are only looking at relative changes. Thus compensating errors cancel out" That logic, whether stated or not, has been what has happened for the last 25 years, and it is unacceptable. Further, it will lead to either trashing the entire planning process or to potentially very costly pursuits of less than optimal strategies.

Other Issues:

The sub work group addressing Reasonable Further Progress has discussed the use of modeling to evaluate alternative strategies in order to identify those which would provide an effective pathway to meet the RFP metric or metrics on the way towards attainment. Thus, it is important that the modeling system have undergone rigorous performance with an adequate data base in order to be used for this purpose. This group has also incorporated the idea of the iterative process as discussed in 4.8.

I will provide other comments on the topics at a later date. However, I wanted these comments to get to you prior to the conference call on 18 March, as I may not be able to get on that call.

Steve Ziman