National Beach Guidance and Required Performance Criteria - Appendix 4B1: Data Quality and Sampling Design Considerations

This appendix provides additional information on basic data quality planning elements, as well as sampling design considerations. It provides supplemental discussions, examples, and additional references that may be helpful to beach program managers. It does not create additional requirements beyond those in the main guidance document.

1. Data Quality
   • 1.1 General Considerations
   • 1.2 Quality System Documentation
   • 1.3 Quality Assurance Project Plan
   • 1.4 Data Quality Objectives
   • 1.5 Standard Operating Procedures
2. Sampling and Monitoring Design Considerations
   • 2.1 Improving Usefulness of Monitoring Information
   • 2.2 Sampling Design Considerations
3. References

---

1. Data Quality

—1.1 General Considerations

In its Office of Water Quality Management Plan (USEPA, 2001c), EPA established a quality policy for Office of Water (OW) Programs. Several key concepts from that document are summarized below because they guide EPA's review of its own quality programs and others.

• Goal: The goal of OW is as follows: "Environmental decisions shall be based on data of known and documented quality, such that the decisions are scientifically, and where necessary, legally defensible and able to withstand public scrutiny."
• Basic tenet: A basic tenet of OW's quality system is that "the level of effort needed to manage the quality of any activity depends on:
  • The importance of the activity,
  • The risk of decision error,
  • The schedule for completion, and
  • The available resources.
• Quality policy: OW's quality policy is based on the goal and basic tenet described above. The OW Quality Management Plan provides a succinct statement of priorities and a detailed guide to components of the quality approach. The quality policy stresses the need for systematic, up-front planning and the use of a graded approach to quality management.
• Graded Approach: The graded approach to quality management might be the most important part of OW's policy. The basic philosophy behind the graded approach is to recognize that "quality" is not an objective attribute that remains constant. Rather, quality is a subjective attribute of a process or product that must be established in the context of that process or product. Therefore, the quality of the data and the effort expended to manage the quality of the data and the decisions should be based on the end goal of the decision. "Good" quality data are those that enable the user to make the decision at hand with an acceptable risk of error within the required time frame.

—1.2 Quality System Documentation

An important part of the grant application process is documenting the monitoring program's quality management practices as they pertain to the collection and analysis of water samples. The documentation should address the following:

• Who are the project manager, the sponsoring organization, the responsible individual within that organization, the project personnel, the "customers," and the "suppliers?" How are the customers and suppliers involved?
• What are the project objectives, and what questions and issues will be addressed?
• What are the project schedule, resources, budget, and milestones? Are there any applicable requirements such as regulatory or contractual requirements?
• What types of data does the project require? How will those data support the project objectives?
• How was the quantity of data needed determined? How were the criteria for the quality of the data determined?
• How, when, and from where were the data obtained, including existing data? What are the constraints on the data collection process?
• What activities during data collection will provide the information used to assess data quality (field or laboratory quality control operations, audits, technical assessments)?
• How will the data for the project will be analyzed and evaluated? How will they be assessed to determine how well they serve their intended use and the performance criteria established?

—1.3 Quality Assurance Project Plan

Typically, the written documentation takes the form of a quality assurance project plan (QAPP). A QAPP typically details the technical activities and quality assurance (QA) and quality control (QC) procedures that should be implemented to ensure the data meet the specified standards. The QAPP should identify who will be involved in the project and their responsibilities; the nature of the study or monitoring program; the questions to be addressed or decisions to be made based on the data collected; where, how, and when samples will be taken and analyzed; the requirements for data quality; the specific activities and procedures to be performed to obtain the requisite level of quality, including QC checks and oversight; and how the data will be managed, analyzed, checked to ensure that it meets the project goals, and reported. The QAPP should be implemented to ensure that data collected and analytical data generated are complete, accurate, and suitable for the intended purpose. EPA has provided requirements and guidance for the preparation of QAPPs in USEPA (1998, 2001b).

—1.4 Data Quality Objectives

EPA has published a planning tool to help develop DQOs that are included in the quality system documentation. This tool, guidance on the DQO Process, recommends a process that usually consists of the following three activities (USEPA, 2000):

• Define the decision to be made.
• Clarify the information needed for this decision.
• Design the data collection program on the basis of the decision rule and the tolerable limits of decision error.

This process should include preparing a clear statement of the problem, identifying the decision(s) to be made using the data, identifying of the information needed to make the decision(s) (e.g., previously collected data, new environmental measurements), defining the spatial and temporal boundaries of the study, developing a decision rule that will describe a logical basis for choosing an appropriate action based on study results, specifying the limits on decision errors, and optimizing the design for obtaining data. Several iterations of this process might be required to specify the DQOs for a project. Because DQOs are continually reviewed during data collection activities, any needed corrective action can be planned and executed to minimize problems before they become significant. General guidance and examples of planning for monitoring programs are also provided in Monitoring Guidance for the National Estuary Program (USEPA, 1991) and Monitoring Guidance for Determining the Effectiveness of Nonpoint Source Controls (USEPA, 1997b).

—1.5 Standard Operating Procedures

Grant applicants should also document their methods and assessment procedures in the quality system documentation they submit. For routine implementation of these methods, standard operating procedures (SOPs), which can be referenced in and provided with the quality system documentation, provide a tool to assist the person(s) performing the activities. An SOP typically presents in detail the method for a given technical (not administrative) operation, analysis, or action in sequential steps, and it includes specific facilities, equipment, materials and methods, QA and QC procedures, and other factors necessary to perform the operation, analysis, or action. By following the SOP, the operation should be performed the same way every time, that is, it is standardized. Such activities may include, but are not limited to, field sampling, laboratory analysis, software development, and database management. EPA presents examples of the format and content of SOPs in (USEPA, 2001a). The format and content requirements for an SOP are flexible because the content and level of detail in SOPs vary according to the nature of the procedure. SOPs should be revised when new equipment is used, when comments by personnel indicate that the directions are not clear, or when a problem occurs. The grantee should ensure that obsolete documents are removed and that the revised SOPs are used in subsequent tasks.

Top of page

---

2. Sampling and Monitoring Design Considerations

—2.1 Improving Usefulness of Monitoring Information

The National Research Council (NRC, 1990a, 1990b) has evaluated marine monitoring programs and practices and has made a series of recommendations to improve the usefulness of monitoring information. EPA (USEPA, 1991) suggested the following steps based on the NRC's findings for designing successful monitoring programs. These steps can be used to develop a beach monitoring program.

Step 1. Develop monitoring objectives

Clear objectives should be developed for each component of the monitoring program. The objectives should include detecting exceedances and notifying the public when an exceedance is detected. Microbiological monitoring of recreational waters, in most cases, is undertaken to establish the degree of allowable microbiological pollution to protect public health and the environment. For beach management programs, recreational waters should attain criteria as protective as those EPA established in Ambient Water Quality Criteria for Bacteria 1986 (USEPA, 1986). Although an advisory should be considered when a sample exceeds water quality standards, it is ultimately a state or local decision to determine when to issue an advisory or closing.

Step 2. Establish testable hypotheses and select statistical methods

Monitoring program objectives should be translated into statistically testable hypotheses. Establishing testable hypotheses ensures that the results of the monitoring program will be unambiguous and that the objectives of the program can be met. This approach results in the creation of a threshold level for determining when to record an exceedance and notify the public.

Step 3. Select analytical methods and alternative sampling designs

Detailed specifications for the analysis of each environmental variable of the monitoring program should be developed, including field and laboratory protocols and quality assurance/quality control procedures. In addition, alternative spatial and temporal sampling designs should be devised. The sampling designs should specify the number and location of sampling points, sample frequency, and level of sample replication. This information should then be used in the next step to evaluate expected monitoring program performance and to select the most efficient sampling design among the alternatives.

Step 4. Evaluate expected monitoring program performance

Evaluating monitoring program performance might be the most important step in the design and review process (USEPA, 1991). Before the program begins, an evaluation of alternative sampling designs assists in the selection of the most appropriate design for cost-effective sampling that meets the program objectives. During the course of the monitoring effort, performance evaluations provide a systematic procedure for measuring success in terms of the ability to continue to meet program goals. The periodic evaluation process should also identify the need to modify the sampling design and methods. Without this evaluation, there is a risk of collecting and analyzing too few or too many samples. The results of this evaluation should be used to identify the modifications to the initial design necessary to increase monitoring program effectiveness.

Step 5. Implement data analysis

The development of a data management system is an essential task in the design of monitoring programs, and sufficient funds should be provided to cover data analysis. The data management system should be operational before the monitoring program is implemented. In addition to specifying data analysis methods, an expeditious timetable for analyzing the data, and the procedures for reporting and communicating the results, the data management system should be used to assess implementation progress and monitoring program performance. The results of the performance assessment can be used to refine the program objectives and to modify individual study elements to satisfy those objectives.

—2.2 Sampling Design Considerations

Sampling design considerations that might be helpful when establishing a monitoring program include the following:

• Identify the decision maker and program personnel.
• Clarify monitoring program goals and objectives.
• Describe the monitoring program.
• Identify the type of data needed and the sampling design.
• Establish quality objectives and criteria.

Identify the Decision Maker and Program Personnel

A beach water quality monitoring program requires the efforts of program managers, technical staff, and potentially other interested parties or stakeholders. The team involved in planning and implementing the program might include senior government officials from offices established to protect health or environmental quality; technical experts familiar with the issues and methods to be used; data analysts; data users, including risk assessors and the manager or program leader who will make the advisory or closing decision; and quality assurance specialists. Individuals or organizations that might be directly affected by the decision also should be involved in planning the monitoring program to improve communication and build consensus. The members of this group will be able to offer different perspectives and assist in solving problems. They might be involved in development of the plan at different stages and participate in meetings or other activities.

Some personnel manage or perform the work of the monitoring program, while other personnel who do not actually do the work are needed to provide oversight and ensure the quality of the work being performed. Quality control (QC) is a system of technical procedures and activities developed and implemented to produce measurements of requisite quality. Quality assurance (QA) is an integrated system of management procedures and activities used to verify that the QC system is operating within acceptable limits. QA oversight is important to maintain the credibility of a beach monitoring program. QA personnel should be identified at the planning stage and included during program operation program to assess all aspects of data collection.

Clarify Monitoring Program Goals and Objectives

A clear statement of the purpose of the monitoring program and the program's objectives prevents confusion and the waste of resources. As noted in EPA's monitoring guidance (USEPA, 1997b), monitoring programs can be undertaken for different reasons and to answer different questions. The types, quantity, and quality of the data can vary considerably to meet different goals. A conceptual model of the potential environmental hazard should be prepared. This model can be in the form of a diagram illustrating known or suspected sites of contamination at one or more beaches, sources of microorganisms, and exposure scenarios (e.g., children playing in sand or shallow water, swimming, or surfing). The problem to be investigated needs to be defined. The following are examples of monitoring program goals:

• Determine whether an impairment exists.
• Determine the spatial and temporal extent of the impairment.
• Determine the causes and sources of the impairment.

An example of the first type of program goal is routine monitoring to protect human health by comparing levels of indicator bacteria to the ambient water quality criteria for bacteria (USEPA, 1986) during the swimming season. This information is used to determine whether an advisory should be posted or the beach closed. The results from initial monitoring might spur intensive monitoring involving the collection of water samples at different times (e.g., daily or only after storm events) and from many locations (e.g., waterbodies downstream from the initial location). Intensive monitoring might be needed while establishing a monitoring program to pinpoint the most appropriate locations for the routine sampling effort and the depths, times, and procedures needed to collect the samples. Such monitoring data might be needed during the program to evaluate whether the sampling design continues to protect human health. Intensive monitoring can determine the most appropriate sampling frequency needed to assess standards. It might also be desirable or necessary to identify the point and nonpoint sources that could be responsible for waterbody impairment, or to evaluate the influence of rainfall on the bacterial load at a particular beach. Extensive sampling is needed to develop predictive tools using statistical analysis or mathematical models.

This guidance focuses on routine monitoring for beach advisory or closing decisions. An example of a principal study question is

Could levels of bacteria in the water at Bayside Beach affect swimmers' health?

Examples of alternative actions that might be considered if the answer to this question is "yes" include the following:

• Post an advisory at the beach to warn swimmers of the potential hazard.
• Close the beach and do not permit swimming until further notice.
• Conduct a sanitary survey to identify point and nonpoint sources of bacteria.
• Take no action.

The following is an example of a decision statement for this type of program:

Determine whether bacterial indicator levels require taking action to protect human health.

Decision rules developed for this program at a freshwater lake might include the following examples:

If the density of enterococci in any one sample exceeds the EPA instantaneous (single- sample) criterion of 61 per 100 mL, the water is sampled again.

If the density of enterococci in the second sample exceeds the EPA instantaneous criterion, the beach is closed.

If the running geometric mean of enterococcal densities from five sequential samples taken during the previous 30 days is greater than the EPA averaging period criterion of 33 per 100 mL, the beach is closed.

If the density of indicator bacteria does not exceed the criteria under the above conditions, swimmers are not at risk and the beach remains open.

Describe the Monitoring Program

The planning team should discuss what information is needed to make the decision. In the above example, bacterial densities lead to the decision. Also useful are measurements of other environmental factors, such as temperature, nutrients, dissolved oxygen, salinity, turbidity, or water flow, which might provide evidence of a problem or show the seriousness of the exceedance.

The regulatory basis for the decision-in this case, EPA's ambient water quality criteria for bacteria-should be documented. In addition, spatial and temporal boundaries for the monitoring program should be examined. For example, a beach might extend for many miles along the coastline of a jurisdiction, but swimmers have access to only a few hundred feet of shoreline at the end of one road. In addition, the presence of a storm water outfall on the beach might be the focus of sampling.

One or more members of the planning team should document these elements of the program in the monitoring plan. The team also should review available resources, relevant deadlines, the budget, the availability of personnel, and schedule requirements to determine how they will affect sampling at the beach(es) in question. This information should be evaluated along with the proposed sampling design and the boundaries of the monitoring program (see below) to assess how well the program objectives can be met within the various technical and cost limitations.

Identify the Type of Data Needed and the Sampling Design

Various sampling designs have been used for monitoring recreational waters adjacent to bathing beaches. The sampling design specifies the number, location, and types of samples to be collected. It provides the conditions under which they should be collected, the analyses to be performed, and the QA and QC procedures necessary to ensure that the tolerable decision error rates specified in the DQOs.

Because enterococci and E. coli are commonly found in the feces of humans and other warm-blooded animals, the presence of enterococci or E. coli in water is an indicator of fecal pollution and the possible presence of enteric bacteria that pose a risk to human health. Epidemiological studies have led to established recreational water standards based on the documented relationship between health effects and water quality (chapter 1). According to studies of marine and freshwater bathing beaches, the amount of enterococci or E. coli in the water is directly related to the incidence of swimming-associated gastroenteritis (Cabelli, 1983; Dufour, 1984).

Although statistical or probabilistic sampling designs are highly desirable, not every sampling problem can be solved with these designs. Moreover, local limitations in staff and funding might lower the number of samples that can be analyzed during the swimming season. Basic sampling design should address the following seven aspects (Bartram and Rees, 2000):

1. Reasons to sample
2. What to sample
3. How to sample
4. When to sample
5. Where to sample
6. How many samples to take
7. Sampling evaluation

A sampling and analysis plan should include the location of sampling sites, frequency of sampling, duration of the sampling period, and depth of sampling. For each recreational waterbody, the plan also should include other pertinent information, such as the types of containers to be used for sampling, how to package samples for transport, references for analytical methods, how to report data, and requirements for repeat sampling. The plan should be developed in conjunction with the local laboratory that will conduct the bacteriological analyses (CADHS, 1999).

It is difficult to decide the optimum number of samples to take and the most suitable locations to characterize the water quality in the most meaningful way. Sampling marine and estuarine waters requires considering tidal cycles, current patterns, bottom currents, countercurrents, stratification, seasonal fluctuations, dispersion of discharges, multidepth sampling, and many other factors. Sampling lakes and rivers adjacent to beaches requires considering wind and flow and whether the beach is upstream or downstream of pollution sources, as well as time of day (see box). Determining the most appropriate, cost-effective use of the resources available for a monitoring program is also difficult. The following aspects of sampling are presented for consideration when developing a monitoring plan.

Location. Sampling locations are chosen based on historical records, usage, current situations, concentration of bathers, pollution sources, accessibility, and other factors. Areas known to be chronically contaminated, as well as areas that typically have the highest bather density, should be included in the sampling plan. An area close to a storm water outfall might have high counts of bacteria, but it might not be an area commonly used for swimming. Therefore, the priority might be to sample in the area with more swimmers to obtain a better estimate of risk to human health. Ultimately, these decisions are appropriate for the beach manager to make. Table 1 in chapter 4 should be consulted for guidance. In addition, other criteria for sampling might be defined, such as obtaining the sample at a specified distance from swimmers and animals and not in the "swash zone" area of low waves near the shore (IITF, 1999), as well as spacing samples at specified intervals for lengthy stretches of beach.

Frequency. Ideally, when first establishing a recreational water quality monitoring program, the optimum sampling frequency is daily and samples of estuarine or marine bathing waters should be obtained at high tide, ebb tide, and low tide to determine the cyclic water quality and deterioration that should be monitored during the swim season (Bordner et al., 1978; see box below). Lakes and rivers might also be sampled at different times, for example, during calm versus windy days or during low-flow versus storm-flow conditions. If a beach monitoring program does not have the resources to sample this often, a minimum frequency of sampling should be established based on historical records, usage, current situations, the potential for health hazards and the number of samples required by the water quality standards being used. Highly populated or high-risk areas, require more frequent sampling, as shown by the tiered approach (Table 1). Sampling might be needed under special circumstances, such as at locations where no sanitary facilities are provided at a beach or when toilets at the beach are not open or not operational.

Subsequent sampling also might be needed to determine when to reopen a recreational area after a beach closing. Sampling frequency can be related to the peak bathing period, which is generally in the afternoon, but preferably samples are collected in both the morning and afternoon (Bartram and Rees, 2000), at least for beaches classified as Tier 1. Weekends and holidays should be considered in the sampling program. To characterize the water quality at the beach before the weekend crowd arrives, a sample also could be taken on Thursday so that the results are ready by Friday. To characterize the water quality at the beach after the weekend crowd has left, a sample could be taken late Sunday or on Monday. The frequency of sampling might change according to a beach classification.

Sampling Depth. The primary factor for determining the depth of sampling is the users at risk. Samples of ankle- and/or knee-depth water might be more appropriate for children and infants, whereas waist- and/or chest-depth samples might be more appropriate for adults (refer to Table 3-1). Sampling from boats is usually inadequate for beach monitoring because water depths would exceed those common to beach-related recreational activities, especially for young children (CADHS, 1999). Local health agencies, however, might desire to assess water quality away from the shore in additional areas where surfing, windsurfing, or other activities occur.

Sampling Time. The most appropriate time of sampling is that which best estimates water quality conditions during the highest periods of risk. Wave and tidal actions affect bacteria levels, as do the number of bathers during sampling and before and after sampling; the water temperature; and the recent, current, and predicted weather conditions (e.g., wind, rain). Bacteria levels change frequently, based on these types of environmental conditions. This factor should be taken into account when formulating a sampling design and when interpreting sampling results and analyses. If information on the conditions of a beach when the most people are in the shallow waters is of interest, sampling should be conducted during high tide when bacteria levels might be higher near the shore (see Table 1). To estimate how water quality is affected by the number of swimmers in the water, the water should be sampled during the time of day when there is the highest bather density at a beach.

In addition, sampling after the weekend might capture the conditions of the water after the highest bather density. Samples could also be taken on Thursday to inform weekend visitors of water quality before they swim on the weekend. (This type of sampling is recommended for use only on a temporary basis if resources prevent routine daily sampling. It should be done only to better understand indicator occurrence patterns, which are used to develop a more minimalistic sampling approach that best represents those patterns.) Ideally, sampling should be done throughout the day and week to look for patterns of bacteria levels. However, it is important to remember that the results of the laboratory test will take about 24 hours.

The final sampling design should be carefully documented in a sampling and analysis plan or incorporated into a QAPP. (Refer to USEPA, 1998 and 2000, for further information on QAPP preparation.) The plan should include a rationale and listing of the location of all sampling sites and stations within a site, the frequency of sampling at each station, the depth of water sample collection, and the duration of the sampling period (e.g., one time only, 2 weeks in July, during the open swimming season). The plan should also include the procedures for obtaining the samples and analyzing them for bacterial indicator(s), procedures for collecting other data from the field, the schedule for repeat sampling, and how and to whom data will be reported. SOPs should be prepared for all activities that need to be performed the same way every time. Each SOP should include details on the method for a given operation, analysis, or action in sequential steps, as well as the facilities, equipment, materials and methods, QA and QC procedures, and other factors required to perform the operation, analysis, or action.

Establish Quality Objectives and Criteria

Data quality standards define the way the sample is collected and analyzed, and they provide performance criteria that, if met, ensure that the data are acceptable and usable by the decision maker. As part of the DQO process, the planning team should establish program and measurement quality objectives to enable the data user to understand any errors or uncertainties associated with the data. Two categories of errors are commonly recognized sampling error and measurement error. Sampling error is the difference between sample values and in situ "true" values, and it results from unknown biases due to sampling design, including natural variability due to spatial heterogeneity and temporal variability in microorganism abundance and distribution. Measurement error is the difference between sample values and in situ "true" values associated with the measurement process, including bias and imprecision associated with sampling methodology, specification of the sampling unit, sample handling, storage, preservation, identification, instrumentation, and other factors.

The monitoring program should specify methods and procedures to reduce the magnitude and frequency of measurement error. For example, using trained staff to perform the data collection and analyses and following standardized, repeatable procedures for data and sample collection can help eliminate sloppy, inconclusive work. Uncertainty in the data because of sampling and measurement errors or errors introduced during data manipulation could result in identifying a risk to human health when one does not exist (i.e., the true density of bacteria is not greater than the criterion) or not identifying a risk when one does exist (i.e., the true density of bacteria exceeds the criterion). Data entry, transfer, calculation, and reporting mistakes can compound these issues. Data entries and the procedures for calculating results must be carefully checked for accuracy and completeness.

Measurement performance criteria are qualitative and quantitative statements used to interpret the degree of acceptability or utility of the data to the user. These criteria, also known as data quality indicators (DQIs), include the following:

• Precision
• Bias
• Representativeness
• Completeness
• Comparability

Sometimes DQIs for some parameters cannot be expressed in terms of precision and bias (accuracy) or completeness. In these cases a full description of the method by which the data will be obtained should be included in the plan. The various measurement performance criteria that should be established for beach water quality monitoring parameters are discussed in the following subsections.

Precision. Precision is defined as the degree of mutual agreement or consistency between individual measurements or enumerated values of the same property of a sample. Obtaining an estimate of precision provides information on the uncertainty due to natural variation, sampling error, and analytical error. The precision of sampling methods is estimated by taking two or more samples at the same sampling site at approximately 10 percent of the sites. The precision of laboratory analyses is estimated by analyzing two or more aliquots of the same water sample. This data quality indicator is obtained from two duplicate samples by calculating the relative percent difference (RPD) as follows:

where C₁ is the first of the two values and C₂ is the second value. Because the absolute value of the numerator is calculated, the RPD is always a positive number. If it is to be calculated from three or more replicate samples, the relative standard deviation (RSD) is used and is calculated as

where s is the standard deviation and x is the mean of repeated samples. The standard deviation or the standard error of a sample mean(s) is calculated as

where X₁ is the measured value of the replicate, x is the mean of repeated sample measurements, and n is the number of replicates. Precision can also be expressed in terms of the range of measurement values.

Because of the heterogeneity of populations of bacteria in surface waters, an RPD of less than or equal to 50 percent between field duplicates for microbiological analyses might be considered acceptable. In laboratory analyses, the precision among laboratories following EPA Method 1600 for detecting enterococci from separate aliquots of the same sample was determined to be 2.2 percent for marine water samples and 18.9 percent for fresh surface water samples (USEPA, 1997a). Analysts should be able to duplicate bacterial colony counts on the same membrane within 5 percent and the counts of other analysts within 10 percent; otherwise, procedures should be reviewed and corrected (IITF, 1999).

Accuracy. Accuracy is the degree of agreement between an observed value and an accepted reference or true value. Accuracy is a combination of random error (precision) and systematic error (bias), both of which are due to sampling and analytical operations. Bias is the systematic distortion of a measurement process that causes errors in one direction so that the expected sample measurement is always greater or lesser to the same degree than the sample's true value. Because accuracy is the measurement of a parameter and comparison to a "truth" and the true values of environmental, physicochemical, and biological characteristics cannot be known, use of a surrogate is required.

The accuracy of field measurements is usually evaluated by analyzing samples prepared from known concentrations of the pollutant(s) of interest or by adding known concentrations of the pollutant(s) of interest to field-collected samples (known as "spiked" samples). In studies following Method 1103.1 (USEPA, 1985) to estimate densities of E. coli, use of samples prepared from known quantities of freeze-dried and cultured E. coli as a surrogate resulted in 97.9 percent recovery of the bacteria from water samples. Based on the mTEC medium, bias was determined to be 2 percent of the true value. This information is helpful in establishing the most appropriate methods to be followed. Accuracy, defined as the similarity of a repeated entity to its original form, such as information, data entry, and calculations, can be controlled by double-checking sources, manual data entries, or electronic data transfers and performing recalculations. Figure H-1 is a graphical representation of the relationship between bias and precision, and accuracy.

Representativeness. Data representativeness is defined as the degree to which data accurately and precisely represent the characteristics of a population, and therefore it addresses the natural variability or the spatial and temporal heterogeneity of a population. It is not quantitative but descriptive in nature, and it can be assessed only by evaluating the sampling design with respect to the particular features of the water at each beach. It is possible to quantitatively estimate sample sizes using estimates of variance and selecting acceptable levels of false positive and false negative error.

In the sampling design, care should be taken to define the area of sample collection and determine whether it is typical and representative of each area of concern. For swimming beaches less than 30 meters in length, a single sample taken from water at the midpoint of the beach site might suffice. For lengthy beaches, establishing the correct number and location of samples is more difficult, because the sampling needs to ensure that the estimated bacterial densities provide a reasonable representation of the potential risk from waterborne pathogens. For example, the monitoring program might decide to sample from the middle of the area where most swimmers congregate and then 15 m on either side of that first sampling station to obtain an average value of bacterial densities for comparison against the standard. Alternatively, each individual sample result might be compared to the standard. At beaches where a known point source of pathogens, such as a storm water outfall, enters the water, the sample might be drawn from stations within 15 m of the point source or where swimmers might be considered to be at greatest risk from exposure.

As noted above, an initial intensive sampling study might be necessary to help decide where and how often samples need to be routinely collected to address bacterial heterogeneity. If sufficient resources are not available to collect and analyze multiple samples along a beach, the monitoring program plan should justify the decision and note the extent of the area that might be affected by an advisory or closing if bacterial densities at a single station exceed the standards.

Completeness. Completeness is defined as the percentage of measurements made that are judged to be valid according to specific criteria and entered into the data management system. Accidental or inadvertent loss of samples during transport or lab activities should be avoided because the loss of the original samples will result in irreparable loss of data. Lack of data entry into the database will reduce the ability to perform analyses, integrate results, and prepare reports. Thus, controlling sample loss by using unbreakable containers, careful sample management (e.g., assigning serial laboratory numbers, completing log books), and tracking samples through analysis and data entry is important. Percent completeness (%C) for measurement parameters can be defined as follows:

where v is the number of measurements judged valid and T is the total number of measurements. Most monitoring programs should try to achieve a level of completeness in which no less than 95 percent of samples are judged to be valid.

Comparability. Two data sets are considered comparable when the two sets can be considered equivalent with respect to the measurement of a specific variable or group of variables. Comparability of data is not defined quantitatively; it is ensured by similarity in sampling based on geographic, seasonal, and method characteristics; the uniform training and experience of field sampling and laboratory personnel; and the use of standardized, repeatable methods for analysis of bacterial indicator densities. This document should help improve comparability among beach water quality monitoring programs by establishing comparable sampling and analysis procedures so that the meaning of the results can be more easily understood by the public nationwide.

Additional Factors Affecting Sampling Design. By establishing the "rules" for data quality at the planning stage, the number of samples that need to be collected and analyzed is adjusted to obtain data that will be used to judge the quality of the data obtained. For example, a duplicate water sample should be collected from at least 10 percent of the sites in the study to calculate precision. Under some conditions, more frequent collection of duplicate samples might be advised. Monitoring programs need to carefully balance their needs to sample from multiple areas and their resource limitations with the need for data quality. If only one sample is collected from every site for analysis, an agency might cover more territory, but it will not be able to detect errors during sampling, inadvertently reducing the density of bacteria in the sample or showing that the particular patch of water sampled contained an unusually high number of fecal bacteria, but was not representative of the entire area. Thus, inappropriate decisions might be made based on erroneous results.

For the same cost, the number of sites sampled could be reduced while including some QC samples to provide a means to double check the results, both from the field sampling effort and from analyses of duplicate aliquots of single samples in the laboratory. This approach can increase the level of confidence in the data produced and help detect unusual conditions that might lead to errors in decision making.

Top of page

---

3. References

1. Bartram, J., and G. Rees. 2000. Monitoring Bathing Waters: A Practical Guide to the Design and Implementation of Assessments and Monitoring Programmes. E & FN SPON, London.
2. Bordner, R., J.A. Winter, and P.V. Scarpino, eds. 1978. Microbiological Methods for Monitoring the Environment, Water and Wastes. EPA 600/8-78-017. U.S. Environmental Protection Agency, Washington, DC.
3. Brenniman, G.R., S.H. Rosenberg, and R.L. Northrop. 1981. Microbial sampling variables and recreational water quality standards. American Journal of Public Health 71(3):283-289.
4. Cabelli, V.J. 1983. Health Effects Criteria for Marine Recreational Waters. EPA 600/1-80-03. U.S. Environmental Protection Agency, Cincinnati, OH.
5. CADHS. 1999. Health and Safety Code Section 115875-115915. California Department of Health Services, Sacramento, CA.
6. Dufour, A.P. 1984. Health Effects Criteria for Fresh Recreational Waters. EPA 600/1-84-004. U.S. Environmental Protection Agency, Cincinnati, OH.
7. Fleisher, J.M. 1990. The effects of measurement error on previously reported mathematical relationships between indicator organism density and swimming-associated illness: A quantitative estimate of the resulting bias. International Journal of Epidemiology 19(4):1100-1106.
8. Gilbert, R.O. 1987. Statistical Methods for Environmental Pollution Monitoring. Van Nostrand Reinhold Company, New York, NY.
9. IITF. 1999. Standard Operating Procedure for Recreational Water Collection and Analysis of E. coli on Streams, Rivers, Lakes and Wastewater. Indiana Interagency Task Force on E. coli. LaPorte County Health Department, Laporte, IN.
10. NRC. 1990a. Monitoring Troubled Waters: The Role of Marine Environmental Monitoring. National Research Center. National Academy Press, Washington, DC.
11. NRC. 1990b. Monitoring Southern California's Coastal Waters. National Research Center. National Academy Press, Washington, DC.
12. USEPA. 1985. Test Methods for Escherichia coli and Enterococci in Water by the Membrane Filter Procedure. EPA 600/4-85-076. U.S. Environmental Protection Agency, Washington, DC.
13. USEPA. 1986. Ambient Water Quality Criteria for Bacteria 1986. U.S. Environmental Protection Agency, Office of Research and Development, Microbiology and Toxicology Division, and Office of Water Regulations and Standards, Criteria and Standards Division, Washington, DC.
14. USEPA. 1991. Monitoring Guidance for the National Estuary Program. EPA 503/8-91-002. U.S. Environmental Protection Agency, Office of Water, Washington, DC.
15. USEPA. 1997a. Method 1600: Membrane Filter Test Method for Enterococci in Water. EPA-821/R-97-004. U.S. Environmental Protection Agency, Office of Water, Washington, DC.
16. USEPA. 1997b. Monitoring Guidance for Determining the Effectiveness of Nonpoint Source Controls. EPA 841/B-96-004. U.S. Environmental Protection Agency, Office of Water, Washington, DC.
17. USEPA. 1998. EPA Guidance for Quality Assurance Project Plans, EPA QA/G-5. EPA/600-R-98-018. U.S. Environmental Protection Agency, Office of Research and Development, Washington, DC.
18. USEPA. 2000. Guidance for the Data Quality Objectives Process, EPA QA/G-4. EPA 600/R-96-055. U.S. Environmental Protection Agency, Office of Environmental Information, Washington, DC.
19. USEPA. 2001a. Guidance for Preparing Standard Operating Procedures (SOPs), EPA QA/G-6. EPA 240/B-01-004. U.S. Environmental Protection Agency, Office of Environmental Information, Washington, DC.
20. USEPA. 2001b. EPA Requirements for Quality Assurance Project Plans, EPA QA/R-5. EPA 240/B-01-003. U.S. Environmental Protection Agency, Office of Environmental Information, Washington, DC.
21. USEPA. 2001c. Office of Water Quality Management Plan. EPA 800/R-95-001. July, 2001. U.S. Environmental Protection Agency, Office of Water, Washington, DC.

22. Top of page