United States Office of Research and EPA/600/R-02/057 Environmental Protection Development September 2002 Agency Washington, D.C. 20460 4>EPA Environmental Technology Verification Report Lead in Dust Wipe Measurement Technology Palintest Scanning Analyzer SA-5000 System on\l Oak Ridge National Laboratory ------- THE ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM ET AM^ V r/EPA oml Oak Ridge National Laboratory Verification Statement TECHNOLOGY TYPE: APPLICATION: TECHNOLOGY NAME: COMPANY: ADDRESS: WEB SITE: E-MAIL: ANODIC STRIPPING VOLTAMMETRY MEASUREMENT OF LEAD IN DUST WIPES Scanning Analyzer SA-5000 System Palintest 21 Kenton Lands Road Erlanger, KY 41018 www.palintestusa.com info@palintestusa.com PHONE: (800)835-9629 FAX: (859) 341-2106 The U.S. Environmental Protection Agency (EPA) has created the Environmental Technology Verification Program (ETV) to facilitate the deployment of innovative or improved environmental technologies through performance verification and dissemination of information. The goal of the ETV Program is to further environmental protection by substantially accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-quality, peer-reviewed data on technology performance to those involved in the design, distribution, financing, permitting, purchase, and use of environmental technologies. ETV works in partnership with recognized standards and testing organizations and stakeholder groups consisting of regulators, buyers, and vendor organizations, with the full participation of individual technology developers. The program evaluates the performance of innovative technologies by developing test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance protocols to ensure that data of known and adequate quality are generated and that the results are defensible. Oak Ridge National Laboratory (ORNL) is one of the verification organizations operating under the Advanced Monitoring Systems (AMS) Center. AMS, which is administered by EPA's National Exposure Research Laboratory (NERL), is one of six technology areas under ETV. In this verification test, ORNL evaluated the performance of lead in dust wipe measurement technologies. This verification statement provides a summary of the test results for Palintest's Scanning Analyzer SA-5000 system. EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002 ------- VERIFICATION TEST DESCRIPTION This verification test was designed to evaluate technologies that detect and measure lead in dust wipes. The test was conducted at the Capitol Community Technical College in Hartford, CT, from November 5 through November 9, 2001. The vendors of commercially-available, field portable technologies blindly analyzed 160 dust wipe samples containing known amounts of lead, ranging in concentration from <2 to 1,500 |_lg/wipe. The experimental design was particularly focused on important clearance standards, such as those identified in 40 CFR Part 745.227(e)(8)(viii) of 40 |_lg/ft2 for floors, 250 |_lg/ft2 for window sills, and 400 |-lg/ft2 for window troughs. The samples included wipes newly-prepared and archived from the Environmental Lead Proficiency Analytical Testing Program (ELPAT). These samples were prepared from dust collected in households in North Carolina and Wisconsin. Also, newly-prepared samples were acquired from the University of Cincinnati (UC). The UC dust wipe samples were prepared from National Institute of Standards and Technology (NIST) Standard Reference Materials (SRMs). The results of the lead analyses generated by the technology were compared with results from analyses of similar samples by conventional laboratory methodology, in a laboratory that was recognized as proficient by the National Lead Laboratory Accreditation Program (NLLAP) for dust testing. Details of the test, including a data summary and discussion of results, may be found in the report entitled Environmental Technology Verification Report: Lead in Dust Wipe Detection Technology— Palintest, Scanning Analyzer SA-5000 System, EPA/600/R-02/057. TECHNOLOGY DESCRIPTION The Scanning Analyzer SA-5000 system uses the electrochemical technique of stripping analysis to specifically determine the concentration of lead in a solution. Anodic stripping analysis is a two step process. The first step is called the deposition step and involves the electro-deposition of lead on to a disposable mercury-film electrode. The deposition is achieved by cathodic deposition at a fixed potential and time. Following the fixed deposition time, the system enters the second step, the stripping or measurement step. The stripping step involves scanning the potential anodically using a potential-time waveform. During this anodic scan the deposited lead is reoxidized and stripped out of the electrode. The current and potential are measured during the anodic scan and the resulting voltammogram contains a peak whose potential is specific to lead and whose height is proportional to the concentration of lead in the solution. The peak height is converted from a current to a concentration using one of many calibration curves stored in the instrument. No user calibration is required because each batch of electrodes is checked during manufacture and assigned an eight figure calibration code. The calibration code is used to select the calibration curve which matches the electrode batch. Reporting limits during this verification test were 25 |_lg/wipe. VERIFICATION OF PERFORMANCE The following performance characteristics of the SA-5000 were observed: Precision: Precision, based on the average percent relative standard deviation (RSD), was 5% for the ELPAT samples and 8% for the UC samples. A technology's performance is considered very precise if the average RSD is less than 10%, but acceptable as long as the average RSD is less than 20%. Accuracy: Accuracy was assessed using the estimated concentrations of the ELPAT and UC samples. The number of results for the ELPAT samples that were reported within the acceptance ranges that have been established for those samples is one measure of accuracy. The SA-5000 reported results within the acceptance ranges for all 72 ELPAT samples (> 25 |_lg/wipe). The average percent recovery value (SA- 5000 reported result/estimated ELPAT concentration) for all samples reported above 25 |_lg/wipe was 91%. For the UC samples, the average percent recovery was 80%. This negative bias was statistically significant, but within the acceptable bias range of 100% ± 25%. For the NLLAP laboratory results, the EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002 ------- average percent recovery values were 98% and 88%, respectively, for the ELPAT and UC samples. The negative bias for both the ELPAT and UC samples was statistically significant. Comparability: A comparison of the SA-5000 results and the NLLAP-recognized laboratory results was performed for all samples (ELPAT and UC) that were reported above 25 |_lg/wipe. The correlation coefficient (r) for the comparison of UC samples was 1.000 [slope (m) = 0.839, intercept = 5.539], and for the ELPAT samples was 0.995 [m = 0.926, intercept = 6.506]. While the slopes for both the ELPAT and UC samples were statistically different than 1.00, the correlation coefficients show a strong linear agreement (i.e., r values greater than 0.990) with the NLLAP laboratory data. Detectable blanks: All twenty samples prepared at concentrations < 2 |_lg/wipe were reported correctly by the SA-5000 as < 25 |-lg/wipe. Performance was also assessed at concentrations near the reporting limits of the technology. Two sets of four ELPAT samples with estimated concentrations of 16.9 and 17.6 |-lg/wipe were all reported by the SA-5000 as < 25 |_lg/wipe. For the set of four ELPAT samples at 29.8 |_lg/wipe, the SA-5000 reported results between 28 and 32 |_lg/wipe. False positive results: A false positive (fp) result is one in which the technology reports a result that is above the clearance level when the true (or estimated) concentration is actually below. For the ELPAT samples, the SA-5000 did not produce any fp results out of a possible 12 results. For the UC samples, the SA-5000 did not produce any out of a possible 38 fp results. By comparison, the NLLAP laboratory did not report any fp results on the UC, but had 2 of!2 possible fp results on the ELPAT samples. False negative results: A false negative (fn) result is one in which the technology reports a result that is below the clearance level when the true (or estimated) concentration is actually above. For the ELPAT samples, the SA-5000 reported 17 of a possible 28 m results. For the UC samples, the SA-5000 reported 22 out of a possible 22 fn results. By comparison, the NLLAP laboratory reported 7 out of a possible 28 m results for the ELPAT samples, and 16 out of a possible 19 fn results for the UC samples. Completeness: Completeness is defined as the percentage of measurements that are judged to be usable (i.e., the result is not rejected). An acceptable completeness rate is 95% or greater. The SA-5000 generated results for all 160 dust wipe samples, for a completeness of 100%. Sample Throughput: Sample throughput is a measure of the number of samples that can be processed and reported by a technology in a given period of time. A single analyst (a Palintest expert) was able to prepare and analyze 80 samples per 12-hour day. EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002 ------- Overall Evaluation: The overall performance was characterized as having an acceptable amount of negative bias, very precise, and in good agreement with an NLLAP-recognized laboratory's results. The verification team found that the SA-5000 was relatively simple for the trained analyst to operate in the field, requiring less than an hour for initial setup. As with any technology selection, the user must determine if this technology is appropriate for the application and the project data quality objectives. Additionally, ORNL and ETV remind the reader that, while the ETV test provides valuable information in the form of a snapshot of performance, state, tribal, or federal requirements regarding the use of the technologies (such as NLLAP recognition for analysis of clearance samples where required) need to be followed. For more information on this and other verified technologies, visit the ETV web site at http: //www. epa. gov/etv. Gary J. Foley, Ph.D. Director National Exposure Research Laboratory Office of Research and Development W. Franklin Harris, Ph.D. Associate Laboratory Director Biological and Environmental Sciences Oak Ridge National Laboratory NOTICE: EPA verifications are based on evaluations of technology performance under specific, predetermined criteria and appropriate quality assurance procedures. EPA and ORNL make no expressed or implied warranties as to the performance of the technology and do not certify that a technology will always operate as verified. The end user is solely responsible for complying with any and all applicable federal, state, and local requirements. Mention of commercial product names does not imply endorsement or recommendation. EPA-VS-SCM-50 The accompanying notice is an integral part of this verification statement. September 2002 ------- EPA/600/R-02/057 September 2002 Environmental Technology Verification Report Lead in Dust Wipe Measurement Technology Palintest Scanning Analyzer SA-5000 System By Amy B. Dindal Charles K. Bayne, Ph.D. Roger A. Jenkins, Ph.D. Oak Ridge National Laboratory Oak Ridge, Tennessee 37831-6120 Eric N. Koglin U.S. Environmental Protection Agency Environmental Sciences Division National Exposure Research Laboratory Las Vegas, Nevada 89193-3478 ------- Notice The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development (ORD), funded and managed, through Interagency Agreement No. DW89937854 with Oak Ridge National Laboratory, the verification effort described herein. This report has been peer and administratively reviewed and has been approved for publication as an EPA document. Mention of trade names or commercial products does not constitute endorsement or recommendation for use of a specific product. ------- Table of Contents Notice ii List of Figures v List of Tables vi Acknowledgments vii Abbreviations and Acronyms viii Section 1 — Introduction 1 Section 2 — Technology Description 2 General Technology Description 2 Analytical Procedure 2 Section 3 — Verification Test Design 3 Objective 3 Testing Location and Conditions 3 Drivers and Objectives for the Test 3 Summary of the Experimental Design 3 University of Cincinnati Sample Description 4 Sample Randomization 5 Precision 6 Accuracy 6 Comparability 7 Detectable Blanks 7 False Positive/Negative Results 7 Completeness 8 Sample Throughput 8 Ease of Use 8 Cost 8 Miscellaneous Factors 8 Section 4 — Laboratory Analyses 9 Background 9 NLLAP Laboratory Selection 9 Laboratory Performance 10 Section 5 — Technology Evaluation 16 Objective and Approach 16 Precision 16 Comparability 17 False Positive/False Negative Results 18 Completeness 21 Sample Throughput 21 Ease of Use 21 Cost Assessment 21 iii ------- SA-5000 Costs 21 Labor 22 Equipment 22 Laboratory Costs 22 Sample Shipment 22 Labor, Equipment, and Waste Disposal 22 Cost Assessment Summary 23 Miscellaneous Factors 23 Summary of Performance 23 Section 6 — References 25 Appendix 26 IV ------- List of Figures 1. Palintest's Scanning Analyzer SA-5000 System 2 2. Distribution of sample concentrations 6 3. Plot of DataChem reported values versus estimated values, shown for concentrations < 500 |_lg/wipel 1 4. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 40 |_lg/wipe 13 5. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 250 |_lg/wipe 14 6. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 400 |_lg/wipe 14 7. Plot of the SA-5000 average concentration versus the DataChem average concentration for all samples (n=21), shown for concentrations < 500 |_lg/wipe 18 8. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target concentration level of 40 |_lg/wipe 19 9. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target concentration level of 250 |_lg/wipe 20 10. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target concentration of 400 |_lg/wipe 20 ------- List of Tables 1. Summary of DataChem Pre-Test Results 10 2. Summary of DataChem percent recovery values by sample source 11 3. Summary of DataChem precision estimates by sample source 11 4. False Positive/False Negative Error Rates for DataChem Measurements 13 5. Summary of the Linear Regression Constants and Recovery Data for DataChem's Measurements Versus the Estimated Concentrations at the Clearance Levels 15 6. Precision of the SA-5000 Analyzer 16 7. Accuracy of SA-5000 Analyzer 16 8. Linear regression constants for the plots of the SA-5000 versus the estimated values and versus the DataChem average measurements 17 9. False Positive/False Negative Error Rates for SA-5000 Measurements 19 10. Summary of the Linear Regression and Recovery Data for the SA-5000 Response versus the Estimated Concentrations 21 11. Estimated analytical costs for lead dust wipe samples 22 12. Performance Summary for the Scanning Analyzer SA-5000 System 24 VI ------- Acknowledgments The authors wish to acknowledge the support of all those who helped plan and conduct the verification test, analyze the data, and prepare this report. In particular, we recognize: Patricia Lindsey, Capitol Community Technical College, for providing a site for the verification test; Sandy Roda (University of Cincinnati) and Laura Hodson (Research Triangle Institute) for coordination of sample preparation; the inorganic analytical laboratory in EPA/Region 1 (North Chelmsford, MA) for the analysis of quality control samples; and the expertise of the technical advisory panel, including Kenn White (American Industrial Hygiene Association), John Schwemberger, Dan Reinhart, Oksana Pozda, and Darlene Watford (EPA/Office of Pollution Prevention and Toxics), Paul Carroll (EPA/Region 1), Sharon Harper (EPA/Research Triangle Park), Peter Ashley, Warren Friedman, Gene Pinzer, and Emily Williams (U.S. Department of Housing and Urban Development); Paul Halfmann and Sharon Cameron (Massachusetts Childhood Lead Poisoning Prevention Program), Kevin Ashley (National Institute for Occupational Safety and Health); Walt Rossiter and Mary McKnight (National Institute of Standards & Technology); Bill Gutknecht (Research Triangle Institute), and Bruce Buxton (Battelle Memorial Institute). The authors would specifically like to thank Kevin Ashley and John Schwemberger for serving as peer reviewers of this report. The authors also acknowledge the participation of Palintest, in particular, Steve Birch and David Miller. For more information on the Lead in Dust Wipe Measurement Technology Verification contact: Eric N. Koglin Roger A. Jenkins Project Technical Leader Program Manager Environmental Protection Agency Oak Ridge National Laboratory Environmental Sciences Division Chemical Sciences Division National Exposure Research Laboratory P.O. Box 2008 P.O. Box 93478 Oak Ridge, TN 37831-6120 Las Vegas, Nevada 89193-3478 (865) 574-4871 (702) 798-2332 jenkinsra@ornl.gov koglin.eric@epa.gov For more information on Palintest's Scanning Analyzer SA-5000 System, contact: David Miller Palintest USA 21 Kenton Lands Road Erlanger, KY41018 1-800-835-9629 info@palintestusa.com www.palintestusa.com ------- Abbreviations and Acronyms AIHA American Industrial Hygiene Association AMS Advanced Monitoring Systems Center, ETV ASTM American Society for Testing and Materials ASV Anodic Stripping Voltammetry CDC Centers for Disease Control and Prevention CFR Code of Federal Regulations CL Clearance level for lead (40, 250, or 400 |_lg/wipe) ELPAT Environmental Lead Proficiency Analytical Testing program EPA U. S. Environmental Protection Agency ETV Environmental Technology Verification Program ETVR Environmental Technology Verification Report fn false negative result fp false positive result ICP-AES Inductively coupled plasma-atomic emission spectrometry NERL National Exposure Research Laboratory, U.S. EPA NIOSH National Institute for Occupational Safety and Health, CDC NIST National Institute of Standards & Technology NLLAP National Lead Laboratory Accreditation Program OPPT Office of Pollution Prevention and Toxics, U.S. EPA ORNL Oak Ridge National Laboratory QA quality assurance QC quality control RSD relative standard deviation RTI Research Triangle Institute SD standard deviation SRM Standard Reference Material UC University of Cincinnati ------- Section 1 — Introduction The U.S. Environmental Protection Agency (EPA) created the Environmental Technology Verification Program (ETV) to facilitate the deployment of innovative or improved environmental technologies through performance verification and dissemination of information. The goal of the ETV Program is to further environmental protection by substantially accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing high-quality, peer-reviewed data on technology performance to those involved in the design, distribution, financing, permitting, purchase, and use of environmental technologies. ETV works in partnership with recognized standards and testing organizations and stakeholder groups consisting of regulators, buyers, and vendor organizations, with the full participation of individual technology developers. The program evaluates the performance of innovative technologies by developing verification test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer- reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA) protocols to ensure that data of known and adequate quality are generated and that the results are defensible. ETV is a voluntary program that seeks to provide objective performance information to all of the participants in the environmental marketplace and to assist them in making informed technology decisions. ETV does not rank technologies or compare their performance, label or list technologies as acceptable or unacceptable, seek to determine "best available technology," or approve or disapprove technologies. The program does not evaluate technologies at the bench or pilot scale and does not conduct or support research. Rather, it conducts and reports on testing designed to describe the performance of technologies under a range of environmental conditions and matrices. The program now operates six centers covering a broad range of environmental areas. ETV began with a 5-year pilot phase (1995-2000) to test a wide range of partner and procedural alternatives in various technology areas, as well as the true market demand for and response to such a program. In these centers, EPA utilizes the expertise of partner "verification organizations" to design efficient processes for conducting performance tests of innovative technologies. These expert partners are both public and private organizations, including federal laboratories, states, industry consortia, and private sector entities. Verification organizations oversee and report verification activities based on testing and QA protocols developed with input from all major stakeholder/customer groups associated with the technology area. The verification described in this report was administered by the Advanced Monitoring Systems (AMS) Center, with Oak Ridge National Laboratory (ORNL) serving as the verification organization. (To learn more about ETV, visit ETV's Web site at http://www.epa.gov/etv.) The AMS Center is administered by EPA's National Exposure Research Laboratory (NERL). The verification of a field analytical technology for measurement of lead in dust wipe samples is described in this report. The verification test was conducted in Hartford, Connecticut, from November 5 through November 9, 2001. The performance of the Palintest's Scanning Analyzer SA-5000 system was determined under field conditions. The technology was evaluated by comparing its results to estimated concentration values and with results obtained on similar samples using a recognized laboratory analytical method. For background information, reports on performance of this and other anodic stripping voltammetry systems can be found in other published reports (EPA, 1996, Ashley etal.,2001). ------- Section 2 — Technology Description In this section, the vendor (with minimal editorial changes by ORNL) provides a description of the technology and the analytical procedure used during the verification testing activities. General Technology Description The Scanning Analyzer SA-5000 system (Figure 1) uses the electrochemical technique of anodic stripping analysis to specifically determine the concentration of lead in a solution. Stripping analysis is a two step process. The first step is called the deposition step and involves the electro-deposition of lead on to a disposable mercury-film electrode. The deposition is achieved by cathodic deposition at a fixed potential and for a fixed FiS"re L palintest's , ., r-. Scanning Analyzer SA- length of time. _... „ ° J IT iT *u f j 5000 System. Following the faxed deposition time, the system enters the second step, the stripping or measurement step. The stripping step involves scanning the potential anodically using a potential-time waveform. During this anodic scan, the deposited lead is reoxidized and stripped off of the electrode. The current and potential are measured during the anodic scan and the resulting voltammogram contains a peak whose potential is specific to lead and whose height is proportional to the concentration of lead in the solution. The peak height is converted from a current to a concentration using one of many calibration curves stored in the instrument. No user calibration is required because each batch of electrodes is checked during manufacturing and assigned an eight figure calibration code. The calibration code is used to select the calibration curve which matches the electrode batch. Analytical Procedure The following is the procedure that was followed by Palintest during the verification test. 1. Place the dust wipe into a 50 mL sonicator tube. Using apipettor add 15 mL (3x5 mL) of 25% nitric acid to the sonicator tube. Use a new crushing rod to push the wipe down into the tube ensuring it is covered by the acid. Continue to push the wipe into the acid solution until trapped air bubbles in the wipe have been released. 2. Place the tube in the ultrasonicator. Fill the ultrasonicator with warm water (45-50 °C) so that the level of water in the sonicator is at least 1 cm above the level of liquid in the tube. 3. Sonicate the tube for 30 minutes then remove the tube and place in a rack. 4. Take the same crushing rod as previously used and repeat the mixing of the wipe in the tube. Replace the tube in the ultrasonicator and sonicate for an additional 15 minutes. 5. Remove the cap and carefully add deionized or distilled water to the 50 mL mark. Using the same crushing rod as previously used, mix the wipe and solution to ensure complete distribution of the extract. Replace the cap and mix well by shaking. 6. Take a 5 mL screw capped test tube and pour a portion of the solution into the tube filling to the 5 mL mark. 7. Add one SoluPrep SP-B tablet, crush and mix until completely dissolved. 8. Test the sample with the scanning analyzer. Switch on the instrument. Select Dust from the menu. Key in the correct calibration code shown on the electrode pack. Open the foil strip containing an electrode and insert into the connector. Insert the electrode into the sample. The instrument automatically starts the test and the result is displayed after 45 seconds. ------- Section 3 — Verification Test Design Objective The purpose of this section is to describe the verification test design. It is a summary of the test plan (ORNL, 2001). Testing Location and Conditions The verification of field analytical technologies for lead in dust wipes was conducted at the Capitol Community Technical College in Hartford, Connecticut. The test was conducted in the basement of a classroom building. The temperature and relative humidity were monitored during field testing, but remained fairly constant. The average temperature and relative humidity over the four days of testing were 68 °F and 32%, respectively. Drivers and Objectives for the Test The purpose of this test was to evaluate the performance of field analytical technologies that are capable of analyzing dust wipe samples for lead contamination. This test provides information on the potential applicability of field technologies to EPA standards for dust clearance testing. The experimental design was designed around the three clearance standards of 40 |-lg/ft2 for floors, 250 |~lg/ft2 for window sills, and 400 |-lg/ft2 for window troughs that are outlined in 40 CFR Part 745.227(e)(8)(viii) (CFR, 2001). The primary objectives of this verification were to evaluate the field analytical technologies in the following areas: (1) how well each performs relative to a conventional, fixed-site analytical method for the analysis of dust wipe samples for lead; (2) how well each performs relative to results generated in previously rounds of ELPAT testing (described in the next section), and (3) the logistical and economic resources necessary to operate the technology. Secondary objectives for this verification were to evaluate the field analytical technology in terms of its reliability, ruggedness, cost, range of usefulness, sample throughput, data quality, and ease of use. Note that this verification test does not provide an assessment of the selection of locations for dust samples in a facility or an assessment of the way that dust samples are collected. The planning for this verification test follows the guidelines established in the data quality objectives process. Summary of the Experimental Design All of the samples analyzed in this verification test were prepared gravimetrically. At the time of the test, both of the wipes utilized in the test (PaceWipe™ and Palintest Dust Wipe™) were on the list of wipes recommended for lead testing by the American Society for Testing and Materials (ASTM, 1996). Initial consideration was given to conducting the test in a real-world situation, where the technologies would have been deployed in a housing unit that had been evacuated due to high levels of lead contamination. In addition to the safety concern of potentially subjecting participants to lead exposure, the spatial variability of adjacent samples would have been expected to be so great that it would be much larger than the anticipated variability of these types of technologies, thereby making it difficult to separate instrument/method variability and sampling variability. The availability of we 11-characterized samples derived from "real- world" environments made the use of proficiency testing samples (so-called "ELPAT" samples) and other prepared samples an attractive alternative. ELPAT and Blank Sample Description In 1992, the American Industrial Hygiene Association (AIHA) established the Environmental Lead Proficiency Analytical Testing (ELPAT) program. The ELPAT Program is a cooperative effort of the American Industrial Hygiene Association (AIHA), and researchers at the Centers for Disease Control and Prevention (CDC), National Institute for Occupational Safety and Health (NIOSH), and the EPA Office of Pollution Prevention and Toxics (OPPT). The ELPAT program is designed to assist laboratories in improving their analytical performance, and therefore, does not specify use of a particular analytical method. Participating laboratories are sent samples to analyze on a quarterly basis. The reported values must fall within a range of acceptable values in order for the laboratory to be deemed proficient for that quarter. Research Triangle Institute (RTI) in Research Triangle Park, NC, is contracted to prepare and distribute the lead-containing paint, soil, and dust ------- wipe ELPAT samples. For the rounds of testing which have occurred since 1992, archived samples are available for purchase. Some of these samples were used in this verification test. Because the samples have already been tested by over one- hundred laboratories, a certified concentration value is supplied with each sample. This certified value represents a pooled measurement of all of the results submitted, with outliers excluded from the calculation. The following description, taken from an internal RTI report, briefly outlines how the samples were prepared. RTI developed a repository of real-world housedust, collected from multiple homes in the Raleigh/Durham/Chapel Hill area, as well as from an intervention project in Wisconsin. After collection, the dust was sterilized by gamma irradiation, and sieved to 150 |_lm. A PaceWipe™ was prepared for receiving the dust by opening the foil pouch, removing the wet folded wipe and squeezing the excess moisture out by hand over a trash can. The wipe was then unfolded and briefly set on a Kimwipe™ to soak up excess moisture. The PaceWipe was then transferred to a flat plastic board to await the dust. After weighing a 0.1000 ± 0.0005 g portion of dust on weighing paper, the pre- weighed dust was gently tapped out onto the PaceWipe. The wipe was then folded and placed in a plastic vial, which was then capped. All vials containing the spiked wipes were stored in a cold room as a secondary means of retarding mold growth until shipment. Before use in the ELPAT program, RTI performed a series of analyses to confirm that the samples were prepared within the quality guidelines established for the program. The data quality requirements for the ELPAT samples were: 1) the relative standard deviation of the samples analyzed by RTI must be 10% or less; 2) the measured concentrations must be within 20% of the target value that RTI was intending to prepare; and 3) analysis by an accredited laboratory must yield results within ± 20% of the RTI result. Ten samples were analyzed by RTI and nine samples were sent to the Wisconsin Occupational Health Laboratory for independent, confirmatory analysis. All ELPAT samples used in this test met the data quality requirements described above. The estimated concentration for an ELPAT sample used in this evaluation was the certified ("consensus") value (i.e., an analytically derived result). RTI prepared the blank samples using the same preparation method as the ELPAT samples, but the concentration of lead was less than 2 |_lg/wipe, well below the expected reporting limits of the participant technologies. University of Cincinnati Sample Description The ELPAT samples consisted of dust mounded in the center of a PaceWipe. The University of Cincinnati (UC) prepared "field QC samples" where the dust was sprinkled over the wipe, more similar to how a wipe would look when a dust wipe sample is collected in the field. The sample was prepared by weighing, so the concentrations can be estimated. In a typical scenario, UC sends these control samples to a laboratory along with actual field-collected samples as a quality check of the laboratory operations. Because the samples are visually indistinguishable from an actual field sample, are prepared on the same wipe, and are shipped in the same packaging, the laboratory blindly analyzes the control samples. This provides the user with an independent assessment of the quality of the laboratory's data. A cluster of twenty UC samples prepared at the key clearance levels were added to the experimental design, primarily so that an abundance of data would exist near the clearance levels, in order to assess false positive and false negative error rates. For Palintest, the UC samples were prepared on their own Palintest DustWipes™. The UC wipe samples were prepared using National Institute of Standards & Technology (NIST) Standard Reference Materials (SRMs). NIST SRM 2711 was used to prepare the 40 |_lg/wipe samples, and NIST SRM 2710 was used to prepare the 250 and 400 |-lg/wipe samples. Both SRM 2711 and SRM 2710 are Montana Soil containing trace concentrations of multiple elements, including lead. Some NIST SRM materials that are spiked on dust wipes are known to have low extraction recoveries when prepared by standard analytical methods (e.g., lead silicates cannot be extracted unless hydrofluoric acid is used) (Ashley et al., 1998). These particular SRMs are not known to contain lead silicates or to give lower lead recoveries. However, it is important to note the ------- possibility of such when using NIST SRMs for lead dust wipe analysis, since similar SRMs (e.g., Buffalo river sediment from Wyoming) do show recoveries in the low 90% range (Ashley et al., 1998). Because accurate and precise estimated concentrations for the UC samples were imperative, ORNL imposed the following data quality requirements for the UC-prepared wipe samples: 1) each estimated concentration had to be within a ± 10% interval of the target clearance level; 2) additional quality control (QC) samples (at least 5% of the total samples ordered) were to be prepared and analyzed by UC as a quality check prior to shipment of the samples; and 3) the relative standard deviation of the QC samples had to be < 10%. It is important to note here the reason why the data quality requirements between the UC and ELPAT samples were different. The data quality requirements for the ELPAT samples (i.e., ± 20% of the target value) were established by the ELPAT program. Since archived samples were being used, those data quality requirements could not be changed. As a quality check of the sample preparation process, UC prepared an additional nine samples (5% of the total number ordered). UC extracted and analyzed the samples following internal procedures (nitric/hydrochloric acid extraction, followed by atomic absorption spectrometry - see EPA, 1996 for Methods 3 05 OB and 601 OB) and provided those results to ORNL. For the nine samples (three at each of the three clearance levels), the average percent recovery (i.e., UC measured concentration/UC estimated concentration x 100%) was 97% (median value = 96%, standard deviation = 3%, range = 94% to 103%). Additionally, 18 randomly-selected samples (six at each of the three clearance levels) were analyzed by the EPA Region 1 laboratory in North Chelmsford, MA, as an independent quality control check of the accuracy and precision of UC's sample preparation procedure (nitric acid digestion followed by ICP/AES analysis - EPA 1996). The average percent recovery (EPA Region 1 reported concentration/UC estimated concentration x 100%) was 91% (median 90%, standard deviation = 3%), with a range of values from 86% to 97%. The average recovery determined from the EPA Region 1 analyses (91%) was lower than that which was determined by UC (97%), but both values were within the data quality requirement of 100 ± 10%. Based on these data, ORNL determined that the UC sample preparation process met the established data quality criteria and was deemed acceptable for use in the determination of false positive/false negative error rates. Distribution and Number of Samples A total of 160 samples were analyzed in the verification test. Figure 2 is a plot containing the distribution of the sample concentrations that were analyzed in this study. Twenty samples were prepared by the University of Cincinnati at +/- 10% of each of the three clearance levels (3 test levels x 20 samples = 60 samples total). Research Triangle Institute prepared 20 "blanks" at lead concentrations < 2 |_lg/wipe. These samples are noted as such in Figure 2. The remaining samples in Figure 2 are ELPAT samples. For most of the ELPAT samples, four samples were analyzed at each concentration level (16 test levels x 4 samples each = 64 samples total). There were two concentration levels (at 49 and 565 |_lg/wipe) where eight samples were analyzed. While the set of samples at each concentration level were prepared using homogeneous source materials and an identical preparation procedure, ELPAT samples cannot be considered true "replicates" because each sample was prepared individually. However, these samples represent four samples prepared similarly at a specified target concentration, with an estimated value calculated from more than 100 analyses of similarly prepared samples. Sample Randomization The samples were packaged in 20-mL plastic scintillation vials and labeled with a sample identifier. Each participant received the same suite of samples, but in a randomized order. The samples were distributed in batches of 16. Completion of chain-of-custody forms documented sample transfer. ------- 25 n Clearance Levels at 40, 250, 400 ug/wipe 20 — "E. S cs ^ St. o L. £ I 15 — 10 — 5-- 'in Jk c _ro 1- 00 0 ro O j <3- ^ - CD CD in 00 00 i- T — CM CO s •\l CD ro M O ) in .n •\i CO •N p r? CD oo in in ro <-> 0 T- CD 0 00 ^ ^f ^f in oo ^3- 0 m CM v o in CM o o Estimated Concentrations of Test Samples (ug/wipe) Figure 2. Distribution of sample concentrations. Description of Performance Factors In Section 5, technology performance is described in terms of precision, accuracy, completeness, and comparability, which are indicators of data quality (EPA, 1996). False positive and negative results, sample throughput, and ease of use are also described. Each of these performance characteristics is defined in this section. Precision Precision is the reproducibility of measurements under a given set of conditions. Standard deviations estimated at each concentration level can be used to establish the relationship between the uncertainty and the average lead concentration. Standard deviation (SD) and relative standard deviation (RSD) for "replicate" results are used to assess precision, using the following equation: RSD = (SD/average concentration) * 100% . (Eq. 1) The overall RSD is characterized by two summary values: mean — i.e., average; range — i.e., the highest and lowest RSD values that were reported. The average RSD may not be the best representation of precision, but it is reported for convenient reference. An average RSD value less than 10% indicates that the measurements are very precise. RSDs greater than 20% should be viewed as indicators of larger variability and possibly non- normal distributions. The uncertainty in the analytical measurements will include influences from both the preparation (i.e., extraction) and measurement steps. Accuracy Accuracy is a measure of how close the measured lead concentrations are to estimated values of the true concentration. The estimated values for the ELPAT samples are the certificate values that are reported on the certificate of analysis sheet provided with the samples. The ELPAT certified values represent an average concentration determined by more than 100 accredited laboratories that participated in previous rounds of ELPAT testing. The UC estimated value is the concentration reported by UC for individual samples, calculated by the amount of NIST-traceable material loaded on the dust wipes. The accuracy and precision of the UC value was assessed by an independent laboratory analyzing randomly selected QC samples. An EPA laboratory in Region 1 analyzed 10% of the total number of samples prepared by UC at each of ------- the three concentration levels and confirmed that the process used to prepare the samples met the pre- determined data quality objective of accuracy within a ± 10% interval of the estimated value. Accuracy of the field technology measurements was statistically tested using t-tests or non-parametric tests at the 5% significance level. These statistical tests compared the average results with the overall estimated values using the precision of the sample measurements. Bias was quantified by computing the percent recovery for four similar samples or a single sample using the equation: percent recovery = [measured amount(s)/estimated value] x 100% (Eq. 2) Accuracy was assessed using both the ELPAT and UC estimated concentrations. The comparison to the ELPAT value represents how close the technology reported results to the consensus value, which represents the amount of "recoverable" lead in the sample. Because the UC samples were prepared gravimetrically from samples of known lead content, the comparison to the UC samples represents how close the technology reported results to an absolute lead value. Comparison to the gravimetric values reveals any bias imposed by the tested sampling and analytical method. The optimum percent recovery value is 100%. Percent recovery values greater than 125% indicate results that are biased high, and values less than 75% indicate results that are biased low. A small but statistically significant bias may be detectable for a field technology if precision is high (i.e., low standard deviation). The field technology can still have acceptable bias with an average percent recovery in the interval of 75% to 125%. Bias within the acceptable range can usually be corrected to 100% by modification of calibration methods. Comparability Comparability refers to how well the field tech- nology and conventional laboratory data agree. The difference between accuracy and comparability is that accuracy is judged relative to a known value, comparability is judged relative to the results of a laboratory procedure, which may or may not report the results accurately. Because true "replicates" were not available for use in this study, the averages from similar samples measured by the technology were compared with corresponding averages measured by the laboratory for all target concentration levels. A correlation coefficient quantifies the linear relationship between two measurements (Draper and Smith, 1981). The correlation coefficient is denoted by the letter r; its value ranges from -1 to +1, where 0 indicates the absence of any linear relationship. The value r = -1 indicates a perfect negative linear relation (one measurement decreases as the second measurement increases); the value r = +1 indicates a perfect positive linear relation (one measurement increases as the second measurement increases). Acceptable r values are 0.990 or greater. The slope of the linear regression line, denoted by the letter m, is related to r. Whereas r represents the linear association between the vendor and laboratory concentrations, m quantifies the amount of change in the vendor's measurements relative to the laboratory's measurements. A value of+1 for the slope indicates perfect agreement. Values greater than 1 indicate that the vendor results are generally higher than the laboratory, while values less than 1 indicate that the vendor results are usually lower than the laboratory. Detectable Blanks Twenty samples in the test were prepared at < 2 |J,g/wipe, below the anticipated reporting limits of both the field technologies and the laboratory. Any reported lead for these samples is considered a "detectable blank". Performance was also assessed at concentrations near the reporting limits of the technology. False Positive/Negative Results A false positive (fp) result is one in which the technology detects lead in the sample above a clearance level when the sample actually contains lead below the clearance level (Keith et al., 1996). A false negative (fh) result is one in which the technology indicates that lead concentrations are less than the clearance level when the sample actually contains lead above the clearance level. For example, if the technology reports the sample concentration to be 35 |_lg/wipe, and the true concentration of the sample is 45 |_lg/wipe, the technology's result would be considered a fh at the 40 |_lg/wipe clearance level. Accordingly, if the technology reports the result as 45 |_lg/wipe and the true concentration is 35 |_lg/wipe, the technology's result would be a fp at the 40 |_lg/wipe clearance level. ------- A primary objective for this verification test was to assess the performance of the technology at each of the three clearance levels of 40, 250, and 400 |J,g/wipe, and estimate the probability of the field technology reporting a fp or fn result. For each clearance level, the probabilities of fn were estimated as curves that depend on a range of concentrations reported about the clearance level. These error probability curves were calculated from the results on the 60 UC samples at concentrations ± 10% of each clearance level. In order to generate probability curves to model the likelihood of false negative results, it was assumed that the estimated concentration provided by UC was the true concentration. However, this evaluation did not include the gravimetric preparation uncertainty in the UC estimated concentration. This error is likely to be much smaller than other sources of measurement error (e.g., extraction efficiency and analytical). The fp/fn evaluation also included a comparison to the ELPAT sample results. The "estimated" value for the UC and ELPAT samples are defined differently. The UC value is based on weight of the NIST-traceable material, while the ELPAT estimated value is the average analytical reported value from more than 100 accredited laboratories. The UC sample estimated lead content is determined gravimetrically, which should be closer to the "true" concentration than an analytical measurement that includes preparation and instrumental errors. In contrast, determining the technology's fp/fn error rates relative to the ELPAT estimated concentrations represents a comparison to typical laboratory values. One limitation of using the ELPAT sample is that concentrations covered a wider overall distribution of lead levels. Thus, the availability of sample concentrations that were tightly (i.e., +/- 10%) clustered about the clearance levels was limited. In order to perform a broader fp/fn analysis, the range of lead levels in the ELPAT samples that bracketed the pertinent clearance levels was extended to ± 25% of the target concentration. Completeness Completeness is defined as the percentage of measurements that are judged to be usable (i.e., the result is not rejected). An acceptable completeness rate is 95% or greater. Sample Throughput Sample throughput is a measure of the number of samples that can be processed and reported by a technology in a given period of time. Sample throughput is reported in Section 5 as number of samples per day per number of analysts. Ease of Use A significant decision factor in purchasing an instrument is how easy the technology is to use. Several factors are evaluated and reported on in Section 5: • What is the required operator skill level (e.g., technician or advanced degree)? • How many operators were used during the test? • Could the technology be run by a single person? • How much training would be required in order to run this technology? • How much subjective decision-making is required? Cost An important factor in the consideration of whether to purchase a technology is cost. Costs involved with operating the technology and a typical laboratory analyses are estimated in Section 5. To account for the variability in cost data and assumptions, the economic analysis is presented as a list of cost elements and a range of costs for sample analysis. Several factors affect the cost of analysis. Where possible, these factors are addressed so that decision makers can independently complete a site- specific economic analysis to suit their needs. Miscellaneous Factors Any other information that might be useful to a person who is considering purchasing the technology is documented in Section 5 under "Observations". Examples of information that might be useful to a prospective purchaser are the amount of hazardous waste generated during the analyses, the ruggedness of the technology, the amount of electrical or battery power necessary to operate the technology, and aspects of the technology or method that make it user-friendly or user-unfriendly. ------- Section 4 — Laboratory Analyses Background EPA regulations (40 CFRPart 745.227(e)(8)(vii)) specify that residences and child occupied facilities built before 1978 that have undergone an abatement must pass clearance testing (CFR 2001). These EPA regulations also state in 40 CFR Part 745.227(f)(2) that dust samples for clearance must be analyzed by a laboratory recognized by EPA (CFR 2001). Many EPA-authorized state and tribal lead programs have the same or similar requirements. EPA's vehicle for recognizing laboratory proficiency is the National Lead Laboratory Accreditation Program (NLLAP). Although the NLLAP was initially designed to accredit fixed site laboratories, in August 1996 the NLLAP was modified so that mobile laboratory facilities and testing firms operating portable testing technologies could also apply for accreditation. Despite this modification, the NLLAP list of accredited laboratories has almost exclusively consisted of fixed site laboratories. One possible outcome of this ETV test is that more mobile laboratory facilities and testing firms operating portable testing technologies will apply for NLLAP accreditation. In order to assess whether the field portable technologies participating in this verification test produce results that are comparable to NLLAP-recognized data, an NLLAP-recognized laboratory was selected to analyze samples concurrently with the field testing. NLLAP Laboratory Selection NLLAP was established by the EPA Office of Pollution Prevention and Toxics under the legislative directive of Title X, the Lead-Based Paint Hazard Reduction Act of 1992. In order for laboratories to be recognized under the NLLAP, they must successfully participate in the ELPAT Program and undergo a systems audit. The acceptable range for the ELPAT test samples is based upon the reported values from participating laboratories. Acceptable results are within three standard deviations from the consensus value. A laboratory's performance is rated as proficient if either of the following criteria are met: (1) in the last two rounds, all samples are analyzed and the results are 100% acceptable; or (2) three-fourths (75%) or more of the accumulated results over four rounds are acceptable. The NLLAP required systems audit must include an on-site evaluation by a private or public laboratory accreditation organization recognized by NLLAP. Some of the areas evaluated in the systems audit include laboratory personnel qualifications and training, analytical instrumentation, analytical methods, quality assurance procedures, and record keeping procedures. The list of recognized laboratories is updated monthly. ORNL obtained the list of accredited laboratories in July 2001. The list consisted of approximately 130 laboratories. Those laboratories which did not accept commercial samples and those located on the U.S. west coast were automatically eliminated as potential candidates. ORNL interviewed at random approximately ten laboratories and solicited information regarding cost, typical turnaround time, and data packaging. Based on these interviews and discussions with technical panel members who had personal experience with the potential laboratories, ORNL selected DataChem (Cincinnati, OH) as the fixed- site laboratory. As a final qualifying step, DataChem blindly analyzed 16 samples (8 ELPAT and 8 prepared by UC) in a pre-test study. As shown in Table 1 below, DataChem passed the pre-test by reporting concentrations that were within 25% of the estimated concentration for samples above the reporting limit. Laboratory Method The laboratory method used by DataChem was hot plate/nitric acid digestion, followed by inductively coupled plasma-atomic emission spectrometry (ICP- AES) analysis. The preparation and analytical procedures, as supplied by DataChem, can be found in the test plan (ORNL, 2001). To summarize the procedure, the wipe was digested in 2 mL of nitric acid, heated in a hotblock for 1 hour at 95 °C, diluted to 20 mL with distilled water, and analyzed by ICP-AES. DataChem's procedures are modifications of Methods 3 05 OB and 601 OB of EPA SW-846 Method Compendium for the preparation and analysis of metals in environmental matrices (EPA, 1996). Other specific references for the preparation and analysis of dust wipes are available from the American Society for Testing and Materials (ASTM, 1998). ------- Table 1. Summary of DataChem Pre-Test Results Sample Type ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT UC UC UC UC UC UC UC UC DataChem Reported Cone (|_lg/wipe) <20 <20 41 44 190 210 440 450 <20 <20 25 38 150 200 250 310 Estimated Cone (|_lg/wipe) 2.12 2.12 41.3 41.3 201.6 201.6 408.7 408.7 10.3 5.9 29.9 44 172.4 237.5 327.3 379 Percent Recovery n/a n/a 99% 107% 94% 104% 108% 110% n/a n/a 84% 86% 87% 84% 76% 82% Analysis Order 16 12 6 3 15 9 2 13 4 1 14 10 11 7 5 8 Laboratory Performance ORNL validated all of the laboratory data according to the procedure described in the verification test plan (ORNL, 2001). During the validation, the following aspects of the data were reviewed: completeness of the data package, correctness of the data, correlation between "replicate" sample results, and evaluation of QC sample results. Each of these categories is described in detail in the verification test plan. An evaluation of the performance of the laboratory results through statistical analysis of the data was performed and is summarized below. (See Section 3 for a detailed description of how the performance factors are defined and the calculations that are involved.) In Table 2, DataChem's reported values are compared to the estimated values to determine percent recovery (i.e., accuracy of the DataChem results) for both the ELPAT and the UC samples. The results are also shown graphically in Figure 3. The average percent recovery for the ELPAT samples was 98%, while the average for the UC samples was 88%. Both Table 2 and Figure 3 indicate that the analytical results from the University of Cincinnati wipe samples were generally reported lower than the estimated value, while the results for the ELPAT samples were closer to the estimated value. The better agreement with the ELPAT samples is not unexpected, given that the ELPAT estimated concentrations represent analytical consensus values that include typical extraction inefficiencies and instrumental error. The negative bias observed with the UC and the ELPAT samples was statistically significant. The cause of the negative bias for the UC samples could be related to: 1) extraction inefficiencies (due to the use of NIST SRMs that contain lead that is unrecoverable with the extraction procedure which was used) and/or, 2) typical analytical variation due to preparation and measurement errors. Another indication of accuracy is the number of individual ELPAT results which were reported within the acceptance ranges that have been established for those samples. For the 72 ELPAT samples (> 20 |-lg/wipe), DataChem reported 71 (99%) within the acceptable ranges of values. The precision assessment presented in Table 3 indicates that the analyses were very precise. The average RSD for the ELPAT samples was 7%, while the average RSD for the UC samples was 8%. The variability of the UC sample preparation process, provided for reference of the minimal achievable RSD for the UC samples, was 6%. A single estimate of the ELPAT variability was not determined, since the ELPAT samples were comprised of 20 different batches of samples. DataChem reported all 20 detectable blank samples correctly as < 20 |-lg/wipe. In addition, DataChem reported seven of the eight samples with estimated concentrations of either 16.9 |-lg/wipe or 17.6 |-lg/wipe as less than their reporting limit of 20 |-lg/wipe and only one was incorrectly reported as 30 |-lg/wipe. 10 ------- Tabl e 2. Summary of DataChem percent recovery values by sam Statistic n a average % recovery std dev minimum % recovery maximum % recovery ELPAT 72 98 9 81 143 uc 60 88 5 74 96 1 excludes estimated values <20 |_lg/wipe(n=28) le source 500 -i 400 - 300 - g 200 H 4* § 100 H "8 0 n D ELPAT (n=72) • UnivofCinci(n=60) 100 200 300 400 Estimated value (ug/wipe) 500 Figure 3. Plot of DataChem reported values versus estimated values, shown for concentrations < 500 |_lg/wipe. Table 3. Summary of DataChem precision estimates by sample source Sample Source ELPAT UC UC preparation a Number of sample sets 18 a 3b 3C average RSD 7 8 6 Min RSD 2 6 5 Max RSD 21 9 7 " 4 replicates in each sample set b 20 replicates in each sample set c The value represents the variability in the sample preparation process. 11 ------- An important evaluation parameter for the analysis of dust wipe samples is how the method performs at the clearance levels and the method's likelihood of reporting false positive (fp) and false negative (fn) results. Recall from the experimental design that 20 UC samples were prepared at ± 10% of each clearance level of 40, 250, and 400 |_lg/wipe, for a total of 60 UC samples. The ELPAT samples covered a wider range of concentrations. There was a total of 40 ELPAT samples that fell within a ±25% interval of the target values that could be used for the fp/fn assessment. The number of false negative and false positive results reported by DataChem relative to the UC and ELPAT estimated concentrations is summarized in Table 4. There are a specific number of possible fp and fn results. For example, if the estimated lead level on the wipe is less than the clearance level (CL), then it is not possible to produce a false negative result; only a false positive (i.e., > 40) result is possible. For the UC samples, in every case where the estimated concentration was less than the CL, DataChem reported a result for that was also less than the CL, indicating no fp results at any of the three CL. DataChem reported two fp results for the ELPAT samples out of a possible 12. When the estimated concentration was above the clearance level, however, DataChem sometimes reported results as less than the clearance level. DataChem reported a higher rate of fn results for the UC samples than the ELPAT samples (16 of 19 vs 7 of 28 possible fn results, respectively). This finding is not surprising, since the results reported above indicated that DataChem's results were negatively biased, or reported lower than the estimated values for the UC samples. As stated in Section 3, it is important to note that in this evaluation, the estimated concentration of the UC samples is assumed to be the "true"concentration, and the uncertainty in gravimetric preparation for the UC estimated concentration is not considered in the evaluation. Figures 4, 5, and 6 show models of the likelihood of DataChem reporting a false negative result at each of the clearance levels versus the true concentrations of the UC samples. (Note that only the UC samples must be used in generation of probability curves because these estimated values are a closer representation of the true lead concentration than the ELPAT estimated concentration. See Song et al., 2001.) These figures indicate that the likelihood of DataChem reporting false negative results for the UC samples at the exact clearance level is high, near 100% in all three cases. This means, for example, that if DataChem reported a value as exactly 250 |-lg/wipe, the probability that the true concentration is >250 is essentially 100%. Again, this is due to the negative bias that was observed in the measurement of the UC samples. The plots also demonstrate that, due to the relatively high level of precision of results reported by DataChem, the performance is very minimally impacted by performing replicate analyses, as the distribution of false negative probabilities is very similar whether 1 or 5 measurements (in Figures 4, 5, and 6, delineated as N = 1, N = 2, etc.) are performed. The interpretation of these curves for use in a "real- world" situation can be demonstrated by the following example. Suppose that a user decides that an acceptable level of risk for having false negative results is 5%. Using Figure 4, 5% FN probability (y = 0.05) corresponds to a "true" lead concentration of 54 |_lg/wipe (meaning if the true concentration of the sample is 54 |_lg/wipe, there is only a 5% chance/risk that DataChem will report the value as < 40 |-lg/wipe.) By plotting DataChem's measured values versus the estimated concentrations, the equations of the linear regression lines can be calculated for each of the three CL. The slope, intercept, and correlation coefficient for the ELPAT and UC samples are presented in Table 5. The user might like to know at what reported value (and at what associated probability) will DataChem be likely to report a "clean" sample (i.e., there is a high probability that the true concentration is < CL). For example, for the UC samples, we know that a value reported by DataChem as 39 |_lg/wipe is biased low and will have a true concentration of > 40 (44.7 |_lg/wipe, using the linear regression equation in Table 5). A true concentration of 40 |_lg/wipe for a UC sample would correspond to a reported value rounded to the nearest whole number of 35 |_lg/wipe (see Table 5). For an ELPAT sample, a true concentration of 40 |J,g/wipe corresponds to a DataChem reported value of 40 |_lg/wipe, because the negative bias was not as large for the ELPAT samples. Estimates of the reported concentration at the 250 and 400 |_lg/wipe levels are reported in Table 5. In both cases, the reported concentrations for the ELPAT samples are higher (i.e., closer to the clearance level) than those of the UC samples. (Recall that the ELPAT estimated values are consensus values of more than 12 ------- 100 analytical measurements, while the UC estimated values are weighed values, so this finding is not surprising.) The user is reminded that the data obtained during this verification test represent performance at one point in time. The data produced by DataChem at some other time after the writing of this report may or may not be similar to what has been produced here. To understand a method's performance at critical clearance levels, it is recommended that the user perform their own assessment of the method's performance by including samples of known concentration (at or near the clearance levels) along with the analysis of "real-world" samples. Table 4. False Positive/False Negative Error Rates for DataChem Measurements Evaluation Parameter fp: # samples where DataChem reported the result as >CLa of the # samples where the estimated concentration was < CL fn: # samples where DataChem reported the result as < CL of the # samples where the estimated concentration was > CL Sample Source UC ELPAT UC ELPAT Number of Samples 40 |_lg/wipe Oof 15 Oof 4 4 of 5 lof 12 250 |J,g/wipe Oof 12 2 of 8 6 of 8 5 of 8 400 |J,g/wipe Oof 14 OofOb 6 of 6 lof 8 Total Oof 41 2 of 12 16 of 19 7 of 28 a CL = clearance level b Because all eight ELPAT values were above 400 |_ig/wipe, no samples were available to assess fp results at this level. True Pb Concentration (ug/wipe) Figure 4. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 40 |ig/wipe. 13 ------- 0.0 240 250 260 270 280 290 300 310 320 True Pb Concentration (ug/wipe) Figure 5. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 250 |ig/wipe. 400 410 420 430 440 450 460 470 480 True Pb Concentration (ug/wipe) Figure 6. False negative probabilities for DataChem reporting average concentrations at a target concentration level of 400 |ig/wipe. 14 ------- Table 5. Summary of the Linear Regression Constants and Recovery Data for DataChem's Measurements Versus the Estimated Concentrations at the Clearance Levels Evaluation Parameter n slope intercept correlation coefficient average % recovery SD of % recovery Reported concentration at theCL 40 |_lg/wipe UC 20 0.787 3.844 0.705 89% 6% 35 l-ig/wipe ELPAT 16 1.612 -6.182 0.840 101% 13% 40 l-ig/wipe 250 |J,g/wipe UC 20 0.912 -10.169 0.759 87% 5% 218 l-ig/wipe ELPAT 16 0.578 90.826 0.549 96% 9% 234 l-ig/wipe 400 |J,g/wipe UC 20 0.956 -26.826 0.916 89% 2% 355 l-ig/wipe ELPAT 8 2.394 -575.771 0.492 100% 5% 382 l-ig/wipe 15 ------- Section 5 — Technology Evaluation Objective and Approach The purpose of this section is to present a statistical evaluation of the SA-5000 data and determine the technology's ability to measure lead in dust wipe samples. This section includes an evaluation of comparability to NLLAP-recognized laboratory data. Other aspects of the technology (such as accuracy, precision, cost, sample throughput, hazardous waste generation, and logistical operation) are also evaluated in this section. The Appendix contains the raw data provided by the vendor during the verification test that were used to assess the performance of the SA-5000. Precision Precision is the reproducibility of measurements under a given set of conditions. Precision was determined by examining the results of blind analyses for replicate samples with estimated concentrations greater than the SA-5000's reporting limits (25 |_lg/wipe). For the ELPAT samples, precision was measured on each set of four samples from a particular round of ELPAT archives. For the 18 sets of samples, the SA-5000's average RSD value was 5%, with a range from 2 to 8%, indicating that the SA-5000 measurements of the ELPAT samples were very precise (see Table 6). For the UC samples, 20 samples were analyzed at each of three target concentration levels of 40, 250, and 400 |J,g/wipe. The average precision of the UC sample measurements by the SA-5000 was 8% RSD. With the expectation that UC was to prepare the samples as close to the target concentrations as possible, the allowable variability was 10% RSD. The actual variability of the UC preparation process was an average of 6% RSD. Accuracy Accuracy represents the closeness of the SA-5000's measured concentrations to the estimated content of spiked samples. One measure of accuracy is the number of results for the ELPAT samples that were reported within the acceptance ranges that have been established for those samples. The SA-5000 reported the results for all 72 ELPAT samples (> 25 |J,g/wipe) within the acceptance ranges. The results reported by the SA-5000 can also be compared to the ELPAT certificate value, i.e., the average Table 6. Precision of the SA-5000 Analyzer Source ELPAT UC UC prep c No. of sample sets 18 a 3b o 6 RSD, % Average 5 8 6 Min 2 8 5 Max 8 9 7 " 4 replicates in each sample set b 20 replicates in each sample set c precision of UC sample preparation process concentration reported by 100+ laboratories who participated in previous rounds of ELPAT testing. The average percent recovery of 91% reported by the SA-5000 for the 72 ELPAT samples indicates a statistically significant negative bias, but such is well within the acceptable bias limits of 100 ± 25% (Table 7). The recovery values range from 74 to 107%. The results for the UC samples were also biased low, with an average percent recovery of 80%, and a range of values from 65 to 95%. When comparing these results to those of the NLLAP laboratory that was presented in Table 2, a similar trend is observed, in that the UC sample results were, on average, biased 10% lower than the ELPAT results. The possible explanations for this difference in performance include: 1) that ELPAT "estimated" values are in fact consensus values from a large number of laboratories that may be similar in performance to DataChem and to the SA-5000; and 2) the reference material used to prepare the UC samples may be more challenging than the ELPAT reference material. Table 7. Accuracy of SA-5000 Analyzer Statistic na average % recovery standard deviation minimum % recovery maximum % recovery % recovery ELPAT 72 91 8 74 107 UC 60 80 7 65 95 1 Excludes estimated values < 25 |_ig/wipe 16 ------- Another way to assess accuracy is to plot the results obtained from the SA-5000 versus the estimated values that are > 25 |_lg/wipe. The linear regression constants for the plot of the ELPAT and UC data are listed in Table 8. As expected, the conclusions produced from this assessment are similar to the above conclusions regarding the percent recovery calculations. The UC samples were generally reported lower than the ELPAT samples relative to the estimated concentrations, but overall the sample results had an acceptable amount of bias. Comparability Comparability refers to how well the SA-5000 and the NLLAP-recognized laboratory data agreed. In this evaluation, the laboratory results are not presumed to be the "correct" answers. Rather, these results represent what a typical fixed laboratory would report for these types of samples. A direct comparison of the SA-5000 results and the laboratory results was performed for all ELPAT (> 25 |_lg/wipe) and UC samples. Because each wipe was prepared individually, a true one-to-one matching of SA-5000 and laboratory results could not be performed. However, the average concentrations of the samples prepared at specific levels was compared for the SA-5000 and laboratory results. In Table 8, the regression constants for the average SA-5000 results versus the average DataChem results for both the ELPAT and UC values are presented. The difference between the regression slopes (m = 0.926 for ELPAT and m = 0.839 for UC) and a slope with a perfect agreement line (m = 1.000) is statistically significant, but the correlation coefficients (r = 0.990 for ELPAT and r = 1.000 for UC) show a strong linear relationship between DataChem and SA-5000 results. To illustrate the strong linear agreement between the SA-5000 and NLLAP laboratory results, Figure 7 is a plot of the average SA-5000 results versus the average DataChem results for both ELPAT and UC data. For clarity, only those values < 500 |_lg/wipe are shown. Detectable Blanks Of the samples that were prepared at < 2 |_lg/wipe, the SA-5000 correctly reported all 20 as < 25 |J,g/wipe, so no detectable blanks were reported. Performance was also assessed at concentrations near the reporting limit of 25 |_lg/wipe. Two sets of four ELPAT samples with estimated concentrations of 16.9 and 17.6 |_lg/wipe were all reported by the SA-5000 as < 25 |_lg/wipe. For the set of four ELPAT samples at 29.8 |lg/wipe, the SA-5000 reported results between 28 and 32 |_lg/wipe. Table 8. Linear regression constants for the plots of the SA-5000 versus the estimated values and versus the DataChem average measurements Statistic n slope (standard error) intercept (standard error) r versus estimated values UC 60 0.754 (0.010) 3.466 (2.773) 0.995 ELPAT 72 0.916 (0.011) 3.757 (5.151) 0.995 versus DataChem average concentrations UC 3 0.839 (0.001) 5.539 (0.224) 1.000 ELPAT 18 0.926 (0.023) 6.506 (10.860) 0.995 17 ------- 500 n 400 - It 300 e 2°° H 100 0 0 100 200 300 400 DataChem average concentration (ug/wipe) 500 Figure 7. Plot of the SA-5000 average concentration versus the DataChem average concentration for all samples (n=21), shown for concentrations < 500 |ig/wipe. False Positive/False Negative Results Similar to the evaluation described and presented in Section 4 for DataChem, the number of false negative and false positive results reported by the SA-5000 relative to the estimated concentrations of both ELPAT and UC samples is summarized in Table 9. For every case where the estimated concentration was less than the clearance level (CL), the SA-5000 reported a result that was also less than the CL, indicating no fp results at any of the three CL for both ELPAT and UC samples. When the estimated concentration was greater than the clearance level, however, the SA-5000 reported many of the results as less than the clearance level. The SA-5000 had more false negative errors relative to the UC estimated concentrations (22 of 22 possible fn results) than when compared to the ELPAT estimated concentrations (17 of 28 possible fn results). This finding is not surprising, since the accuracy results reported above indicated that the SA-5000 results were more negatively biased for the UC samples. In Figures 8, 9, and 10, the false negative probabilities at the three clearance levels are compared for the DataChem and SA-5000 results for the UC samples only (see previous discussion on false negatives in Section 4). In these figures, the two-sided 90% confidence intervals (not shown for clarity) are used to express uncertainty on the false negative curves. These confidence intervals overlap for the SA-5000 and DataChem at the 40 |_lg/wipe and the 250 |-lg/wipe clearance levels over the range of true lead concentrations shown in the Figures 8 and 9. The overlapping confidence intervals indicate the two methods are comparable when considering their uncertainty. In Figure 10, the 90% confidence intervals for the two methods only overlap for part of the true lead concentration range (425 to 472 |-lg/wipe). This result indicates that the SA-5000 appears to be more prone to false negatives at the 400 |-lg/wipe clearance level, when considering the uncertainty of the two methods. Table 10 contains the linear regression constants for SA-5000 measured concentration versus estimated concentration for the three CLs, average percent recovery values and standard deviations, and an estimate of the reported SA-5000 concentration at the clearance levels for both the ELPAT and UC samples. The UC samples average recoveries indicate that the SA-5000 results were more negatively biased for the 250 and 400 |-lg/wipe levels than for the 40 |-lg/wipe level. This is also apparent in the estimated concentration that a user might require from the SA-5000 in order to be reasonably confident that the true result is below the clearance level. For the UC samples, both DataChem and the SA-5000 results reported as 35 |-lg/wipe correspond to a true concentration of 40 |J,g/wipe, but the reported values at the CLs for the 250 and 400 |-lg/wipe clearance levels are lower for 18 ------- the SA-5000 (189 and 308 |_lg/wipe) than for DataChem (218 and 355 |_lg/wipe). The SA-5000 ELPAT sample results were negatively biased relative to the estimated concentrations, but not as low as the UC samples, so the estimates of a reported concentration at the CL are slightly higher for the ELPAT samples. Regardless of analytical technique, there is some uncertainty in assessing false positive and false negative error rates around critical action levels due to "normal" levels of variability (Song et al, 2001). Analytical values falling near the level of interest should be interpreted with care for both fixed-laboratory and field-based analytical methods. Table 9. False Positive/False Negative Error Rates for SA-5000 Measurements Evaluation Parameter fjp: # samples where SA-5000 reported the result as > CLa of the # samples where the estimated concentration was < CL fn: # samples where SA-5000 reported the result as < CL of the # samples where the estimated concentration was > CL Sample Source UC ELPAT UC ELPAT Number of Samples 40 |_lg/wipe Oof 14 Oof 4 6 of 6 4 of 12 250 |J,g/wipe Oof 11 Oof 8 9 of 9 8 of 8 400 |J,g/wipe Oof 13 OofOb 7 of 7 5 of 8 Total Oof 38 Oof 12 22 of 22 17 of 28 a CL = clearance level b Because all eight ELPAT values were above 400 |_ig/wipe, no samples were available to assess fp results at this level. True Pb Concentration (ug/wipe) Figure 8. Comparison of the false negative probabilities for the SA- 5000 and DataChem at a target concentration level of 40 |ig/wipe. 19 ------- s. 225 250 275 300 325 350 375 400 425 450 True Pb Concentration (ug/wipe) Figure 9. Comparison of the false negative probabilities for the SA-5000 and DataChem at a target concentration level of 250 |ig/wipe. True Pb Concentration (ug/wipe) Figure 10. Comparison of the false negative probabilities for the SA- 5000 and DataChem at a target concentration of 400 |ig/wipe. 20 ------- Table 10. Summary of the Linear Regression and Recovery Data for the SA-5000 Response versus the Estimated Concentrations Evaluation Parameter n slope intercept correlation coefficient average % recovery SD of % recovery Reported concentration atCL 40 |_lg/wipe UC 20 0.652 8.880 0.694 88% 5% 35 l-ig/wipe ELPAT 16 0.559 13.791 0.904 90% 9% 36 l-ig/wipe 250 |J,g/wipe UC 20 0.764 -1.898 0.616 76% 5% 189 l-ig/wipe ELPAT 16 0.570 78.537 0.697 90% 7% 221 l-ig/wipe 400 |J,g/wipe UC 20 0.883 -44.860 0.780 77% 4% 308 l-ig/wipe ELPAT 8 1.170 -95.766 0.323 94% 4% 372 l-ig/wipe Completeness Completeness is defined as the percentage of measurements that are judged to be usable (i.e., the result was not rejected). Valid results were obtained by the technology for all 160 dust wipe samples. Therefore, completeness was 100%. Sample Throughput Sample throughput is representative of the estimated amount of time required to prepare and analyze the samples and perform the data analysis. Operating in the field, one Palintest analyst accomplished a sample throughput rate of approximately eighty samples per day. The analyst prepared and analyzed all 160 dust wipe samples in two 12-hour days, completing duplicate analysis on each sample and operating two instruments simultaneously. Ease of Use The technology can be operated by a single person. Users unfamiliar with the technology may need approximately one-half day of additional training to operate the instrument. No particular level of educational training is required for the operator. The analyst who operated the instrument during the verification test was a Palintest expert. Cost Assessment The purpose of this economic analysis is to estimate the range of costs for analysis of lead in dust wipe samples using the SA-5000 and a conventional analytical laboratory method. The analysis was based on the results and experience gained from this verification test, costs provided by Palintest, and representative costs provided by the NLLAP analytical laboratory to analyze the samples. To account for the variability in cost data and assumptions, the economic analysis is presented as a list of cost elements and a range of costs for sample analysis by the SA-5000 instrument and by the laboratory. Several factors affected the cost of analysis. Where possible, these factors were addressed so that decision makers can complete a site-specific economic analysis to suit their needs. The following categories are considered in the estimate: • sample shipment costs, • labor costs, and • equipment costs. Each of these cost factors is defined and discussed and serves as the basis for the estimated cost ranges presented in Table 11. This analysis assumed that the individuals performing the analyses were fully trained to operate the technology. Costs for sample acquisition and pre-analytical sample preparation, tasks common to both methods, were not included in this assessment. SA-5000 Costs The costs associated with using the instrument included labor and equipment costs. No sample shipment charges were associated with the cost of operating the instrument because the samples were analyzed on site. 21 ------- Table 11. Estimated analytical costs for lead dust wipe samples Analysis method: Analyst/manufacturer: Sample throughput: Cost category Sample shipment Labor Rate Equipment Instrument purchase price Reagents/supplies Waste Disposal SA-5000 Palintest 80 samples/day Cost ($) 0 50-100/h per analyst 3850 7 per sample * 250 Analysis method: NLLAP Laboratory: Actual turnaround: Cost category Sample shipment Labor Overnight shipping Labor Rate Equipment Waste Disposal EPASW8466010b DataChem 18 working days Cost ($) 100-200 50-100 30 per sample Included " Included ""Included" indicates that the cost is included in the labor rate. * Price per sample when purchased as one 10-test kit. Discounts are given for higher volume purchases. Labor Labor costs included on-site labor to perform the analyses. The cost of the on-site labor was estimated at a rate of $50-100/h, depending on the required expertise level of the analyst. This cost element included the labor involved during the entire analytical process, comprising sample preparation, sample management, analysis, and reporting. If the user would have to travel to the site, the cost of mobilization and demobilization, travel, and per diem expenses should also be considered. However, in a typical application where the SA-5000 might be used, the analysis would usually be carried out by a person located on site. Equipment Equipment costs included purchase of equipment and the reagents and other consumable supplies necessary to complete the analysis. • Instrument purchase. The instrument can be purchased for $3,850. This includes the SA- 5000 Scanning Analyzer, deluxe carrying case, 1-mL and 5-mL pipettors, and an ultrasonicator. • Reagents and supplies. The dust sample preparation and electrode pack costs $72.45 for one pack often tests and contains: electrodes, SoluPrep SP-B tablets, crushing and stirring rods, and sonicator tubes. Discounts are given for larger volume purchases. Laboratory Costs Sample Shipment The costs of shipping samples to the laboratory included overnight shipping charges as well as labor charges associated with the various organizations involved in the shipping process. • Labor. This cost element included all of the tasks associated with shipping the samples to the NLLAP laboratory. Tasks included packing the shipping coolers, completing the chain-of- custody documentation, and completing the shipping forms. The estimate to complete this task ranged from 2 to 4 h, at $50 per hour. • Overnight shipping. The overnight express shipping service cost was estimated to be $50 - 100 for two boxes of samples. Labor, Equipment, and Waste Disposal The labor quotes from commercial analytical laboratories that offered to perform the analyses for this verification test ranged from $20 to $30 per sample, with turnaround time estimates ranging from 7 to 14 days. Some laboratories can provide a 1-2 day turnaround, but the quick turnaround was not necessary for this test. The quotes were dependent on many factors, including the perceived difficulty of the sample matrix, the current workload of the laboratory, data packaging, and the competitiveness of the market. This rate was a fully loaded analytical cost that included equipment, 22 ------- labor, waste disposal, and report preparation. The cost for DataChem to analyze samples for this verification test was $30 per sample, with a turnaround time of 18 working days. Cost Assessment Summary An overall cost estimate for use of the SA-5000 instrument versus use of the NLLAP- laboratory was not made because of the extent of variation in the different cost factors, as outlined in Table 11. The overall costs for the application of any technology would be based on the number of samples requiring analysis, the sample type, and the site location and characteristics. Decision-making factors, such as turnaround time for results, must also be weighed against the cost estimate to determine the value of the field technology's providing immediate answers versus the laboratory's provision of reporting data within 18 days of receipt of samples. Miscellaneous Factors The following are general observations regarding the field operation and performance of the SA-5000 instrument: The SA-5000 required no electrical power and worked continuously through a 12-hour workday without the need for recharging the battery. The Palintest analyst was ready for the first set of samples within 1 h of arriving on site. If the nitric acid is pre-prepared, and the equipment is packaged in a carrying case and brought to the test site, atypical set-up time is 15 min. The set- up took a little longer for the verification test because the equipment had to be unpacked from sealed boxes and the concentrated nitric acid had to be diluted. Tests with the SA-5000 generated a 5-gal bucket of vials containing dilute nitric acid waste, which cost approximately $250 to dispose by a commercial vendor. The actual volume of waste was approximately 2.1 gallons (8 L). The disposable electrodes have passed leaching tests for mercury and silver, so they were thrown away in the regular trash. The Palintest analyst analyzed all of the samples in duplicate (i.e., analyzed each sample with two different electrodes). The duplicate analysis was performed for Palintest's interests only, so the result reported was the first result of the duplicate measurements. Even though this added some time to the analysis, the analyst confirmed each of the results and still was able to analyze all 160 samples in a two working days because he operated two instruments. The shelf-life of the electrodes is approximately 18 months. Solu-Prep tablets are added to each vial prior to analysis. The main ingredient is potassium chloride. The tablet is colored red so the analyst can quickly know that the right pellet is being used. On the last day of testing, 15 potential users attending a nearby conference on lead-safe housing observed the technology in operation and completed a survey about its user friendliness. Most (n=10) thought the system was user-friendly and commented that the relatively low cost (<$4,000) was attractive when considering purchase options. Some of the participants (n=6) stated they would consider purchasing or using this instrument based on their observations and felt a new user could be trained in 2 to 4 hours. A few observers reported that the SA-5000 seemed like it should be operated in a laboratory rather than in a field setting because of the 45 min digestion period and the hazardous waste that is generated. Summary of Performance A summary of performance is presented in Table 12. Note that performance is based on the specific protocols employed for this verification test. If different testing protocols are used, different performance results may be obtained. The verification test found that the SA-5000 instrument was relatively simple for a trained analyst to operate in the field, requiring less than an hour for initial setup. The sample throughput of the SA-5000 was eighty samples per day with a single operator. The overall performance of the SA-5000 for the analysis of lead in dust wipe samples was characterized as having an acceptable amount of negative bias, very precise, and has a strong linear relationship with the NLLAP-laboratory results. ORNL and ETV remind the reader that, while the ETV test provides valuable information in the form of a snapshot of performance, state, tribal, or federal requirements regarding the use of the technologies (such as NLLAP recognition for analysis of clearance samples where required) need to be followed. 23 ------- Table 12. Performance Summary for the Scanning Analyzer SA-5000 System Feature/parameter Precision : average RSD Accuracy: average % recovery Positive results on "detectable blank" samples (< 2 jag/wipe) False positive results False negative results Comparison with slope laboratory results intercept l-ig/wipe samples) correlation coefficient Overall evaluation Completeness Size and Weight Sample throughput (1 analyst) Power requirements Training requirements Cost Waste generated Performance summary UC Samples 8% 80% n/a DataChem SA-5000 Oof 41 Oof 38 DataChem SA-5000 16 of 19 22 of 22 0.839 5.539 1.000 - Statistically significant negative bias but within the acceptable bias range - Very Precise - Strong linear relationship to the NLLAP lab results - No fp results - Higher number of fn results ELPAT Samples 5% 91% 0 of 20 samples DataChem SA-5000 2 of 12 0 of 12 DataChem SA-5000 7 of 28 17 of 28 0.926 6.506 0.995 - Statistically significant negative bias but within the acceptable bias range - Very Precise - Strong linear relationship to the NLLAP lab results - No fp results - Higher number of fn results 100% of 160 dust wipe samples 6 3/4" x 5 1/8" x 2 1/8"; 13.3 oz 80 samples/12-hr day battery operated (eight - AA) One-half day instrument-specific training Purchase: $3,850 Reagents/Supplies: $72.45 for one pack of 10 tests (discounts for larger volume purchases) 5 -gal bucket of vials of diluted nitric acid/extract dust wipes 2.1 gallons (8 L) for 160 samples analyzed 24 ------- Section 6 — References American Society for Testing and Materials. 1996. "Specification E1792-96a: Standard Specification for Wipe Sampling Materials for Lead in Surface Dust" in ASTM Standards on Lead Hazards Associated with Buildings. West Conshohocken, PA. American Society for Testing and Materials. 1998. "Practice El644: Standard Practice for Hot Plate Digestion of Dust Wipe Samples for the Determination of Lead" in ASTM Standards on Lead Hazards Associated with Buildings. West Conshohocken, PA. Ashley, Kevin, Korrie Mapp, and Mark Millson. 1998. "Ultrasonic Extraction and Field-Portable Anodic Stripping Voltammetry for the Determination of Lead in Workplace Air Samples." AIHA Journal. 59(10), 671-679. Ashley, K., T.J. Wise, W. Mercado, and D.B. Parry. 2001. "Ultrasonic Extraction and Field-Portable Anodic Stripping Voltammetric Measurement of Lead in Dust Wipe Samples." Journal of Hazardous Materials. 83, p 41-50. Code of Federal Regulations. 2001. "Identification of Dangerous Levels of Lead", Final Rule, 40 CFR Part 745,January. Draper, N. R., and H. Smith. 1981. Applied Regression Analysis. 2nd ed. John Wiley & Sons, New York. EPA (U.S. Environmental Protection Agency). 1996. "Method 3050B-1: Acid Digestion of Sediment, Sludge, and Soils." In Test Methods for Evaluating Solid Waste: Physical/ Chemical Methods, Update II. SW-846. U.S. Environmental Protection Agency, Washington, D.C., December. EPA (U.S. Environmental Protection Agency). 1996. "Method 6010B-1: Inductively Coupled Plasma-Atomic Emission Spectrometry." In Test Methods for Evaluating Solid Waste: Physical/ Chemical Methods, Update II. SW-846. U.S. Environmental Protection Agency, Washington, D.C., December. EPA (U.S. Environmental Protection Agency). 1996. "Evaluation of the Performance of Reflectance and Electrochemical Technologies for the Measurement of Lead in Characterized Paints, Bulk Dusts, and Soils." EPA 600/R-95/093. U.S. Environmental Protection Agency, Research Triangle Park, NC. Keith, L.H., G. L. Patton, D.L. Lewis and P.O. Edwards. 1996. Chapter 1: Determining What Kinds of Samples and How Many Samples to Analyze, pp. 19. In Principles of Environmental Sampling. Second Edition. Edited by L. H. Keith, ACS Professional Reference Book, American Chemical Society, Washington, DC. ORNL (Oak Ridge National Laboratory). 1998. Quality Management Plan for the Environmental Technology Verification Program 's Site Characterization and Monitoring Technologies Pilot. QMP-X-98- CASD-001, Rev. 0. Oak Ridge National Laboratory, Oak Ridge, Tenn., November. ORNL (Oak Ridge National Laboratory). 2001. Technology Verification Test Plan: Evaluation of Field Portable Measurement Technologies for Lead in Dust Wipes. Chemical Sciences Division, Oak Ridge National Laboratory, Oak Ridge, Tenn., November. Song, Ruiguang, Paul C. Schlecht, and Kevin Ashley. 2001. "Field Screening Test Methods: performance criteria and performance characteristics." Journal of Hazardous Materials. 83, 29-39. 25 ------- Appendix Palintest's SA-5000 Results Compared with Laboratory Results Sample Analysis Order 138 133 35 12 9 77 100 o J 112 11 19 59 152 82 40 92 41 48 139 25 34 126 104 74 44 72 5 128 65 7 79 10 Source ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT Rep 1 2 o 3 4 1 2 3 4 1 2 o 5 4 1 2 3 4 1 2 o 3 4 1 2 o J 4 1 2 3 4 1 2 3 4 Palintest SA-5000 Result |ig/wipe <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 <25 30 32 30 28 Estimated |ig/wipe o .J .3 o .J .3 .3 o .J .3 o .J o .J .3 o .J .3 .3 o .J .3 o .J o .J .3 o .J .3 16.9 16.9 16.9 16.9 17.6 17.6 17.6 17.6 29.8 29.8 29.8 29.8 DataChem Result |ig/wipe <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 <20 30 <20 <20 <20 33 26 28 28 Estimated |ig/wipe o .J .3 o .J .3 .3 o .J .3 o .J o .J .3 o .J .3 .3 o .J .3 o .J o .J .3 o .J .3 16.9 16.9 16.9 16.9 17.6 17.6 17.6 17.6 29.8 29.8 29.8 29.8 26 ------- Sample Analysis Order 14 125 143 58 78 4 151 30 105 53 134 109 135 103 123 95 27 158 119 145 149 147 108 1 144 142 45 84 102 38 51 66 57 121 55 94 Source UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT Rep 1 2 o 6 4 1 2 3 4 1 2 o J 4 1 2 3 4 1 2 o J 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 o J 4 Palintest SA-5000 Result |ig/wipe 38 32 38 36 36 38 30 32 32 34 36 34 34 38 34 30 36 34 36 30 36 36 40 40 42 38 42 40 38 40 42 45 47 51 44 47 Estimated |ig/wipe 39.9 35.1 43.3 42.1 38.8 42.8 35.0 36.4 38.5 38.9 38.8 37.6 43.1 41.1 37.3 35.1 44.5 36.4 38.8 39.2 41.3 41.3 41.3 41.3 49.0 49.0 49.0 49.0 49.1 49.1 49.1 49.1 58.6 58.6 58.6 58.6 DataChem Result |ig/wipe 33 32 31 29 32 38 37 36 37 37 33 41 32 38 30 35 36 31 34 34 37 42 44 41 43 52 49 48 70 54 48 44 64 55 56 52 Estimated |ig/wipe 35.4 35.7 38.5 36.4 35.1 40.7 39.4 41.0 41.0 38.8 39.3 44.7 36.0 44.7 39.9 37.5 37.4 36.7 35.8 39.7 41.3 41.3 41.3 41.3 49.0 49.0 49.0 49.0 49.1 49.1 49.1 49.1 58.6 58.6 58.6 58.6 27 ------- Sample Analysis Order 62 136 80 56 90 70 6 155 43 68 106 20 111 69 97 115 96 33 75 150 49 37 114 99 91 29 2 76 88 71 130 83 18 64 54 124 156 17 36 67 Source ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB Rep 1 2 o 5 4 1 2 o 5 4 1 2 o J 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 o 6 4 1 2 3 4 1 2 o 3 4 1 2 3 4 Palintest SA-5000 Result |ig/wipe 75 71 65 67 96 104 110 100 155 162 157 157 200 186 194 192 225 194 221 231 164 182 168 178 186 176 211 192 225 198 170 166 213 205 196 176 178 194 192 180 Estimated |ig/wipe 88.0 88.0 88.0 88.0 117.0 117.0 117.0 117.0 162.3 162.3 162.3 162.3 201.6 201.6 201.6 201.6 239.0 239.0 239.0 239.0 249.5 239.0 245.1 225.7 244.0 269.4 258.9 250.0 270.0 263.9 240.1 237.3 255.6 262.8 258.9 225.7 228.5 240.6 255.0 241.2 DataChem Result |ig/wipe 82 83 79 100 120 120 120 110 150 160 150 160 200 190 200 220 230 250 250 230 210 250 230 230 200 240 210 210 220 220 230 170 190 210 210 250 220 210 210 220 Estimated |ig/wipe 88.0 88.0 88.0 88.0 117.0 117.0 117.0 117.0 162.3 162.3 162.3 162.3 201.6 201.6 201.6 201.6 239.0 239.0 239.0 239.0 244.0 274.4 252.8 258.9 241.7 274.9 244.5 236.2 244.0 242.3 260.0 228.5 242.3 267.2 236.2 275.5 262.2 226.3 227.4 243.4 28 ------- Sample Analysis Order 46 110 52 116 89 120 85 113 26 23 22 153 86 8 50 73 107 160 127 140 93 117 60 122 148 137 129 32 118 21 39 42 132 98 131 141 Source ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB UCLAB ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT Rep 1 2 o 6 4 1 2 o 6 4 1 2 o 6 4 1 2 3 4 1 2 o 5 4 1 2 3 4 1 2 o 6 4 1 2 o J 4 1 2 3 4 Palintest SA-5000 Result |ig/wipe 200 207 211 239 225 246 223 246 285 293 303 266 318 311 274 307 274 274 339 278 317 278 299 324 337 274 315 322 409 388 378 355 407 378 403 386 Estimated |ig/wipe 256.7 256.7 256.7 256.7 260.8 260.8 260.8 260.8 366.2 388.9 401.6 372.3 429.3 391.7 370.1 401.1 376.7 367.9 383.9 385.0 388.3 365.1 385.0 421.5 428.2 367.3 405.5 403.8 408.7 408.7 408.7 408.7 418.1 418.1 418.1 418.1 DataChem Result |ig/wipe 290 240 230 250 220 250 210 210 320 360 350 340 350 340 370 340 370 340 370 390 330 320 330 360 340 360 390 330 360 430 410 410 440 410 430 420 Estimated |ig/wipe 256.7 256.7 256.7 256.7 260.8 260.8 260.8 260.8 377.8 395.0 399.4 385.0 395.5 382.8 413.8 374.0 426.5 378.9 401.1 423.2 372.9 362.9 384.5 411.0 397.2 393.3 437.6 375.1 408.7 408.7 408.7 408.7 418.1 418.1 418.1 418.1 29 ------- Sample Analysis Order 63 157 81 101 87 47 15 13 61 159 24 16 154 31 28 146 Source ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT ELPAT Rep 1 2 o 5 4 1 2 o 5 4 1 2 o J 4 1 2 3 4 Palintest SA-5000 Result |ig/wipe 581 515 551 558 523 521 576 541 809 780 817 841 1257 1322 1236 1416 Estimated |ig/wipe 561.9 561.9 561.9 561.9 564.7 564.7 564.7 564.7 805.1 805.1 805.1 805.1 1482.6 1482.6 1482.6 1482.6 DataChem Result |ig/wipe 580 540 560 540 560 560 570 530 760 770 760 740 1500 1500 1500 1400 Estimated |ig/wipe 561.9 561.9 561.9 561.9 564.7 564.7 564.7 564.7 805.1 805.1 805.1 805.1 1482.6 1482.6 1482.6 1482.6 30 ------- |