How to improve reliability in groundwater analysis: over a decade of experience with external quality control in field campaigns on volatile halogenated compounds

Abstract

The reliability in measurement results obtained during environmental monitoring is crucial for the assessment and further planning of remediation efforts on the respective contaminated sites by the responsible authorities. A case study concerned with groundwater contaminated with perchloroethylene, trichloroethylene and 1,1,2-trichlorotrifluoroethane including their degradation products which involves private contract laboratories and an independent provider of quality assurance (QA) is presented. The experience gained with biannual monitoring campaigns over 14 years indicates that the selection of contractors on basis of accreditation status and successful performance in interlaboratory comparisons are not sufficient. Rather the auditing of the contractors by the QA provider prior to each campaign and the crosschecking of selected monitoring samples by the QA provider led to a lasting improvement of reliability in the contractors' measurement results. A mean deviation of 20% from the reference value determined by the QA provider for the crosschecked samples was reached.

---

Environmental impact

Volatile haloorganics belong to the major contaminants in groundwater aquifers and affect the production of drinking water. Remediation efforts are costly and often have to be maintained over years and the success needs to be monitored by regular measurement campaigns conducted by commercial contract laboratories. The authorities responsible for the rehabilitation of contaminated sites need to ensure comparability of measurement results obtained over years from different contract laboratories and under varying conditions. These are compound-specifically decreasing contaminant levels, the emerging of additional contaminants, improvements in analytical procedures and interferences by matrix components. The presented case study of a major rehabilitation project summarises the quality control measures taken and the respective stepwise improvement of the reliability of monitoring results.

1. Introduction

1.1. Background

The remediation of contaminated groundwater especially in areas where drinking water is produced from groundwater poses a challenge to the responsible authorities lasting over decades. The success of the costly measures taken for clean-up and rehabilitation has to be monitored as basis for further decision making. Information on the contaminant levels both spatially and temporarily over the years of remediation effort is achieved by periodical monitoring campaigns on the quantification of the key pollutants. In the case study outlined here, commercial analytical laboratories have been contracted for this purpose by order of the Senate of Berlin, Germany. At the same time external expertise from BAM Federal Institute for Materials Research and Testing has been consulted to ensure reliability of analytical data. This independent quality control has to deal with fluctuation of contract laboratories due to regularly renewed calls for tender and should accompany the campaigns with appropriate measures of quality control. This requires competence in providing quality assurance including interlaboratory comparisons, development of analytical protocols and auditing their implementation in the field laboratories as well as analytical capabilities to conduct selective control measurements. In the following, experiences with the improvement of external quality control measures in close co-operation with, however, independent from the entrusting authorities and contracted laboratories are comprised.

1.2. Survey of the remediation project

The megasite remediation project “Ökologisches Großprojekt Industriegebiet Spree” is located in the southeast of Berlin, Germany, near the river Spree and between the Britz Canal and the Teltow Canal. The area covers more than 19 km² of industrial and residential land (Fig. 1). The industrial land use dates back to 1846 and since then the groundwater has extensively been contaminated with volatile halogenated organic compounds (VHOC). Among these are most prominently perchloroethylene (PCE) and trichloroethylene (TCE) along with their degradation products, cis-1,2-dichloroethylene (cDCE) and vinyl chloride (VC).^1–4 Additionally, chlorofluorohydrocarbons (H-CFC) such as the refrigerant 1,1,2-trichlorotrifluoroethane (Freon R113) and its degradation products 1,2-dichlorotrifluoroethane (R123) and chlorotrifluoroethene (R1113)^4–6 contribute to the contamination. Increasing water demand during the early 20^th century leads to the establishment of the two major waterworks Johannisthal (1901–1908) and Wuhlheide (1914–1916). Since 1994 a total of 30 groundwater pumps and treat remediation plants have been established and have removed to date approximately 100 t of VHOC and 2.5 t of H-CFC. Currently 18 plants are in operation with a total groundwater throughput of 1000 m³ h⁻¹. The actual VHOC and H-CFC contamination levels range between detection limits in peripheral sections of the area to more than 2 mg L⁻¹ at the hot spots.⁷ The monitoring campaigns are conducted biannually each May and October. About 550 monitoring points with a total of 1.200 single monitoring screens are distributed over the redevelopment area. Depending on the status of the remediation measures and the actual level of water pollution, a selection of wells is sampled during each campaign by one of two contracted laboratories. Until now in the spring-campaigns (May) a larger number of wells has been included than in the autumn-campaigns (October). The data obtained by these laboratories are evaluated with regard to further planning of the remediation by the consulting geological engineers (TAUW GmbH, Berlin, Germany) and documented using GeODin®, a modular data management system (FUGRO CONSULT GmbH, Berlin, Germany). As the contracts between the Senate of Berlin and the analytical laboratories expire after two years, guaranteeing the comparability of the data obtained over the years is a challenge and crucial for the assessment of redevelopment efforts.

Fig. 1 Remediation area.

2. Instruments for external quality assurance

2.1. Selection of contractors by proficiency testing

From the beginning of the project until the beginning of the contract period 2007/2008 BAM conducted interlaboratory comparisons every two years on behalf of the responsible authority (Senate of Berlin, Germany) for the determination of VHOC and H-CFC in synthetic and real-world groundwater samples according to ISO/IEC Guide 43-1⁸ and ISO 13528.⁹ Accreditation for field sampling and trace analysis of VHOC and H-CFC in water was prerequisite for participation. The performance of the 20–25 field laboratories (small and medium private enterprises) was assessed on basis of assigned values (mass fraction added to the synthetic samples) or on basis of reference values obtained at BAM with a number of independent analytical procedures (Table 1). Successful participation was the prerequisite to apply for a two-yearly contract on the monitoring campaigns (call for tender by the authorities) and was achieved by 80–90% of participants in the rounds conducted so far. As a matter of experience the successful participation in intercomparison was not sufficient to guarantee constant reliability in monitoring results over a longer period under real-life conditions by a given laboratory. Therefore, further accompanying quality control measures were required.

Table 1 Reference methods for detection of VHOC and H-CFC at BAM

No	Sampler on GC	GC system and column	Detection	Analyte	Standard procedure
1	Headspace (CombiPAL, CTC Analytics)	HP6890 (Agilent) DB624 and DBWax (parallel) (Varian)	μECD, FID	VHOC	DIN ISO 10301
2	Headspace (MPS2, Gerstel)	HP6890 (Agilent) VF624 ms (Varian)	μECD/FID	H-CFC, VHOC	DIN ISO 10301
3	Headspace (MPS2X, Gerstel)	HP6890 (Agilent), VF624 ms (Varian)	MS (MSD 5973, Agilent)	H-CFC, VHOC	In house procedure on basis of DIN ISO 10301
4	Headspace (Transferline, Perkin Elmer)	GC CP3800 (Varian), DB624 (Varian)	PID (OI Analytical)	Unsaturated VHOC and H-CFC	In house procedure on basis of DIN ISO 10301
5	Purge & trap (Teckmar 2000, Trap VOCARB 3000)	GC 8065 (Fisons), DB624 (Varian)	MS (MD 800, Fisons)	H-CFC, VHOC	ISO 15680

---

2.2. Control measurements on field samples in the monitoring campaign

Certain samples from a monitoring campaign were selected by the consulting engineers to be cross-checked by BAM. The selection was based on specific items such as the importance of the samples in the remediation concept, strong deviations from earlier campaigns, matrix-related problems in past monitoring campaigns at this specific site or in cases where laboratories were unable to apply the standardised methods for analysis (e.g. in the case of H-CFC). A few control measurements were also selected at random. BAM had the capabilities to analyse the water samples with a variety of independent analytical methods (Table 1). The VHOCs were analysed by the headspace and purge & trap technique for sampling and with separation on different chromatographic conditions and detection systems FID, ECD, PID and MS according to standard methods DIN ISO 10301¹⁰ (No 1 and 2) and ISO 15680¹¹ (No 5).

A GC/MS-screening (methods 3 and 5, Table 1) was done on each sample as well. The results gave information about sources of systematic errors.

2.3. Reference samples within the monitoring campaign

Synthetic water samples fortified with H-CFC and VHOC were given to the contract laboratories to be analysed along with the field samples during the monitoring campaign. The concentrations of the spiked compounds ranged between LOD and the respective medium concentration levels observed on the site (for details on the concentration ranges see Table 2 below). Therefore two samples with different concentration levels were prepared, had to be analysed with the methods to be used in the campaign and results had to be returned within 48 h. This allowed evaluation of the degree of equivalence between contractors' results and reference values at the beginning of the campaign and to exclude e.g. systematic bias in quantification.

Table 2 Concentration levels and cut-off criteria for contractors' deviation from the reference value (crosscheck measurements)

Level	Analytes	Concentration range/μg L^−1	Cut-off criteria (%)
<3 LOD	TCE, PCE, R113	0.1–0.3	30
	VC, cDCE, R1113, R123	1–3
Low: 3–10 LOD	TCE, PCE, R113	0.3–1	20
	VC, cDCE, R1113, R123	3–10
Medium: 10–100 LOD	TCE, PCE, R113	1–100	10
	VC, cDCE, R1113, R123	10–100
High: >100 LOD	TCE, PCE, R113	>100	20
	VC, cDCE, R1113, R123

---

2.4. Auditing of contract laboratories

During the first monitoring campaign of each two year contract period the authorities and the consulting engineers initiated a contractor audit involving a technical expert from BAM to bridge the gap between the analysts in the laboratories and the geologists among the initiators. The technical capabilities were reviewed and specific requirements for the monitoring campaign were implemented.

3. Results and discussion

3.1. Development of external quality assurance measures

The development of the external quality assurance (QA) measures can be divided into three phases. In the first phase (1997–2000) field laboratories to be contracted for the monitoring campaigns were selected exclusively on basis of their proficiency proven in the interlaboratory comparison. However, any intercomparison is chronologically and organisationally disconnected from the monitoring campaign and does not reflect the performance of a laboratory under routine conditions. In order to avoid doubts regarding the reliability of measurement results the crosschecking of selected find samples (see Section 2.2) was introduced as additional QA already at the beginning of the project.

The second phase from 2001–2005 was characterised by the stepwise introduction of additional QA measures such as the application of reference samples (see Section 2.3) and auditing of the contractors (see Section 2.4). This included a learning process on the side of the authorities and the external QA provider how to identify and eliminate sources of implausible results and how to impose consequences on contractors in the case of non-compliance with stipulations.

The third phase since 2005 proceeded with a completed external QA system including the selection of contractors, their auditing and the problem-related supervision of the monitoring campaigns. The reliability of results is regarded as satisfactory and occasional implausibilities can be handled appropriately. Further QA measures are not believed to yield additional benefit at reasonable effort. A flow chart on the organisation of the monitoring campaigns can be found in the ESI (Fig. S1†). The sequence of QA measures is depicted along with the deviation of contractors' results from the BAM reference by way of example for vinyl chloride in Fig. 2.

Fig. 2 Development of QA measures and deviation of contractors' vinyl chloride results from the reference.

It should be noted that the documentation of remediation progress by the geologists requires exactly one quantification result per analyte and well. No uncertainty information from the contractors is taken into consideration though the contractors are obliged to conduct a double determination per analyte and well. Therefore, the external QA measures provided by BAM aimed at restricting the measurement uncertainty and the bias of the contractors within a set range regarded as acceptable for the assessment of remediation progress and not further improvable by external QA with realistic effort. This range covers the measurement uncertainty as technically feasible with the employed standard procedures (Table 1) during the assignment of reference values for the control samples at BAM, the measurement uncertainty of the contractors as evaluated during the audit, and the uncertainty contribution in cases observed due to matrix effects.

3.2. Improvement of the reliability of contractors' results

The basic tools for the assessment of the reliability of measurement results of the contractors during the monitoring campaigns and their improvement were the split samples cross-checked at BAM (see Section 2.2). Aside from implausible contractor results during the campaigns which raised suspicion about their reliability the deviation of the contractor's results from the results obtained at BAM clearly indicated the extent of the technical problems in the contractors' laboratories and eventually led to their successful treatment.

Among the problems during the campaigns were most prominently the false-positive detection of compounds (CFCH and VC), systematically too high quantification of VC, strong fluctuations of results in the case of higher levels (cDCE, TCE, PCE, R113) and other implausible fluctuations of the results regarding the contamination profile of specific groundwater wells in the area over the years.

Therefore, certain field samples were split and analysed in parallel to the contractors' laboratories also at BAM. Here, with the reduced number of samples a higher technical effort could be done to ensure the trueness of determined results, such as screening (GC/MS), quantification with independent sampling and chromatographic systems, using different detectors (Table 1), standard addition of matrix samples and using different calibration standards.

Reasons for false-positive quantifications of VC by the contractors were basically the small differences in the retention times to those of H-CFC, methane and ethylene. A successful separation is depicted in the ESI, Fig. S2†.

Further information about additional contaminants in the groundwater was obtained during the screening routinely done at BAM on each sample to be crosschecked. The H-CFCs R123 and R1113 were determined only from 2001 onwards because until 2000 there was no information about a contamination of the groundwater with these compounds. False-positive detection of R1113 with non-specific detectors (FID, ECD, methods 1 and 2, Table 1) by contractors could be shown to occur in the case of presence of methane and other unspecified highly volatile gases in a given water sample. The use of MS for peak identification or quantification solved this problem.¹²VC results being systematically too high often were due to inappropriate handling of VC standard solution. Commercial stock solutions need to be handled at low temperatures to avoid evaporation during dilution to the calibration solutions.¹²

The absolute deviations of the contract laboratories in percent of the reference values determined at BAM on the cross-checked samples were collected according to the concentration range of the analytes and the three phases of the project. The concentration ranges were defined on bases of the respective analyte's limit of detection (LOD) as specified in the respective standard procedure.¹⁰ The concentration ranges were “low level” (from 3× LOD to 10× LOD), “medium level” (between 10× LOD and 1000× LOD) and “high level” (above 1000× LOD) (Table 2).

The cut-off limit for a still tolerable deviation of a contractor's result from the reference value of BAM was set differently for the compounds and the concentration levels in order to consider the general improvement of reproducibility with increasing analyte concentration. A difference of 30% of the single result from the reference near the detection limit (<3 LOD), 20% for low level concentrations and 10% for medium level concentrations was regarded as analytically feasible and tolerable for the assessment of remediation measures. High level concentrations required dilution of the samples which may increase measurement uncertainty and deviation between contractors' results and reference values. Therefore, the tolerance for the deviation was set to 20%.

As Fig. 3 indicates by way of example for VC the deviation of the contractors' results from the BAM reference tended to decrease during the three phases.

Fig. 3 Robust means of relative deviations of contractors' results from the reference (n = number of results) for the determination of vinyl chloride.

In 2001 and later it was seen that the cases of implausible data were further reduced when it had become usage to split all field samples and have them collected at BAM, however, still only selected samples—not known to the contractors—were cross-checked as described above.

The general trend towards improved reliability (=decreasing deviation from the BAM reference) also led to a reduced portion of unacceptable contractors' results above the cut-off-limits (Fig. 4).

Fig. 4 Number of contractors' results above the cut-off limit (in brackets: total number of results).

A detailed statistical analysis of the normalised deviations was carried out. A complete description of the aims and tasks of the data assessment, the data (transformed from the original measurement results) assessed, the parameters and tools used for statistical decision-making can be found in the ESI†. Data pre-processing as described in Clause S3 revealed distributions of normalised deviations for the three phases under investigation as comprehensively depicted in Fig. 5.

Fig. 5 Overall distribution densities of observations within the defined deviation classes of 10% to 90% for the three phases.

It is obvious from Fig. 5 that the overall performance of the laboratories improved significantly from phase 1 to 3: while in the beginning, the distribution is flat and rather similar to a rectangular (in other words, noise-like), it becomes much more pronounced at the later stages, with more and more observations within a limited, and allocated around the reference value, deviation interval not much larger than 20%. As Fig. 5 demonstrates, it may be attained on average for all of the analytes but not without constant, expert-knowledge based supervision and step-by-step improvement of the methods applied.

It can clearly be seen from Fig. 5 that the latest phase represents best performance of the laboratories, close to an “ideal” model where the value found by the field laboratory deviates only randomly from the reference, with no persistent bias and a standard deviation very close to 20%.

Analyte-specific deviations from this overall performance characteristic are depicted in Fig. 6 which resembles, and explains to a large extent, Fig. 4: the distributions of normalized deviations fit the ideal model the better the later the data were obtained, i.e. even given certain problems with specific analytes, the match with the reference value significantly improves from phase one (1997–2000) to phase three (2005–2010) due to the impact and result of the QA measures applied. While within phase one the field-laboratory determinations for all of the analytes show a rather noise-like distribution with (particularly very) large chi-square values, one sees (i) a substantial reduction of the chi-square values and (ii) an approximate normal distribution for the determination of the analytes in phase three. In particular, this explains the huge fraction of “non-compliant” results attained in the beginning. For details of Fig. 6, refer to ESI, Clause S3.3†.

Fig. 6 Per-analyte chi-square values for the deviation of contractors' results from the reference for the three phases.

It is believed that aside from the crosschecking of field samples by the QA provider the contractor audit and the mandatory reference sample contributed essentially to the observed improvements.

The first contractor audit was done before the May monitoring in 2000. Then, a number of technical and organizational problems during the campaigns became obvious. Requirements regarding equipment and QA were not fulfilled and time management was not quite in accordance with the requirements of standard methods. The mandatory analysis of reference samples within the campaigns (see Section 2.3) was introduced at the beginning of the autumn monitoring campaign in 2000. The contractors, selected on basis of the interlaboratory comparison, had to prove their proficiency on reference samples from this point onwards at the beginning of each monitoring campaign. The contract period was prolonged from one year to two years after 2001 in order to make longer use of the once achieved reliability in a given contractor's results.

It was seen that the improving quality of the laboratories' services was not associated with any improving performance in the interlaboratory comparisons but with implementation of the directly problem-related QA measures cross-checking of field samples (Section 2.2), reference samples during the campaigns (Section 2.3), and contractor audit (Section 2.4). Therefore, since 2009 the responsible authority in accordance with BAM resigned the interlaboratory comparison. Since then, candidate laboratories are selected for the call for tender by means of a prequalification procedure. In this procedure, technical aspects with regard to the laboratory skills and experience directly associated with the specific analytical challenges on the site are evaluated. Contractor audit and mandatory analysis of reference samples remain in place as crucial measures to maintain the reliability in contractor results arrived at over the years.

4. Conclusion

The implementation of external QA measures in co-operation with responsible authorities and contracted field laboratories in environmental monitoring campaigns was outlined. It was seen that accreditation status of applicants and proficiency testing as selection criteria were a basis from which further external quality assurance measures endorsed by an independent provider with their own analytical capabilities can significantly improve the reliability in measurement results of the contractors during the field campaigns. Maintaining reliability on a constantly acceptable level requires independent external QA change especially in the case of periodical change of contracted laboratories.

Acknowledgements

This project is financed by Senate for Health, Environment and Consumer Protection, Berlin and the Federal Institute for Special Tasks Arising from Unification (BvS).

References

1. J. W. Washington, Ground Water, 1995, 33, 415–424.
2. D. Hunkeler, R. Aravena, K. Berry-Spark and E. Cox, Environ. Sci. Technol., 2005, 39, 5975–5981.
3. F. Aulenta, A. Bianchi, M. Majone, M. Papini, M. Potalivo and V. Tandoi, Environ. Int., 2005, 31, 185–190.
4. A. Deipser and E. Stegmann, Environ. Sci. Pollut. Res., 1997, 4, 209–216.
5. A. Deipser, Waste Manage. Res., 1998, 16, 330–341.
6. Ch. Balsiger, Ch. Holliger and P. Hohener, Chemosphere, 2005, 61, 361–373.
7. J. Naumann, 15 Jahre Ökologisches Großprojekt Berlin—Rückschau und Ausblick, Altlastensymposium 26-27 Juni 2008, Senatsverwaltung für Gesundheit, Umwelt und Verbraucherschutz, Berlin, Tagungsband, 2008.
8. ISO/IEC Guide 43-1:1997-Proficiency testing by interlaboratory comparisons—Part 1: Development and operation of proficiency testing schemes.
9. ISO 13528:2005, Statistical methods for use in proficiency testing by interlaboratory comparisons.
10. Din ISO 10301:1997, Bestimmung leichtflüchtiger halogenierter Kohlenwasserstoffe (LHKW).
11. ISO 15680: 2003, Water quality—Gas-chromatographic determination of a number of monocyclic aromatic hydrocarbons, naphthalene and several chlorinated compounds using purge-and-trap and thermal desorption; German version: DINEN ISO 15680: 2003.
12. U. Dorgerloh, U. Erhardt, S. Hilbert, M. Mechler, I. Nehls, R. Schmieder and T. Win, Vom Wasser, 2004, 102, 12–15.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: 10.1039/c1em10460h

---