Traceable assigned values in external quality assessment schemes compared to those obtained by alternative procedure: a case study for Cu, Se and Zn in serum

Abstract

International standards for the recognition of the competence of testing laboratories require that measurement results should be traceable to a conventionally agreed reference. This should be achieved by appropriate calibration of equipment and method validation involving analysis of certified reference materials (CRM). However, these are costly and for many analytical procedures, few are available. Participation in external quality assessment schemes (EQAS) may provide a mean to support the laboratory traceability statement, if the values assigned to test samples are traceable to a stated reference. Values may be assigned to EQAS test samples by a variety of techniques but there has been no direct comparison of results obtained when these procedures are applied to the same samples. In this study, traceable values for Cu, Se and Zn concentrations were assigned to three batches of EQAS serum samples, by analysis by expert laboratories together with CRMs, and compared with those obtained by three other of the approaches described in ISO 13528; analysis by a definitive method (ID-ICP-MS); determination of robust consensus mean from the results of expert laboratories; robust consensus mean of results from EQAS participants. The assigned values (μmol L⁻¹) ± expanded uncertainty (%) for the low, medium and high pools obtained by ID-ICP-MS were: Cu 13.37 ± 1.2, 21.03 ± 1.8, 28.73 ± 1.2; Se 0.74 ± 3.5, 1.51 ± 3.4, 3.11 ± 3.6; Zn 9.69 ± 4.9, 22.52 ± 1.5, 30.85 ± 3.8. Concentrations determined using the three other approaches were similar but the uncertainties increased as the methodologies became increasingly less rigorous.

---

Introduction

The reliability and comparability of measurements is essential in clinical science. Comparability and reliability of measurements can be achieved if measurements are traceable to conventionally agreed standards, e.g. the International System (SI) of units, by means of an unbroken chain of comparisons, all having stated uncertainties, as in the case of physical measurements (Fig. 1). Guidance on how to estimate the uncertainty of measurements has been given by ISO¹ and the EURACHEM/CITAC Guide² but, accurate representations of the SI units, which are an essential requisite to establish traceability, are not readily available. The international standards for testing, calibrating and clinical laboratories^3,4 indicate that competent laboratories should use appropriate calibrated equipment, standards and reference materials to demonstrate the traceability of their measurements. However, for reasons of availability and cost, it is not possible for laboratories to include certified reference materials (CRMs) together within each series of analyses. Therefore the accuracy of a laboratory method is demonstrated by the strength of a traceability chain, i.e. the chain of comparisons linking a laboratory result to an appropriate representation of the SI unit, e.g. a CRM. The intermediate steps of the chain are represented by reference materials, to which values have been assigned by comparison with CRMs (Fig. 1).

Fig. 1 The traceability chain. Left – measurement of weight. Right – analytical chemistry.

For many analytical procedures, including the determination of copper, selenium and zinc in plasma and serum, there are few suitable CRMs. However, as a consequence of the IMEP-17 project,⁵ a limited number of vials of two serum-based materials are available with concentrations (which include those for copper, selenium and zinc) traceable to SI units, assigned using methods of higher metrological order. These materials were certified by the Institute for Reference Materials and Measurements (IRMM) on the basis of measurements performed by reference methods by IRMM itself and other metrological institutes and are described as ‘certified test samples’. Table 1 shows the certified values and their expanded uncertainties (k = 2) for copper, selenium and zinc.

Table 1 Concentrations and expanded uncertainties (U) in the two IMEP-17 materials

		Material 1		Material 2
		Conc.	U	Conc.	U
Cu	μmol L^−1	17.57	0.10	16.48	0.12
Se	μmol L^−1	1.022	0.035
Zn	μmol L^−1	16.32	0.35	29.38	0.29

---

Because of the small number of vials that exist it is incumbent upon the scientific community to use them in such ways that will provide benefit to as large a number of laboratories and for as long a period of time, as possible. Therefore, the network of EQAS (external quality assessment schemes) Organisers for Occupational and Environmental Laboratory Medicine have worked together to provide traceability for their own EQAS specimens to the IMEP-17 materials via a set of secondary reference materials which are available in greater abundance. The traceable EQAS specimens then offer to participant laboratories an objective reference to support their traceability statements.

Notwithstanding the possibility to provide traceability in this way there are many assays where there are no suitable CRMs. Since the introduction of EQA more than 50 years ago, scheme organisers have developed a number of procedures to set the target concentration of a test material, against which the performance of participants may be assessed. The ISO standard Conformity Assessment – General Requirements for Proficiency Testing⁶ describes five ways to set these assigned values i.e.,

(1) By formulation (e.g. manufacture or dilution);

(2) By certification, as determined by definitive test or measurement methods (for quantitative tests);

(3) By determination by analysis, measurement or comparison of the proficiency test item alongside a reference material or standard, traceable to a national or international standard;

(4) By consensus of results from expert laboratories; and

(5) By consensus of results from all participants, using statistical methods described in ISO 13528 and with consideration of the effects of outliers.

Statistical methods to determine assigned values according to these procedures, together with their standard uncertainties, are elaborated in ISO 13528.⁷ This document suggests that assigned values given by consensus are the least reliable. However, in practice these approaches are the most widely used and, for many analytes, where no CRMs are available to be used for EQAS or to validate the concentration, there is no practical alternative.

A small study which looked at data given by formulation, certification and consensus of results from expert laboratories was reported for lead in blood⁸ but the organisers of EQAS for occupational and environmental laboratory medicine are not aware of any systematic comparison of all methods to set assigned values and to calculate their standard uncertainties. The network of expert and other laboratories represented by the schemes and their participants, and with access to the technique of isotope dilution inductively coupled plasma mass spectrometry (ID-ICP-MS), a rigorous investigation of procedures to set assigned values has been undertaken using the measurement of copper, selenium and zinc in serum as a model.

Materials and methods

Reference materials

The IMEP-17 materials, 20 vials of each, were kindly donated by the Institute for Reference Materials and Measurements, Geel, Belgium.

Three secondary reference materials (2^ry RMs) were prepared as previously described⁹ except that pooled human serum rather than bovine serum was used and the starting material was not treated with Chelex. The batch volumes were sufficient to give approximately 2000 individual vials. The initial pool was divided into three equal portions two of which were spiked with standard solutions of Cu, Se and Zn (Merck) and thoroughly mixed. The calculated increases in concentration (μmol L⁻¹) above the endogenous values, were 7.69 and 15.38 (Cu), 0.777 and 2.330 (Se), 13.46 and 21.15 (Zn). Aliquots (2 mL), were dispensed into NUNC Cryovials (Sanbio) and stored at approximately −80 °C.

Homogeneity and stability

The prepared samples were stored at approximately −80 °C until distributed for analysis. Homogeneity and stability testing (at 25 °C, 4 °C, −20 °C and −80 °C for up to 6 months) was carried out as described in ISO 13528.⁷

Selection of expert laboratories

Laboratories that had previously demonstrated consistently good EQAS performance were nominated by scheme organisers and invited to participate in the study. Those that agreed were asked, on two occasions, to analyse 5 specimens of sera with known concentrations of Cu, Se and Zn, to confirm that current performance was appropriate for this project. Those whose results throughout the two trials were consistently within two standard deviations of the robust mean values¹⁰ were included in the study. Of the original 11 laboratories, the numbers of participants who demonstrated minimal bias and low uncertainty were 7, 8 and 6 for Cu, Se and Zn, respectively.

Determination of assigned values

The 2^ry RMs were analysed in a series of discrete projects.

Certification. The concentrations of copper, selenium and zinc were determined by ID-ICP-MS at LGC Teddington, UK, the designated National Measurement Institute for chemical and biochemical analysis for the UK. Full details of the methodology are given in the report by Turner et al.¹¹

Analysis together with CRMs. This project was represented by the analysis of the IMEP-17 materials and the 2^ry RMs by the expert laboratories for the traceability study. The techniques employed were FAAS, ETAAS, and ICP-MS. These laboratories analysed the IMEP-17 samples and the 2^ry RMs within the same analytical event. Samples were analysed five times under repeatability conditions on two separate occasions. Data from the expert laboratories were then combined to assign concentrations and their standard uncertainties, to the 2^ry RMs, with adjustment using the certified values for the IMEP samples as described in ISO 13528, Section 5.4.⁷

Consensus of expert laboratories. The robust mean values of the non-adjusted analytical results reported by the expert laboratories for the 2^ry RMs were calculated using the Algorithm A in ISO 13528.⁷

Consensus of results by all participants. The three 2^ry RMs were included as specimens distributed by eight of the schemes within the network and also analysed by two other laboratories represented within the group. The total number of laboratories was 295.

Results

The results of the homogeneity and stability experiments satisfied the criteria given in ISO 13528 (ref. 7) Appendix B, confirming the suitability of these 2^ry RMs for assessing traceability and for use in EQA schemes.

Table 2 shows the assigned values and standard uncertainties for the three 2^ry RMs determined by ID-ICP-MS and the traceable robust mean and uncertainties i.e. with IMEP adjustments (‘reference values’). Concentrations, calculated as the consensus from results reported by the expert laboratories and the scheme participants, are also given in the table.

Table 2 Assigned values and standard uncertainties given by the compared procedures. u = standard uncertainty. Factors for conversion of ng g⁻¹ to μmol L⁻¹; weight of 1 mL serum = 1.0213 g, Cu = 63.55, Se = 78.96, Zn = 65.39

		Low		Medium		High
		Value	u	Value	u	Value	u
(a) Copper
ID-ICPMS	μmol L^−1	13.37	0.08	21.03	0.13	28.73	0.17
	ng g^−1	868	5.21	1365	8.20	1865	11.18
Reference values	μmol L^−1	14.10	0.13	21.43	0.13	28.40	0.26
	μg L^−1	896	8.26	1362	8.26	1805	16.52
Expert laboratories	μmol L^−1	13.40	0.517	20.54	0.919	27.85	0.792
	μg L^−1	852	32.86	1306	58.40	1770	50.33
Consensus from participants (robust mean)	μmol L^−1	13.81	1.09	21.29	1.64	29.21	2.15
	μg L^−1	878	69.27	1353	104.22	1856	136.63
(b) Selenium
ID-ICPMS	μmol L^−1	0.74	0.014	1.51	0.025	3.11	0.059
	ng g^−1	59.6	1.10	122	2.05	251	4.74
Reference values	μmol L^−1	0.72	0.028	1.46	0.03	2.99	0.09
	μg L^−1	56.85	2.21	115.28	2.37	236.09	7.11
Expert laboratories	μmol L^−1	0.692	0.047	1.389	0.128	2.842	0.287
	μg L^−1	54.64	3.71	109.68	10.11	224.40	22.66
Consensus from participants (robust mean)	μmol L^−1	0.74	0.10	1.52	0.15	3.10	0.36
	μg L^−1	58.43	7.90	120.02	11.84	244.78	28.43
(c) Zinc
ID-ICPMS	μmol L^−1	9.69	0.24	22.52	0.65	30.85	0.76
	ng g^−1	647	15.82	1504	43.35	2060	50.48
Reference values	μmol L^−1	10.19	0.20	24.95	0.27	32.27	0.26
	μg L^−1	666	13.08	1631	17.66	2110	17.00
Expert laboratories	μmol L^−1	9.45	0.482	22.68	1.578	30.00	1.789
	μg L^−1	618	31.52	1483	103.12	1962	116.98
Consensus from participants (robust mean)	μmol L^−1	9.95	0.84	23.26	2.73	31.06	3.76
	μg L^−1	651	54.93	1515	178.51	2025	245.86

---

The concentrations of Cu, Se and Zn reported by the EQAS participants are summarised in Table 3 which shows the number of results and (i) the mean, median and standard deviation of the values for all results, (ii) the robust mean and standard deviation (Algorithm A) for all results. The dispersions of results are shown in Fig. 2.

Table 3 Consensus of results, mean, median and standard deviation (SD) reported by all EQAS participants. Concentrations are in μmol L⁻¹

	Element	n	Low	n	Medium	n	High
Mean (SD), all	Cu	266	14.09 (2.55)	266	21.48 (3.37)	266	29.31 (4.28)
Median, all		266	13.8	266	21.2	266	29.2
Robust mean (SD)			13.81 (1.24)		21.29 (1.89)		29.21 (2.44)
Mean (SD), all	Zn	291	10.12 (1.88)	294	23.18 (2.73)	293	30.97 (3.76)
Median, all		291	10.0	294	23.3	293	31.0
Robust mean (SD)			9.95 (0.96)		23.26 (1.84)		31.06 (2.35)
Mean (SD), all	Se	148	0.76 (0.21)	148	1.55 (0.29)	148	3.14 (0.54)
Median, all		148	0.74	148	1.51	148	3.07
Robust mean (SD)			0.74 (0.11)		1.52 (0.17)		3.10 (0.40)

---

Fig. 2 Dispersion of results reported by all participants for the low, medium and high samples.

Recoveries of the trace elements added to the original serum pool to prepare the medium and high samples were in good agreement with the amounts introduced (Table 4).

Table 4 Recoveries (%) of added Cu, Se and Zn as given by the 4 procedures to determine assigned values

Procedure to determine assigned value	Copper		Selenium		Zinc
	Medium	High	Medium	High	Medium	High
ID-ICPMS	95.72	97.95	99.59	101.87	95.34	100.03
Reference values	95.3	93.0	95.2	97.4	109.7	104.4
Expert laboratories	92.8	94.0	89.7	92.3	98.3	97.2
Consensus from participants	97.3	100.1	100.4	101.3	98.9	99.8

---

Z-Scores were calculated as follow: Z = (x – x*)/s* where x = participant result, x* = consensus robust mean and s* = consensus robust standard deviation (Table 5).

Table 5 Mean Z-scores in the respective schemes (mean/median of 3 samples L–M–H). Schemes which sent only one result have been excluded

Scheme number	Copper		Zinc		Selenium
	n	Mean Z-score	n	Mean Z-score	N	Mean Z-score
1	20	0.006	20	0.352	13	0.566
2	48	0.175	47	0.594	31	−0.003
3	32	0.266	31	−0.294	21	1.326
4	19	−1.947	22	−1.118	9	1.700
5	23	1.658	25	0.486	5	−1.599
6	19	−1.377	19	0.709	8	1.278
7	79	1.001	95	−0.007	41	−0.477
8	24	−0.006	32	−0.289	19	2.315
All	266	0.277	294	0.056	148	0.538

---

Discussion

Traceability

Accreditation to standards such as ISO/IEC 17025:2005 (ref. 3) and ISO 15189:2012 (ref. 4) require that laboratories should be able to demonstrate the traceability of their results. Within clinical sectors this is relevant to situations where a patient may have repeated investigations performed at different laboratories, and unless results are traceable to a standard unit of amount, direct comparison of results may not be possible. Traceability is also essential to confirm the accuracy of a measurement where a diagnosis or a treatment decision is made on the basis of a laboratory result.

Investigations performed in clinical laboratories using multi-channel analysers use methods and reagents provided by the instrument manufacturers, and the reagents include calibrants that have certificates stating their traceability. For other investigations traceability will need to be established by an alternative procedure such as inclusion of CRMs within every series of analyses.

However, CRMs are costly and not always available. Secondary RMs such as those produced as described in this work may be used to show a metrologically traceable link between the test results and a stated reference. Laboratories may use 2^ry RMs as if they are CRMs, and when the result given and it's uncertainty overlaps with the stated uncertainty of the certified value it may be inferred that the patients' results within that analytical series are also traceable. An alternative approach is to show traceability through performance in external quality assessment schemes. When a result and it's uncertainty reported by a participant overlaps the uncertainty of the assigned value of the scheme's test sample, a traceable link between the laboratory results and the test sample has been demonstrated. When the assigned value has been shown by the scheme organiser to be traceable to a stated reference then the laboratory results are also traceable, via this chain, to the same reference (Fig. 1). For the same reasons that laboratories cannot analyse CRMs with each series of measurements, so scheme organisers do not usually distribute CRMs to participants or determine the assigned value by reference to a CRM. A traceable link can, however, be established if an expert laboratory(ies) analyse the test sample together with a 2^ry RM, to define the assigned value. With these links in place laboratory results can be shown to be traceable, or otherwise, to the amount of substance. The advantage of using 2^ry RMs in this way is that traceability of several laboratories can be shown using a limited resource.

Assigned value

Although formulation of the test samples is one of the methods described in ISO/IEC 17043 to assign values to EQAS samples,⁶ this is not possible when the base material contains an unknown endogenous concentration. A calculation of the endogenous concentration can be made where recovery of the added measurand is determined but this does not account for any interference from the sample matrix unit.

The results from this study indicate that the assigned values given by the investigated procedures are comparable, taking into account the respective uncertainties, which, of course increase as the methodologies become less rigorous. The importance of robust calculations to eliminate outliers is indicated by the data in Table 3 where it is seen that the standard deviations are reduced by approximately 50% when the reported results are evaluated using Algorithm A compared with a simple calculation using all values.

It might be inferred from these results that, for practical purposes, EQA scheme organisers can use the robust mean of participant's results as the assigned value for test items. However, when the data were re-examined by comparing results from the individual schemes participating in this study (Table 5), it was apparent that this conclusion is not necessarily correct. In a separate study of EQA schemes for measurements of Al in serum, the robust means from different schemes were significantly different, as indicated by the Z-scores.¹² A number of possible explanations may be advanced as causing a bias. In some schemes the participants are more likely to be specialist trace element centres while in other schemes the profile may be biased towards general clinical laboratories. There could be an influence associated with the analytical techniques used by the scheme participants.¹³ The number of results used to calculate the scheme-mean is a further possible factor. Ideally therefore, the robust consensus mean should only be used when there is independent verification of its accuracy, traceable to an international or national standard. Where this is not possible, results from reference (metrological) or selected expert laboratories should be recommended.

Acknowledgements

Results from the Institut National De Santé Publique Du Québec, Canada were provided by Dr Jean-Philippe Weber, who has since retired.

References

1. ISO/IEC Guide 98-3:2008 Uncertainty of measurement – Part 3: Guide to the expression of uncertainty in measurement (GUM:1995).
2. EURACHEM/CITAC Guide, Quantifying Uncertainty in Analytical Measurement, 3rd edn, 2012.
3. ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories.
4. ISO 15189, Medical laboratories - Particular requirements for quality and competence specifies the quality management, 2012.
5. Institute for Reference Materials and Measurements 2003, International Measurement Evaluation Programme IMEP-17, Trace and minor constituents in human serum, EUR 20657, Reports Part 1, 2 and 3.
6. ISO/IEC 17043 (2010), Conformity assessment – General requirements for proficiency testing.
7. ISO 13528 (2005), Statistical methods for use in proficiency testing by interlaboratory comparisons.
8. R. Braithwaite and A. Girling, Fresenius' Z. Anal. Chem., 1988, 332, 704–709.
9. A. Taylor and R. J. Briggs, J. Anal. At. Spectrom., 1986, 1, 391–395.
10. M. J. R. Healey, Clin. Chem., 1979, 25, 675–677.
11. J. Turner, L. Evans, R. Hearn and A. Taylor, Agilent Technologies Application No. 5989-7429EN 2007.
12. J. Arnaud, J.-P. Weber, C. Weykamp, P. J. Parsons, J. Angerer, E. Mairiaux, S. Valkonen, A. Menditto, M. Patriarca and A. Taylor, Clin. Chem., 2008, 54, 1892–1899.
13. J. Arnaud, I. Coté, P. J. Parsons, M. Patriarca, A. Taylor and C. Weykamp, Accredit. Qual. Assur., 2014, 19, 169–174.

---

Footnote

† The authors are members of the Thematic Network “Organisers of external quality assessment/proficiency testing schemes related to occupational and environmental laboratory medicine”.

---