Analytical Methods Committee AMCTB No. 105

Received
30th April 2021
, Accepted 30th April 2021

First published on 10th June 2021

It has become accepted practice for laboratories to report the value of the uncertainty of each measured quantity value (i.e. measurement uncertainty, MU). Informally, the MU expresses the range of values within which the true value of the analyte concentration (i.e. the value of the measurand) is asserted to lie. It is much less widely appreciated that the estimate of MU is also not an exact or ‘true’ value. The quoted MU is actually only an estimate, and has its own uncertainty which also can be expressed using a confidence interval (CI) for a specified confidence level (e.g. 95%).

It was perhaps understandable that the existence of a confidence interval (CI) for each value of measurement uncertainty (MU) was not widely discussed in the initial phase of getting the concept of MU accepted by both laboratories and their customers. Now, however, making sure that the CI of the MU is small enough, can be very important for making reliable decisions based upon measurement results. This document aims to explain the existence of a CI on every MU estimate, and how it can be evaluated, using a worked example that includes primary sampling within the measurement process. In particular this Technical Brief aims to explain how and when it is helpful to calculate the CI of MU estimates using the software RANOVA3. Furthermore, situations will be identified for which a knowledge of this CI is important.

For example, if we have 10 observations (3.3, 4.6, 3.5, 6.6, 6.9, 4.1, 5.3, 4.8, 4.9, 4.9) generated for a population with a mean (μ) of 5 and SD (σ) of 1, the estimated SD (s) of this ‘sample’ is 1.17. We can calculate the confidence limits of s using the chi-squared distribution^{1} (χ^{2}), with the equations:

LCL_{s} = √(νs^{2}/χ^{2}_{(1−α/2),ν}) = 0.80 |

UCL_{s} = √(νs^{2}/χ^{2}_{(α/2),ν}) = 2.14 |

When an estimate of SD is used to estimate MU, the CI of the SD (or more strictly on the variance, [SD]^{2}) can be used to express explicitly the CI of the MU estimate (CI_{U}), which is bound by LCL_{U} and UCL_{U}. In many situations with low degrees of freedom e.g. <30, we use percentage points of the Student’s ‘t’ rather than the normal distribution, to calculate what is effectively expanded uncertainty, to implicitly allow for the uncertainty in the estimated SD. However, Student’s ‘t’ is not applicable to more complex situations, such as the output from analysis of variance (ANOVA) discussed below.

Using the modelling (or ‘bottom up’) approach to estimating MU, it should be possible to enter the CI of each of the component variances into a summation, or a Monte Carlo simulation, to calculate CI_{U} on the resultant estimate of the overall MU.

Sample target | S1A1 | S1A2 | S2A1 | S2A2 |
---|---|---|---|---|

A | 3898 | 4139 | 4466 | 4693 |

B | 3910 | 3993 | 4201 | 4126 |

C | 5708 | 5903 | 4061 | 3782 |

D | 5028 | 4754 | 5450 | 5416 |

E | 4640 | 4401 | 4248 | 4191 |

F | 5182 | 5023 | 4662 | 4839 |

G | 3028 | 3224 | 3023 | 2901 |

H | 3966 | 4283 | 4131 | 3788 |

Measurement uncertainty (MU) was calculated by placing these 32 measured quantity values (Table 1) into RANOVA3. Robust ANOVA was selected as there was an evident outlier in the sample (target C). The MU is expressed in Table 2 as both standard uncertainty (u = SD) and expanded relative uncertainty (U′ = 100 × 2 × SD/mean). The respective confidence intervals are expressed as the 95% confidence limits (LCL_{U}, UCL_{U}). The basic interpretation of the estimates of the expanded relative uncertainty (U′) for all 8 sampling targets (Table 2), ignoring CI_{U}, is that the U′ estimate for the whole measurement process is 16.4%, whilst that for sampling alone is 14.5%. However, when the CI_{U} of is examined (Table 2) it becomes clear that the population value of lies somewhere between 13.7% and 35.3%. This CI_{U} is strongly asymmetric, with a positive skew, as the MU estimate (16.4%) is much closer to the LCL_{U} (13.7%) than to the UCL_{U} (35.3%). This skewed CI is typical of all of these uncertainty estimates, both classical and robust, and is caused by its frequency distribution (which is either exactly or approximately chi-squared).

Sampling | Confidence limits | Analysis | Confidence limits | Measurement | Confidence limits | |
---|---|---|---|---|---|---|

u (SD) | 319 | (251, 762) | 168 | (140, 208) | 361 | (301, 777) |

U′ (95%) | 14.5 | (11.4, 34.6) | 7.6 | (6.3, 9.4) | 16.4 | (13.7, 35.3) |

Interestingly, the CI_{U} for lies between a similarly wide 11.4% to 34.6%. This CI_{U} overlaps substantially with that for showing that no significant difference has been found between these two estimates, of (16.4%) and (14.5%). By contrast, the CI_{U} of does not overlap with that for (7.6%), which lies between 6.3% and 9.4%, indicating that their population values are significantly different from each other.

Incidentally, it is worth pointing out that, although the appears to be only twice the size of it actually contributes four times as much variance to because they add using their variances, i.e.,

Generally, the width of the CI_{U} reduces as the number of duplicated measurements used in the estimation process increases, but is more marked for the sampling uncertainty (Fig. 1).

Fig. 1 Estimates of uncertainty, with their CI_{U}, arising from sampling (u_{samp}) and analysis (u_{anal}) from a previous study of glasshouse-grown lettuce for nitrate (mg kg^{−1}). Standard, rather than relative uncertainty, and a log scale, are used to enable the comparison. Increasing the number of duplicate samples clearly reduces the CI of the uncertainty estimates in both cases, but is more marked for the sampling uncertainty.^{7} |

The estimate of MU can be used in compliance assessment, by comparing the measured quantity value (x) against a threshold value. This requires the use of the confidence interval of the concentration estimate (CI_{x}), which is bound by LCL_{x} (e.g. x − U_{meas}) and UCL_{x} (e.g. x + U_{meas}). For the example of nitrate in lettuce, the regulatory threshold is 4500 mg kg^{−1}. For rejection of a batch with 97.5% confidence (at the lower tail of distribution), the LCL_{x} of the concentration estimate (x) for the single composite sample with single analysis routinely taken (e.g. S1A1) needs to exceed this threshold value. Applying this criterion to the eight batches of lettuce (Table 1), seven batches would have been accepted for human consumption. Only one batch (C, x = 5708 mg kg^{−1}) would have been rejected, using of 16.4%, giving the LCL_{x} as 4774 mg kg^{−1} (i.e. 5708 × 1 − [/100]), which is above 4500 mg kg^{−1}.

The suitability of a minimum of 8 duplicated samples is confirmed by the fact that this compliance decision is barely affected using any of the different estimates of within its CI_{U}. However, if a smaller number of targets had been used in the estimation of MU, to apparently save money, it is clear from Fig. 1 that there would have been both a different estimate of MU, but more importantly a much wider CI_{U}, making this estimate much less reliable. For example, if only four duplicated samples were used, the CI_{U} is substantially widened to the point where an estimate of U_{meas} could arise that would cause the rejection of a second batch (F). This erroneously rejected batch of 20000 lettuces, caused by the insufficiently reliable estimate of MU, would be worth far more than the small apparent saving achieved by taking fewer duplicated samples.

A second advantage of knowing CI_{U}, is for the comparison of estimates of MU made by different approaches, to see whether they are significantly different. This topic will be discussed in a subsequent and related Technical Brief.

The task of combining CI_{U} into the uncertainty statement for the measurand, if required, will need further research.

Michael H. Ramsey

This Technical Brief was prepared for the Analytical Methods Committee with contributions from members of the AMC Sampling Uncertainty and Statistics Expert Working Groups, and the Eurachem Working Group on Uncertainty from Sampling, and approved on 19^{th} March 2021.

- P. D. Rostron, T. Fearn and M. H. Ramsey, Confidence intervals for robust estimates of measurement uncertainty, Accreditation and Quality Assurance: Journal for Quality, Comparability and Reliability in Chemical Measurement, 2020, DOI:10.1007/s00769-019-01417-4.
- AMC, The Duplicate Method for the estimation of measurement uncertainty arising from sampling, Technical Brief No. 40, 2009, https://rsc.li/amc.
- AMC, TB Robust statistics: a method of coping with outliers, Technical Brief No. 6, 2001, https://rsc.li/amc.
- AMC, Why do we need the uncertainty factor? Technical Brief No. 88, Anal. Methods, 2019, 11, 2105–2107, 10.1039/C9AY90050K.
- https://www.rsc.org/Membership/Networking/InterestGroups/Analytical/AMC/Software/ .
- Eurachem/EUROLAB/CITAC/Nordtest/AMC Guide: Measurement uncertainty arising from sampling: a guide to methods and approach, ed. M.H.Ramsey, S. L. R. Ellison and P. Rostron, Eurachem, ISBN 978-0-948926-35-8 http://www.eurachem.org/index.php/publications/guides/musamp, Example A1, 2nd edn, 2019, pp.39–42 Search PubMed.
- J. A. Lyn, M. H. Ramsey, S. Coad, A. P. Damant, R. Wood and K. A. Boon, The duplicate method of uncertainty estimation: are eight targets enough?, Analyst, 2007, 132, 1147–1152, 10.1039/b702691a.

This journal is © The Royal Society of Chemistry 2021 |