Weighted least-squares in calibration: What difference does it make?

Abstract

In univariate calibration, an unknown concentration or amount x₀ is estimated from its measured response y₀ by comparison with a calibration data set obtained in the same way for known x values. The calibration function y = f(x) contains parameters obtained from a least-squares (LS) fit of the calibration data. Since minimum-variance estimation requires that the data be weighted inversely as their true variances, any other weighting leads to predictable losses of precision in the calibration parameters and in the estimation of x₀. Incorrect weighting also invalidates the apparent standard errors returned by the LS calibration fit. Both effects are studied using Monte Carlo calculations. For the strongest commonly encountered heteroscedasticity, proportional error (σ_i ∝ y_i), neglect of weights yields as much as an order of magnitude precision loss for x₀ in the small x region, but only nominal loss in the calibration mid-range. Use of replicates gives great improvement at small x but can underperform unweighted regression in the mid-to-large x region. Variance function estimation approximates minimum-variance, even though the true variance functions are not well reproduced. A relative error test applied to the calibration data themselves is predisposed to favor 1/y² (or 1/x²) weighting, even if the data are homoscedastic. This predisposition weakens when replicate measurements are taken and disappears when the test is applied to an independent set of data. The distinction between a priori and a posteriori parameter standard errors is emphasized. Where feasible, the a priori approach permits reliable assignment of weights, application of a χ² test, and use of the normal distribution for confidence limits.