Influence of precision, sample size and design on the beta error of linearity tests

Abstract

The beta error of two linearity tests for the detection of lack-of-fit to a straight line calibration model based on small sample size (6≤n≤10) has been evaluated namely, the ANOVA lack-of-fit to the first degree model and the test of significance of b₂, the second degree coefficient from the second degree model fitted through the data. For these investigations different models, designs and measurement precisions were considered by means of simulations. The effect of heteroscedasticity was also evaluated. The results obtained indicated that the test of significance of b₂ in general performs better than the ANOVA test. The designs with three concentration levels (measurements positioned at both extremes and at concentration in the middle of the calibration range) give the best results. Heteroscedasticity and low measurement precision increase the probability that lack-of-fit is not detected. Two means to predict the beta error for the test of significance of b₂ are proposed: (i) prediction derived from the plots of the beta error versus the ratio of the deviation from linearity (expressed as the average relative prediction error) to the measurement precision (expressed as %RSD at the concentration in the middle of the calibration range) and (ii) prediction using the formula. The formula to determine the sample size that allows the detection of a certain deviation from linearity with a specified beta error is also given.