title: Characterization of heteroscedastic measurement noise in the absence of replicates.
abstract: A method is described for the characterization of measurement errors with non-uniform variance (heteroscedastic noise) in contiguous signal vectors (e.g., spectra, chromatograms) that does not require the use of replicated measurements. High-pass digital filters based on inverted Blackman windowed sinc smoothing coefficients are employed to provide point estimates of noise from measurement vectors. Filter parameters (number of points, cutoff frequency) are selected based on the amplitude spectrum of the signal in the Fourier domain. Following this, noise estimates from multiple signals are partitioned into bins based on a variable that correlates with the noise amplitude, such as measurement channel or signal intensity. The noise estimates in each bin are combined to estimate the standard deviation and, where appropriate, a functional model of the noise can be obtained to characterize instrumental errors (e.g., shot noise, proportional noise). The proposed method is demonstrated and evaluated with both simulated and experimental data sets, and results are compared with replicated measurements. Experimental data includes fluorescence spectra, ion chromatograms from liquid chromatography/mass spectrometry, and UV-vis absorbance spectra. The limitations and advantages of the new method compared to replicate analysis are presented.
authors.affiliation: Trace Analysis Research Centre, Department of Chemistry, Dalhousie University, P.O. Box 15000, Halifax, NS B3H 4R2, Canada. Electronic address: peter.wentzell@dal.ca.
authors.forename: Peter D
authors.initials: PD
authors.lastname: Wentzell
authors.affiliation: Trace Analysis Research Centre, Department of Chemistry, Dalhousie University, P.O. Box 15000, Halifax, NS B3H 4R2, Canada.
authors.forename: Anthony C
authors.initials: AC
authors.lastname: Tarasuk
keywords: Digital filters; Error models; Heteroscedastic noise; Limit of detection; Measurement errors; Variance estimation