Ultraviolet spectral transfer based on a convolutional variational autoencoder model for detecting chemical oxygen demand in rivers

Abstract

The spectral data shift generated using two different ultraviolet (UV) instruments to detect chemical oxygen demand (COD) in rivers often invalidates existing predictive models. To solve this problem, this study proposes a non-linear spectral transfer method based on a convolutional variational autoencoder (CVAE). This method extracts key features from spectral data using an encoder and reconstructs the original data using a decoder, which significantly reduces the impact of spectral shifts caused by differences in the resolution of measurement devices. Specifically, the encoder of the CVAE combines the convolutional layers with a sigmoid activation function to effectively capture the non-linear relationship between the spectra of the master and slave devices. Furthermore, in the face of the problem that traditional autoencoders lack the modeling ability for data distribution, this method adopts the idea of variational inference to learn the potential representation of COD spectral signals, thereby effectively capturing the global features of the COD spectral signals. Finally, we input the transformation spectra derived from the spectral space transformation, piecewise direct standardization, deep autoencoders, convolutional autoencoder, and the proposed method into the partial least squares multivariate calibration model. The results show that the proposed method reduces the root mean square error of the prediction by 1.0242, 0.4571, 0.4201, and 0.1965, respectively, compared to the other four methods. The above results indicate that the proposed method effectively solves the problem of model sharing between different instruments and provides new perspectives for research and applications in related fields.