Aspects of multi-layer feed-forward neural networks influencing the quality of the fit of univariate non-linear relationships

Abstract

In analytical chemistry, multi-layer feed-forward (MLF) neural networks are increasingly used as a technique to model (univariate) non-linear relationships. This paper investigates the influence of the MLF network parameters (transfer function, learning rate, momentum factor, number of hidden units, and weight initialization) and their interactions on the network performance.

The choice of the transfer function depends greatly on the nature of the non-linearity to be modelled. For data sets containing periodicities, results indicate that a sigmoid function yields the best network performance. Application of a sine function results in a slightly lower performance, but the MLF network optimization process is more robust with respect to the other network parameters. Finally, a procedure is outlined which might serve as a recipe to construct decision tree-like graphs which yield effective MLF network parameters sets for various problem domains.