Applicability domain of a calibration model based on neural networks and infrared spectroscopy

Artificial neural networks are used as calibration models in routine analytical determinations that involve spectroscopic data. To ensure that the model will generate reliable predictions for new samples, the applicability domain must be well defined. This article describes a strategy for establishing the limits of the applicability domain when the calibration model is a feed-forward neural network. The applicability domain was defined by two limits: 1) the 0.99 quantile of the squared Mahalanobis distance calculated from the network activations of the training set and 2) the 0.99 quantile of the reconstruction error of the training spectra using either an autoencoder network or a decoder network. A new sample with a squared Mahalanobis distance and/or spectral residuals beyond these limits is said to be outside the applicability domain, and the prediction is questionable. The approach was illustrated by predicting the density of diesel fuel samples from mid-infrared spectra and the fat content in meat from near-infrared spectra. The methodology could correctly detect anomalous spectra in prediction using either the autoencoder or the decoder. • A new method to establish the applicability domain (AD) of a regression network. • One limit of the AD derived from the Mahalanobis distance of network activations. • The other limit of the AD derived from spectral residuals from an autoencoder. • A neural network was used to predict the density of diesel samples using IR spectra. • A neural network was used to predict the fat content of meat using NIR spectra.