Unboxing deep learning models using gradient-based methods — A tutorial

Deep learning models, particularly convolutional neural networks, have demonstrated remarkable performance in regression and classification tasks involving spectroscopic data. However, their black-box nature is considered a major drawback limiting the applicability of the methods. This paper provides a comprehensive tutorial on gradient-based methods that address this problem by providing information about the contribution of individual predictors to the predicted value. These methods are based on understanding how the data flows inside a neural network model and thereby can be used to evaluate variable importance and the validity of the model as a whole. The paper introduces the concept of gradients in neural networks, presents detailed derivations of saliency maps (vanilla gradients) and integrated gradients, supplemented by numerical examples that illustrate the computational procedures step by step. The theoretical part is followed by examples based on both simulated data and two well-known spectroscopic datasets (Tecator and Beer). The examples demonstrate that integrated gradients provide stable and interpretable importance scores that are consistent across different model architectures and training iterations when the underlying model performs well. The paper also proposes a small modification based on the gradient-response product (GIR) that improves the informativeness of the method for noisy datasets and datasets with systematic but irrelevant variation. The main content is accompanied by Python code in the form of a Jupyter notebook, making the methods accessible for practical implementation.