Extending Layer-wise Relevance Propagation in Neural Networks using Semiring Annotations

Recently, neural networks have allowed computers to solve numerous problems in various fields such as natural language processing and computer vision.Compared to traditional algorithms, machine learning models have proven to be both more successful and more difficult to interpret.Neural networks are considered as black boxes that are unable to easily explain themselves, i.e. justifying the reasons that led them to make a prediction.Layer-wise Relevance Propagation (LRP) is a technique that has been introduced to ensure explainability by identifying the input features relevant to the output choice.In parallel, research in provenance theory has developed annotation techniques, which can for instance be used to compute query provenance in databases.In this paper, we extend LRP propagation rules to semiring-based provenance annotations of a network and implement semiringbased propagation rules for computer vision models of different scales.We show that different semirings lead to different types and granularities of explanations, and that such techniques can be applied to perform tasks like image mask computation and neural network pruning.