Instance-based visualization and analysis of neural networks

Neural network models are widely used, and visualization helps to understand the black-box behavior of these models. Current visualization methods mainly focus on neural networks trained on data with an intrinsic representation (image, text, speech) and depend on human interpretation of the data. However, generic multivariate data is the most commonly used form of data, and neural network visualization options are limited. Furthermore, current methods mainly focus on showing the final learned weights and filters. In contrast, we propose an instance-based approach and show the flow of instances through the neural network to explain model behavior. The visualization method is centered around selecting instances of interest and showing the propagation of weight and activation contribution to the final classifications. This enables users to explore and understand both global and local model behavior by inspecting varying groups of instances. Combined automated and interaction techniques enable tracing importance-scored paths to explore and understand feature importance. The effectiveness of the visualization method is shown through examples and use cases on real-world classification datasets and compared with insights from computational explainability methods. Additionally, a qualitative user study confirms the effectiveness and value in analyzing neural networks using our instance-based visualization approach.