Améliorer l’interprétabilité des réseaux de neurones par l’apprentissage de représentations structurées

Deep learning has revolutionized artificial intelligence, enabling advances in many application domains. However, this performance often comes at the cost of transparency. Due to their complexity, neural models are frequently perceived as black boxes, which is a major obstacle to their understanding and adoption in sensitive sectors where trust and explainability are crucial.This thesis addresses the central challenge of interpretability by focusing on learning structured internal representations. Our hypothesis is that by constraining how models encode information, it is possible to design architectures that are intrinsically more transparent. Our contributions are organized around two main directions: learning discrete representations for sequential models, and learning disentangled representations for complex data.The first part of this thesis aims to replace the continuous representations traditionally used in neural networks with discrete or quantized representations. By forcing a model’s internal states to belong to a finite set of symbols, we transform its functioning to resemble that of logical and symbolic systems. We explore this idea in the context of recurrent neural networks (RNNs), developing methods to discretize their latent representations, either during or after training.This approach allows us to establish a formal equivalence between an RNN and a finite-state automaton. Such a symbolic structure turns a black box into a white box. This opens the way for new explainability strategies by enabling exhaustive analysis of the model’s behavior. Furthermore, we show that an additional major benefit of discretization is a speed-up in inference. Complex matrix operations are replaced by simple table lookups, making these models particularly suited to resource or latency constrained environments.The second part of the thesis focuses on learning disentangled representations, where each factor of variation in the data is encoded independently. In the context of generative models based on variational autoencoders (VAEs), we introduce and formalize the concept of hallucination. Although this phenomenon has been studied in the field of text generation, we adapt it here to image generation to refer to the creation of unrealistic or inconsistent attributes during latent space manipulations.We propose a metric to quantify this phenomenon and show empirically that, while related, disentanglement and hallucination are two distinct properties of a representation. Finally, we revisit the dominant disentanglement paradigm, which assumes that each generative factor should be captured by a single latent dimension, and propose a different approach. Noting the limitations of this approach for complex factors, we propose a vector-wise disentanglement framework, in which each generative factor is represented by a group of latent components. We adapt state-of-the-art metrics to this new setting and demonstrate its improved disentangling capability on real-world data.