Explaining Representations in Correlation-based Deep Multiview Representation Learning

Multiview representation learning techniques based on deep correlation maximization have become increasingly popular for learning meaningful and compact representations from multiview data. Even though their performance is state-of-the-art in many interpretability-critical fields, their black-box behavior poses a problem and restricts their usability. To overcome this restriction, we propose XDCCA (eXplanations for Deep Canonical Correlation Analysis), an explanation strategy using characteristic rules in combination with SHAP that exploits the inherent structure of latent spaces created by correlation maximization techniques. We demonstrate how XDCCA allows for interpreting learned representations and their correlation using real medical time series and synthetic image data.