The Hierarchical Disentangled Information framework enables the discovery of meaningful structures

This paper introduces Hierarchical Disentangled Information (HDI), an unsupervised hierarchical framework that leverages information theory to disentangle atomic features across different hierarchical levels, uncovering meaningful latent variables beyond pre-defined labels. HDI facilitates the discovery of complex information structures without reliance on labeled data by effectively managing information flow across hierarchy levels, maximizing mutual information between them, and minimizing redundancies within each level. A significant contribution of this work is the introduction of the Hierarchical Disentanglement Score (HDS), a novel suite of metrics designed to address the critical gap in evaluating hierarchically disentangled representations, moving beyond "flat" disentanglement assessments. We evaluate HDI on four widely used benchmarks-MNIST, CIFAR10, STL-10, and SVHN-and our comprehensive experimental results showcase its efficiency in unraveling complex information structures. HDI demonstrates a strong balance between disentanglement and reconstruction, achieving high MIG scores and a remarkably low TC-Proxy value while maintaining a low reconstruction loss. This quantitative evidence is complemented by compelling visualizations, where HDI's latent dimensions exhibit clearly separated, unimodal peaks, confirming its ability to capture isolated and interpretable factors of variation. Critically, HDI achieves superior and stable performance across all HDS metrics, significantly outperforming other VAE models, including those designed for disentanglement. With a top HDSrc score, HDI proves its unparalleled ability to learn robustly disentangled, hierarchically structured, complete, and reconstructive latent representations. Although trade-offs between disentanglement and reconstruction are inherent, our holistic evidence from novel metrics, established benchmarks, and qualitative analysis solidifies HDI's exceptional performance in learning truly disentangled and interpretable representations.