HiLite: Hierarchical Level-implemented Architecture Attaining Part-Whole Interpretability

Beyond the traditional CNN structure, we have recently witnessed lots of breakthroughs in computer vision architectures such as Vision Transformer, MLP-Mixer, SNN-MLP, and so on. However, many efforts in developing novel architectures for vision tasks are heavily focused on achieving powerful performances, and how to attain interpretability in a trained neural network remains an open question. Inspired by the imaginary system GLOM, we present HiLite : <u>Hi</u>erarchical <u>L</u>evel-<u>i</u>mplemented Archi<u>te</u>cture attaining Part-Whole Interpretability, where islands of identical vectors can provide unprecedented interpretability. In our column-like structure, each level is a layer of a part-whole hierarchy composed of multiple neurons, and the function to define the neural field along an image input patch is initialized as the level vector inside the model. We propose two-column networks (Top-Down (TD) and Bottom-Up (BU)) that allow inter-level communication between adjacent levels on a specific patch and propose Gated Consensus Attention to perform intra-level communication on different patches within the level. At each time step, the level vector and outputs from different networks are combined into a weighted sum and passed to the next step, and outputs from the final time step are utilized as representation vectors. Here, supervised contrastive learning is used to find the relationship of meaningful patches in each class, where negative examples contribute to preventing representation collapse between neighboring patches. HiLite shows a possibility of performance through a quantitative evaluation on four image classification datasets as well as two metrics for assessing representation quality and showcases the intrinsic interpretability by simply generating a visual cue. We believe that our work is a solid step towards novel research on neural architectures attaining interpretability.