Decoding Mouse Visual Tasks via Hierarchical Neural-Information Gradients

Understanding how the brain encodes and decodes dynamic neural responses to visual stimuli is crucial for revealing visual information representation. Currently, most methods (including deep neural networks, DNNs) often overlook the dynamic generation process of neural data, such as hierarchical visual data, within the brain’s structure. In this work, we introduce two decoding paradigms: fine-grained decoding tests (single brain regions) and coarse-grained decoding tests (multiple regions). Using the Allen Institute’s Visual Coding Neuropixel dataset, we propose the Adaptive Topological Vision Transformer (AT-ViT), which exploits a biologically calibrated cumulative hierarchy derived from single-area decoding performance to adaptively decode topological relationships across brain regions. Extensive experiments confirm the ‘Information-Gradient Hypothesis’: single-area decoding accuracy should recovers the anatomical visual hierarchy, and AT-ViT achieves maximal performance when this data-driven gradient is respected. AT-ViT outperforms non-hierarchical baselines (ada-PCA/SVM) by 1.08–1.93% in natural scenes and 2.46–3.34% in static gratings across sessions, peaking at hierarchy 3 (visual cortex + thalamus/midbrain) with up to 96.21%, but declining 1–2% when including hippocampus data, highlighting its random, performance-hindering nature. This work demonstrates hierarchical networks’ superiority for brain visual tasks and opens avenues for studying hippocampal roles beyond visual decoding.