Multi-Resolution TopK Sparse Autoencoders Improve Reconstruction Transfer but Not Object-Level Sparse Identity

Sparse autoencoders are increasingly used as interpretability interfaces for vision representations, where individual sparse latents can act as handles for recurring visual features. We study a deliberately narrow setting: TopK sparse autoencoders trained on DINOv2 ViT-S/14 layer-9 patch features from one COCO train2017 subset and evaluated on COCO val2017 objects. We ask whether training on mixed input resolutions improves the stability of sparse feature handles across resolutions. Multi-resolution training gives competitive reconstruction transfer, but does not preserve object-local sparse feature identity as well as high-resolution single-resolution baselines. A three-seed core check supports the main high-resolution baseline gap, while focused controls for per-resolution exposure, dictionary size, and active TopK budget improve reconstruction more readily than object-level sparse-support stability. The result is a scoped cautionary finding: for TopK SAEs on this backbone and layer, reconstruction transfer and object-level sparse identity should be evaluated separately when using sparse latents as feature-tracking tools.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-07-01
DOI
https://doi.org/10.5281/zenodo.21112011
Primary Topic
Generative Adversarial Networks and Image Synthesis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Multi-Resolution TopK Sparse Autoencoders Improve Reconstruction Transfer but Not Object-Level Sparse Identity

Arsenii Shvetsov
Zenodo (CERN European Organization for Nuclear Research)
Generative Adversarial Networks and Image Synthesis
preprint

Multi-Resolution TopK Sparse Autoencoders Improve Reconstruction Transfer but Not Object-Level Sparse Identity

Arsenii Shvetsov
preprint en

Abstract

Sparse autoencoders are increasingly used as interpretability interfaces for vision representations, where individual sparse latents can act as handles for recurring visual features. We study a deliberately narrow setting: TopK sparse autoencoders trained on DINOv2 ViT-S/14 layer-9 patch features from one COCO train2017 subset and evaluated on COCO val2017 objects. We ask whether training on mixed input resolutions improves the stability of sparse feature handles across resolutions. Multi-resolution training gives competitive reconstruction transfer, but does not preserve object-local sparse feature identity as well as high-resolution single-resolution baselines. A three-seed core check supports the main high-resolution baseline gap, while focused controls for per-resolution exposure, dictionary size, and active TopK budget improve reconstruction more readily than object-level sparse-support stability. The result is a scoped cautionary finding: for TopK SAEs on this backbone and layer, reconstruction transfer and object-level sparse identity should be evaluated separately when using sparse latents as feature-tracking tools.

Zenodo (CERN European Organization for Nuclear Research)
Generative Adversarial Networks and Image Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.