Decodable but Not Clusterable: Coarse Supervision, Within-Class Geometry, and a Label-Free Diagnostic for Recoverable Subclass Order
Abstract Networks trained with coarse labels often keep fine-label information that a linear probe can read but that clustering cannot recover. We give this dissociation a mechanism, a test and a remedy. Within a coarse class, the two-means optimum is captured by the leading nuisance eigendirection once the ratio of that eigenvalue to the semantic variance, the spectral dose , passes a crossover between 1 and π/2; beyond π the optimum is provably blind to the fine label. Within-class whitening removes this phase transition and recovers the Bayes semantic split. A label-free monitor, the k-means gain relative to a Gaussian with the same covariance, is bounded by 1 + π/(2·dose) and yields a repair rule that needs no fine labels. Across seven settings and 647 class snapshots, cross-entropy fine-tuning from in-domain self-supervised bases raised the dose and cut clusterability by 59–63% while the probe stayed flat; ImageNet and DINOv2 bases lowered the dose and gained. The monitor tracked clusterability changes (Spearman 0.77), and whitening recovered 48–59% of the loss on CIFAR-100 and all of it on Fashion-MNIST. The rule still helped when the number of clusters was also estimated, whereas covariance-adaptive clustering stayed trapped.
Authors
- Volkan Oban (ORCID: https://orcid.org/0000-0003-1046-9155)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-13
- DOI
- https://doi.org/10.21203/rs.3.rs-11708078/v1
- Primary Topic
- Domain Adaptation and Few-Shot Learning
- Type
- preprint
Funders
- Nvidia