A Color Ceiling in Unsupervised Clustering of Texture Images: Validation Against Expert Annotations on Held-Out Domains

Unsupervised clustering of deep image features is often proposed for unlabeled image archives, yet it is usually judged by internal criteria rather than by agreement with expert annotations. In a single domain of texture images, namely full-size rock-core scans, we evaluate unsupervised partitions against depth-registered expert descriptions of 20 wells (nine classes), holding out entire wells. Color statistics, texture descriptors, VGG16, two DINOv2 models and SimCLR encoders pre-trained on the images are combined with k-means, Gaussian mixtures and SCAN, several seeds, and pre-training runs. Across all combinations, the mean class accuracy plateaus at 0.35–0.40 (chance level 0.12), and six color statistics reach this plateau with a Gaussian mixture model (0.37). Apparent gains of in-domain self-supervised features depend on the clustering algorithm and the pre-training run. The plateau is robust to the evaluation protocol, a coarser class scheme and more diverse fitting wells, and persists on color-normalized and grayscale images. Linear probes on the same frozen features reach 0.47–0.54, which locates the bottleneck in the unsupervised objectives. Fitting the clustering transductively on all unlabeled images lifts in-domain features to 0.42–0.48, but this gain does not transfer to unseen wells. The conclusions are specific to this domain.