The Target Comes From the Data: What Self-Supervised Learning Actually Learns

Self-supervised learning is often described as learning from data without labels. This description is useful but incomplete. A self-supervised encoder still depends on a chosen pretext target, representation, corruption rule, architecture, latent dimension, validation split, and audit boundary. This paper evaluates 72 self-supervised encoders on the 307 development vehicles from the complete-case Auto MPG data. Four representations are crossed with masked reconstruction, denoising reconstruction, and contrastive learning, with six parameter settings per family. Five expanding validation years select one encoder per representation-family group, yielding 12 primary models without consulting mpg, origin, or the later population. Across the 12 primary encoders, mean cross-encoder 10-nearest-neighbor Jaccard agreement is 0.350, with a minimum of 0.069. Mean pairwise-distance Spearman is 0.834. Mean input-embedding neighborhood fidelity is 0.590, but the minimum is 0.068. Under 20 deterministic 1% retraining perturbations, mean neighborhood stability is 0.844, while the least stable primary encoder falls to 0.280. The best frozen mpg linear probe reaches temporal R² 0.158, compared with 0.057 for the standardized input baseline. The strongest frozen origin probe reaches macro-F1 0.536. These probes are downstream audits only: mpg and origin do not select, tune, or train the self-supervised encoders. The findings show that a target generated from the data is still a target. Self-supervision can preserve useful information, but it does not define a unique representation by itself. The deposit includes the complete paper and a reproducibility package containing source code, data, machine-readable results for all 72 candidates, 12 selected encoders, frozen embeddings, two 12 × 12 agreement matrices, five generated figures, 20 deterministic perturbation audits per primary encoder, a SHA-256 integrity manifest, and an automated verifier.