Compute-Optimal Scaling of JEPA Compared with Masked Autoencoders and Contrastive Learning

Scaling studies often compare methods at a single model size, which can reverse the ranking at larger scale. We study how the downstream performance of JEPA, masked autoencoders (MAE), and contrastive learning scales with compute, and we estimate the compute-optimal model size for each method. We use fixed data and budget grids, five seeds per point, and a synthetic data family with controllable complexity to isolate the effect of scale from data. A random-feature pilot shows a test-error minimum at intermediate width; the compute-matched JEPA comparison is pending. Preprint. Results in this version come from small-scale pilot experiments; the full experimental protocol is described in the text and is pending.