One-Sample Tilted Empirical Risk Minimization: Finite-Sample Tracking, Denominator Order, and Robustness

Background/Aim: Classical tilted empirical risk minimization (TERM) assigns loss-dependent weights through a global normalizer. Direct evaluation on one observation removes relative weighting. We ask how closely a strict one-current-sample approximation tracks finite-sample TERM and how its finite-step behavior depends on denominator order, normalizer rate, and tilt.Methods: A scalar moving-average normalizer carries cross-sample information between updates. We compared a previous-state denominator with a matched current-sample-first order on a toy problem using full-batch oracles, seven normalizer rates, and 20 shared streams. Scale-calibrated tilts were then evaluated with Polyak–Ruppert-averaged stochastic gradient descent (SGD) under four Breast Cancer Wisconsin Diagnostic (WDBC) label-noise conditions and seven Abalone response-contamination conditions.Results: Both orders remained close to the toy finite-sample references. They were exactly identical at zero tilt; at nonzero tilt, their local low-rate discrepancy was approximately proportional to the product of learning and normalizer rates and changed direction with tilt. No order was uniformly superior. Negative tilt improved on empirical risk minimization (ERM) in specified corrupted conditions. Representative task-specific baselines were stronger in the most corrupted conditions, while the finite-sample TERM tracking findings were unchanged.Conclusion: One persistent scalar can preserve tilt-dependent one-sample updates and closely track classical TERM in the tested finite-dimensional settings. The evidence is empirical and finite-horizon.