Delta-Iterations: A General-Purpose Linear Regression Solver for Huber's Penalty, $ε$-Insensitive Loss, and Other Moreau Envelope-Based Loss Functions

A general-purpose linear regression solver for a class of smooth loss functions is presented. The method applies to loss functions of the Moreau envelope type, which are continuously differentiable and have Lipschitz-continuous first derivatives. This class includes piecewise-defined cost functions consisting of constant, linear and quadratic segments, such as Huber's loss, insensitive least squares, one-sided least squares. The convergence guarantee provided by the associated proximal operator, the simplicity of the method and its versatility in handling various cost functions within a unified framework are its main advantages. MATLAB code for reproducing the numerical results presented in this article is provided.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-12
DOI
https://doi.org/10.48550/arxiv.2611.16558
Primary Topic
Matrix Theory and Algorithms
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Delta-Iterations: A General-Purpose Linear Regression Solver for Huber's Penalty, $ε$-Insensitive Loss, and Other Moreau Envelope-Based Loss Functions

Çağatay Candan
arXiv (Cornell University)
Matrix Theory and Algorithms
preprint

Delta-Iterations: A General-Purpose Linear Regression Solver for Huber's Penalty, $ε$-Insensitive Loss, and Other Moreau Envelope-Based Loss Functions

Çağatay Candan
preprint en

Abstract

A general-purpose linear regression solver for a class of smooth loss functions is presented. The method applies to loss functions of the Moreau envelope type, which are continuously differentiable and have Lipschitz-continuous first derivatives. This class includes piecewise-defined cost functions consisting of constant, linear and quadratic segments, such as Huber's loss, insensitive least squares, one-sided least squares. The convergence guarantee provided by the associated proximal operator, the simplicity of the method and its versatility in handling various cost functions within a unified framework are its main advantages. MATLAB code for reproducing the numerical results presented in this article is provided.

arXiv (Cornell University)
Matrix Theory and Algorithms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon