Generalization memorization machine for regression

In machine learning, the pursuit of minimizing empirical risk may lead to overfitting, especially when dealing with small datasets. To overcome this problem, regularization techniques are commonly employed; however, these techniques may also reduce the memorization capacity of the model such as support vector machine (SVM) and support vector regression (SVR). The recently proposed Generalization Memorization Machine (GMM) solves these problems and achieves zero empirical risk while demonstrating better generalization ability. In this paper, the generalization memorization machine (GMM) is applied into regression problems to propose a generalization memorization machine for regression (GMMR). Our GMMR memorizes samples that are outside an ε-tube. Two models of GMMR, i.e., hard and soft GMMRs (called HGMMR and SGMMR) are presented, where HGMMR guarantees the ε learning errors and the SGMMR relaxes the learning errors. The optimization problems in the proposed models are quadratic programming problems that can be solved efficiently. Experimental results on benchmark datasets show that our HGMMR and SGMMR have significant improvements in memorization and generalization compared to some state-of-the-art support vector regressions.