Optimization Problem of Constructing Linear Regressions with a Minimum Value of the Mean Absolute Error on Test Sets

<p>This article is devoted to the problem of selecting a given number of the most informative regressors in linear regressions. When using the ordinary least squares method, the exact solution to this problem by the criterion of maximizing the coefficient of determination when using the entire data set can be obtained as a result of solving a specially formulated mixed 0-1 integer linear programming problem. However, in machine learning, an important stage in creating a reliable and efficient model is its construction based on the training set and checking the accuracy of its prediction based on the test set. Therefore, in this article formulates an optimization problem for subset selection in linear regressions based on the criterion of minimizing the mean absolute error on the test set. The formulation is based on a well-known technique, according to which absolute errors should be presented as the difference between two non-negative variables. Computational experiments were carried out using the statistical data on athletes' salaries stored into the Gretl package and the LPSolve optimization problem solver. For this purpose, the training set was formed from 70%, 75%, and 80% of observations. In all these cases, the average decrease in the value of the coefficient of determination of the models was 24.76%, 18.4%, and 12.22%, but the mean absolute error decreased by 24.8%, 26.3%, and 21.05%, respectively. Experiments showed that the average time to solve problems when minimizing the mean absolute error on test sets was 2.33–2.85 times higher than the time to solve problems when maximizing the coefficient of determination on training sets.</p>