Comparative Analysis of One-Dimensional Regression Techniques

This research paper presents a comprehensive comparison of eight regression techniques applied to a one-dimensional dataset. The study evaluates linear regression, polynomial regression, ridge regression, lasso regression, decision tree regression, random forest regression, Support Vector Regression (SVR), and gradient boosting regression. Using a dataset that correlates years of experience with salary, we assess the performance of each method based on key metrics such as Mean Absolute Error (MAE), Mean Squared Error (MSE), and R-squared (R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>). Our findings reveal significant differences in the accuracy and robustness of these models. Linear and polynomial regression provide a baseline for comparison, while regularization methods such as ridge and lasso regression demonstrate improved stability against overfitting. Decision tree and random forest regression capture non-linear relationships with varying degrees of success. SVR, with its kernel-based approach, adapts well to complex patterns, and gradient boosting regression shows superior predictive performance through ensemble learning. This study delves into the strong points and weaknesses of eight regression techniques, guiding practitioners in selecting appropriate models for their specific applications.