Enhanced weighted least squares regression: A robust approach for managing outliers and heteroscedasticity

The Ordinary Least Squares (OLS) approach performs best in the best-case scenario, but when the homoscedasticity and outlier-absence assumptions are broken, it performs noticeably worse. With an emphasis on Robust Weighted Least Squares (RWLS), M-estimation, and Least Trimmed Squares (LTS), this paper assesses robust alternatives to OLS. OLS was demonstrated to be extremely sensitive to outliers using a dataset on state-level crime rates in the US; the Mean Squared Error (MSE) increased from 5.480 (original data) to 12.580 (with outliers). On the other hand, RWLS using Tukey's Bisquare function produced the most consistent coefficient estimates and the smallest MSE, increasing from 4.520 to 5.340. In comparison to OLS, M-estimation and LTS also demonstrated lower MSE and higher resilience. The findings show how well robust approaches—in particular, RWLS with Tukey's Bisquare—address heteroscedasticity and outliers, which makes them essential for practical regression analysis.