Detecting overfitting by examining residuals in autoregressive models

The aim of this study is to see how overfitting can be detected using non-rigorous analysis of residuals. The well-known statistical packages was used to simulate data from stationary autoregressive models with Gaussian white noise with mean 0 and variance one. In order to see the effect of realization size on our findings, the sample size 50 was used as an example of small realization and the sample size 500 was used as an example of large realization. The method of maximum likelihood was used in the fitting of autoregressive models to the simulated data which is available in the statistical package R. Interesting and promising results were obtained. Our study seems to suggest that comparing estimates with their standard errors is the only reliable criterion in spotting or detecting overfitting. To make sure that the defect in the behavior of the residuals is due only to the over, we used only the same class of models in the simulation and the fitting.