Nonparametric assessment of the predictive accuracy of models using time series of methane concentrations in the atmospheric air of the Arctic Island of Bely

Over the past few decades, the number, variety, and complexity of time series forecasting models have grown. There has also been an increased interest among researchers in comparing and assessing the predictive accuracy and performance of models, and in determining which models are more accurate. The researchers propose using the hypothesis testing approach to assess the performance of a time series forecast model. They obtained data for the study while monitoring the dynamics of ground-level concentrations of the main greenhouse gases on the Arctic Island of Bely, Yamalo-Nenets Autonomous Area, Russia. A total of three models based on autoregressive neural networks with exogenous input (NARX) were considered to predict changes in methane concentration in the surface layer of atmospheric air. The performance of the models was assessed using 8 “traditional” indices (correlation and determination coefficients, Wilmott goodness-of-fit indices, mean absolute error, relative mean square error, etc.) and the proposed permutational approach using the hypothesis testing method. In general, based on the data under study, the estimates of the permutation approach and other indicators of accuracy and error coincide, but there are discrepancies. Effect sizes do not always determine the statistical significance of differences.