Evaluating hv-block cross-validation for dependence-aware model selection in reference evapotranspiration estimation

Reference evapotranspiration (ET o ) estimation by machine learning techniques, e.g. Random Forest (RF), requires reliable model selection under temporally autocorrelated hydro-meteorological conditions. The hv-block cross-validation procedure 1 has been proposed as an assumption-light, dependence-aware alternative to classical information criteria. On the other hand, its consistency has been criticized because the procedure’s block construction does not satisfy the combinatorial balance of a Balanced Incomplete Block Design (BIBD) on which the original consistency argument relied, so that its theoretical consistency remains unassessed. The present research addressed this gap through a two-phase empirical design. In the first phase, a controlled Monte Carlo simulation (R = 500 replications per autocorrelation strength \(\phi \) ) with a known linear data-generating process was performed to test whether hv-block’s over-selection bias is detectable, persists as sample size scales from n = 4,000 to n = 10,000 and then n = 20,000, and is related to the structural design of the procedure or to autocorrelated noise. In the second phase, the same hv-block framework, using both PACF-based and cyclical exclusion windows, was applied to a real-world, 4000-day ET o dataset from two weather stations in Iran. RF was employed as the base model, and out-of-bag error and 5-fold cross-validation were utilized as temporally naïve benchmarks. As the true input set is unknown for real data, this phase assessed the practical consequences of the criterion choice rather than over-selection. The obtained results revealed that hv-block showed a higher over-selection rate (23–30%) than BIC and AIC at all tested autocorrelation levels, and than 5-fold CV in most configurations. Further, no bias diminishing was detected within the investigated sample-size range ( n = 4000–20,000). Control experiments (ϕ = 0 and h = 0) indicated that the over-selection is mainly associated with the structural design of the procedure (the exclusion gap and the resulting fold geometry and replication imbalance), while the autocorrelated noise showed a secondary role. Nevertheless, the resulting performance accuracy losses were negligible in both the simulation (RMSE differences below 0.002) and real-data applications (RMSE differences lower than 0.01 mm day⁻ 1 at both stations). Overall, while the results partially confirmed the theoretical concern regarding the consistency of hv-block cross-validation, its practical consequences for operational ET o estimation and irrigation water management applications were minimal.