Assessing grain production resilience under climate stress: temporal holdouts and cross-regional validation

Accurately identifying grain production resilience is a prerequisite for improving agricultural risk governance under climate change. The relationship between predictive accuracy for capacity-based indices and observed performance under climate shocks requires direct testing. Building on existing distinctions between potential and revealed resilience, this study compares capacity-based resilience with outcome-based resistance. We integrate grain production, socioeconomic, and climate data for Chinese prefecture-level cities, construct a provincial capacity-based composite index and a measure of resistance to climate shocks based on expected yield estimated using past information, and conduct a machine learning audit using strict temporal holdouts, transfer to nonoverlapping regions, nationwide replication, information-set ablation, probability calibration, and spatial residual diagnostics. The findings are threefold. First, capacity-based composite indices are readily predicted under random splits, but strict temporal and spatial validation substantially weakens this advantage. Direct tests of construct alignment reveal almost no stable correlation between capacity scores or their components and realized resistance. Second, the full city-level model has limited ability to predict continuous variation in realized resistance. Grain production history provides most of the information for below-trend classification, and probability calibration needs improvement. Third, aggregate discrimination is similar across the major producing areas, strict external sample, and nationwide replication, but annual AUC values differ. Comparisons of feature importance indicate similar patterns of information dependence, whereas spatial clustering of test residuals reveals regional prediction biases. We therefore argue that machine learning studies of agricultural resilience should assess construct validity, simple benchmarks, validation that prevents information leakage, external transfer, and calibration before interpreting features. High accuracy is substantively meaningful only when the measurement target aligns with the intended application.