AI-imputed and crowdsourced price data show strong agreement with traditional price surveys in data-scarce environments

Continuous access to up-to-date food price data is crucial for monitoring food security and responding swiftly to emerging risks. However, in many food-insecure countries, price data is often delayed, lacks spatial detail, or is unavailable during crises when markets may become inaccessible, and rising prices can rapidly exacerbate hunger. Recent innovations, such as AI-driven data imputation and crowdsourcing, present new opportunities to generate continuous, localized price data. This paper evaluates the reliability of these approaches by comparing them to traditional enumerator-led data collection in northern Nigeria, a region affected by conflict, food insecurity, and data scarcity. The analysis examines crowdsourced prices for two staple food commodities, maize and rice, submitted daily by volunteers through a smartphone application over 36 months (2019-2021), and compares them with data collected concurrently by trained enumerators during the final eight months of 2021. Additionally, the crowdsourced dataset is compared to AI-imputed prices from the World Bank's Real-Time Prices (RTP) database. Data from the alternative methods reflected similar price inflation trends during the COVID-19 pandemic. Pearson's correlation coefficients indicate strong statistical agreement between crowdsourced and enumerator-collected prices (r = 0.94 for yellow and white maize, r = 0.96 for Indian rice, and r = 0.78 for Thailand rice). Furthermore, the crowdsourced data shows a high correlation with the AI-imputed prices (r = 0.99 for maize, and r = 0.94 for rice). The results from additional statistical tests of normality and paired means shows that the discrepancies between price datasets are consistent with measurement error rather than differences in actual price dynamics. Further tests of equivalence confirmed that enumerator and crowdsourced prices represent the same underlying market processes for specific commodity subtypes, and connotes that crowdsourced price data is a credible reference for validating AI-imputed estimates. The results support the use of AI imputation and crowdsourcing methods to improve price data collection and track market dynamics in near real time. These data innovations can be particularly valuable in areas that are underrepresented in national aggregate data due to limited monitoring capacity, and where high-frequency local data can aid targeted interventions.