A new era in poverty diagnostics: Experiential knowledge from machine learning to improve the efficiency of social programs in combating poverty

Public social programs—conceived as a response to societal demands for broader welfare coverage not only in Brazil but also worldwide—rely heavily on self-reported data from registries of low-income households to identify beneficiaries. Consequently, reliance on self-reporting possibly introduces biases and inefficiencies, undermining scarce public resources. To counter this, we embarked on designing a concise questionnaire for diagnosing poverty, which has the potential to increase precision, reduce administrative costs, and improve the cost-effectiveness of social programs—and thus ensure a wider social impact. The current study creates a tool-using machine-learning (ML) to better identify who needs help by using a quick, affordable, and databased questionnaire that can sort and rank households by their risk of poverty. Unlike traditional methods of means test (MT), it avoids questions about household income. Instead, it relies on education level, the number of rooms, and the ownership of durable goods—all verifiable characteristics for a probabilistic poverty risk index. This report draws on the Poverty Probability Index (PPI), the Survey of Well-being via Instant and Frequent Tracking (SWIFT), and the World Bank’s "Pover-T Tests: Predicting Poverty" competition. The proposed approach introduces an innovation using a probabilistic risk index generated by a predictive model, promising to ensure social impact and a step towards innovative data-driven solutions. Ultimately, this research offers policymakers a robust instrument for the evaluation and optimization of social welfare programs, thus contributing more effectively to the fundamental goals of poverty reduction.