Challenges and Experiences in Data Integration to Support Research on Nonprofit Organizations

Nonprofit organizations are important contributors to the US economy and social well-being. The Nonprofit Organization Research Panel Project (NORPP) Manager has been developing and sharing datasets and software tools to facilitate data-driven research on nonprofit organizations. The project has two major thrusts: (1) large-scale survey panels, e.g., the Annual National Survey of Nonprofit Trends and Impacts, from 2021 to the current (2024); and (2) NORPP Analytics Integration Platform (NAIP) to collect, process, and query a wide range of socioeconomic indicator datasets, such as IRS 990 form and census data. Technical challenges that include data heterogeneity, data quality, and protection of sensitive data have made the expansion and maintenance of NAIP datasets both labor-intensive and time-consuming. We are currently exploring new technologies, including Large Language Models such as GPT series, to automate database query generation, schema adaptation, and data quality assurance processes.