Can ChatGPT accurately identify the position of parties? A validation study with an expert survey in France

We evaluate whether ChatGPT can be used to estimate the ideological positions of parties in real time. Compared to other methods, ChatGPT can be used at all times, with minimal cost, no technical skills, and for all parties, including new ones. In a validation exercise, we asked ChatGPT and a group of local experts to simultaneously estimate the positions of French political parties in the context of the 2024 European election on a series of issues from the Chapel Hill Expert Survey. We find that ChatGPT’s estimations are generally close to those of the expert survey, especially for major and salient political issues (left-right, European integration, immigration). However, it provides less accurate estimates on fringe issues (free market, asylum policies, urban–rural interests) and specific party characteristics (within-party division, issue blurring). Thus, ChatGPT shows strong potential as an alternative data source for estimating parties’ positions but should be used with caution.