Pre-Analysis Plan: The Effects of AI-Produced Political Videos on Political Attitudes (Study 2)

Political campaigns now distribute AI-produced videos that put words in the mouths of real politicians, and the consequences for how viewers evaluate the targets are largely unknown. We pre-register a survey experiment on Prolific (N = 3,000) in which respondents are randomly assigned to a placebo video (a Progressive Insurance advertisement) or to one of four AI-produced attack ads, each against a different Democratic nominee for the U.S. Senate (Josh Turek in Iowa, Peggy Flanagan in Minnesota, James Talarico in Texas, and Chris Pappas in New Hampshire), with the placebo arm twice the size of each video arm because all four videos are compared with it. We estimate each video's effect on feelings toward its target with four design-based estimators, two of which adjust for a large language model's predictions of each respondent's potential outcomes, and we compare the estimates with the effects the LLM predicts. Applied to Study 1, an already completed experiment on a single video (N = 3,086), the four estimators agree closely, the LLM predictions add no precision beyond conventional covariates, and the LLM's predicted effects miss two of four experimental estimates by a factor of two or more. We filed this plan on October 30, 2026, the day Study 2 began fielding, and before any of us had seen any of its outcome data. Fielding began before the plan was filed because the study had to be in the field before the November 3, 2026 midterm election. Files: ai_slop_study2_pre_analysis_plan.pdf is the pre-analysis plan. ai_slop_study2_code_and_predictions.zip holds the analysis code (study_2/, run by study_2/run_all.R), the simulated data on which the plan's worked example is run, and the LLM predictions the plan registers (pre_analysis_plan/related_content/final_predictions/ and ads_qsf_predictions/).