Evaluating AI models for food and alcohol advertisement classification against human benchmarks

The growth of food and alcohol marketing on social media creates a need for scalable monitoring methods that go beyond manual processing. This study evaluates whether Large Language Models and Vision-Language Models can recognize advertisements and identify their features in consistence with general public or expert opinion. We collected 1000 Facebook ads from major Belgian brands, and annotated them with 600 crowd workers, three dieticians and four AI models (GPT-4o, Qwen 2.5, Pixtral and Gemma3). Our analysis of the data shows that for single-option advertisement features, like alcohol presence or target group, GPT-4o and Qwen reached agreement with the dietician consensus above 90%, similar to the level of pairwise agreement observed between individual dieticians. Though agreement was lower for multiple choice features, like premium offers and marketing strategies, it was still within the variability observed in crowd raters. The bias analysis revealed how models interpret certain labels, with some being consistently under- or over-detected. Based on these findings, we propose tiered deployment recommendations that distinguish between ad features that MLLMs can already monitor with human-level accuracy, and more complex features requiring expert oversight and taxonomy refinement, like marketing strategies or food categories.