Seeing is Believing? A Survey and Benchmark of Hallucinations in Video Scene Graph Generation

Multimodal Large Language Models (MLLMs) have advanced Video Scene Graph Generation (VidSGG), yet they remain highly susceptible to hallucination—generating visually ungrounded entities and relationships. This issue is particularly severe in dynamic sports videos, where rapid motion and occlusion degrade visual features. In this paper, we propose a comprehensive taxonomy of hallucinations in VidSGG, categorizing errors by type (Object, Identity, Relation, Temporal) and root cause (Language Prior, Visual Bias, Missing Annotations). We design a targeted benchmark protocol on the SoccerNet dataset to evaluate open-source MLLMs under varying grounding conditions. Our empirical analysis of Qwen2-VL reveals that visual-only inference leads to severe repetition and generic hallucination. However, we demonstrate that external grounding via textual hints (simulating ASR and OCR) significantly anchors the model, reducing language prior bias and improving the specificity of generated scene graphs. We conclude by outlining open problems for grounding and structural consistency in future VidSGG systems. Author's Note: During the preparation of this manuscript, the author used AI for LaTeX formatting, code debugging, and language polishing. The author conducted all experiments, analyzed the data, and takes full responsibility for the final content.