From Pixels to Concepts: How <scp>AI</scp> Interprets Visual Concepts in Social Media Videos? A Case Study of Mental Health <scp>YouTube</scp> Shorts

ABSTRACT Visual information on social media is increasingly central to how people share and seek knowledge across health, political, and everyday domains. As video content proliferates, AI‐assisted analysis offers new possibilities for social media research and information retrieval. This study examines the potential of multimodal large language models (MLLMs) to interpret conceptual content in user‐generated videos. Using LLaVA‐1.6 Mistral 7B, we analyzed 725 keyframes from 142 depression‐related YouTube Shorts to evaluate MLLM performance on abstract social concepts. Focusing on four dimensions of video‐based self‐disclosure, we found that greater prompt detail did not necessarily improve human‐AI alignment. Factors such as concept complexity and genre diversity also influenced outcomes. Our findings highlight both the promise and limits of MLLMs in visual content analysis and call for more human‐centered evaluation and prompt design strategies when using AI in multimodal contexts.