IntentionGPT: Boosting E-commerce User Intent Recognition with Self-data Augmentation and Structure-aware Retrieval

In recent years, Vision Language Models (VLMs) have significantly advanced multi-modal reasoning and generation tasks, offering promising solutions for e-commerce applications like multi-modal user intent recognition. However, VLMs trained on open-domain datasets often struggle with understanding fine-grained e-commerce semantics. This limitation stems primarily from two key factors: the scarcity of labeled multi-modal data for training and the lack of domain-specific knowledge to effectively reason. To overcome the above problems, we introduce IntentionGPT, a novel framework for multi-modal user intent recognition in e-commerce with limited labeled data. Specifically, we first propose a self-data augmentation method to generate diverse synthetic samples from minimal seed labeled samples to tackle data scarcity challenge. Meanwhile, a new collaborative filtering mechanism is proposed to filter out noise samples to enhance data quality. Second, a structure-aware retrieval method is proposed for retrieving domain knowledge to enhance the model's multi-modal reasoning ability. Third, the model soups method are utilized to fuse models from different training stages, combining their strengths and mitigating potential biases for robust performance. This work systematically addresses both data scarcity and domain adaptation challenges, and achieves first place in the WWW2025 multi-modal dialogue system intent recognition challenge.