Adchat-TVQA: Innovative Application of LLMs-Based Text-Visual Question Answering Method in Advertising Legal Compliance Review

Advertising legal compliance reviews have always been time-consuming and labor-intensive, and existing Large Language Models(LLMs) are far worse performing than senior industry experts. In this paper, we propose a novel LLMs-based text-visual question answering method called Adchat-TVQA for advertising legal compliance review. After reading an advertising image and the corresponding review question, it will first understand the content on the image through multimodal learning, then optimize the question with Zero-Shot Chain-of-Thought (CoT) prompting method based on predefined industry expert experience, and finally input the augmented question into the large language model for inquiry. The feasibility of this method is verified through system prototype implementation and end-to-end functional tests. In addition, we summarized the characteristics of advertising images and added image segmentation and text box transpose to the data processing process for system performance improvement. Performance testing of the text visual cognitive question task on the AIWIN2021 dataset shows that the method scores higher than Microsoft's LayoutLM, with an increasement of the score from 51.52% to 52.74%.