A Large Language Model-Powered Bot for Calligraphy Knowledge Q&A and Appreciation

In the paper, we propose an approach of multimodal RAG (Retrieval-Augmented Generation) on the LLM(Large Language Model) of Qwen-VL to implement a bot for knowledge Q&A(Question and Answer) and appreciation of Chinese brush calligraphy. Firstly, electronic files on calligraphy, which contain a large number of images of brush strokes and texts, are divided into multiple chucks and vectorized, and then stored in a vector library, as knowledge bas. When a calligraphy enthusiasts input an image or some text in the bot dialog for knowledge questioning or art appreciation, the image features are extracted using the CLIP model, and the text is encoded using the word embedding model. Then the proposed method is used to retrieve the appropriate text and images from the knowledge base and pass them to the LLM, which will answer the questions of the enthusiasts in a multimodal way. In terms of system implementation, the paper uses LangChain framework to orchestrate the task flow, and uses Gradio to build a lightweight interactive interface, which supports mixed graphical input and visualized output. The bot developed in the paper is an assistant for calligraphy enthusiasts to learn and appreciate Chinese calligraphy.