Evaluating the performance of ChatGPT in the ceramic knowledge Q&A

Ceramics, as an intangible cultural heritage of humanity, carry rich historical legacy, cultural significance, and unique craftsmanship. However, the study and transmission of ceramic knowledge face significant barriers, including high learning thresholds and a lack of professional practitioners, which limits the dissemination of related knowledge. This severely restricts the popularization and innovative development of ceramic techniques. In recent years, large language models such as ChatGPT have gained widespread attention for their impressive knowledge reasoning and expressive capabilities, potentially aiding in the transmission of ceramic knowledge. This paper presents an empirical study to explore the application of ChatGPT in the field of ancient Chinese ceramics. Our findings show that, despite ChatGPT's strong generative abilities, it exhibits significant deficiencies in foundational professional knowledge within the ceramics field. Furthermore, ChatGPT's performance varies when addressing different types of ceramic-related questions, with the strongest performance in questions related to Yaozhou kiln ceramics and the weakest in those related to Ge kiln ceramics. Overall, our research provides valuable insights into the strengths and limitations of large language models, particularly ChatGPT, in the field of ceramics.