Use of Artificial Intelligence in Metadata Creation in the Social Science Japan Data Archive: Metadata Extraction in Social Surveys Using Large Language Models

To improve the effectiveness of metadata production in Japanese for social surveys, the Social Science Japan Data Archive (SSJDA) at the University of Tokyo's Institute of Social Science has built a metadata extraction method using OpenAI's API. Since April 1998, SSJDA has been steadily providing data by making metadata manually. However, with the increase in deposited data, we need to improve the efficiency of metadata production. The system demonstrated considerable improvements in processing time and cost efficiency compared to the manual approach, while also showing a high level of concordance with expert-created metadata. In the future SSJDA will implement LLMs in local environments and develop a user-friendly GUI, which is expected to yield further enhancements and to support self-archiving.