From Challenges to Solutions: Insights from Our Journey of Developing a Topic Modeling Tool for English and Chinese

Our presentation aims to detail the development journey of a user-friendly topic modeling web tool designed for both English and Chinese texts. This tool was developed over the course of one semester, from September to December 2024, and leverages powerful techniques to sift through extensive document collections to unveil prevailing themes, thus enriching textual data analysis. The initial phase of our project involved extensive study of multiple algorithms, including the traditional Latent Dirichlet Allocation (LDA) and its variants, as well as more advanced models like BERTopic, Embedded Topic Model (ETM), and Contextualized Topic Model (CTM). Our goal was to identify the most effective model that meets our unique requirements, particularly the challenges posed by the morphological differences between English and Chinese. Throughout the development process, we faced numerous challenges, such as model selection and integration of different tools, especially for Chinese, for which necessitated additional steps in preprocessing and embedding are needed because of the inherent differences in language structure between Chinese and western alphabetic languages. In our presentation, we will discuss the various difficulties encountered during model selection and the coding phase, and how these setbacks were instrumental in our iterative development approach. This exploration into the trial and error inherent in digital tool development was crucial for refining our tool's accuracy and usability. Moreover, we will outline our strategies for preserving the tool and its outputs, ensuring that our contributions are sustained within the Digital Humanities field. Our goal is to spark a conversation about the significance of process reflection in DH projects and to engage with fellow researchers in this dynamic community. We are enthusiastic about the possibility of presenting our work at the symposium and eagerly anticipate the opportunity to connect with like-minded professionals and scholars.