Semi-Automatic Hierarchical Taxonomy Creation from Existing Taxonomies with Large Language Models

Abstract The development of taxonomies is critical for organizing knowledge in the field of Information Systems, particularly in hierarchical structures that align with human cognitive and navigational preferences. Traditional manual methods for creating taxonomies require substantial expert involvement, which is often impractical under resource constraints. Moreover, these methods are frequently reported without sufficient documentation or evaluation. Therefore, a semi-automatic method is proposed for refining overly detailed taxonomies, where the initial groups are iteratively consolidated and generalized, resulting in a more abstract and practical taxonomy. The method uses large language models (LLMs) to automate the process by leveraging their contextual understanding and generative capabilities. Particularly, LLMs are employed to: (i) refine the taxonomy’s perspective, (ii) decide on merging groups with siblings based on semantic similarity, and (iii) propose representative labels for merged groups. The method uses expert input to validate and fine-tune the various steps of the process. The proposed method was then applied to develop a taxonomy of innovations, using the Cooperative Patent Classification schema, a widely employed classification system for patent documents, as a case study. Since the taxonomy was intended to assist in organizing and classifying patents, the results were compared to those of a manually created taxonomy based on classification performance. The classifier using the taxonomy generated by the proposed method performed comparably to the manually created taxonomy (i.e., Micro-F scores of 0.70 vs. 0.71 and Macro-F1 scores of 0.87 vs. 0.86). Moreover, both taxonomies exhibited similar structural features and groups. Beyond creating an innovation taxonomy and using it for patent classification, the method has broader implications for efficiently generating taxonomies across different domains, offering a transparent and replicable approach.